Testing dependencies in Docker

[marissafujimoto]

• rearranging
• Fix Bowtie2 installation command
• Add seqtk installation to dockerfile
• Add hello world bash script
• Call dependencies in bash script
• Pass arguments in dependency POC

Description

Addressing #2. Right now just focused on quick and fast test of dependencies. Running through bash with

docker run -it $(docker build -q --platform linux/amd64 .) bash

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Terminal output

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[cansavvy]

Thanks for getting this started! This is great! We're moving forward.

I need to check with Berger lab that the the fasta files I'm adding here are the correct ones for the samples we're processing for the example but it does work if from the extdata directory I run:

docker run -it $(docker build -q --platform linux/amd64 .) bash /home/scripts/dependency-poc.sh

Couple overall thoughts we should think about after getting the proof of concept here through:

1. Probably will have users use volumes instead of copying the files over to the image. This way users won't have to rebuild the image each time they want to run it. Plus it will allow files to be saved more easily. https://docs.docker.com/engine/storage/volumes/

2. Probably will want these software programs to be called from anywhere as opposed to /home/biodocker/bin/bowtie2 type deals so that way if "power" users decide to use the image interactively that doesn't become a roadblock for them.

[marissafujimoto]

Thanks for getting this started! This is great! We're moving forward.

I need to check with Berger lab that the the fasta files I'm adding here are the correct ones for the samples we're processing for the example but it does work if from the extdata directory I run:

docker run -it $(docker build -q --platform linux/amd64 .) bash /home/scripts/dependency-poc.sh

Couple overall thoughts we should think about after getting the proof of concept here through:

1. Probably will have users use volumes instead of copying the files over to the image. This way users won't have to rebuild the image each time they want to run it. Plus it will allow files to be saved more easily. https://docs.docker.com/engine/storage/volumes/
2. Probably will want these software programs to be called from anywhere as opposed to /home/biodocker/bin/bowtie2 type deals so that way if "power" users decide to use the image interactively that doesn't become a roadblock for them.

1. I was actually testing earlier with volumes so I think that shouldn't be an issue for the final version.
2. I can organize the dependencies on the docker image if that's a use case. Just basically would move the binaries to /usr/local/bin and add them to the path. Maybe we could also move any source code to /usr/src just to make the directories a little cleaner. Let me know what you think!

Anything blocking this pr though?

[marissafujimoto]

@cansavvy small update here: 4778b3d to sym link / install all the dependencies to /usr/local/bin. Just in case anyone needs to run them within the container.

[marissafujimoto]

I just moved this up so that the image layers are doing the more general / slow steps first. Just might save someone a couple minutes if they need to add a dependency later. https://docs.docker.com/get-started/docker-concepts/building-images/understanding-image-layers/

[marissafujimoto]

Though looking at it now we are definitely pretty far from the most efficient dockerfile...

[cansavvy]

Yeah, it's okay, we will polish. we're still in the stage of figuring out what is needed here.

[cansavvy]

I have further edits here that we'll need to incorporate at some point: https://github.com/FredHutch/pgmap/pull/8/files#diff-25e231b330e6b4280e139dcfb0868218749cdbd08abb6c4baa396b8478642a2d

but this will still be in flux for a bit I suspect.

[cansavvy]

@cansavvy small update here: 4778b3d to sym link / install all the dependencies to /usr/local/bin. Just in case anyone needs to run them within the container.

Oh great. This is good. thanks!

[cansavvy]

Yeah, it's okay, we will polish. we're still in the stage of figuring out what is needed here.