Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New multipackage container containing python packages for PrepareAA s… #2588

Open
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

DSchreyer
Copy link
Contributor

See title. PrepareAA (https://github.com/jluebeck/AmpliconSuite-pipeline/tree/45cb915702df56d72fa337c93107adb8bb4fe1b8) uses additional python packages for its scripts in the newest version.

Having a conda package for it is sadly not possible as it requires licenses to run, which are free for academic use. Therefore, a multi package container is the only option atm to use it with singularity and docker

@@ -392,3 +392,4 @@ bioconductor-clusterprofiler=4.6.0,bioconductor-org.hs.eg.db=3.16.0
xopen=1.1.0,pysam=0.16.0
bioconductor-purecn=2.4.0,bioconductor-txdb.hsapiens.ucsc.hg38.knowngene=3.16.0,bioconductor-org.hs.eg.db=3.16.0,bioconductor-txdb.hsapiens.ucsc.hg19.knowngene=3.2.2
python=3.6.13,pandas=1.1.5,numpy=1.19.5,jinja2==2.11.3,pyyaml=5.3,fire=0.3
python=2.7,pysam=0.20.0,flask=1.1.2,numpy=1.16.5,matplotlib=2.2.4,scipy=1.2.0,mosek::mosek=8.0.60,intervaltree=3.0.2,future=0.18.2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

python2? Its not supported anymore, should we really create container for it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the only way the software dependencies were working for PrepareAA, because it uses mosek 8 and multiple conflicts arose. I tried many combinations, but only the versions for python2 worked

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll see if I find another solution

@@ -392,4 +392,4 @@ bioconductor-clusterprofiler=4.6.0,bioconductor-org.hs.eg.db=3.16.0
xopen=1.1.0,pysam=0.16.0
bioconductor-purecn=2.4.0,bioconductor-txdb.hsapiens.ucsc.hg38.knowngene=3.16.0,bioconductor-org.hs.eg.db=3.16.0,bioconductor-txdb.hsapiens.ucsc.hg19.knowngene=3.2.2
python=3.6.13,pandas=1.1.5,numpy=1.19.5,jinja2==2.11.3,pyyaml=5.3,fire=0.3
python=3.7,pysam=0.16.0,flask=2.2.2,numpy=1.21.6,matplotlib=3.2.2,scipy=1.7.3,mosek::mosek=8.1.83,intervaltree=3.1.0,future=0.18.2
python=3.7,pysam=0.16.0,flask=2.2.2,numpy=1.21.6,matplotlib=3.2.2,scipy=1.7.3,mosek::mosek=8.1.83,anaconda::intervaltree=3.1.0,future=0.18.2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah now I see. Mosek is not coming from conda-forge or bioconda. That is probably the problem. Any chance to move this to conda-forge or bioconda?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sadly no. I asked the developers a few months ago and they are not eager to put it onto conda-forge or bioconda. This is an issue that always comes up and mosek 8 is required for the software. Newer versions of mosek do not work

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need permission. It would be nice if they would maintain it in conda-forge but it is not needed, you can do that or anyone else.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to do it myself with the mosek pip version, but there only version 9 is available. I am unsure how I can do it with the older versions which I only find on their website or their mosek repository

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And you need version 8.1? 9 is not working?
We should create a feedstock with version 9 and get the ball rolling, then we can use the feedstock to try to get an older version deployed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bgruening I now upgraded to python3 and mosek9 which seems to work with the software. I will let you know if this will break something based on comments from the software maintainers. Would this be an adequate solution for you to consider merging ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bgruening I had a discussion with the software maintainer and it is indeed usable with mosek9, which fixes all dependencies on python2 (nf-core/circdna#52) . I also tested it in the nf-core pipeline and the multi container seems to be working.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bgruening is anything else needed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are still using this other conda repository. We only support conda-forge and bioconda, I'm really hesitant to break this rule.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, because it is not possible to put it onto conda-forge. The python mosek package does not have a source distribution available which is necessary to build the recipe and create a feedstock. I tried multiple solutions, but it does not seem to work.

However, mosek is already in multiple multi-package containers and now the newer version would be usable with python3. It is also a conda and python package, so I am not sure what kind of security issues could exist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants