-
Notifications
You must be signed in to change notification settings - Fork 220
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New multipackage container containing python packages for PrepareAA s… #2588
base: master
Are you sure you want to change the base?
Conversation
combinations/hash.tsv
Outdated
@@ -392,3 +392,4 @@ bioconductor-clusterprofiler=4.6.0,bioconductor-org.hs.eg.db=3.16.0 | |||
xopen=1.1.0,pysam=0.16.0 | |||
bioconductor-purecn=2.4.0,bioconductor-txdb.hsapiens.ucsc.hg38.knowngene=3.16.0,bioconductor-org.hs.eg.db=3.16.0,bioconductor-txdb.hsapiens.ucsc.hg19.knowngene=3.2.2 | |||
python=3.6.13,pandas=1.1.5,numpy=1.19.5,jinja2==2.11.3,pyyaml=5.3,fire=0.3 | |||
python=2.7,pysam=0.20.0,flask=1.1.2,numpy=1.16.5,matplotlib=2.2.4,scipy=1.2.0,mosek::mosek=8.0.60,intervaltree=3.0.2,future=0.18.2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
python2? Its not supported anymore, should we really create container for it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is the only way the software dependencies were working for PrepareAA, because it uses mosek 8 and multiple conflicts arose. I tried many combinations, but only the versions for python2 worked
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll see if I find another solution
combinations/hash.tsv
Outdated
@@ -392,4 +392,4 @@ bioconductor-clusterprofiler=4.6.0,bioconductor-org.hs.eg.db=3.16.0 | |||
xopen=1.1.0,pysam=0.16.0 | |||
bioconductor-purecn=2.4.0,bioconductor-txdb.hsapiens.ucsc.hg38.knowngene=3.16.0,bioconductor-org.hs.eg.db=3.16.0,bioconductor-txdb.hsapiens.ucsc.hg19.knowngene=3.2.2 | |||
python=3.6.13,pandas=1.1.5,numpy=1.19.5,jinja2==2.11.3,pyyaml=5.3,fire=0.3 | |||
python=3.7,pysam=0.16.0,flask=2.2.2,numpy=1.21.6,matplotlib=3.2.2,scipy=1.7.3,mosek::mosek=8.1.83,intervaltree=3.1.0,future=0.18.2 | |||
python=3.7,pysam=0.16.0,flask=2.2.2,numpy=1.21.6,matplotlib=3.2.2,scipy=1.7.3,mosek::mosek=8.1.83,anaconda::intervaltree=3.1.0,future=0.18.2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah now I see. Mosek is not coming from conda-forge or bioconda. That is probably the problem. Any chance to move this to conda-forge or bioconda?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sadly no. I asked the developers a few months ago and they are not eager to put it onto conda-forge or bioconda. This is an issue that always comes up and mosek 8 is required for the software. Newer versions of mosek do not work
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't need permission. It would be nice if they would maintain it in conda-forge but it is not needed, you can do that or anyone else.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried to do it myself with the mosek pip version, but there only version 9 is available. I am unsure how I can do it with the older versions which I only find on their website or their mosek repository
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And you need version 8.1? 9 is not working?
We should create a feedstock with version 9 and get the ball rolling, then we can use the feedstock to try to get an older version deployed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bgruening I now upgraded to python3 and mosek9 which seems to work with the software. I will let you know if this will break something based on comments from the software maintainers. Would this be an adequate solution for you to consider merging ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bgruening I had a discussion with the software maintainer and it is indeed usable with mosek9, which fixes all dependencies on python2 (nf-core/circdna#52) . I also tested it in the nf-core pipeline and the multi container seems to be working.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bgruening is anything else needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are still using this other conda repository. We only support conda-forge and bioconda, I'm really hesitant to break this rule.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, because it is not possible to put it onto conda-forge. The python mosek package does not have a source distribution available which is necessary to build the recipe and create a feedstock. I tried multiple solutions, but it does not seem to work.
However, mosek is already in multiple multi-package containers and now the newer version would be usable with python3. It is also a conda and python package, so I am not sure what kind of security issues could exist.
See title. PrepareAA (https://github.com/jluebeck/AmpliconSuite-pipeline/tree/45cb915702df56d72fa337c93107adb8bb4fe1b8) uses additional python packages for its scripts in the newest version.
Having a conda package for it is sadly not possible as it requires licenses to run, which are free for academic use. Therefore, a multi package container is the only option atm to use it with singularity and docker