bioconda install: Illegal instruction #41
Comments
|
Hi Ben, Thanks for reporting the issue, and lscpu is really helpful. It looks like you have SSE instruction sets, AVX, and pclmul but not AVX2. Could you try a couple things to help me figure out what's going on? gcc -march=native -dM -E - < /dev/null | grep -i "SSE\|AVX" | sortIt could also help if you tried the statically-linked prebuilt binaries. My guess (from the above) is that it will run using the 128-bit version but crash on the 256-bit version. The other thing you could try is Thanks! Daniel |
|
Thanks for the quick reply. The make seemed to complete successfully, but produce unusable binaries: |
|
Try What's your OS? We've tried under Arch, Ubuntu and CentOS. |
|
I'm on Ubuntu 18.04. I have another 18.04 install on a newer laptop and that works fine, I guess because it has the required instructions. I seem to be having problems with zlib, as you say. But not sure it is a race condition since it still occurs without |
|
Hi, It seems that dashing_128 was using AVX registers, even though avx2 was disabled. This modified version might work now. And I've modified how zstd is built, so if that doesn't work, I'd recommend cloning from scratch and trying to recompile. Sorry for the hassle, and thanks again for the feedback. |
|
Hi, That modified version threw an illegal instruction. The |
Merged
|
Glad to hear it! I was also able to find what was causing the problem in parallel build. ( |
|
Thanks, afraid I still see dashing_s128 throwing an illegal instruction error though. |
|
Hi, Unfortunately it seems like dashing 0.4.0 from bioconda still fails with illegal instruction. I wonder if simply updating the bioconda package to the newest version might help things? |
|
Hi ben, Sure, let me try to get some updates into conda. Last I was working with them, the build was a bit more complex than they tend to prefer (due to simd detection), and I'll update the builtin binaries as well. Thanks for letting me know. Daniel |
|
Thanks for the efforts. Ideally, would it be possible to compile dashing in such a way that there is only a single binary, but one that calls different compiled code based on what instruction set it finds at runtime? I believe https://github.com/bbuchfink/diamond does this (both in distributed binary and through conda), for instance, but could be wrong. |
|
I think it's Diamond is compiling from source by specifying architecture, e.g., https://github.com/bbuchfink/diamond/blob/master/Dockerfile#L11, which is simply using cmake and building for bare x86-64 (https://github.com/bioconda/bioconda-recipes/blob/master/recipes/diamond/build.sh). I'll work with bioconda again for an update; it should be able to just clone recursively and make. And I've updated the statically-linked binaries here, which I'll merge into main assuming that CI tests pass. I've seen some tools that use fat binaries and select at runtime, but some OS's don't like running executables with any unsupported operations. And the way that Dashing is written, it would be difficult to separate every function using SIMD that way. I think the easiest way would be to install a wrapper script and all statically-linked binaries as a Python package; I'll think about that. tl;dr: Binaries are updated as of this commit, and I'll work with bioconda for updated releases as well as consider a the wrapper script in the next couple of weeks. |
|
re diamond, I think that you are right - without bothering to dig through the source code, at http://www.diamondsearch.org/index.php?pages/installation/ it says
which suggests it looks for architecture at runtime. At the bioconda build file https://github.com/bioconda/bioconda-recipes/blob/master/recipes/diamond/build.sh it removes the Anyway, that is all of little help to here, if choosing at runtime is hard to code. I had the impression that anaconda could build for multiple architectures, but am not sure on the specifics - python wrapper sounds fine though sounds like you'd be re-inventing the wheel somewhat (though more useful for non-bioconda peoples). Instead of asking for a recursive git clone you might also just release a proper source archive, not just the one github auto-generates, and then bioconda can grab that. Anyway, I'll stop typing half-random thoughts now. Thanks. |
|
Hi, Server info: and: Thanks. |
|
Hi alienzj -- Thanks for the report! I'll give it a look. The flags should help. I'm not surprised that 512 failed, since your machine doesn't have avx512 support. In the meantime, can you try Also, on this matter, I'm planning on using a wrapper like MMSeqs2's script which compiles every possible version and uses a script to select the correct hardware version, which I'll be working on in the next week or so. |
|
Hi again, Sorry to bring this old thing up again. Unfortunately dashing in conda (v1.0) is a regression - I'm getting illegal instruction on EPYC servers, and dashing v2 doesn't appear to have a bioconda package at all. v0.4.0 on bioconda works though, hence why I say regression. It would be great if the package could choose the correct architecture at runtime e.g. like DIAMOND does. Thanks, ben |
|
Hi Ben, I agree - it would be nice. It would be possible to put together a wrapper that selects the right version. And thanks for letting me know about illegal instruction on EPYC; the 256-bit vector version should work on it, but if it's not there's a bug. I'll see if I can figure out what the problem is. If I can't find it, I'll comb through the binary for a wrong instruction. When I get time, I'll try to set up bioconda for Dashing2. Hopefully it's a bit more flexible than it was the last time I tried. Thanks, Daniel |
|
If you have time, could you try the s256 binary from this folder on your EPYC server? I tried something that might have fixed the problem, but I don't have one to test on. |
|
HI, thanks for the quick reply and fix. That s256 exe seems good to me. I guess the idea is to make a new release? |
bernt-matthias
added a commit
to bernt-matthias/bioconda-recipes
that referenced
this issue
Jan 23, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
Trying to use dashing as it seems very useful, thanks.
Not sure if this issue belongs here but on my system the bioconda install gives an illegal instruction error:
Specifically I have
installed. I presume this is an issue with dashing compilation itself, but not 100% sure on that.
Any suggestions? On my other computer it works without issue. If it helps this is my cpu specs
Thanks, ben.
The text was updated successfully, but these errors were encountered: