Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update wheels to use same BLAS / LAPACK as numpy #130

Open
tskisner opened this issue Aug 11, 2022 · 0 comments
Open

Update wheels to use same BLAS / LAPACK as numpy #130

tskisner opened this issue Aug 11, 2022 · 0 comments
Assignees

Comments

@tskisner
Copy link
Member

Within a single python process there may be multiple compiled extensions that link to BLAS/LAPACK. When the first package is imported (e.g. numpy), the lapack library used by numpy will be dlopen'ed by the linker. Later, when another package using lapack is imported (e.g. scipy, so3g, toast, etc), if the lapack library has the same name, the linker will use the previously loaded lapack.

This is an issue across the scientific python stack, and has been solved by the community in the following way:

  • Within the conda ecosystem, package variants can be pinned to a particular blas implementation. This way scipy (and toast) use the same blas as numpy.
  • When building wheels for PyPI, numpy and scipy (and toast) use the OpenBLAS multilib package hosted by Anaconda (https://anaconda.org/multibuild-wheels-staging/openblas-libs/files), which guarantees compatibility across the stack.

We should update the so3g wheels to use this same technique. As a bonus, it will save subtantial CI build time avoiding compilation of OpenBLAS.

At some point in the future we could also consider making conda-forge packages of so3g / sotodlib, but that is a separate exercise.

@tskisner tskisner self-assigned this Aug 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant