add xformers to cirun

[h-vetinari]

xformers compiles a lot of symbols per architecture, and times out with our default set of architectures, which should look like this:

    if [[ ${cuda_compiler_version} == 11.8 ]]; then
        export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0;7.5;8.0;8.6;8.9+PTX"
    elif [[ ${cuda_compiler_version} == 12.0 ]]; then
        export TORCH_CUDA_ARCH_LIST="5.0;6.0;6.1;7.0;7.5;8.0;8.6;8.9;9.0+PTX"

To avoid timing out with the 6h limit on azure, we're currently just able to build

    if [[ ${cuda_compiler_version} == 11.8 ]]; then
        export TORCH_CUDA_ARCH_LIST="5.0;8.9+PTX"
    elif [[ ${cuda_compiler_version} == 12.0 ]]; then
        export TORCH_CUDA_ARCH_LIST="5.0;7.0;8.0;9.0+PTX"

i.e. 2-4 instead of 9 GPU architectures. I think it would be good to build the full set of architectures on the cirun server. Shouldn't need a particularly beefy machine, just more time.

CC @jaimergp @isuruf @jakirkham @conda-forge/xformers

[jaimergp]

If you just need more time, shouldn't it suffice with cpu-large?

[h-vetinari]

If you just need more time, shouldn't it suffice with cpu-large?

It's a CUDA build, so I think it needs the GPU agents? Not sure if ptxas works on a CPU agent, but happy to try. 🤷

It's possible that things might also work with a large agent, but given the length of compilation (probably close to ~24h in AZP-equivalent runtime), I think using xlarge is justified.

[jakirkham]

CUDA builds typically can be done on CPU. This actually is how RAPIDS builds its packages

Think ptxas should work on a CPU only worker. Even if we find issues, it may be worth the effort to fix them since CPU workers are simply more common. Plus it would make it easier to debug builds offline when needed

That said, if there is a testing need, it could make sense to use the GPU runners for that

On a different note, there are a few other feedstocks here that could be moved to CPU only. Noted them in a separate issue: #1012

[h-vetinari]

Are we OK now with trying the CPU runners? Or should we add both in this PR (but prefer the CPU one on the feedstock if it works)?

[jaimergp]

Oops, almost forgot about this. Merging!

[jaimergp]

A new PR will pop up in https://github.com/conda-forge/.cirun/pulls. Feel free to merge that if I don't see it before :P

[jaimergp]

Done.