Slow setup times on windows

[LLukas22]

Setting up cuda-toolkit on a normal windows github actions runner can take >>15 min. I tried to speed this up by using the network installer and specify only the sub-packages i actually need but that also doesn't make any difference. The same setup on linux takes ~2:30 min also via the network method. Am i doing something wrong? What is the recommended way to install only specific packages on windows?

[bvnp43]

ubuntu-20.04
cuda: 12.2.0
method: network
sub-packages: '["nvcc", "cudart"]'

28s

windows-latest
cuda: 12.2.0
method: network
sub-packages: '["nvcc", "cudart", "visual_studio_integration"]'

17m 36s

@Jimver

[bullno1]

I tried messing around with the script: https://github.com/bullno1/cuda-toolkit/commits/master/.
Newer installer, -n don't seem to make a difference.

The actual silent installation in my machine is very fast.
If you look at the log,:when cached, even the local installer takes very little time to download.
It just spends forever to install.

My guess is that it tries to extract the archive and the CI machine hardware is bad for that.
Either spinning disk or throttled CPU.

The installer is also just an archive appended to an executable as most of them are.
The official doc also mentions that you can extract it manually.

Maybe I'll attempt to do just that:

• Extract the few packages that I actually need.
• Repack in a different format.
• Extract on CI.

[bullno1]

It's only 6 minutes today: https://github.com/bullno1/hey/actions/runs/7282548943/job/19845086235#step:4:1

I have a feeling the issue is entirely with Github

[rickardp]

Was this ever looked into?

[bvnp43]

It is visual_studio_integration component, that takes 20 minutes to install every time, other ones seems are pretty fast.

v0.2.14

[rickardp]

This is also what we found. Removing it speeds things up significantly but breaks some tooling. Is the same thing experienced on a Windows desktop? (Don't have one so cannot try)

Maybe it's possible to just unzip the installer files instead if the bug is in the installer itself?

[bullno1]

It is visual_studio_integration component, that takes 20 minutes to install every time, other ones seems are pretty fast.

v0.2.14

Bummer, I think I need that or msbuild won't find the toolchain.

This is also what we found. Removing it speeds things up significantly but breaks some tooling. Is the same thing experienced on a Windows desktop? (Don't have one so cannot try)

Maybe it's possible to just unzip the installer files instead if the bug is in the installer itself?

The installer can be extracted, the problem is the transitive dependencies that you need to also install.
AFAIK, there is very little doc on the structure of the archived files and where to extract them.

[thewh1teagle]

That's so slow! it takes me ~25 minutes only for setup nvidia step in my builds.
Is there any workaround?

Looks like the installer itself is very heavy, I think that it tried to install the driver although I don't need it to compile with cuda, and also it does many checks before install it.

[thewh1teagle]

I had this problem with cuda setup taking forever, like 30 minutes. So I made a simple script setup_cuda.ps1,
and now it finishes in just 2 minutes! ✨

[ichinii]

I had this problem with cuda setup taking forever, like 30 minutes. So I made a simple script setup_cuda.ps1, and now it finishes in just 2 minutes! ✨

Do you mind if we copy your script to our project?

[thewh1teagle]

I had this problem with cuda setup taking forever, like 30 minutes. So I made a simple script setup_cuda.ps1, and now it finishes in just 2 minutes! ✨

Do you mind if we copy your script to our project?

You can. Consider it as MIT license