Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing CUDA 11.8? #213

Closed
RaulPPelaez opened this issue Mar 17, 2023 · 5 comments · Fixed by #214
Closed

Missing CUDA 11.8? #213

RaulPPelaez opened this issue Mar 17, 2023 · 5 comments · Fixed by #214

Comments

@RaulPPelaez
Copy link

Hi, we use your github action in NNPOps, we are trying to write workflows for testing with CUDA 11.8 openmm/NNPOps#94 , would you consider adding it?
Thanks.

@Jimver
Copy link
Owner

Jimver commented Mar 18, 2023

Thx for the report! This is available in version v0.2.10 now.

@RaulPPelaez
Copy link
Author

Thanks for the effort!

@iiSeymour
Copy link

I just updated to v0.2.10 for CUDA 11.8 but I'm running into disk limitations - any ideas?
https://github.com/nanoporetech/dorado/actions/runs/4512457985/jobs/7945932085

/usr/bin/sudo /opt/hostedtoolcache/cuda_installer-linux/11.8.0/x64/cuda_installer-linux-5.15.0-1034-azure_11.8.0.run --silent --toolkit --samples
terminate called after throwing an instance of 'boost::filesystem::filesystem_error'
  what():  boost::filesystem::copy_file: No space left on device: "./builds/nsight_systems/host-linux-x64/python/bin/python", "/usr/local/cuda-11.8/nsight-systems-2022.4.2/host-linux-x64/python/bin/python"
Aborted (core dumped)
Warning: Error during installation: Error: The process '/usr/bin/sudo' failed with exit code [13](https://github.com/nanoporetech/dorado/actions/runs/4512457985/jobs/7945932085#step:4:14)4

@Jimver
Copy link
Owner

Jimver commented Mar 24, 2023

I just updated to v0.2.10 for CUDA 11.8 but I'm running into disk limitations - any ideas?
https://github.com/nanoporetech/dorado/actions/runs/4512457985/jobs/7945932085

/usr/bin/sudo /opt/hostedtoolcache/cuda_installer-linux/11.8.0/x64/cuda_installer-linux-5.15.0-1034-azure_11.8.0.run --silent --toolkit --samples
terminate called after throwing an instance of 'boost::filesystem::filesystem_error'
  what():  boost::filesystem::copy_file: No space left on device: "./builds/nsight_systems/host-linux-x64/python/bin/python", "/usr/local/cuda-11.8/nsight-systems-2022.4.2/host-linux-x64/python/bin/python"
Aborted (core dumped)
Warning: Error during installation: Error: The process '/usr/bin/sudo' failed with exit code [13](https://github.com/nanoporetech/dorado/actions/runs/4512457985/jobs/7945932085#step:4:14)4

@iiSeymour this is a known issue. Can you check if the fix in #207 (comment) fixes it?

@iiSeymour
Copy link

I need the whole toolkit so ended up going with a different solution. Just clearing out some space manually on the Linux runner solved it:

- name: Clean up diskspace
  if: matrix.os == 'ubuntu-20.04'
  run: sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/share/boost "$AGENT_TOOLSDIRECTORY"

https://github.com/nanoporetech/dorado/pull/135/files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants