You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm running llama.cpp server in a docker compose like the following service. It works on Nvidia 2080 Ti. However it fails when I run the exact service on a Nvidia A16 GPU. I get the error: "CUDA Error: all CUDA-capable devices are busy or unavailable". It works without GPU (llama.cpp:server). Do you have any ideas what could have cause this? The only working process is xorg, which is the GUI of the Ubuntu Desktop I think.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I'm running llama.cpp server in a docker compose like the following service. It works on Nvidia 2080 Ti. However it fails when I run the exact service on a Nvidia A16 GPU. I get the error: "CUDA Error: all CUDA-capable devices are busy or unavailable". It works without GPU (llama.cpp:server). Do you have any ideas what could have cause this? The only working process is xorg, which is the GUI of the Ubuntu Desktop I think.
Beta Was this translation helpful? Give feedback.
All reactions