sudo docker start local-ai Error response from daemon: load library failed: libnvidia-ml.so.1

[baditaflorin]

LocalAI version:

Environment, CPU architecture, OS, and Version:

Linux v81gpu01 6.8.0-50-generic #51-Ubuntu SMP PREEMPT_DYNAMIC Sat Nov  9 17:58:29 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Describe the bug
Trying to run this command

docker run -p 8080:8080 --gpus all --name local-ai -ti localai/localai:latest-aio-gpu-nvidia-cuda-12

Provides me this output

sudo docker start local-ai
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown
Error: failed to start containers: local-ai

To Reproduce

Expected behavior

Logs

Additional context

[mudler]

@baditaflorin sounds you are having installation issues with the Nvidia container toolkit. Did you followed their docs? https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installing-with-ap