Incomplete tutorial

[sleepingcat4]

System Info

3090 GPU (Ampere model)

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

I was following this tutorial and realised it was missing some important pointers, like how I could be using NVIDIA cards for running the tei gpu locally. I was able to install Nvidia container toolkit after reading the Github README file. Yet, my docker was unable to detect the CUDA devices and was running on CPU.

Note: I can use CUDA on my Jupyter Notebook and I havetwo 3090 on my remote node.

https://huggingface.co/docs/text-embeddings-inference/en/local_gpu

Expected behavior

Docker should be able to detect my GPUs