Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error linking when the library version on the host is lower than that in the image #266

Open
elezar opened this issue May 29, 2024 · 0 comments

Comments

@elezar
Copy link
Member

elezar commented May 29, 2024

When the Driver version and library version on the host are lower than the library version included in the image, after ldconfig of libnvidia-container is executed during container creation, the symlinks of the library in the container will be linked to a new version of library in image. This causes corresponding libraries to become unavailable. For example, executing nvidia-smi would result in an error: Failed to initialize NVML: Driver/library version mismatch.

reproduce

Use a host which driver version is lower than 525.105.17.

$ docker pull nsblink/ubuntu:test_nvc
$ docker run --rm -e NVIDIA_VISIBLE_DEVICES=1 -e NVIDIA_DRIVER_CAPABILITIES=compute,utility -it --entrypoint /bin/bash nsblink/ubuntu:test_nvc
$ nvidia-smi
$ cd /lib/x86_64-linux-gnu; ls -lah | grep libnvidia-ml
root@da0fd684b11a:/lib/x86_64-linux-gnu# ls -lah | grep libnvidia-ml
lrwxrwxrwx  1 root root    26 Jul 29 07:59 libnvidia-ml.so.1 -> libnvidia-ml.so.525.105.17
-rw-r--r--  1 root root  1.8M May 12  2022 libnvidia-ml.so.470.129.06
-rw-r--r--  1 root root  1.8M Jul 25 08:32 libnvidia-ml.so.525.105.17

patch

Here I provide a patch !225 to solve this problem by recreating symlinks for libraries related to driver versions after ldconfig execution.

Migrated from https://gitlab.com/nvidia/container-toolkit/libnvidia-container/-/issues/3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant