Replies: 16 comments 1 reply
-
Logs are required for review from WSL teamIf this a feature request, please reply with '/feature'. If this is a question, reply with '/question'. How to collect WSL logsDownload and execute collect-wsl-logs.ps1 in an administrative powershell prompt: The script will output the path of the log file once done. If this is a networking issue, please use collect-networking-logs.ps1, following the instructions here Once completed please upload the output files to this Github issue. Click here for more info on logging View similar issuesPlease view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one and thumbs upping the other issue to help us prioritize it! Open similar issues:
Closed similar issues:
|
Beta Was this translation helpful? Give feedback.
-
|
Bot suggestions are not what I am facing to my knowledge. |
Beta Was this translation helpful? Give feedback.
-
|
These logs will likely be useless. To have the same errors, I will have to reinstall Windows. Everything is permissions denied so I cannot even remove the docker container. It is definitely a file issue but how or what is the cause eludes me. A fresh Windows install should have cleaned house I would have thought to prevent this. The logs I sent, have me getting errors trying to remove the docker container with more permission denials. |
Beta Was this translation helpful? Give feedback.
-
Diagnostic information |
Beta Was this translation helpful? Give feedback.
-
|
@Dart-Sid: based on the error you're seeing, it looks like the root cause is just a missing library inside your distro.
|
Beta Was this translation helpful? Give feedback.
-
|
Initially no. I would be willing to wipe it if you know a way that actually
works. I tried it before but when I started to install Ubuntu again, it
would say I had it installed already. I followed instructions that said to
unregister the distribution and it never seemed to matter it still detected
it somewhere.
…On Wed, Oct 2, 2024, 5:47 PM Blue ***@***.***> wrote:
@Dart-Sid <https://github.com/Dart-Sid>: based on the error you're
seeing, it looks like the root cause is just a missing library inside your
distro.
libnvidia-ml.so.1 seems to be part of cuda. Did you install cuda inside
your distribution ?
—
Reply to this email directly, view it on GitHub
<#12118 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/BLZGQCY2BMP2KXKYKJWKIQDZZRZZ5AVCNFSM6AAAAABPIQKZ3GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBZHAYTONJQHA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
@Dart-Sid did you install CUDA on your WSL? This is from https://github.com/open-webui/open-webui README:
|
Beta Was this translation helpful? Give feedback.
-
|
I will try that in the morning. I am likely going to have to start over
again since the issues started before installation of Cuda in the first
place. I followed nVidia documents when I tried that repair. I didn't want
Cuda support initially and only followed the guide I posted. It just hung
up when it detected the gpu. Does that make sense?
…On Wed, Oct 2, 2024, 10:12 PM Toto ***@***.***> wrote:
@Dart-Sid <https://github.com/Dart-Sid> did you install CUDA on your WSL?
This is from https://github.com/open-webui/open-webui README:
If you wish to utilize Open WebUI with Ollama included or CUDA
acceleration, we recommend utilizing our official images tagged with either
:cuda or :ollama. To enable CUDA, you must install the [Nvidia CUDA
container toolkit](
https://docs.nvidia.com/dgx/nvidia-container-runtime-upgrade/) on your
Linux/WSL system.
—
Reply to this email directly, view it on GitHub
<#12118 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/BLZGQC5OYZP363RT45FOJ23ZZSYYFAVCNFSM6AAAAABPIQKZ3GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJQGQZDCNZXGA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
@Dart-Sid list your
Some Before launching any docker containers make sure your CUDA setup is working. I did a quick test running your Sample docker log output showing what GPU was detected: and the dashboard from Windows side, asking "How to test CUDA on WSL":
|
Beta Was this translation helpful? Give feedback.
-
|
Below is the output from your first 3 requests. There is nothing there or
command not found is all I get. Something is wrong with the way this
machine is built as far as storage is concerned. I do not know how this
even boots. It is strange.
***@***.***:~$ ls -l /usr/lib/wsk/lib/
ls: cannot access '/usr/lib/wsk/lib/': No such file or directory
***@***.***:~$ nvidia-container-cli list
nvidia-container-cli: command not found
***@***.***:~$ nvidia-container-cli info
nvidia-container-cli: command not found
***@***.***:~$
…On Thu, Oct 3, 2024 at 12:37 AM Toto ***@***.***> wrote:
@Dart-Sid <https://github.com/Dart-Sid> list your /usr/lib/wsl/lib/
folder and there are the libraries exported from the Windows side.
libnvidia-ml.so.1 should be there too:
***@***.***:~$ ls -l /usr/lib/wsl/lib/
total 224256
-r-xr-xr-x 1 root root 162552 May 23 17:27 libcuda.so
-r-xr-xr-x 1 root root 162552 May 23 17:27 libcuda.so.1
-r-xr-xr-x 1 root root 162552 May 23 17:27 libcuda.so.1.1
-r-xr-xr-x 1 root root 10524136 May 23 17:27 libcudadebugger.so.1
-r-xr-xr-x 1 root root 801840 Oct 20 2023 libd3d12.so
-r-xr-xr-x 1 root root 6880344 Oct 20 2023 libd3d12core.so
-r-xr-xr-x 1 root root 942048 Mar 31 2024 libdxcore.so
-r-xr-xr-x 1 root root 11742584 May 23 17:27 libnvcuvid.so
-r-xr-xr-x 1 root root 11742584 May 23 17:27 libnvcuvid.so.1
-r-xr-xr-x 1 root root 115888416 May 23 17:27 libnvdxdlkernels.so
-r-xr-xr-x 1 root root 572008 May 23 17:27 libnvidia-encode.so
-r-xr-xr-x 1 root root 572008 May 23 17:27 libnvidia-encode.so.1
-r-xr-xr-x 1 root root 244400 May 23 17:27 libnvidia-ml.so.1
-r-xr-xr-x 1 root root 362960 May 23 17:27 libnvidia-opticalflow.so
-r-xr-xr-x 1 root root 362960 May 23 17:27 libnvidia-opticalflow.so.1
-r-xr-xr-x 1 root root 72656 May 23 17:27 libnvoptix.so.1
lrwxrwxrwx 1 root root 15 Oct 2 21:37 libnvoptix_loader.so.1 -> libnvoptix.so.1
-r-xr-xr-x 1 root root 67702216 May 23 17:27 libnvwgf2umx.so
-r-xr-xr-x 1 root root 715296 May 23 17:27 nvidia-smi
Some nvidia-container-cli output:
***@***.***:~$ nvidia-container-cli list
/dev/dxg
/usr/lib/wsl/drivers/nv_dispwi.inf_amd64_cac9c128a7d2b723/nvidia-smi
/usr/lib/wsl/lib/libnvidia-ml.so.1
/usr/lib/wsl/lib/libcuda.so.1
/usr/lib/wsl/lib/libcudadebugger.so.1
/usr/lib/wsl/lib/libnvidia-encode.so.1
/usr/lib/wsl/lib/libnvidia-opticalflow.so.1
/usr/lib/wsl/lib/libnvcuvid.so.1
/usr/lib/wsl/lib/libdxcore.so
***@***.***:~$ nvidia-container-cli info
NVRM version: 552.55
CUDA version: 12.4
Device Index: 0
Device Minor: 0
Model: NVIDIA RTX A4000
Brand: NvidiaRTX
GPU UUID: GPU-9b5345a4-025d-d2e3-8b1b-fe97775c4935
Bus Location: 00000000:61:00.0
Architecture: 8.6
Before launching any docker containers make sure your CUDA setup is
working.
I did a quick test running your docker run -d -p 3000:8080 --gpus=all -v
ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui
--restart always ghcr.io/open-webui/open-webui:ollama command and the
container launches w/out error.
Sample docker log output showing what GPU was detected:
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA RTX A4000, compute capability 8.6, VMM: yes
llm_load_tensors: ggml ctx size = 0.27 MiB
llm_load_tensors: offloading 32 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 33/33 layers to GPU
llm_load_tensors: CUDA_Host buffer size = 281.81 MiB
llm_load_tensors: CUDA0 buffer size = 4156.00 MiB
llama_new_context_with_model: n_ctx = 8192
llama_new_context_with_model: n_batch = 512
llama_new_context_with_model: n_ubatch = 512
llama_new_context_with_model: flash_attn = 0
llama_new_context_with_model: freq_base = 500000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init: CUDA0 KV buffer size = 1024.00 MiB
llama_new_context_with_model: KV self size = 1024.00 MiB, K (f16): 512.00 MiB, V (f16): 512.00 MiB
llama_new_context_with_model: CUDA_Host output buffer size = 2.02 MiB
llama_new_context_with_model: CUDA0 compute buffer size = 560.00 MiB
llama_new_context_with_model: CUDA_Host compute buffer size = 24.01 MiB
llama_new_context_with_model: graph nodes = 1030
llama_new_context_with_model: graph splits = 2
INFO [main] model loaded | tid="140528950800384" timestamp=1727931994
and the dashboard from Windows side, asking "How to test CUDA on WSL":
open-webui_test.PNG (view on web)
<https://github.com/user-attachments/assets/4f40f9bf-4fc0-431e-9105-6bf4e06dd8ae>
—
Reply to this email directly, view it on GitHub
<#12118 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/BLZGQC7OXK3AMXQ2MYLLWXDZZTJXZAVCNFSM6AAAAABPIQKZ3GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJQGU3DKMBYGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
I ran the log while attempting to add docker support. This log shows me unregistering the Ubuntu install from yesterday and is a reinstall and all the way to doing the steps listed here: https://docs.docker.com/engine/install/ubuntu/ This all fails. I know it sounds simple to some of you but there is something about my version of Windows or other factors that I am not getting results. The Docker items just are not available. I got to where NVIDIA papers led me to use the above link and this link: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installing-the-nvidia-container-toolkit I am no Linux guy or software developer. My intention was to follow a simple guide by Dave and run LLama3.1 The vast array of possibilities are just too much to determine what is actually happening as people keep saying it is no problem for them which isn't helping. I reinstalled Windows to clean up any settings that existed on my computer. Updates ran for both Windows, Chrome, and NVIDIA. Nothing else was installed and I did not alter any Windows Features other than WSL and the VMP. I ran through Dave's process as he was working on a workstation like I am with an NVIDIA card so I assumed all would be the same. Clearly, the only difference being I have Windows 10 Pro and he must be on 11 of some variant yet file access inside the Ubuntu install is lacking. All the links pointing out using NVIDIA specific drivers just circle back to documents I do not understand to know what is right. I run these commands and they fail or are no longer pointing to viable resources which leaves me hanging since I do not know what some of the elements are for. I have logged today's exploits for anyone that can see what they think. I appreciate the help. I hope the higher minds understand you are dealing with a low experience individual. I will need hand holding as they say to get to the root of the problem. I like troubleshooting networks. I have no idea how to track on WSL or Ubuntu (especially since I cannot access the logs) but am willing to learn if pointed to documents that don't read like I know something since I do not. |
Beta Was this translation helpful? Give feedback.
-
Diagnostic information |
Beta Was this translation helpful? Give feedback.
-
|
When you install the Nvidia driver in Windows it adds some libraries that are exported to WSL instances. They should be located under Sample output on my system: |
Beta Was this translation helpful? Give feedback.
-
|
Here is the output from that directory. It matches yours. PS C:\WINDOWS\system32\lxss\lib> ls Mode LastWriteTime Length Name -a---- 9/26/2024 7:17 PM 162664 libcuda.so PS C:\WINDOWS\system32\lxss\lib> |
Beta Was this translation helpful? Give feedback.
-
|
I removed the distro and started again. After installing updates and installing pciutils, update-pciids, and docker, now I am working from https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installing-the-nvidia-container-toolkit. On the first prerequisite section I have run the curl command, the sed command (failed, couldn't open temp file) reran with sudo and it didn't say anything so I assume it completed. Whe I run the sudo apt-get update, it says it cannot verify because the public key wasn't available. See output: myname@DESKTOP-OB0N9MR:~$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
|
Beta Was this translation helpful? Give feedback.
-
|
Looks like pulling Ubuntu with wsl --install -d pulls a simple version or one without certain features. I followed another guide and it worked! Thank you all for your help. It is appreciated. |
Beta Was this translation helpful? Give feedback.
-
Windows Version
Microsoft Windows [Version 10.0.19045.3803]
WSL Version
2.2.4.0
Are you using WSL 1 or WSL 2?
Kernel Version
5.15.153.1-2
Distro Version
Ubuntu 24.04
Other Software
Docker version 24.0.5, build ced0996
Ollama version 0.3.12
Repro Steps
I am attempting to follow a video on Youtube to setup a docker container with Ollama. The steps I tried are as follows.
Installed features for WSL and Virtual Machine platform via Turn Windows features on or off followed by a reboot.
Updated graphics driver from nVidia via GeForce Experience. GTX2060 card.
Open Powershell in administrator mode.
Powershell command: "wsl -install" (which did not work and gave me a list of parameters to use)
Desktop: Installed the WSL2 Linux kernel update package for x64 machines from https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa2h6b3dFNnhMbXQyT1pIaGh0ZnZZYXNwdDZ2QXxBQ3Jtc0trVllyX1gxaDFjdGEwQV9BNF9Fd0RkMGRkZkp5RU9GYUNxM29IN09WdUJ5ellyY1V0QU5tZGM2QU1fUFQyNHRQV3VNNFRUZ1Y4bDYwRlNqQ1l2Ukx0STFHU1pYS3JWYklYOG8yMUgzbnVweUlKU1BBVQ&q=https%3A%2F%2Flearn.microsoft.com%2Fen-us%2Fwindows%2Fwsl%2Finstall-manual%23step-4---download-the-linux-kernel-update-package&v=DYhC7nFRL5I.
Powershell command: "wsl --set-default-version 2"
Powershell command: "wsl --install -d Ubuntu"
Ubuntu operation: Setup user and password.
Ubuntu command: "curl -fsSL https://ollama.com/install.sh | sh"
Ubuntu command: "ollama serve"
Ubuntu command: "ollama pull llama3.1:latest"
Ubuntu command: "sudo snap install docker"
Ubuntu command: "sudo docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama"
Expected Behavior
I should be able to run these steps and have a docker container accessible on port 3000 of my local machine and via its IP that serves Ollama.
Actual Behavior
I tried this installation with an installation of Windows (updated to the latest patch) I had been using for some time with issues arising that seemed to be related to file access either permissions or expected location on the virtual disk. In an attempt to alleviate previous settings, I reinstalled Windows which also updated to the latest patch and attempted the same process. When running the ollama serve command, it would start the process and load up to the graphics card. I have listed the output in the logs as section 1.
It never was able to move past this screen. I researched and attempted to remove and reinstall Ollama which led to this command erroring out due to the port being in use by its previous install. I continued to the next step ollama pull llama3.1:latest which was successful. I tested with ollama run llama3.1:latest and was able to work with the prompts on Ubuntu. I then attempted sudo snap install docker which was successful. Next I ran the command "sudo docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama" which output the second section of the logs. I attempted to run docker with docker run ollama /bin/sh -c "cat /etc/release" with the following error:
docker: permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/create": dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'. All this is new to me and I may have to install differently than the guide provided by https://www.youtube.com/watch?v=DYhC7nFRL5I due to Windows versioning or something else. I cannot find anything related to my issues directly. I am unable to access commands listed due to the installation type of Ubuntu. I added tools for nVidia in an attempt to fix the first issue as I thought it was driver related. It appears to be file related but I am am not sure how to address it. Logs for ollama were empty. If there is anything I can do to help figure this out, let me know what I can do. I have limited knowledge in the space and that is why I went with a walkthrough video.
Diagnostic Logs
Section1:
$ ollama serve
2024/10/02 11:03:50 routes.go:1153: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/jamie/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2024-10-02T11:03:50.292-07:00 level=INFO source=images.go:753 msg="total blobs: 0"
time=2024-10-02T11:03:50.292-07:00 level=INFO source=images.go:760 msg="total unused blobs removed: 0"
time=2024-10-02T11:03:50.293-07:00 level=INFO source=routes.go:1200 msg="Listening on 127.0.0.1:11434 (version 0.3.12)"
time=2024-10-02T11:03:50.300-07:00 level=INFO source=common.go:135 msg="extracting embedded files" dir=/tmp/ollama3671702458/runners
time=2024-10-02T11:04:05.305-07:00 level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[cpu_avx2 cuda_v11 cuda_v12 rocm_v60102 cpu cpu_avx]"
time=2024-10-02T11:04:05.306-07:00 level=INFO source=gpu.go:199 msg="looking for compatible GPUs"
time=2024-10-02T11:04:07.342-07:00 level=INFO source=types.go:107 msg="inference compute" id=GPU-8b9047fd-c962-cba0-0479-a483168f84cc library=cuda variant=v12 compute=7.5 driver=12.7 name="NVIDIA GeForce RTX 2060" total="6.0 GiB" available="5.0 GiB"
Section 2:
sudo docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama
Unable to find image 'ghcr.io/open-webui/open-webui:ollama' locally
ollama: Pulling from open-webui/open-webui
a2318d6c47ec: Pull complete
40d734479f14: Pull complete
0b61b7b259eb: Pull complete
081a3493c0e7: Pull complete
ccc2c5b1cfa8: Pull complete
4f4fb700ef54: Pull complete
437d5c5d5b79: Pull complete
b392a61474cc: Pull complete
88c22d59192b: Pull complete
675cda623171: Pull complete
b803d29c1d3d: Pull complete
81fdc9fe4355: Pull complete
834975622fe2: Pull complete
776d131ba774: Pull complete
bec920c09511: Pull complete
Digest: sha256:fd4902daca0e8358f456ea4d48abceaf40edd3f9c8ba1b8614e912820bee427e
Status: Downloaded newer image for ghcr.io/open-webui/open-webui:ollama
75ab12e9bd8a39d9e3760db22736a009bae56792f33e77654e3347a72fdbd599
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown.
Beta Was this translation helpful? Give feedback.
All reactions