can't use custom CUDA install path with CUDA_PATH and cuDNN_PATH

[SimonRelu]

Hey,

I'd like to be able to point to a custom install location for cuda and cudnn (not /usr/local/cuda). I believed this should be possible with the CUDA_PATH and cuDNN_PATH env but this doesn't seem to work. I'm using onnxruntime v1.13.1 and build using the following options:

./build.sh --config Release \
           --parallel \
           --use_cuda \
           --cudnn_home $cuDNN_PATH \
           --cuda_home $CUDA_PATH \
           --build_wheel \
           --enable_pybind \
           --disable_ml_ops \
           --skip_tests \
           --cmake_extra_defines 'CMAKE_CUDA_ARCHITECTURES=60;61;70;75;86'

building goes well but when I try to create an inference session I get the following error:

[E:onnxruntime:Default, provider_bridge_ort.cc:1271 TryGetProviderInfo_CUDA] /app/onnxruntime/onnxruntime/core/session/provider_bridge_ort.cc:1074 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory

Any idea why this doesn't work? The ${CUDA_PATH}/lib64/libcublasLt.so.11 file does exist.

When I put the cuda files in /usr/local/cuda it does work, even when CUDA_PATH is not equal to /usr/local/cuda

Here are the different env vars I've tried:

os.environ["CUDA_PATH"] = "/app/lib/cuda"
os.environ["CUDA_HOME"] = "/app/lib/cuda"
os.environ["cuDNN_PATH"] = "/app/lib/cuda"
os.environ["CUDNN_PATH"] = "/app/lib/cuda"
os.environ["CUDNN_HOME"] = "/app/lib/cuda"
os.environ["cuDNN_HOME"] = "/app/lib/cuda"
os.environ["PATH"] = "/app/lib/cuda/lib64:" + os.environ["PATH"]
os.environ["LD_LIBRARY_PATH"] = "/app/lib/cuda/lib64:" + os.environ["LD_LIBRARY_PATH"]

NOTE: running this fixes it:

export LD_LIBRARY_PATH=/app/lib/cuda/lib64:${LD_LIBRARY_PATH}
ldconfig

but that is not a solution since I'd like to install this on machines locally without modifying the root files.

Additionally I'd like to be able to do this for Windows as well.

[BengtGustafsson]

We hve similar problems. We need to ship all .so files needed to our customers. We can't rely on what's pre-installed on the machine or be affected by it. This works for onnxruntime.so itself but for secondary .so files we need to be able to set RPATH to . in onnxrutime.so so that it finds its onxxruntime_providers_cuda so file next to itself and the rpath of onxxruntime_providers_cuda so for it to find cuDNN's so files next to itself.

On Windows we have a similar problem with the fact that Windows now comes with an old DirectML.dll which is not compatible with latest onnxruntime.dll but will be found instead of the one we distribute if the executable is not in the same directory as the dlls. In this case it seems that we have to somehow rename the file from DirectML.dll and load it under this new name, via the onnxruntime cmake system. I can't see another solution for this.