How to properly install wheel file built from source?

[mattroos]

I know this is really a python/pip question, but hopefully someone can help...

I've built onnxruntime from source,

 ./build.sh --config Release --build_shared_lib --parallel --build_wheel

and then installed the wheel file...

pip install ./build/Linux/Release/dist/onnxruntime_gpu_tensorrt-1.7.0-cp36-cp36m-linux_x86_64.whl

But it fails to import into python.

>>> import onnxruntime
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'onnxruntime'

If I then do a pip uninstall, it indicates that the installed files were in the expected directories (in the virtual python environment in which I did the pip install).

mroos@synapse:~$ pip uninstall ./build/Linux/Release/dist/onnxruntime_gpu_tensorrt-1.7.0-cp36-cp36m-linux_x86_64.whl 
Found existing installation: onnxruntime-gpu-tensorrt 1.7.0
Uninstalling onnxruntime-gpu-tensorrt-1.7.0:
  Would remove:
    /home/mroos/python_envs/env_p3.6.9_tf2.2/bin/onnxruntime_test
    /home/mroos/python_envs/env_p3.6.9_tf2.2/lib/python3.6/site-packages/onnxruntime/*
    /home/mroos/python_envs/env_p3.6.9_tf2.2/lib/python3.6/site-packages/onnxruntime_gpu_tensorrt-1.7.0.dist-info/*
Proceed (y/n)? y
  Successfully uninstalled onnxruntime-gpu-tensorrt-1.7.0

What am I doing wrong? Why can't python find and import the module?

[ytaous]

hi, @mattroos - usually I do:

1 cd to dist dir
2. pip install onnx onnxruntime_gpu_tensorrt-1.7.0-cp36-cp36m-linux_x86_64.whl --force
3. verify with import again

[snnn]

@yufenglee Any idea about the QDQ test failure?

[yufenglee]

The error says "Unknown model file format version.". ONNX has a IR update recently, but it is not pulled and supported in onnxruntime. You can resolve this issue by rebuilding and reinstall your onnx from the stable release branch .

[yufenglee]

I have a PR to avoid this kind of failure.

[mattroos]

Thanks, @yufenglee! I got pulled onto something else for a few days but will git a go after that.

[mattroos]

Hi, @snnn, @yufenglee, @pranav-prakash, and @ytaous! I'm back on this effort after a hiatus.

After building and installing onnx 1.9.0 and the most resent onnxruntime pull, I'm able to import the CPU version of both into python. I'm now trying to build the TensorRT version. I initially tried with the recent tensorrt version 8.0.0.3. Here is my build command:

./build.sh \
--config Release \
--build_shared_lib \
--parallel \
--build_wheel \
--use_tensorrt \
--tensorrt_home /usr/src/tensorrt \
--cudnn_home /usr/lib/x86_64-linux-gnu \
--cuda_home /usr/local/cuda

I get this missing library error:

-- Found TensorRT headers at /usr/include/x86_64-linux-gnu
-- Find TensorRT libs at /usr/lib/x86_64-linux-gnu/libnvinfer.so;/usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so;TENSORRT_LIBRARY_MYELIN-NOTFOUND
-- Could NOT find TENSORRT (missing: TENSORRT_LIBRARY) 
ERRORCannot find TensorRT library.

Do I need a lower version of TensorRT? Here they state that onnxruntime was built and tested with tensorrt 7.2.2 and 7.2.3.