Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to install TensorRT-LLM #2597

Open
gowthamtupili opened this issue Dec 20, 2024 · 1 comment
Open

Unable to install TensorRT-LLM #2597

gowthamtupili opened this issue Dec 20, 2024 · 1 comment

Comments

@gowthamtupili
Copy link

Device Details:
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal

I am trying to build tensorRT-LLM for whisper, and I have followed the steps as mentioned in https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper#readme, but I am getting the following errors,
When I am using python version 3.8 and pip install -r requirements.txt command, it is giving me the following error
Collecting tensorrt_llm==0.17.0.dev2024121700 (from -r requirements.txt (line 1))
Using cached tensorrt_llm-0.17.0.dev2024121700.tar.gz (1.4 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [54 lines of output]
INFO:wheel-stub:Testing wheel tensorrt_llm-0.17.0.dev2024121700-cp310-cp310-linux_x86_64.whl against tag cp310-cp310-linux_x86_64
INFO:wheel-stub:Testing wheel tensorrt_llm-0.17.0.dev2024121700-cp312-cp312-linux_aarch64.whl against tag cp312-cp312-linux_aarch64
INFO:wheel-stub:Testing wheel tensorrt_llm-0.17.0.dev2024121700-cp312-cp312-linux_x86_64.whl against tag cp312-cp312-linux_x86_64
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 235, in download_wheel
return download_manual(wheel_directory, distribution, version, config)
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 173, in download_manual
raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
Traceback (most recent call last):
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 235, in download_wheel
return download_manual(wheel_directory, distribution, version, config)
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 173, in download_manual
raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
RuntimeError: Didn't find wheel for tensorrt-llm 0.17.0.dev2024121700

  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/home/mcw/anaconda3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/mcw/anaconda3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/mcw/anaconda3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
      whl_basename = backend.build_wheel(metadata_directory, config_settings)
    File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/buildapi.py", line 29, in build_wheel
      return download_wheel(pathlib.Path(wheel_directory), config_settings)
    File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 237, in download_wheel
      report_install_failure(distribution, version, config, exception_context)
    File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/error.py", line 67, in report_install_failure
      raise InstallFailedError(
  wheel_stub.error.InstallFailedError:
  *******************************************************************************
  
  The installation of tensorrt-llm for version 0.17.0.dev2024121700 failed.
  
  This is a special placeholder package which downloads a real wheel package
  from https://pypi.nvidia.com/. If https://pypi.nvidia.com/ is not reachable, we
  cannot download the real wheel file to install.
  
  You might try installing this package via
  ```
  $ pip install --extra-index-url https://pypi.nvidia.com/ tensorrt-llm
  ```
  
  Here is some debug information about your platform to include in any bug
  report:
  
  Python Version: CPython 3.8.3
  Operating System: Linux 5.15.0-125-generic
  CPU Architecture: x86_64
  Driver Version: 560.28
  CUDA Version: 12.6
  
  *******************************************************************************
  
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

However, when I change the python version from 3.8 to 3.10, and run the same command

/home/mcw/anaconda3/envs/py310/compiler_compat/ld: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so: undefined reference to `opal_list_t_class'
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
error: Cannot link MPI programs. Check your configuration!!!

ERROR: Failed building wheel for mpi4py

My understanding is that
Problem:
mpi4py only works with python version 3.8 or else it will get error
tensorrt-llm only works with python version 3.10

@zhangts20
Copy link

You can try adding the directory containing libopen-rte.so.40 to the LD_LIBRARY_PATH.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants