Qwen2-VL Model support #1

atlury · 2024-09-23T13:46:31Z

Hello

Will it be possible to include support for Qwen2-VL model? Thank you

zhaocc1106 · 2024-09-24T02:05:14Z

Hello

Will it be possible to include support for Qwen2-VL model? Thank you

It maybe difficult now because trtllm do not support M-ROPE(NVIDIA/TensorRT-LLM#2183). I will follow up continuously.

atlury · 2024-09-24T02:22:48Z

Thank you!

Dimensionzw · 2024-11-25T13:24:15Z

Hello
Will it be possible to include support for Qwen2-VL model? Thank you

It maybe difficult now because trtllm do not support M-ROPE(NVIDIA/TensorRT-LLM#2183). I will follow up continuously.

qwen2-vl and m-rope have been supported in the latest tensorrtllm master. Will grps consider supporting it? refer to：
https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/multimodal

zhaocc1106 · 2024-11-26T01:37:18Z

Hello
Will it be possible to include support for Qwen2-VL model? Thank you

It maybe difficult now because trtllm do not support M-ROPE(NVIDIA/TensorRT-LLM#2183). I will follow up continuously.

qwen2-vl and m-rope have been supported in the latest tensorrtllm master. Will grps consider supporting it? refer to： https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/multimodal

Implementing the Qwen2-VL processor in C++ is a bit complicated, but I’m trying to support it.

zhaocc1106 · 2024-12-13T10:41:10Z

beta is ok, i will push in recent days.

zhaocc1106 · 2024-12-17T12:35:59Z

Master branch have support. Docker image is beta version at now and will update in the futrue.

Dimensionzw · 2024-12-18T03:09:52Z

Master branch have support. Docker image is beta version at now and will update in the futrue.主分支有支持。 Docker 镜像目前是 beta 版本，将来会更新。

Thank you very much, but the image registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12 cannot be pulled, and an error is reported：

Login Succeeded
image registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
image_name_tag grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
name_space: opengrps
image_name: grps_gpu
docker pull registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
Error response from daemon: manifest for registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12 not found: manifest unknown: manifest unknown
Error: Process completed with exit code 1.

zhaocc1106 · 2024-12-18T03:18:42Z

Master branch have support. Docker image is beta version at now and will update in the futrue.主分支有支持。 Docker 镜像目前是 beta 版本，将来会更新。

Thank you very much, but the image registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12 cannot be pulled, and an error is reported：
Login Succeeded
image registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
image_name_tag grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
name_space: opengrps
image_name: grps_gpu
docker pull registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
Error response from daemon: manifest for registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12 not found: manifest unknown: manifest unknown
Error: Process completed with exit code 1.

Current use image beta version: registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12_beta

Docker image will be update in the future.

Dimensionzw · 2024-12-23T02:50:30Z

Master branch have support. Docker image is beta version at now and will update in the futrue.主分支有支持。 Docker 镜像目前是 beta 版本，将来会更新。

Thank you very much, but the image registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12 cannot be pulled, and an error is reported：
Login Succeeded
image registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
image_name_tag grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
name_space: opengrps
image_name: grps_gpu
docker pull registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
Error response from daemon: manifest for registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12 not found: manifest unknown: manifest unknown
Error: Process completed with exit code 1.
Current use image beta version: registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12_beta

Docker image will be update in the future.

During the execution of the command “grpst archive .”, an error keeps reporting：

>>>> Archiving project(.) to ./server.mar.
>>>> Check inference conf(./conf/inference.yml).
>>>> Check inference conf success.
>>>> Check server conf(./conf/server.yml).
>>>> Check server conf success.
>>>> Build customized project and unit test.
WARNING 2024-12-23 10:06:50 rm -rf /opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/build 
INFO 2024-12-23 10:06:51 INFO read config from .config file 
INFO 2024-12-23 10:06:51 INFO cuda_enable=1 torch_enable=0 tf_enable=0 trt_enable=0 libtorch_path= libtensorflow_path= libtensorrt_path= 
-- The C compiler identification is GNU 13.2.0
-- The CXX compiler identification is GNU 13.2.0
-- The CUDA compiler identification is NVIDIA 12.6.85
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- CMAKE_BUILD_TYPE: RelWithDebInfo
CMake Warning (dev) at CMakeLists.txt:68 (find_package):
  Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
  Run "cmake --help-policy CMP0074" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  Environment variable CUDA_ROOT is set to:

    /usr/local/cuda

  For compatibility, CMake is ignoring the variable.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at CMakeLists.txt:68 (find_package):
  Policy CMP0146 is not set: The FindCUDA module is removed.  Run "cmake
  --help-policy CMP0146" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda (found version "12.6")
-- CUDA_TOOLKIT_ROOT_DIR: /usr/local/cuda
-- CUDA_VERSION: 12.6
-- CUDA_INCLUDE_DIRS: /usr/local/cuda-12.6/targets/x86_64-linux/include/
-- CUDA_LIBRARIES: /usr/local/cuda/lib64/libcudart_static.a;Threads::Threads;dl;/usr/lib/x86_64-linux-gnu/librt.a
-- CUDA_CUBLAS_LIBRARIES: /usr/local/cuda/lib64/libcublas.so
-- CUDA_CUBLASLT_LIBRARIES: /usr/local/cuda/lib64/libcublasLt.so
-- GPU_ARCHS is not defined. Generating CUDA code for default SMs: 35;37;52;60;61;70;75;80;86;89
-- CMAKE_CUDA_ARCHITECTURES: 35;37;52;60;61;70;75;80;86;89
-- CMAKE_CUDA_FLAGS:  -Xcompiler -Wno-deprecated-declarations
-- CUDA_NVCC_FLAGS: -std=c++17;-Xcompiler=-fno-gnu-unique;;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_89,code=sm_89
-- Found MPI_C: /opt/hpcx/ompi/lib/libmpi.so (found version "3.1")
-- Found MPI_CXX: /opt/hpcx/ompi/lib/libmpi.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Using MPI_INCLUDE_PATH: /opt/hpcx/ompi/include;/opt/hpcx/ompi/include/openmpi;/opt/hpcx/ompi/include/openmpi/opal/mca/hwloc/hwloc201/hwloc/include;/opt/hpcx/ompi/include/openmpi/opal/mca/event/libevent2022/libevent;/opt/hpcx/ompi/include/openmpi/opal/mca/event/libevent2022/libevent/include
-- Using MPI_LIBRARIES: /opt/hpcx/ompi/lib/libmpi.so
-- system-nameLinux
CMake Deprecation Warning at third_party/tokenizers-cpp/msgpack/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
CMake Deprecation Warning at third_party/tokenizers-cpp/sentencepiece/CMakeLists.txt:15 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- VERSION: 0.2.00
-- Found Protobuf: /usr/local/lib/libprotobuf.so (found version "3.9.2")
-- CMAKE_CXX_FLAGS_RELWITHDEBINFO: -O2 -g -DNDEBUG
-- USE_CXX11_ABI: ON
-- ========================= Importing and creating target nvinfer ==========================
-- Looking for library nvinfer
-- Library that was found /usr/local/tensorrt/targets/x86_64-linux-gnu/lib/libnvinfer.so
-- ==========================================================================================
-- Building for TensorRT version: 10.7.0, library version: 10
-- TRT_INCLUDE_DIR: /usr/local/tensorrt/include, TRT_LIB_DIR: /usr/local/tensorrt/targets/x86_64-linux-gnu/lib, TRT_LIB: nvinfer
-- TensorRT version 10 >= 10, int64 dimension is enabled
-- COMMON_HEADER_DIRS: /opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/src;/include;/usr/local/tensorrt/include
-- Configuring done (7.6s)
-- Generating done (1.0s)
-- Build files have been written to: /opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/build
[  5%] Running cpp protocol buffer compiler on sentencepiece_model.proto
[  5%] Running cpp protocol buffer compiler on sentencepiece.proto
[  7%] Generating release/libtokenizers_c.a
[ 12%] Building CXX object third_party/tokenizers-cpp/CMakeFiles/tokenizer_cpp_objs.dir/src/sentencepiece_tokenizer.cc.o
[ 12%] Building CXX object third_party/tokenizers-cpp/CMakeFiles/tokenizer_cpp_objs.dir/src/rwkv_world_tokenizer.cc.o
[ 15%] Building CXX object third_party/tokenizers-cpp/CMakeFiles/tokenizer_cpp_objs.dir/src/huggingface_tokenizer.cc.o
[ 17%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/bpe_model.cc.o
[ 25%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/sentencepiece_model.pb.cc.o
[ 25%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/char_model.cc.o
[ 25%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/sentencepiece.pb.cc.o
    Updating crates.io index
warning: spurious network error (3 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
warning: spurious network error (2 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
[ 28%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/error.cc.o
[ 30%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/filesystem.cc.o
warning: spurious network error (1 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
[ 33%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/model_factory.cc.o
[ 35%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/model_interface.cc.o
[ 38%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/normalizer.cc.o
[ 41%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/sentencepiece_processor.cc.o
[ 43%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/unigram_model.cc.o
[ 46%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/util.cc.o
error: failed to get `serde` as a dependency of package `tokenizers-c v0.1.0 (/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/rust)`

Caused by:
  download of config.json failed

Caused by:
  failed to download from `https://index.crates.io/config.json`

Caused by:
  [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
gmake[2]: *** [third_party/tokenizers-cpp/CMakeFiles/tokenizers_c.dir/build.make:71: third_party/tokenizers-cpp/release/libtokenizers_c.a] Error 101
gmake[1]: *** [CMakeFiles/Makefile2:228: third_party/tokenizers-cpp/CMakeFiles/tokenizers_c.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
[ 48%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/word_model.cc.o
[ 51%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/absl/flags/flag.cc.o
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc: In function ‘std::vector<std::basic_string_view<char> > sentencepiece::{anonymous}::ToPieceArray(const std::vector<std::__cxx11::basic_string<char> >&)’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:54:21: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<std::__cxx11::basic_string<char> >::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
   54 |   for (int i = 0; i < v.size(); ++i) out[i] = v[i];
      |                   ~~^~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc: In function ‘void sentencepiece::{anonymous}::ConvertToUnicodeSpansInternal(sentencepiece::SentencePieceText*)’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:67:26: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
   67 |     for (int i = prev; i < prev + mblen; ++i) {
      |                        ~~^~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc: In member function ‘sentencepiece::util::Status sentencepiece::SentencePieceProcessor::PopulateSentencePieceText(std::string_view, std::string_view, const std::vector<long unsigned int>&, const sentencepiece::EncodeResult&, sentencepiece::SentencePieceText*) const’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:578:27: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::basic_string_view<char>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  578 |         for (int i = 0; i < w.size(); ++i) {
      |                         ~~^~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:589:17: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::basic_string_view<char>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  589 |           if (i == w.size() - 1) {
      |               ~~^~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc: In lambda function:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:853:27: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
  853 |         for (int j = 0; j < consumed; j++) {
      |                         ~~^~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:857:17: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
  857 |           if (j == consumed - 1) {
      |               ~~^~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc: In member function ‘std::vector<std::pair<std::vector<sentencepiece::unigram::Lattice::Node*>, float> > sentencepiece::unigram::Lattice::NBest(size_t, bool, float)’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:432:25: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<sentencepiece::unigram::Lattice::Node*>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  432 |       for (int i = 0; i < end_nodes(node->pos).size(); i++) {
      |                       ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:443:25: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<double>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  443 |       for (int i = 0; i < adjusted_probs.size(); i++) {
      |                       ~~^~~~~~~~~~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:454:23: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<sentencepiece::unigram::Lattice::Node*>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  454 |     for (int i = 0; i < end_nodes(node->pos).size(); i++) {
      |                     ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc: In member function ‘virtual sentencepiece::NBestEncodeResult sentencepiece::unigram::Model::SampleEncodeAndScore(std::string_view, float, int, bool, bool) const’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:779:25: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<std::pair<std::vector<sentencepiece::unigram::Lattice::Node*>, float> >::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  779 |       for (int i = 0; i < nbest_samples.size(); i++) {
      |                       ~~^~~~~~~~~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:789:25: warning: comparison of integer expressions of different signedness: ‘const int’ and ‘std::vector<std::pair<std::vector<sentencepiece::unigram::Lattice::Node*>, float> >::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  789 |       if (index_of_best != nbest_samples.size()) {
      |           ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:830:27: warning: comparison of integer expressions of different signedness: ‘std::vector<std::pair<std::vector<std::pair<std::basic_string_view<char>, int> >, float> >::size_type’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
  830 |     while (results.size() < samples) {
      |            ~~~~~~~~~~~~~~~^~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc: In member function ‘sentencepiece::EncodeResult sentencepiece::unigram::Model::EncodeOptimized(std::string_view) const’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:970:20: warning: comparison of integer expressions of different signedness: ‘std::size_t’ {aka ‘long unsigned int’} and ‘const int’ [-Wsign-compare]
  970 |     while (key_pos < size) {
      |            ~~~~~~~~^~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:991:40: warning: comparison of integer expressions of different signedness: ‘const long unsigned int’ and ‘const int’ [-Wsign-compare]
  991 |         if (!has_single_node && length == mblen) {
      |                                 ~~~~~~~^~~~~~~~
[ 51%] Built target tokenizer_cpp_objs
[ 53%] Linking CXX static library libsentencepiece.a
[ 53%] Built target sentencepiece-static
gmake: *** [Makefile:156: all] Error 2
[  7%] Built target tokenizer_cpp_objs
[ 10%] Generating release/libtokenizers_c.a
    Updating crates.io index
warning: spurious network error (3 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
warning: spurious network error (2 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
warning: spurious network error (1 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
error: failed to get `serde` as a dependency of package `tokenizers-c v0.1.0 (/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/rust)`

Caused by:
  download of config.json failed

Caused by:
  failed to download from `https://index.crates.io/config.json`

Caused by:
  [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
gmake[2]: *** [third_party/tokenizers-cpp/CMakeFiles/tokenizers_c.dir/build.make:71: third_party/tokenizers-cpp/release/libtokenizers_c.a] Error 101
gmake[1]: *** [CMakeFiles/Makefile2:228: third_party/tokenizers-cpp/CMakeFiles/tokenizers_c.dir/all] Error 2
gmake: *** [Makefile:156: all] Error 2

Do you have any ideas to solve this problem? The Chinese mirror who replaced Cargo.toml in the tokenizers-cpp project rust folder also reported the same error.

zhaocc1106 · 2024-12-23T06:17:54Z

Master branch have support. Docker image is beta version at now and will update in the futrue.主分支有支持。 Docker 镜像目前是 beta 版本，将来会更新。

Thank you very much, but the image registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12 cannot be pulled, and an error is reported：
Login Succeeded
image registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
image_name_tag grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
name_space: opengrps
image_name: grps_gpu
docker pull registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12
Error response from daemon: manifest for registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12 not found: manifest unknown: manifest unknown
Error: Process completed with exit code 1.
Current use image beta version: registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.5_cudnn9.2_trtllm0.16.0_py3.12_beta
Docker image will be update in the future.

During the execution of the command “grpst archive .”, an error keeps reporting：

>>>> Archiving project(.) to ./server.mar.
>>>> Check inference conf(./conf/inference.yml).
>>>> Check inference conf success.
>>>> Check server conf(./conf/server.yml).
>>>> Check server conf success.
>>>> Build customized project and unit test.
WARNING 2024-12-23 10:06:50 rm -rf /opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/build 
INFO 2024-12-23 10:06:51 INFO read config from .config file 
INFO 2024-12-23 10:06:51 INFO cuda_enable=1 torch_enable=0 tf_enable=0 trt_enable=0 libtorch_path= libtensorflow_path= libtensorrt_path= 
-- The C compiler identification is GNU 13.2.0
-- The CXX compiler identification is GNU 13.2.0
-- The CUDA compiler identification is NVIDIA 12.6.85
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- CMAKE_BUILD_TYPE: RelWithDebInfo
CMake Warning (dev) at CMakeLists.txt:68 (find_package):
  Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
  Run "cmake --help-policy CMP0074" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  Environment variable CUDA_ROOT is set to:

    /usr/local/cuda

  For compatibility, CMake is ignoring the variable.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at CMakeLists.txt:68 (find_package):
  Policy CMP0146 is not set: The FindCUDA module is removed.  Run "cmake
  --help-policy CMP0146" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda (found version "12.6")
-- CUDA_TOOLKIT_ROOT_DIR: /usr/local/cuda
-- CUDA_VERSION: 12.6
-- CUDA_INCLUDE_DIRS: /usr/local/cuda-12.6/targets/x86_64-linux/include/
-- CUDA_LIBRARIES: /usr/local/cuda/lib64/libcudart_static.a;Threads::Threads;dl;/usr/lib/x86_64-linux-gnu/librt.a
-- CUDA_CUBLAS_LIBRARIES: /usr/local/cuda/lib64/libcublas.so
-- CUDA_CUBLASLT_LIBRARIES: /usr/local/cuda/lib64/libcublasLt.so
-- GPU_ARCHS is not defined. Generating CUDA code for default SMs: 35;37;52;60;61;70;75;80;86;89
-- CMAKE_CUDA_ARCHITECTURES: 35;37;52;60;61;70;75;80;86;89
-- CMAKE_CUDA_FLAGS:  -Xcompiler -Wno-deprecated-declarations
-- CUDA_NVCC_FLAGS: -std=c++17;-Xcompiler=-fno-gnu-unique;;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_89,code=sm_89
-- Found MPI_C: /opt/hpcx/ompi/lib/libmpi.so (found version "3.1")
-- Found MPI_CXX: /opt/hpcx/ompi/lib/libmpi.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Using MPI_INCLUDE_PATH: /opt/hpcx/ompi/include;/opt/hpcx/ompi/include/openmpi;/opt/hpcx/ompi/include/openmpi/opal/mca/hwloc/hwloc201/hwloc/include;/opt/hpcx/ompi/include/openmpi/opal/mca/event/libevent2022/libevent;/opt/hpcx/ompi/include/openmpi/opal/mca/event/libevent2022/libevent/include
-- Using MPI_LIBRARIES: /opt/hpcx/ompi/lib/libmpi.so
-- system-nameLinux
CMake Deprecation Warning at third_party/tokenizers-cpp/msgpack/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
CMake Deprecation Warning at third_party/tokenizers-cpp/sentencepiece/CMakeLists.txt:15 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- VERSION: 0.2.00
-- Found Protobuf: /usr/local/lib/libprotobuf.so (found version "3.9.2")
-- CMAKE_CXX_FLAGS_RELWITHDEBINFO: -O2 -g -DNDEBUG
-- USE_CXX11_ABI: ON
-- ========================= Importing and creating target nvinfer ==========================
-- Looking for library nvinfer
-- Library that was found /usr/local/tensorrt/targets/x86_64-linux-gnu/lib/libnvinfer.so
-- ==========================================================================================
-- Building for TensorRT version: 10.7.0, library version: 10
-- TRT_INCLUDE_DIR: /usr/local/tensorrt/include, TRT_LIB_DIR: /usr/local/tensorrt/targets/x86_64-linux-gnu/lib, TRT_LIB: nvinfer
-- TensorRT version 10 >= 10, int64 dimension is enabled
-- COMMON_HEADER_DIRS: /opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/src;/include;/usr/local/tensorrt/include
-- Configuring done (7.6s)
-- Generating done (1.0s)
-- Build files have been written to: /opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/build
[  5%] Running cpp protocol buffer compiler on sentencepiece_model.proto
[  5%] Running cpp protocol buffer compiler on sentencepiece.proto
[  7%] Generating release/libtokenizers_c.a
[ 12%] Building CXX object third_party/tokenizers-cpp/CMakeFiles/tokenizer_cpp_objs.dir/src/sentencepiece_tokenizer.cc.o
[ 12%] Building CXX object third_party/tokenizers-cpp/CMakeFiles/tokenizer_cpp_objs.dir/src/rwkv_world_tokenizer.cc.o
[ 15%] Building CXX object third_party/tokenizers-cpp/CMakeFiles/tokenizer_cpp_objs.dir/src/huggingface_tokenizer.cc.o
[ 17%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/bpe_model.cc.o
[ 25%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/sentencepiece_model.pb.cc.o
[ 25%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/char_model.cc.o
[ 25%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/sentencepiece.pb.cc.o
    Updating crates.io index
warning: spurious network error (3 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
warning: spurious network error (2 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
[ 28%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/error.cc.o
[ 30%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/filesystem.cc.o
warning: spurious network error (1 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
[ 33%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/model_factory.cc.o
[ 35%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/model_interface.cc.o
[ 38%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/normalizer.cc.o
[ 41%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/sentencepiece_processor.cc.o
[ 43%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/unigram_model.cc.o
[ 46%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/util.cc.o
error: failed to get `serde` as a dependency of package `tokenizers-c v0.1.0 (/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/rust)`

Caused by:
  download of config.json failed

Caused by:
  failed to download from `https://index.crates.io/config.json`

Caused by:
  [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
gmake[2]: *** [third_party/tokenizers-cpp/CMakeFiles/tokenizers_c.dir/build.make:71: third_party/tokenizers-cpp/release/libtokenizers_c.a] Error 101
gmake[1]: *** [CMakeFiles/Makefile2:228: third_party/tokenizers-cpp/CMakeFiles/tokenizers_c.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
[ 48%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/word_model.cc.o
[ 51%] Building CXX object third_party/tokenizers-cpp/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/absl/flags/flag.cc.o
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc: In function ‘std::vector<std::basic_string_view<char> > sentencepiece::{anonymous}::ToPieceArray(const std::vector<std::__cxx11::basic_string<char> >&)’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:54:21: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<std::__cxx11::basic_string<char> >::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
   54 |   for (int i = 0; i < v.size(); ++i) out[i] = v[i];
      |                   ~~^~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc: In function ‘void sentencepiece::{anonymous}::ConvertToUnicodeSpansInternal(sentencepiece::SentencePieceText*)’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:67:26: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
   67 |     for (int i = prev; i < prev + mblen; ++i) {
      |                        ~~^~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc: In member function ‘sentencepiece::util::Status sentencepiece::SentencePieceProcessor::PopulateSentencePieceText(std::string_view, std::string_view, const std::vector<long unsigned int>&, const sentencepiece::EncodeResult&, sentencepiece::SentencePieceText*) const’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:578:27: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::basic_string_view<char>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  578 |         for (int i = 0; i < w.size(); ++i) {
      |                         ~~^~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:589:17: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::basic_string_view<char>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  589 |           if (i == w.size() - 1) {
      |               ~~^~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc: In lambda function:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:853:27: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
  853 |         for (int j = 0; j < consumed; j++) {
      |                         ~~^~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/sentencepiece_processor.cc:857:17: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
  857 |           if (j == consumed - 1) {
      |               ~~^~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc: In member function ‘std::vector<std::pair<std::vector<sentencepiece::unigram::Lattice::Node*>, float> > sentencepiece::unigram::Lattice::NBest(size_t, bool, float)’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:432:25: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<sentencepiece::unigram::Lattice::Node*>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  432 |       for (int i = 0; i < end_nodes(node->pos).size(); i++) {
      |                       ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:443:25: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<double>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  443 |       for (int i = 0; i < adjusted_probs.size(); i++) {
      |                       ~~^~~~~~~~~~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:454:23: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<sentencepiece::unigram::Lattice::Node*>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  454 |     for (int i = 0; i < end_nodes(node->pos).size(); i++) {
      |                     ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc: In member function ‘virtual sentencepiece::NBestEncodeResult sentencepiece::unigram::Model::SampleEncodeAndScore(std::string_view, float, int, bool, bool) const’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:779:25: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<std::pair<std::vector<sentencepiece::unigram::Lattice::Node*>, float> >::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  779 |       for (int i = 0; i < nbest_samples.size(); i++) {
      |                       ~~^~~~~~~~~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:789:25: warning: comparison of integer expressions of different signedness: ‘const int’ and ‘std::vector<std::pair<std::vector<sentencepiece::unigram::Lattice::Node*>, float> >::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
  789 |       if (index_of_best != nbest_samples.size()) {
      |           ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:830:27: warning: comparison of integer expressions of different signedness: ‘std::vector<std::pair<std::vector<std::pair<std::basic_string_view<char>, int> >, float> >::size_type’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
  830 |     while (results.size() < samples) {
      |            ~~~~~~~~~~~~~~~^~~~~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc: In member function ‘sentencepiece::EncodeResult sentencepiece::unigram::Model::EncodeOptimized(std::string_view) const’:
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:970:20: warning: comparison of integer expressions of different signedness: ‘std::size_t’ {aka ‘long unsigned int’} and ‘const int’ [-Wsign-compare]
  970 |     while (key_pos < size) {
      |            ~~~~~~~~^~~~~~
/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/sentencepiece/src/unigram_model.cc:991:40: warning: comparison of integer expressions of different signedness: ‘const long unsigned int’ and ‘const int’ [-Wsign-compare]
  991 |         if (!has_single_node && length == mblen) {
      |                                 ~~~~~~~^~~~~~~~
[ 51%] Built target tokenizer_cpp_objs
[ 53%] Linking CXX static library libsentencepiece.a
[ 53%] Built target sentencepiece-static
gmake: *** [Makefile:156: all] Error 2
[  7%] Built target tokenizer_cpp_objs
[ 10%] Generating release/libtokenizers_c.a
    Updating crates.io index
warning: spurious network error (3 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
warning: spurious network error (2 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
warning: spurious network error (1 tries remaining): [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
error: failed to get `serde` as a dependency of package `tokenizers-c v0.1.0 (/opt/nfshub245/ml-hub/multimodal/code/grps_trtllm/third_party/tokenizers-cpp/rust)`

Caused by:
  download of config.json failed

Caused by:
  failed to download from `https://index.crates.io/config.json`

Caused by:
  [35] SSL connect error (OpenSSL/1.1.1w: error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
gmake[2]: *** [third_party/tokenizers-cpp/CMakeFiles/tokenizers_c.dir/build.make:71: third_party/tokenizers-cpp/release/libtokenizers_c.a] Error 101
gmake[1]: *** [CMakeFiles/Makefile2:228: third_party/tokenizers-cpp/CMakeFiles/tokenizers_c.dir/all] Error 2
gmake: *** [Makefile:156: all] Error 2

Do you have any ideas to solve this problem? The Chinese mirror who replaced Cargo.toml in the tokenizers-cpp project rust folder also reported the same error.

I haven't meet this. It seems that is your network issue when access crates.io. You can try set network proxy.

zhaocc1106 · 2024-12-24T12:04:04Z

0817b67
Offical v0.16.0 docker image have been push:
registry.cn-hangzhou.aliyuncs.com/opengrps/grps_gpu:grps1.1.0_cuda12.6_cudnn9.6_trtllm0.16.0_py3.12

Docker image size have been reduced.

zhaocc1106 self-assigned this Sep 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Qwen2-VL Model support #1

Qwen2-VL Model support #1

atlury commented Sep 23, 2024

zhaocc1106 commented Sep 24, 2024

atlury commented Sep 24, 2024

Dimensionzw commented Nov 25, 2024

zhaocc1106 commented Nov 26, 2024

zhaocc1106 commented Dec 13, 2024

zhaocc1106 commented Dec 17, 2024

Dimensionzw commented Dec 18, 2024

zhaocc1106 commented Dec 18, 2024

Dimensionzw commented Dec 23, 2024

zhaocc1106 commented Dec 23, 2024

zhaocc1106 commented Dec 24, 2024

Qwen2-VL Model support #1

Qwen2-VL Model support #1

Comments

atlury commented Sep 23, 2024

zhaocc1106 commented Sep 24, 2024

atlury commented Sep 24, 2024

Dimensionzw commented Nov 25, 2024

zhaocc1106 commented Nov 26, 2024

zhaocc1106 commented Dec 13, 2024

zhaocc1106 commented Dec 17, 2024

Dimensionzw commented Dec 18, 2024

zhaocc1106 commented Dec 18, 2024

Dimensionzw commented Dec 23, 2024

zhaocc1106 commented Dec 23, 2024

zhaocc1106 commented Dec 24, 2024