Add vllm #28931

maresb · 2025-01-25T05:02:48Z

Very rough draft. I will almost certainly require help.

Opened on the advice of @h-vetinari in conda-forge/xformers-feedstock#42

Direct and transitive dependencies:

compressed-tensors Add compressed-tensors #28778
partial-json-parser Add partial-json-parser #28793
outlines-core Outlines core #28863
outlines Build for v0.1.11 outlines-feedstock#33
airportsdata Add airportsdata #28861
depyf Add depyf #28865
libnacl Add libnacl #28930
tensorizer Add tensorizer #28932
xgrammar Add xgrammar #28866

Checklist

github-actions · 2025-01-25T05:04:11Z

Hi! This is the staged-recipes linter and your PR looks excellent! 🚀

conda-forge-admin · 2025-01-25T05:04:23Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/vllm/recipe.yaml) and found some lint.

Here's what I've got...

For recipes/vllm/recipe.yaml:

❌ license_file entry is missing, but is required.
❌ Non noarch packages should have python requirement without any version constraints.
❌ Non noarch packages should have python requirement without any version constraints.

For recipes/vllm/recipe.yaml:

ℹ️ Please depend on pytorch directly. If your package definitely requires the CUDA version, please depend on pytorch =*=cuda*.
ℹ️ Use importlib-metadata instead of importlib_metadata
ℹ️ PyPI default URL is now pypi.org, and not pypi.io. You may want to update the default source url.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/12962027449. Examine the logs at this URL for more detail.}

conda-forge-admin · 2025-01-25T05:20:54Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/vllm/recipe.yaml) and found it was in an excellent condition.

maresb · 2025-01-25T14:41:57Z

Interesting, we're getting different results between CUDA 11.8 and 12.0.

Both fail in the following command:

['cmake', '$SRC_DIR', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', 
'-DVLLM_TARGET_DEVICE=cuda', '-DVLLM_PYTHON_EXECUTABLE=$PREFIX/bin/python', 
'-DVLLM_PYTHON_PATH=$PREFIX/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process:$PREFIX/lib/python39.zip:$PREFIX/lib/python3.9:$PREFIX/lib/python3.9/lib-dynload:$PREFIX/lib/python3.9/site-packages:$PREFIX/lib/python3.9/site-packages/setuptools/_vendor', 
'-DFETCHCONTENT_BASE_DIR=$SRC_DIR/.deps', '-DNVCC_THREADS=1',
'-DCMAKE_JOB_POOL_COMPILE:STRING=compile', '-DCMAKE_JOB_POOLS:STRING=compile=2']

12.0 fails earlier at CUDA detection:

 │ │       -- Caffe2: Found protobuf with new-style protobuf targets.
 │ │       -- Caffe2: Protobuf version 28.2.0
 │ │       -- Could NOT find CUDA (missing: CUDA_INCLUDE_DIRS) (found version "12.0")
 │ │       CMake Warning at $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:31 (message):
 │ │         Caffe2: CUDA cannot be found.  Depending on whether you are building Caffe2
 │ │         or a Caffe2 dependent library, the next warning / error will give you more
 │ │         info.
 │ │       Call Stack (most recent call first):
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
 │ │         CMakeLists.txt:84 (find_package)
 │ │       
 │ │       
 │ │       CMake Error at $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:90 (message):
 │ │         Your installed Caffe2 version uses CUDA but I cannot find the CUDA
 │ │         libraries.  Please set the proper CUDA prefixes and / or install CUDA.
 │ │       Call Stack (most recent call first):
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
 │ │         CMakeLists.txt:84 (find_package)

11.8 gets further:

 │ │       -- Caffe2: Found protobuf with new-style protobuf targets.
 │ │       -- Caffe2: Protobuf version 28.2.0
 │ │       -- Found CUDA: /usr/local/cuda (found version "11.8")
 │ │       -- The CUDA compiler identification is NVIDIA 11.8.89 with host compiler GNU 11.4.0
 │ │       -- Detecting CUDA compiler ABI info
 │ │       -- Detecting CUDA compiler ABI info - done
 │ │       -- Check for working CUDA compiler: $PREFIX/bin/nvcc - skipped
 │ │       -- Detecting CUDA compile features
 │ │       -- Detecting CUDA compile features - done
 │ │       -- Found CUDAToolkit: /usr/local/cuda/include (found version "11.8.89")
 │ │       -- Caffe2: CUDA detected: 11.8
 │ │       -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
 │ │       -- Caffe2: CUDA toolkit directory: /usr/local/cuda
 │ │       -- Caffe2: Header version is: 11.8
 │ │       -- Found Python: $PREFIX/bin/python (found version "3.9.21") found components: Interpreter
 │ │       CMake Warning at $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:140 (message):
 │ │         Failed to compute shorthash for libnvrtc.so
 │ │       Call Stack (most recent call first):
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
 │ │         CMakeLists.txt:84 (find_package)
 │ │       
 │ │       
 │ │       CMake Warning (dev) at $PREFIX/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:441 (message):
 │ │         The package name passed to `find_package_handle_standard_args` (nvtx3) does
 │ │         not match the name of the calling package (Caffe2).  This can lead to
 │ │         problems in calling code that expects `find_package` result variables
 │ │         (e.g., `_FOUND`) to follow a certain pattern.
 │ │       Call Stack (most recent call first):
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:174 (find_package_handle_standard_args)
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
 │ │         CMakeLists.txt:84 (find_package)
 │ │       This warning is for project developers.  Use -Wno-dev to suppress it.
 │ │       
 │ │       -- Could NOT find nvtx3 (missing: nvtx3_dir)
 │ │       CMake Warning at $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:180 (message):
 │ │         Cannot find NVTX3, find old NVTX instead
 │ │       Call Stack (most recent call first):
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
 │ │         CMakeLists.txt:84 (find_package)
 │ │       
 │ │       
 │ │       -- USE_CUDNN is set to 0. Compiling without cuDNN support
 │ │       -- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
 │ │       -- USE_CUDSS is set to 0. Compiling without cuDSS support
 │ │       -- USE_CUFILE is set to 0. Compiling without cuFile support
 │ │       -- Automatic GPU detection failed. Building for common architectures.
 │ │       -- Autodetected CUDA architecture(s): 3.5;5.0;8.0;8.6;8.9;9.0;8.9+PTX;9.0+PTX
 │ │       -- Added CUDA NVCC flags for: -gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_89,code=sm_89;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_89,code=compute_89;-gencode;arch=compute_90,code=compute_90
 │ │       -- Found Torch: $PREFIX/lib/libtorch.so
 │ │       -- CUDA target architectures: 3.5;5.0;8.0;8.6;8.9;9.0
 │ │       -- CUDA supported target architectures: 8.0;8.6;8.9;9.0
 │ │       -- FetchContent base directory: $SRC_DIR/.deps
 │ │       CMake Error at $PREFIX/share/cmake-3.31/Modules/ExternalProject/shared_internal_commands.cmake:943 (message):
 │ │         error: could not find git for clone of cutlass-populate
 │ │       Call Stack (most recent call first):
 │ │         $PREFIX/share/cmake-3.31/Modules/ExternalProject.cmake:3041 (_ep_add_download_command)
 │ │         CMakeLists.txt:29 (ExternalProject_Add)
 │ │       Call Stack (most recent call first):
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
 │ │         $PREFIX/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
 │ │         CMakeLists.txt:84 (find_package)
 │ │       
 │ │       
 │ │       -- USE_CUDNN is set to 0. Compiling without cuDNN support
 │ │       -- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
 │ │       -- USE_CUDSS is set to 0. Compiling without cuDSS support
 │ │       -- USE_CUFILE is set to 0. Compiling without cuFile support
 │ │       -- Automatic GPU detection failed. Building for common architectures.
 │ │       -- Autodetected CUDA architecture(s): 3.5;5.0;8.0;8.6;8.9;9.0;8.9+PTX;9.0+PTX
 │ │       -- Added CUDA NVCC flags for: -gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_89,code=sm_89;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_89,code=compute_89;-gencode;arch=compute_90,code=compute_90
 │ │       -- Found Torch: $PREFIX/lib/libtorch.so
 │ │       -- CUDA target architectures: 3.5;5.0;8.0;8.6;8.9;9.0
 │ │       -- CUDA supported target architectures: 8.0;8.6;8.9;9.0
 │ │       -- FetchContent base directory: $SRC_DIR/.deps
 │ │       CMake Error at $PREFIX/share/cmake-3.31/Modules/ExternalProject/shared_internal_commands.cmake:943 (message):
 │ │         error: could not find git for clone of cutlass-populate
 │ │       Call Stack (most recent call first):
 │ │         $PREFIX/share/cmake-3.31/Modules/ExternalProject.cmake:3041 (_ep_add_download_command)
 │ │         CMakeLists.txt:29 (ExternalProject_Add)
 │ │       
 │ │       
 │ │       -- Configuring incomplete, errors occurred!

conda-forge-admin · 2025-01-25T19:04:34Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/vllm/recipe.yaml) and found some lint.

Here's what I've got...

For recipes/vllm/recipe.yaml:

❌ Selectors in comment form no longer work in v1 recipes. Instead, if / then / else maps must be used. See lines [39, 41, 46, 48, 49].

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/12967644561. Examine the logs at this URL for more detail.}

conda-forge-admin · 2025-01-25T19:07:58Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/vllm/recipe.yaml) and found it was in an excellent condition.

This reverts commit 769e3fb.

h-vetinari · 2025-01-26T04:22:33Z

Thanks @maresb! I had forgotten that there's already #24710, perhaps @mediocretech would be interested in collaborating?

W.r.t. CUDA, we need to move on from 12.0 here, which isn't used anywhere else in conda-forge anymore - it's just that staged-recipes seems to have been forgotten in the context of conda-forge/conda-forge-pinning-feedstock#6630.

maresb · 2025-01-26T04:50:32Z

Oh, I didn't notice that effort, thanks @h-vetinari! Although that's old it looks like @rongou is eager to help! 🚀

Do you think that CUDA 12.0 is actually causing a problem here? I was thinking (i.e. wildly guessing) that we need to patch CMakeLists.txt, but I've never used cmake. 😞

h-vetinari · 2025-01-26T05:02:34Z

Mainly I want to avoid redundant work. As soon as #28938 is in and we have merged main here, I'll be happy to take a look what's going on.

h-vetinari · 2025-01-26T05:05:25Z

In any case, you'll have to address

 │ │       CMake Error at $PREFIX/share/cmake-3.31/Modules/ExternalProject/shared_internal_commands.cmake:943 (message):
 │ │         error: could not find git for clone of cutlass-populate

Add vllm

a11b1c1

Lints

81cb0a8

maresb added 2 commits January 24, 2025 21:32

Fix pytorch matchspec

551ea6d

Remove extraneous? cuda-nvcc

1318567

maresb added 2 commits January 25, 2025 11:12

Try adding a bunch of junk from pytorch recipe

769e3fb

Revert "Try adding a bunch of junk from pytorch recipe"

bb3adea

This reverts commit 769e3fb.

maresb force-pushed the vllm branch from efddfe8 to bb3adea Compare January 25, 2025 19:13

Add git as host dependency

80e4983

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add vllm #28931

Add vllm #28931

maresb commented Jan 25, 2025 •

edited

Loading

github-actions bot commented Jan 25, 2025

conda-forge-admin commented Jan 25, 2025

conda-forge-admin commented Jan 25, 2025

maresb commented Jan 25, 2025

conda-forge-admin commented Jan 25, 2025

conda-forge-admin commented Jan 25, 2025

h-vetinari commented Jan 26, 2025

maresb commented Jan 26, 2025

h-vetinari commented Jan 26, 2025

h-vetinari commented Jan 26, 2025

Add vllm #28931

Are you sure you want to change the base?

Add vllm #28931

Conversation

maresb commented Jan 25, 2025 • edited Loading

github-actions bot commented Jan 25, 2025

conda-forge-admin commented Jan 25, 2025

conda-forge-admin commented Jan 25, 2025

maresb commented Jan 25, 2025

conda-forge-admin commented Jan 25, 2025

conda-forge-admin commented Jan 25, 2025

h-vetinari commented Jan 26, 2025

maresb commented Jan 26, 2025

h-vetinari commented Jan 26, 2025

h-vetinari commented Jan 26, 2025

maresb commented Jan 25, 2025 •

edited

Loading