Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updates for vllm 0.6.2 #12338

Merged
merged 16 commits into from
Nov 12, 2024
Merged

Updates for vllm 0.6.2 #12338

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
b3f1e09
Initial updates for vllm 0.6.2
xiangyuT Sep 29, 2024
30aa803
fix
gc-fu Nov 5, 2024
f5ee889
Change Dockerfile to support v062
gc-fu Nov 6, 2024
c9eab78
Fix
gc-fu Nov 6, 2024
6d14b22
fix examples
gc-fu Nov 6, 2024
e956b7b
Fix
gc-fu Nov 7, 2024
f09963b
done
gc-fu Nov 7, 2024
f91b396
fix
gc-fu Nov 11, 2024
6ad0f20
Update engine.py
gc-fu Nov 12, 2024
ec304eb
Fix Dockerfile to original path
gc-fu Nov 12, 2024
290f085
fix
gc-fu Nov 12, 2024
ab32ca4
add option
gc-fu Nov 12, 2024
0f93953
fix
gc-fu Nov 12, 2024
2eeb2f2
fix
gc-fu Nov 12, 2024
5fd0dce
fix
gc-fu Nov 12, 2024
dbbe5cc
fix
gc-fu Nov 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 17 additions & 4 deletions docker/llm/serving/xpu/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ ARG https_proxy

ENV TZ=Asia/Shanghai
ENV PYTHONUNBUFFERED=1
# To prevent RPC_TIMEOUT ERROR for the first request
ENV VLLM_RPC_TIMEOUT=100000


# Disable pip's cache behavior
Expand Down Expand Up @@ -41,6 +43,14 @@ RUN wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRO
pip install --upgrade colorama && \
# Download all-in-one benchmark and examples
git clone https://github.com/intel-analytics/ipex-llm && \
# The following comment segment is used when building from source...
# cd ipex-llm && \
# git fetch origin pull/12338/head:local_pr && \
# git checkout local_pr && \
# pip uninstall -y ipex-llm && \
# cd python/llm && \
# python setup.py install && \
# cd ../../../ && \
cp -r ./ipex-llm/python/llm/dev/benchmark/ ./benchmark && \
cp -r ./ipex-llm/python/llm/example/GPU/HuggingFace/LLM ./examples && \
# Install vllm dependencies
Expand Down Expand Up @@ -74,13 +84,16 @@ RUN wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRO
rm -rf /tmp/neo && \
mkdir -p /llm && \
cd /llm && \
git clone -b 0.5.4 https://github.com/analytics-zoo/vllm.git /llm/vllm && \
git clone -b 0.6.2 https://github.com/analytics-zoo/vllm.git /llm/vllm && \
cd /llm/vllm && \
pip install -r /llm/vllm/requirements-xpu.txt && \
VLLM_TARGET_DEVICE=xpu python setup.py install && \
pip install setuptools-scm && \
pip install --upgrade cmake && \
VLLM_TARGET_DEVICE=xpu pip install --no-build-isolation -v /llm/vllm && \
# pip install -r /llm/vllm/requirements-xpu.txt && \
# VLLM_TARGET_DEVICE=xpu python setup.py install && \
pip install mpi4py fastapi uvicorn openai && \
pip install gradio==4.43.0 && \
pip install transformers==4.44.2 && \
# pip install transformers==4.44.2 && \
# patch /usr/local/lib/python3.11/dist-packages/fastchat/serve/gradio_web_server.py < /tmp/gradio_web_server.patch && \
pip install ray && \
patch /usr/local/lib/python3.11/dist-packages/fastchat/serve/gradio_web_server.py < /tmp/gradio_web_server.patch
Expand Down
Loading
Loading