Code

Upstream merge 24 12 09#314

Code

Merged

gshtras merged 81 commits intomainROCm/vllm:mainfrom upstream_merge_24_12_09ROCm/vllm:upstream_merge_24_12_09Copy head branch name to clipboardDec 9, 2024

+6,768-4,177

Commits

Commits on Dec 2, 2024

• [Misc][LoRA] Move the implementation of lora bias to punica.py (vllm-project#10829)

  Show description for b45f0d7
  jeejeelee

  authored
  b45f0d7

  Copy full SHA for b45f0d7

• [Misc][XPU] Avoid torch compile for XPU platform (vllm-project#10747)

  Show description for 519cc6c
  

  yma11

  and

  youkaichao

  authored
  519cc6c

  Copy full SHA for 519cc6c

• Fix openvino on GPU (vllm-project#10793)

  janimo

  authored
  9b14d97

  Copy full SHA for 9b14d97

• [Model] Add TP and BNB quantization support to LlavaMultiModalProjector (vllm-project#10834)

  Show description for 4c05edb
  

  Isotr0py

  and

  DarkLight1337

  authored
  4c05edb

  Copy full SHA for 4c05edb

Commits on Dec 3, 2024

• [Bugfix] Prevent benchmark_throughput.py from using duplicated random prompts (vllm-project#10753)

  mgoin

  authored
  4433195

  Copy full SHA for 4433195

• [Model] support bitsandbytes quantization with minicpm model (vllm-project#10842)

  Show description for d746268
  zixuanzhang226

  authored
  d746268

  Copy full SHA for d746268

• [Bugfix] Fix QKVParallelLinearWithShardedLora bias bug (vllm-project#10844)

  Show description for a4cf256
  jeejeelee

  authored
  a4cf256

  Copy full SHA for a4cf256

• [core][distributed] add pynccl broadcast (vllm-project#10843)

  Show description for 21fe7b4
  youkaichao

  authored
  21fe7b4

  Copy full SHA for 21fe7b4

• [torch.compile] remove compilation_context and simplify code (vllm-project#10838)

  Show description for dc5ce86
  youkaichao

  authored
  dc5ce86

  Copy full SHA for dc5ce86

• [Doc] Add github links for source code references (vllm-project#10672)

  Show description for ef51831
  

  russellb

  and

  DarkLight1337

  authored
  ef51831

  Copy full SHA for ef51831

• [Misc] Remove deprecated names (vllm-project#10817)

  Show description for 3257d44
  DarkLight1337

  authored
  3257d44

  Copy full SHA for 3257d44

• [Core][Performance] Add XGrammar support for guided decoding and set it as default (vllm-project#10785)

  Show description for 9323a31
  

  aarnphm

  and

  mgoin

  authored
  9323a31

  Copy full SHA for 9323a31

• [Speculative Decoding] Move indices to device before filtering output (vllm-project#10850)

  Show description for f6084f6
  

  zhengy001

  and

  Yang Zheng(SW)(Alex)

  authored
  f6084f6

  Copy full SHA for f6084f6

• [V1] VLM - Run the mm_mapper preprocessor in the frontend process (vllm-project#10640)

  Show description for 3bc94ca
  

  3 peopleauthored
  3bc94ca

  Copy full SHA for 3bc94ca

• [MISC][XPU] quick fix for XPU CI (vllm-project#10859)

  Show description for 2f2cdc7
  yma11

  authored
  2f2cdc7

  Copy full SHA for 2f2cdc7

• [Bugfix] Only require XGrammar on x86 (vllm-project#10865)

  Show description for 7090c27
  mgoin

  authored
  7090c27

  Copy full SHA for 7090c27

• [Frontend] correctly record prefill and decode time metrics (vllm-project#10853)

  Show description for 7c32b68
  tomeras91

  authored
  7c32b68

  Copy full SHA for 7c32b68

• [Build][Bugfix] Using the correct type hint (vllm-project#10866)

  Show description for a061fe6
  gshtras

  authored
  a061fe6

  Copy full SHA for a061fe6

Commits on Dec 4, 2024

• [Benchmark] Benchmark structured output with datasets (vllm-project#10557)

  Show description for 381ac93
  

  xuechendi

  and

  aarnphm

  authored
  381ac93

  Copy full SHA for 381ac93

• [CI/Build] Replace mean with torch.all in test_pynccl.py (vllm-project#10876)

  Show description for d2bd88b
  tlrmchlsmth

  authored
  d2bd88b

  Copy full SHA for d2bd88b

• Drop ROCm load format check (vllm-project#10767)

  Show description for b5b647b
  wangxiyuan

  authored
  b5b647b

  Copy full SHA for b5b647b

• [ci/build] Change queue name for Release jobs (vllm-project#10875)

  khluu

  authored
  fa2dea6

  Copy full SHA for fa2dea6

• [ci/build] Job to build and push release image (vllm-project#10877)

  khluu

  authored
  c9ca4fc

  Copy full SHA for c9ca4fc

• [bugfix] fixed parameter “n” when set parameter “bestof” > 1 (vllm-project#10854)

  Show description for 8db957e
  o2363286

  authored
  8db957e

  Copy full SHA for 8db957e

• [ci/build] Update vLLM postmerge ECR repo (vllm-project#10887)

  khluu

  authored
  c92acb9

  Copy full SHA for c92acb9

• [LoRA] Change lora_tokenizers capacity (vllm-project#10796)

  Show description for 01d079f
  xyang16

  authored
  01d079f

  Copy full SHA for 01d079f

• [Model] Consolidate ViTs attention implementation without mask (vllm-project#10893)

  Show description for 10398b4
  Isotr0py

  authored
  10398b4

  Copy full SHA for 10398b4

• Benchmark serving structured output (vllm-project#10880)

  Show description for 82eb5ea
  

  xuechendi

  and

  mgoin

  authored
  82eb5ea

  Copy full SHA for 82eb5ea

• [CI/Build] improve python-only dev setup (vllm-project#9621)

  Show description for e4c34c2
  

  dtrifiro

  and

  youkaichao

  authored
  e4c34c2

  Copy full SHA for e4c34c2

Commits on Dec 5, 2024

• [V1] Fix when max_model_len is not divisible by block_size (vllm-project#10903)

  Show description for 2a56e12
  WoosukKwon

  authored
  2a56e12

  Copy full SHA for 2a56e12

• [benchmark] Make H100 benchmark optional (vllm-project#10908)

  khluu

  authored
  7883c2b

  Copy full SHA for 7883c2b

• [Bugfix] Fallback to outlines for complex json schemas (vllm-project#10899)

  Show description for 8d370e9
  mgoin

  authored
  8d370e9

  Copy full SHA for 8d370e9

• [Doc] Create a new "Usage" section (vllm-project#10827)

  Show description for aa39a8e
  DarkLight1337

  authored
  aa39a8e

  Copy full SHA for aa39a8e

• [Bugfix] Fix BNB loader target_modules (vllm-project#10720)

  Show description for 1f958a7
  jeejeelee

  authored
  1f958a7

  Copy full SHA for 1f958a7

• [Misc] Update llama 3.2 template to support system prompt with images (vllm-project#10901)

  Show description for 39c89e7
  tjohnson31415

  authored
  39c89e7

  Copy full SHA for 39c89e7

• [Misc][LoRA] Clean up the function interface of Punica (vllm-project#10917)

  Show description for 571da8f
  jeejeelee

  authored
  571da8f

  Copy full SHA for 571da8f

• [CI/Build] Bump test transformers version (vllm-project#10106)

  Show description for 998eeaf
  

  Isotr0py

  and

  DarkLight1337

  authored
  998eeaf

  Copy full SHA for 998eeaf

• [Misc][Gaudi] Avoid torch.compile and enable lazy collectives (vllm-project#10897)

  Show description for a430652
  kzawora-intel

  authored
  a430652

  Copy full SHA for a430652

• [ci][build] add tests for python only compilation (vllm-project#10915)

  Show description for 9743d64
  youkaichao

  authored
  9743d64

  Copy full SHA for 9743d64

Commits on Dec 6, 2024

• [torch.compile] use size tuning for specific sizes (vllm-project#10933)

  Show description for db87eb6
  youkaichao

  authored
  db87eb6

  Copy full SHA for db87eb6

• [torch.compile] add logging for compilation time (vllm-project#10941)

  Show description for b031a45
  

  youkaichao

  and

  WoosukKwon

  authored
  b031a45

  Copy full SHA for b031a45

• [CI/Build] Fix broken multimodal test (vllm-project#10950)

  DarkLight1337

  authored
  222f5b0

  Copy full SHA for 222f5b0

• [torch.compile] fix deprecated code (vllm-project#10948)

  Show description for a1887f2
  youkaichao

  authored
  a1887f2

  Copy full SHA for a1887f2

• [Core] Support Lark grammars for XGrammar (vllm-project#10870)

  Show description for 8b59631
  mgoin

  authored
  8b59631

  Copy full SHA for 8b59631

• [Doc] add KubeAI to serving integrations (vllm-project#10837)

  Show description for 7406274
  samos123

  authored
  7406274

  Copy full SHA for 7406274

• [misc] fix typo (vllm-project#10960)

  Show description for c05cfb6
  youkaichao

  authored
  c05cfb6

  Copy full SHA for c05cfb6

• [ci] fix broken tests (vllm-project#10956)

  Show description for dcdc3fa
  youkaichao

  authored
  dcdc3fa

  Copy full SHA for dcdc3fa

Commits on Dec 7, 2024

• [Core] Cleanup startup logging a bit (vllm-project#10961)

  Show description for 69d357b
  russellb

  authored
  69d357b

  Copy full SHA for 69d357b

• [Bugfix] Fix test-pipeline.yaml (vllm-project#10973)

  Show description for acf092d
  jeejeelee

  authored
  acf092d

  Copy full SHA for acf092d

• [3/N] Support and implement merged input processor for LLaVA model (vllm-project#10676)

  Show description for 955fa95
  

  DarkLight1337

  and

  ywang96

  authored
  955fa95

  Copy full SHA for 955fa95

• [Build] Fix for the Wswitch-bool clang warning (vllm-project#10060)

  Show description for f13cf9a
  gshtras

  authored
  f13cf9a

  Copy full SHA for f13cf9a

• [Misc][LoRA] Refactor and clean MergedQKVParallelLinearWithLora implementation (vllm-project#10958)

  Show description for b26b4cd
  Isotr0py

  authored
  b26b4cd

  Copy full SHA for b26b4cd

• [Model] Composite weight loading for multimodal Qwen2 (vllm-project#10944)

  Show description for bf0e382
  DarkLight1337

  authored
  bf0e382

  Copy full SHA for bf0e382

• [Doc] Explicitly state that InternVL 2.5 is supported (vllm-project#10978)

  Show description for 1c768fe
  DarkLight1337

  authored
  1c768fe

  Copy full SHA for 1c768fe

• [Model] Update multi-modal processor to support Mantis(LLaVA) model (vllm-project#10711)

  Show description for 39e227c
  DarkLight1337

  authored
  39e227c

  Copy full SHA for 39e227c

• [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (vllm-project#10975)

  Show description for c889d58
  DarkLight1337

  authored
  c889d58

  Copy full SHA for c889d58

• [BugFix][Kernel]: fix illegal memory access in causal_conv1d when conv_states is None (vllm-project#10928)

  Show description for 78029b3
  xffxff

  authored
  78029b3

  Copy full SHA for 78029b3

• [core][executor] simplify instance id (vllm-project#10976)

  Show description for 1b62745
  youkaichao

  authored
  1b62745

  Copy full SHA for 1b62745

• [core][misc] remove use_dummy driver for _run_workers (vllm-project#10920)

  Show description for 7be15d9
  youkaichao

  authored
  7be15d9

  Copy full SHA for 7be15d9

Commits on Dec 8, 2024

• [torch.compile] allow candidate compile sizes (vllm-project#10984)

  Show description for fd57d2b
  youkaichao

  authored
  fd57d2b

  Copy full SHA for fd57d2b

• [V1] Initial support of multimodal models for V1 re-arch (vllm-project#10699)

  Show description for a11f326
  ywang96

  authored
  a11f326

  Copy full SHA for a11f326

• [torch.compile][misc] fix comments (vllm-project#10993)

  Show description for 43b05fa
  youkaichao

  authored
  43b05fa

  Copy full SHA for 43b05fa

Commits on Dec 9, 2024

• [misc] clean up and unify logging (vllm-project#10999)

  Show description for 46004e8
  youkaichao

  authored
  46004e8

  Copy full SHA for 46004e8

• [Doc][V1] Add V1 support column for multimodal models (vllm-project#10998)

  Show description for af7c4a9
  ywang96

  authored
  af7c4a9

  Copy full SHA for af7c4a9

• [torch.compile] add dynamo time tracking (vllm-project#11005)

  Show description for d1c2e15
  youkaichao

  authored
  d1c2e15

  Copy full SHA for d1c2e15

• [V1] Fix Detokenizer loading in AsyncLLM (vllm-project#10997)

  Show description for c690357
  ywang96

  authored
  c690357

  Copy full SHA for c690357

• [Core] Require xgrammar >= 0.1.6 (vllm-project#11021)

  Show description for e691b26
  russellb

  authored
  e691b26

  Copy full SHA for e691b26

• [Platform] Move async output check to platform (vllm-project#10768)

  Show description for aea2fc3
  wangxiyuan

  authored
  aea2fc3

  Copy full SHA for aea2fc3

• [V1] Input Batch Relocation (vllm-project#10962)

  Show description for 25b79d9
  

  varun-sundar-rabindranath

  and

  Varun Sundar Rabindranath

  authored
  25b79d9

  Copy full SHA for 25b79d9

• [ci/build] Recompile CI dependencies list with Python 3.12 (vllm-project#11013)

  Show description for edc4fa3
  khluu

  authored
  edc4fa3

  Copy full SHA for edc4fa3

• [V1] Further reduce CPU overheads in flash-attn (vllm-project#10989)

  Show description for 3b61cb4
  WoosukKwon

  authored
  3b61cb4

  Copy full SHA for 3b61cb4

• [Misc][LoRA] Abstract PunicaWrapper (vllm-project#10955)

  Show description for ca87149
  jeejeelee

  authored
  ca87149

  Copy full SHA for ca87149

• [Model] merged input processor for Phi-3-Vision models (vllm-project#10977)

  Show description for a811dd6
  

  Isotr0py

  and

  DarkLight1337

  authored
  a811dd6

  Copy full SHA for a811dd6

• [Bugfix][Hardware][Gaudi] Bump vllm_hpu_extension version (vllm-project#11028)

  Show description for cbcbdb1
  kzawora-intel

  authored
  cbcbdb1

  Copy full SHA for cbcbdb1

• [v1] fix use compile sizes (vllm-project#11000)

  Show description for 1a2f8fb
  youkaichao

  authored
  1a2f8fb

  Copy full SHA for 1a2f8fb

• [Neuron] Upgrade neuron to 2.20.2 (vllm-project#11016)

  Show description for 9c6459e
  

  xendo

  and

  Jerzy Zagorski

  authored
  9c6459e

  Copy full SHA for 9c6459e

• [ROCm][bugfix] scpecilative decoding worker class (vllm-project#11035)

  Show description for b63ba84
  gshtras

  authored
  b63ba84

  Copy full SHA for b63ba84

• Merge remote-tracking branch 'upstream/main' into develop

  gshtras

  committed
  7c61516

  Copy full SHA for 7c61516

• format

  gshtras

  committed
  401a541

  Copy full SHA for 401a541

• Merge remote-tracking branch 'origin/main' into upstream_merge_24_12_09

  gshtras

  committed
  c9f5c24

  Copy full SHA for c9f5c24

• Merge remote-tracking branch 'upstream/main' into upstream_merge_24_12_09

  gshtras

  committed
  c324ea8

  Copy full SHA for c324ea8