Skip to content

Commit

Permalink
[Bugfix] Significant performance drop on CPUs with --num-scheduler-st…
Browse files Browse the repository at this point in the history
…eps > 1 (vllm-project#11794)
  • Loading branch information
DamonFool authored Jan 8, 2025
1 parent 973f5dc commit a4e2b26
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions vllm/engine/arg_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -1157,6 +1157,12 @@ def create_engine_config(self,
if self.enable_chunked_prefill and self.pipeline_parallel_size > 1:
raise ValueError("Multi-Step Chunked-Prefill is not supported "
"for pipeline-parallel-size > 1")
from vllm.platforms import current_platform
if current_platform.is_cpu():
logger.warning("Multi-Step (--num-scheduler-steps > 1) is "
"currently not supported for CPUs and has been "
"disabled.")
self.num_scheduler_steps = 1

# make sure num_lookahead_slots is set the higher value depending on
# if we are using speculative decoding or multi-step
Expand Down

0 comments on commit a4e2b26

Please sign in to comment.