Skip to content

Commit

Permalink
format
Browse files Browse the repository at this point in the history
Signed-off-by: bjmsong <bjmsong@126.com>
  • Loading branch information
mdattack committed Jan 2, 2025
1 parent 165212d commit baa5b0a
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion vllm/worker/model_runner.py
Original file line number Diff line number Diff line change
@@ -1136,7 +1136,8 @@ def load_model(self) -> None:
self.prompt_adapter_manager.create_prompt_adapter_manager(
self.model))

if self.kv_cache_dtype == "fp8" and (current_platform.is_rocm() or current_platform.is_cuda()):
if self.kv_cache_dtype == "fp8" and (current_platform.is_rocm()
or current_platform.is_cuda()):
# Currently only ROCm accepts kv-cache scaling factors
# via quantization_param_path and this will be deprecated
# in the future.

0 comments on commit baa5b0a

Please sign in to comment.