-
-
Notifications
You must be signed in to change notification settings - Fork 5.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[V1] Improve TP>1 Error Handling + Stack Trace #11721
Merged
robertgshaw2-redhat
merged 44 commits into
vllm-project:main
from
neuralmagic:tp-shutdown
Jan 3, 2025
+40
−21
Merged
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit
Hold shift + click to select a range
2d857cd
remove shutdown from LLMEngine
robertgshaw2-redhat 9e70c5f
format
robertgshaw2-redhat f34875c
no need for shutdown in asyncllm
robertgshaw2-redhat 7a777d9
remove from asyncllm
robertgshaw2-redhat dfc9dee
stash
robertgshaw2-redhat c72b45a
update
robertgshaw2-redhat 4e2dc00
fix
robertgshaw2-redhat 0b4b6af
added back explicit del
robertgshaw2-redhat 4c445af
stash
robertgshaw2-redhat 567b424
working
robertgshaw2-redhat 7d04b98
fix failing test
robertgshaw2-redhat 62e1022
remove explicit shutdown calls
robertgshaw2-redhat 0b0ca08
updated
robertgshaw2-redhat 729938a
pdated
robertgshaw2-redhat 0259241
update
robertgshaw2-redhat 58e4b36
working
robertgshaw2-redhat cacf6b0
updated
robertgshaw2-redhat ccc747d
fixup
robertgshaw2-redhat ddc2a97
fixup
robertgshaw2-redhat af0d529
reduce cruft
robertgshaw2-redhat 17e152b
updated
robertgshaw2-redhat 37859d7
finish
robertgshaw2-redhat c29f329
updated
robertgshaw2-redhat 1c4b92a
updated
robertgshaw2-redhat eb9b00b
stash
robertgshaw2-redhat 1da99a8
updated
robertgshaw2-redhat ca7b92d
Merge branch 'main' into tp-shutdown
robertgshaw2-redhat 2743166
updated
robertgshaw2-redhat 8e257c1
stash
robertgshaw2-redhat b7c50dc
revert spurious change
robertgshaw2-redhat dcfd3b8
updated
robertgshaw2-redhat 6e0e0d4
stash
robertgshaw2-redhat 55a6195
updated
robertgshaw2-redhat aa6954f
updated
robertgshaw2-redhat 1d15ae0
remove cruft
robertgshaw2-redhat 0347baa
Update vllm/v1/executor/multiproc_executor.py
robertgshaw2-redhat 20b8fa2
stash
robertgshaw2-redhat 32840f2
Merge branch 'tp-shutdown' of https://github.com/neuralmagic/vllm int…
robertgshaw2-redhat 884879a
switch to SIGUSR1
robertgshaw2-redhat bb86a03
updated
robertgshaw2-redhat 405bcc1
Update vllm/v1/engine/core_client.py
robertgshaw2-redhat 25e0fea
update message
robertgshaw2-redhat efd6270
updated
robertgshaw2-redhat a5a306e
fixed!
robertgshaw2-redhat File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading status checks…
remove from asyncllm
commit 7a777d9ea0f83f706ac6ce3d6eff23f61d61d933
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NOTE: moved to
CoreClient
so that it can be shared acrossAsyncLLM
andLLMEngine