DM-40434: Prompt Processing continues processing after timeout or termination #182
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These timeouts make it possible to abort processing instead of having to wait until it finishes (as is the case with a Knative timeout).
Optional is deprecated, and typing.Tuple can be replaced with tuple.
kfindeisen
force-pushed
the
tickets/DM-40434
branch
2 times, most recently
from
ba05510
to
8e3b703
Compare
This interrupt can be used to indicate that the app should clean up, then shut down.
This spec is already assumed by the pipeline, and without explicit documentation it's not obvious why no current branch raises RetriableError.
Factoring out the check makes it easier to handle different exceptions slightly differently.
kfindeisen
force-pushed
the
tickets/DM-40434
branch
from
2196f83
to
509d2ff
Compare
kfindeisen
force-pushed
the
tickets/DM-40434
branch
from
509d2ff
to
ae00209
Compare
If pipeline processing is interrupted (the most likely case), retriability is handled with almost the same logic as pipeline errors (the exception is that an interrupt is our first nondeterministic failure mode). Otherwise, the failure is assumed non-retriable just in case.
This makes it possible to adjust the grace period to accommodate the service's long shutdown time (in particular, syncing to the central repo).
This handler lets us retry services that were shut down during prep_butler or while waiting for raws. The outermost handler is still needed to catch interrupts that slip through the cracks.
kfindeisen
force-pushed
the
tickets/DM-40434
branch
from
ae00209
to
48bbde1
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds configurable timeouts and grace periods to the service container, as well as a handler for TERM signals sent by Knative.
Closes #91.