Add _request_timeout
to Kubernetes watches
#15744
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #15622
This is a workaround for an issue in
kubernetes_asyncio
in which a 5 minute request timeout is the default behavior. There is an open PR to resolve there that seems like it'll be released in December at the earliest. In the meantime, we can override the 5 minute timeout by passing aClientTimeout
into_request_timeout
on our watches.This enables
timeout_seconds
to once again do its job of enforcing timeouts on watches.After starting a flow run with a CPU request that was impossible to schedule on my cluster and a 20 minute timeout
pod_watch_timeout_seconds
:Checklist
<link to issue>
"mint.json
.