[v14] Handle resource cleanup on termination within the inventory control stream #44224
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rosstimothy
added
backport
no-changelog
rosstimothy
force-pushed
the
tross/backport-43644/v14
branch
from
9c79e24 to
9366941
Compare
…tream (#43644) Historically, each component responsible for managing a particular resource has been responsible for deleting said resources if Teleport was shutting down for good. This is done to provide a better user experience by trying to eliminate the chance of a stale resource from being present for the full TTL of the backend item. However, the mechanism to do so requires the process responsible for the resources to delete each resource individually. For dynamic resources this could require several thousand Delete RPCs during shutdown. Since the shutdown process is time bound this could result in some deletions from being successful and lead to the same stale resource issues. This also poses a problem and races with any inbound heartbeats being sent via the inventory control stream. All resource deletion mechanism are unary RPCs that get processed outside of the inventory control stream, which without careful coordination could result in all the deletions occurring before any in flight heartbeats are processed by the inventory control stream. In an attempt to simplify the deletion process a new UpstreamInventoryGoodbye message has been added to the inventory control stream in order to allow deletion to occur in-band. Instead of sending individual delete RPCs when a process is terminating it can now send a single UpstreamInventoryGoodbye via the inventory control stream. Any control streams that receive a GoodBye prior to being terminated indicate to auth that it should remove all resources that were being represented by said stream. Additionally, the DownstreamInventoryHello was updated to include supported capabilities to better coordinate backward compatibility and supported features by the auth server. This allows the agent to get a better understanding of what the auth server supports without having to do a series of version checks.
rosstimothy
force-pushed
the
tross/backport-43644/v14
branch
from
9366941 to
5711136
Compare
espadolini
approved these changes
Jul 17, 2024
fspmarshall
approved these changes
Jul 18, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport #43644 to branch/v14