Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Ensure Historical Stream is cleared before pushing new messages #375

Merged
merged 1 commit into from
Nov 9, 2023

Conversation

morgsmccauley
Copy link
Collaborator

On cancelling an existing Historical Process the Redis Stream is cleared/deleted, but this is only the case if the async task is still active and the cancellation token is consumed. This leads to the following potential problems:

  • When restarting Coordinator, if any Historical Processes are currently active, the task and cancellation token will be lost, meaning the new process will start without clearing the existing Stream
  • When any branch within tokio::select! completes, all others are cancelled. Therefore, if the Historical Process finishes successfully, the cancellation branch will never be executed, meaning the Stream won't be cleared. This becomes a problem if the Redis Stream still contains messages when the next process is kicked off.

This PR updates cancellation so that the Stream is always cleared when a new Historical Process is kicked off.

@morgsmccauley morgsmccauley requested a review from a team as a code owner November 8, 2023 04:42
@morgsmccauley morgsmccauley merged commit 208b224 into main Nov 9, 2023
3 checks passed
@morgsmccauley morgsmccauley deleted the fix/historical-cleanup branch November 9, 2023 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant