Add ingestion attempt communication with PMPY #3049
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This new communication channel is a set of keys where we will add the uuid of the recently added entity, with the value 0 that represents the first attempt for ingestion.
lagoan
requested review from
ConnorSheremeta,
jefferya and
pgwillia
as code owners
…albertalib/jupiter into add/entity_ingestion_attempts_priority
This is a temporary meassure so we can go through the rest of the tests. We still need to go back and update the code to work with a newer ruby version.
pgwillia
reviewed
Feb 2, 2023
pgwillia
approved these changes
Feb 3, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
pushmi_pullyu is now re-adding entities that failed to be deposited back into the deposit queue and the deposit is retried very quickly. This is an issue because we don't give time for whatever problem occurred during ingestion to be fixed.
We need another communication channel between jupiter and pushmi_pullyu so they can organize and figure out when we can retry the deposits and when to drop them.
This new communication channel is a set of keys where we will add the uuid of the recently added entity, with the value 0 representing the first attempt for ingestion.
Related to pushmi_pully issue 297
What's New
The change is only to add a key-value pair on redis indicating the new entity to be deposited with its current attempt count.
This release should be done at the same time as pushmi_pullyu changes on PR #298