Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Do not eat bitmap backfill errors #871

Merged
merged 3 commits into from
Jul 16, 2024
Merged

fix: Do not eat bitmap backfill errors #871

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6a2bcf8
fix: Ensure while loop terminates with error instead of skipping
darunrs Jul 16, 2024
f4a9eb5
Do not throw error if bitmap process encounters an error
darunrs Jul 16, 2024
6c55556
Break out of loop after error
darunrs Jul 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion block-streamer/src/block_stream.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ async fn process_bitmap_indexer_blocks(

let mut last_published_block_height: u64 = start_block_height;

while let Some(Ok(block_height)) = matching_block_heights.next().await {
while let Some(block_height) = matching_block_heights.next().await.transpose()? {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing here is we could also trace the error and then proceed to Lake. But I felt this was a situation that deserves a fail fast given that the stored Last Processed Block will allow for graceful restarts and errors in the backfill process should be fixed immediately.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In what case do we get an error? Can we handle it internally with retries?

I'm on the fence about this. We could log it, monitor it and perhaps set up an alert, and then fix, all while not causing disruptions to users?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed in team meeting to go with a resilient approach where possible. I've changed the code to instead log an error and then proceed to lake.

redis
.publish_block(indexer, redis_stream.clone(), block_height, MAX_STREAM_SIZE)
.await?;
Expand Down
Loading