Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Do not eat bitmap backfill errors #871

Merged
merged 3 commits into from
Jul 16, 2024
Merged

fix: Do not eat bitmap backfill errors #871

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6a2bcf8
fix: Ensure while loop terminates with error instead of skipping
darunrs Jul 16, 2024
f4a9eb5
Do not throw error if bitmap process encounters an error
darunrs Jul 16, 2024
6c55556
Break out of loop after error
darunrs Jul 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 19 additions & 9 deletions block-streamer/src/block_stream.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -269,15 +269,25 @@ async fn process_bitmap_indexer_blocks(

let mut last_published_block_height: u64 = start_block_height;

while let Some(Ok(block_height)) = matching_block_heights.next().await {
redis
.publish_block(indexer, redis_stream.clone(), block_height, MAX_STREAM_SIZE)
.await?;
redis
.set_last_processed_block(indexer, block_height)
.await?;

last_published_block_height = block_height;
while let Some(block_height_result) = matching_block_heights.next().await {
match block_height_result {
Ok(block_height) => {
redis
.publish_block(indexer, redis_stream.clone(), block_height, MAX_STREAM_SIZE)
.await?;
redis
.set_last_processed_block(indexer, block_height)
.await?;

last_published_block_height = block_height;
}
Err(err) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should break right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yep we should do so explicitly. If an error happens, the stream just ends anyway, but a break is more explicit.

Copy link
Collaborator

@morgsmccauley morgsmccauley Jul 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I was under the assumption that the error handling here would cause it to continue processing.

With the previous syntax it would continue because you are matching against Ok(), so if it was an Err() the loop condition wasn't met.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I believe in the new case, the next() would complete as the stream terminates due to the thrown error and thus cannot yield anymore.

tracing::error!(
"Backfill using bitmap indexer failed unexpectedly: {:?}",
err
);
}
}
}

Ok(last_published_block_height)
Expand Down
Loading