feat: don't crash relayer due to flaky RPCs #5115
Conversation
|
kamiyaa
force-pushed
the
jeff/bad-rpc
branch
3 times, most recently
from
ca808d3 to
1c9af66
Compare
| let db = dbs.get(origin).unwrap().clone(); | ||
| let metadata_builder = BaseMetadataBuilder::new( | ||
| origin.clone(), | ||
| destination_chain_setup.clone(), | ||
| prover_syncs[origin].clone(), | ||
| validator_announces[origin].clone(), | ||
| validator_annonce.clone(), |
There was a problem hiding this comment.
Since we can no longer assume all mailboxes / validator_announce were successfully instantiated, we should loop through the successfully instantiated ones instead of blindly going through all origin_chains / destination_chains
| /// Any chains that fail to build mailbox will not be included | ||
| /// in the hashmap. Errors will be logged and chain metrics | ||
| /// will be updated for chains that fail to build mailbox. | ||
| pub async fn build_mailboxes( |
There was a problem hiding this comment.
Move logic into functions so we can test them in isolation
- instead of crashing relayer if any RPC calls fail we just log the error and metric
|
|
||
| #[tokio::test] | ||
| #[tracing_test::traced_test] | ||
| async fn test_failed_build_mailboxes() { |
There was a problem hiding this comment.
Turns out we don't need to do too many fancy things to cause an error for build_mailboxes or build_validator_announces.
All we need to do is just not have a ChainConf associated with said chain and it will cause an error in build_mailbox / build_validator_announce
Description
build_mailboxesandbuild_validator_announces, we just log the error and metricDrive-by changes
Related issues
Fixes #4887
Backward compatibility
Are these changes backward compatible? Are there any infrastructure implications, e.g. changes that would prohibit deploying older commits using this infra tooling?
Yes
Testing
Added unittests to test ensure functions run correctly and log errors in metrics