Skip to content

Commit

Permalink
feat: Add "StartBlock::Continue" to registry contract/types (#548)
Browse files Browse the repository at this point in the history
The start block parameter (`start_block_height`) has two states,
signifying "Start from block" and "Start from latest" only. With the
removal of the continuous "real-time" process, which essentially runs
Indexers "From Interruption", the need for a third state arises,
representing "Interruption" arises. This PR expands the registry
contract/types to accommodate this change. This work was a good
opportunity to refactor the existing types, removing unnecessary fields
and tailoring it to the new architecture.

## Registry Types Refactoring
- `filter: IndexerRule` -> `rule: Rule`: `IndexerRule` contained a lot
of noise/data which was not needed. `id`, `name`, and
`indexer_rule_kind` have all been removed. The only useful part here is
`MatchingRule` which has been renamed to just `Rule`.
- `schema` is now non-optional: This field is always required so it
makes sense to convey that in the code. Having it as an `Option` created
lots of unnecessary checks throughout the system.
- `start_block_height` replaced by `start_block`: The latter being an
`enum` to represent "From Latest", "From Block", and "From Interruption"

## Public methods/API
To minimise the disruption of the existing contract consumers, the
public API remains unchanged, i.e. the core methods
(`list_indexer_functions`, `register_indexer_function`, etc.) use/return
the same types. As the underlying data will be migrated to the new
types, these methods infer the old types from the new ones. New methods
have been created to work with the new types (`register`, `list_all`,
etc.) allowing clients to move over when possible, as opposed to
creating a breaking change.
  • Loading branch information
morgsmccauley committed Feb 7, 2024
1 parent 305fc0d commit c92f477
Show file tree
Hide file tree
Showing 10 changed files with 628 additions and 219 deletions.
2 changes: 1 addition & 1 deletion block-streamer/src/block_stream.rs
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,7 @@ mod tests {
)
.unwrap(),
function_name: "test".to_string(),
indexer_rule: registry_types::IndexerRule {
indexer_rule: registry_types::OldIndexerRule {
indexer_rule_kind: registry_types::IndexerRuleKind::Action,
matching_rule: registry_types::MatchingRule::ActionAny {
affected_account_id: "queryapi.dataplatform.near".to_string(),
Expand Down
2 changes: 1 addition & 1 deletion block-streamer/src/indexer_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ use near_lake_framework::near_indexer_primitives::types::AccountId;
use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};

use registry_types::IndexerRule;
use registry_types::OldIndexerRule as IndexerRule;

#[derive(serde::Serialize, serde::Deserialize, Clone, Debug)]
pub struct IndexerConfig {
Expand Down
2 changes: 1 addition & 1 deletion block-streamer/src/rules/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ pub mod outcomes_reducer;
pub mod types;

use near_lake_framework::near_indexer_primitives::StreamerMessage;
use registry_types::{IndexerRule, MatchingRule};
use registry_types::{MatchingRule, OldIndexerRule as IndexerRule};

use types::{ChainId, IndexerRuleMatch};

Expand Down
2 changes: 1 addition & 1 deletion block-streamer/src/rules/outcomes_reducer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ fn build_indexer_rule_match_payload(

#[cfg(test)]
mod tests {
use registry_types::{IndexerRule, IndexerRuleKind, MatchingRule, Status};
use registry_types::{IndexerRuleKind, MatchingRule, OldIndexerRule as IndexerRule, Status};

use crate::rules::outcomes_reducer::reduce_indexer_rule_matches_from_outcomes;
use crate::rules::types::{ChainId, IndexerRuleMatch};
Expand Down
2 changes: 1 addition & 1 deletion block-streamer/src/server/block_streamer_service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use tonic::{Request, Response, Status};

use crate::indexer_config::IndexerConfig;
use crate::rules::types::ChainId;
use registry_types::{IndexerRule, IndexerRuleKind, MatchingRule};
use registry_types::{IndexerRuleKind, MatchingRule, OldIndexerRule as IndexerRule};

use crate::block_stream;
use crate::server::blockstreamer;
Expand Down
2 changes: 1 addition & 1 deletion coordinator/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,7 @@ mod tests {
use mockall::predicate;
use std::collections::HashMap;

use registry_types::{IndexerRule, IndexerRuleKind, MatchingRule, Status};
use registry_types::{IndexerRuleKind, MatchingRule, OldIndexerRule as IndexerRule, Status};

use crate::registry::IndexerConfig;

Expand Down
6 changes: 3 additions & 3 deletions coordinator/src/migration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,7 @@ mod tests {
use std::collections::HashMap;

use mockall::predicate;
use registry_types::{IndexerRule, IndexerRuleKind, MatchingRule, Status};
use registry_types::{IndexerRuleKind, MatchingRule, OldIndexerRule, Status};

use crate::registry::IndexerConfig;

Expand All @@ -296,7 +296,7 @@ mod tests {
function_name: "test".to_string(),
code: String::new(),
schema: Some(String::new()),
filter: IndexerRule {
filter: OldIndexerRule {
id: None,
name: None,
indexer_rule_kind: IndexerRuleKind::Action,
Expand Down Expand Up @@ -369,7 +369,7 @@ mod tests {
function_name: "test".to_string(),
code: String::new(),
schema: Some(String::new()),
filter: IndexerRule {
filter: OldIndexerRule {
id: None,
name: None,
indexer_rule_kind: IndexerRuleKind::Action,
Expand Down
6 changes: 4 additions & 2 deletions coordinator/src/registry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ use near_jsonrpc_client::JsonRpcClient;
use near_jsonrpc_primitives::types::query::QueryResponseKind;
use near_primitives::types::{AccountId, BlockReference, Finality, FunctionArgs};
use near_primitives::views::QueryRequest;
use registry_types::{AccountOrAllIndexers, IndexerRule};
use registry_types::{
OldAccountOrAllIndexers as AccountOrAllIndexers, OldIndexerRule as IndexerRule,
};

use crate::utils::exponential_retry;

Expand Down Expand Up @@ -69,7 +71,7 @@ impl RegistryImpl {

fn enrich_indexer_registry(
&self,
registry: HashMap<AccountId, HashMap<String, registry_types::IndexerConfig>>,
registry: HashMap<AccountId, HashMap<String, registry_types::OldIndexerConfig>>,
) -> IndexerRegistry {
registry
.into_iter()
Expand Down
Loading

0 comments on commit c92f477

Please sign in to comment.