Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Query Hasura Endpoint through Block Streamer #745

Merged
merged 17 commits into from
May 23, 2024

Conversation

darunrs
Copy link
Collaborator

@darunrs darunrs commented May 20, 2024

Block Streamer will be querying block match data from an Indexer's postgres tables through Hasura. Thus, Block Streamer needs to be able to query and parse data returned by Hasura's graphQL APIs.

This PR introduces a crate which can generate code necessary to parse returned data from a graphQL query. I've also created a struct which encapsulates the code for making graphQL calls for ease of use when integrating or mocking this feature in the future.

@darunrs darunrs marked this pull request as ready for review May 21, 2024 01:35
@darunrs darunrs requested a review from a team as a code owner May 21, 2024 01:35
@darunrs darunrs linked an issue May 21, 2024 that may be closed by this pull request
block-streamer/Cargo.toml Show resolved Hide resolved
Comment on lines 34 to 53
let variables = get_bitmaps_exact::Variables {
receiver_ids: Some(receiver_ids),
block_date: Some(block_date),
limit: Some(limit),
offset: Some(offset),
};
let request_body = GetBitmapsExact::build_query(variables);
let res = self
.client
.post(&self.graphql_endpoint)
.header("x-hasura-role", HASURA_ACCOUNT)
.json(&request_body)
.send()
.await
.expect("Failed to query bitmaps for list of exact receivers");
let response_body: Response<get_bitmaps_exact::ResponseData> = res.json().await?;
match response_body.data {
Some(data) => Ok(data.darunrs_near_bitmap_v5_actions_index),
None => Ok([].into()),
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be reduced to the following by using post_graphql, but you'll need to enable the reqwest feature for graphql_client.

post_graphql::<GetBitmapsExact, _>(
    &self.client,
    &self.graphql_endpoint,
    get_bitmaps_exact::Variables {
        receiver_ids: Some(receiver_ids),
        block_date: Some(block_date),
        limit: Some(limit),
        offset: Some(offset),
    },
)
.await?
.data
.ok_or(anyhow!("No data"))
.map(|data| data.darunrs_near_bitmap_v5_actions_index)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh nice callout! Thanks for the optimization.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually can't use that directly has we need to attach headers to the client call and that function doesn't allow for it. I can use your return object optimizations you provided though.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I'll just re-implement their function.

block-streamer/src/graphql/client.rs Outdated Show resolved Hide resolved
block-streamer/src/graphql/queries/get_bitmaps_exact.rs Outdated Show resolved Hide resolved
block-streamer/src/graphql/client.rs Outdated Show resolved Hide resolved
block-streamer/README.md Show resolved Hide resolved
block-streamer/Cargo.toml Outdated Show resolved Hide resolved
graphql_endpoint: String,
}

async fn post_graphql<Q: GraphQLQuery, U: reqwest::IntoUrl>(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's just add this to GraphQLClient, then we don't need to pass the client and url, we can just use &self

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually tried to do this but I was getting into some trouble with the parameter types (Q and U). automock was asking them to be static lifetimes and I couldn't get it to work. Do you know what I should do to make that work?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you just need to add a 'static lifetime to the traitbound - not exactly sure why it's required, but it's fine to add.

async fn post_graphql<Q: GraphQLQuery + 'static>(
        &self,
        variables: Q::Variables,
    ) -> Result<Response<Q::ResponseData>, reqwest::Error> {
        let body = Q::build_query(variables);
        let reqwest_response = self
            .client
            .post(self.graphql_endpoint.to_string())
            .header("x-hasura-role", HASURA_ACCOUNT)
            .json(&body)
            .send()
            .await?;

        reqwest_response.json().await
    }

then using it

pub async fn get_bitmaps_exact(
        &self,
        receiver_ids: Vec<String>,
        block_date: String,
        limit: i64,
        offset: i64,
    ) -> anyhow::Result<Vec<get_bitmaps_exact::GetBitmapsExactDarunrsNearBitmapV5ActionsIndex>>
    {
        self.post_graphql::<GetBitmapsExact>(get_bitmaps_exact::Variables {
            receiver_ids: Some(receiver_ids),
            block_date: Some(block_date),
            limit: Some(limit),
            offset: Some(offset),
        })
        .await
        .expect("Failed to query bitmaps for list of exact receivers")
        .data
        .ok_or(anyhow::anyhow!("No bitmaps were returned"))
        .map(|data| data.darunrs_near_bitmap_v5_actions_index)
    }

Copy link
Collaborator Author

@darunrs darunrs May 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah that's it. The issue I was having was when I retained passing self.client and such to the function, it was saying self would escape the function and need to outlive static. But by making them internally called, the lifetime of self is bound by the call to post_graphql rather than to that function AND get_bitmaps_exact. Awesome, thanks!

block-streamer/src/graphql/client.rs Outdated Show resolved Hide resolved
@darunrs
Copy link
Collaborator Author

darunrs commented May 22, 2024

Should I also move src/graphql/client.rs into the src folder as src/graphql_client.rs instead? The name clashes with the graphql_client crate we are using but Rust doesn't seem to mind when I tried it. It build and ran. We technicalyl no longer need the graphql folder since only the client lives inside it now. Previously it was the codegen files.

@darunrs darunrs merged commit ffced35 into main May 23, 2024
8 checks passed
@darunrs darunrs deleted the block-streamer-bitmap-query branch May 23, 2024 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate graphQL Querying into Block Streamer
2 participants