feat: Support Deployment of Block Streamer

[darunrs]

Block Streamer needs a Dockerfile in order to build the image and be deployable either locally or in GCP. I've created one for the service and updated the compose file to set the correct env variables needed. This also sserves as a record for what ENV variables need to be set during Terraform deployments to GCP.

In addition, there's a small issue with running QueryApi locally where if the user runs Runner locally through yarn start, provisioning of resources through hasura fails because it populates the postgres host as 'localhost' whereas it should be 'postgres', as the calls are processed inside the hasura docker. This is due to PGHOST being both used and also passed to Hasura inside Runner. To fix this, I created a separate env variable called PGHOST_HASURA which can be set to 'postgres'. This way, Runner can use 'localhost' while it passes 'postgres' to hasura's docker.

[darunrs]

The testing I did locally was relatively simple:

1. I built the images using docker compose.
2. I ran the resources, Coordinator V1, and block streamer image locally in Docker.
3. I ran Runner V2 and Coordinator V2 locally through CLI.
4. I hard coded Coordinator V2 to only init one indexer.
5. Verified Block Stream and Executor were initialized, that blocks were in stream, and that Runner was running them alongside V1 executors.

There's probably some edge case testing needed as well. Open to ideas on what things I should try.

Something I noticed though is that I was getting a lot of S3 errors locally when running all current dev indexers in V2. Might be a problem with running locally and not in GCP, but its something to consider. Caching might help if the indexers are reasonably close in terms of what blocks they're querying.

[darunrs]

Current plan for deployment:

1. Get approval for code PR (This one)
2. Publish PR for NearOps Infra.
3. Merge this PR.
4. Deploy dev infra and verify.
5. Get approval for NearOps and merge it.

[morgsmccauley]

Can this still be overridden from the outside?

[darunrs]

So that's a good point. I can look into that.

[darunrs]

It can be overridden using docker compose file's ENV. I'm not sure about GCP deployment. But for GCP it oughta be release anyway.

[morgsmccauley]

I'm referring to RUST_LOG here - let's remove, as it should be set in the terraform config anyway?

[darunrs]

Oh sorry you mean that. I had to put that in because leaving it out prevents tracing logs from appearing inside Docker. Now of course we can set that in docker compose and terraform too if we want. But I felt like it made sense to leave it here as we probably wouldn't override it unless we really wanted to, in which case we totally can by placing it there. Removing it means we need to specify it in deployment explicitly without any default.

Essentially, removing it here means we don't have a default setting. It's probably a minor change but if we want to be explicit about all env variables, doing so in the same place, I'm totally fine going with that.

[morgsmccauley]

Yeah i'd prefer to keep it in one place to avoid confusion

[morgsmccauley]

LGTM just the RUST_LOG and 0.0.0.0 change to make :)

[darunrs]

Block Streamer currently uses [::] which is dual stack 0.0.0.0, meaning it works for both IPv4 and IPv6 addresses. I changed Runner's server to [::] and that seems to have worked as well.

[morgsmccauley]

Block Streamer currently uses [::] which is dual stack 0.0.0.0, meaning it works for both IPv4 and IPv6 addresses. I changed Runner's server to [::] and that seems to have worked as well.

@darunrs I may be wrong, but it seems [::] and 0.0.0.0 are for ipv6 and ipv4 respectively, not the same. If they are, it might be worth checking if specifying ipv6 will conflict with the firewall config, which is set to 0.0.0.0. With this config, I wonder if GCP will only expose ipv4 traffic, and not ipv6.

[darunrs]

Block Streamer currently uses [::] which is dual stack 0.0.0.0, meaning it works for both IPv4 and IPv6 addresses. I changed Runner's server to [::] and that seems to have worked as well.

@darunrs I may be wrong, but it seems [::] and 0.0.0.0 are for ipv6 and ipv4 respectively, not the same. If they are, it might be worth checking if specifying ipv6 will conflict with the firewall config, which is set to 0.0.0.0. With this config, I wonder if GCP will only expose ipv4 traffic, and not ipv6.

Good catch. Our VPC Network currently only supports IPv4. We don't really have a reason to change that. I wasn't able to find anything after a short search that definitively said [::] works dual stack in conjunction with Dual Stack GCP networks. I'll update the addresses to expose on 0.0.0.0 explicitly instead!