Skip to content

Commit

Permalink
test: Make block-streamer unit-testable and add tests (#442)
Browse files Browse the repository at this point in the history
This PR is a combination of refactoring + testing, the former enabling
the latter

## Mock `impl` instead of `trait`
Originally, `trait`s such as `S3ClientTrait` were used to abstract the
behaviour of I/O related `struct`s, allowing for the injection of mock
implementations via `automock`. This was fine, but lead to very verbose
syntax which needed to be propagated throughout the codebase where ever
it was used.

Now, the `impl` itself is mocked, and either the mock or real
implementation is returned dependant on the `cfg(test)` flag. This means
we no longer need to worry about generics and can instead use the
`struct` itself, as we would with non-mocked code.

The trade-off here is that rust now thinks the 'real' code is
dead/unused as `test` is enabled by default.

## Mocking `near-lake-framework`
`aws` exposes an HTTP client which can be configured with mock
request/response values. `aws-sdk-s3` can be configured to use this
client, creating a mock client. Injecting this config into
`near-lake-framework` allows us to create deterministic outputs,
enabling unit-testing.

`aws_config` is now taken as a parameter where relevant so that we can
mock `near-lake-framework` in out tests. The functions for achieving
this have been placed in `test_utils` so they can be reused.

## Test `BlockStreamerService` and `BlockStream`
With `RedisClient`, `S3Client`, and `Lake` now mockable - tests have
been added for these `struct`s. I've also added a Github action which
runs on every PR. All raw block data has been added to `data/`, which
makes up the majority of the diff in this PR.
  • Loading branch information
morgsmccauley committed Dec 14, 2023
1 parent 9c5ee15 commit 28bc9f3
Show file tree
Hide file tree
Showing 34 changed files with 895 additions and 845 deletions.
73 changes: 73 additions & 0 deletions .github/workflows/block-streamer-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: Block Streamer

on:
push:
branches: [ main ]
paths:
- "block-streamer/**"
pull_request:
paths:
- "block-streamer/**"

env:
CARGO_TERM_COLOR: always

jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Protoc
uses: arduino/setup-protoc@v2
- name: Check
working-directory: ./block-streamer
run: cargo check

test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Protoc
uses: arduino/setup-protoc@v2
- name: Test
working-directory: ./block-streamer
run: cargo test


format:
runs-on: ubuntu-20.04
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Install Protoc
uses: arduino/setup-protoc@v2
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
toolchain: 1.70.0
override: true
profile: minimal
components: rustfmt
- name: Check formatting
working-directory: ./block-streamer
run: |
cargo fmt -- --check
clippy:
runs-on: ubuntu-20.04
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Install Protoc
uses: arduino/setup-protoc@v2
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
toolchain: 1.70.0
override: true
profile: minimal
components: clippy
- name: Clippy check
working-directory: ./block-streamer
run: |
cargo clippy
Loading

0 comments on commit 28bc9f3

Please sign in to comment.