Use test config in CI #770
Conversation
github-actions
bot
added
the
crate: evm_arithmetization
.github/workflows/ci.yml
Outdated
|
|
||
| - name: Run specific ignored tests in release mode | ||
| run: | | ||
| RUST_LOG=release cargo test --release \ |
There was a problem hiding this comment.
Did you intend to have RUST_LOG=release? I would expect it to take error,warn,info,debug,trace
There was a problem hiding this comment.
I removed it as it has no effect.
| @@ -52,6 +52,7 @@ type C = KeccakGoldilocksConfig; | |||
| /// `1337` from address `0x5B38Da6a701c568545dCfcB03FcB875f56beddC4` to address | |||
| /// `0xAb8483F64d9C6d1EcF9b849Ae677dD3315835cb2`. | |||
| #[test] | |||
| #[ignore] | |||
There was a problem hiding this comment.
Is erc721 taking that long that we're ignoring it on default?
There was a problem hiding this comment.
Yes, I found that the test running on CI is about 4x slower than on my MacBook. The test time in release mode on my MacBook can be found here: #739
There was a problem hiding this comment.
If we're going to start doing test selection like this, I think we need a better test runner (cargo-nextest)
It allows you to categorise your tests, have different timeouts, profiles for ci etc
There was a problem hiding this comment.
Currently, we only need minimal functionality, so our existing setup works fine. Using cargo-nextest would indeed be beneficial, and we can look at converting to it in a future PR.
scripts/prove_rpc.sh
Outdated
| @@ -97,6 +97,7 @@ if [[ $8 == "test_only" ]]; then | |||
| echo "Proving blocks from ($START_BLOCK) to ($END_BLOCK)" | |||
| command="cargo r --release --package zero --bin leader -- \ | |||
| --test-only \ | |||
| --use-test-config \ | |||
There was a problem hiding this comment.
this step is test-only, i.e. no proving but only witness generation so --use-test-config has no effect
| @@ -134,6 +135,7 @@ else | |||
| echo "Proving blocks from ($START_BLOCK) to ($END_BLOCK)" | |||
| command="cargo r --release --package zero --bin leader -- \ | |||
| --runtime in-memory \ | |||
| --use-test-config \ | |||
There was a problem hiding this comment.
not sure we want the script to always use the testing config, rather take it as optional argument perhaps?
There was a problem hiding this comment.
I assume we always use the testing config in CI tests, while our cron jobs use the standard config?
There was a problem hiding this comment.
I thought we had said in the dev sync we'd use a mix of both in CI? I'm fine either way though
There was a problem hiding this comment.
Done. Keep Execute bash script to generate and verify a proof for a small block. with default config.
sai-deng
requested review from
0xaatif,
atanmarko,
wborgeaud,
LindaGuiga and
paulgoleary
as code owners
.github/workflows/ci.yml
Outdated
| @@ -95,6 +95,26 @@ jobs: | |||
| CARGO_INCREMENTAL: 1 | |||
| RUST_BACKTRACE: 1 | |||
|
|
|||
| run_ignored_tests: | |||
| name: Run Specific Ignored Tests in Release Mode | |||
There was a problem hiding this comment.
nit: I'd just make the job name a tad more explicit, something like this?
| name: Run Specific Ignored Tests in Release Mode | |
| name: Slow evm_arithmetization tests in release mode |
The
two_to_one_blocktest takes approximately 30 seconds to complete on my MacBook in release mode. However, it remains slow in the CI environment because it runs in debug mode by default. To address this, I have added a new CI job that specifically runs longer tests, includingtwo_to_one_block, in release mode. This ensures better performance and faster feedback for these resource-intensive tests.Before:
Continuous Integration / Test evm_arithmetization (pull_request) Successful in 17m
Jerigon Integration / Native tracer proof generation (pull_request) Successful in 14m
Jerigon Integration / Zero tracer proof generation (pull_request) Successful in 14m
After:
Continuous Integration / Slow evm_arithmetization tests in release mode (pull_request) Successful in 5m (new)
Continuous Integration / Test evm_arithmetization (pull_request) Successful in 8m
Jerigon Integration / Native tracer proof generation (pull_request) Successful in 9m
Jerigon Integration / Zero tracer proof generation (pull_request) Successful in 9m