Skip to content

Releases: ethereum/tests

London update 9.0.5: Postfork tests

22 Aug 21:45
@winsvega winsvega
9.0.5
6f41f7b
Compare
Choose a tag to compare
London update 9.0.5: Postfork tests

The recent tests changes:

A few more tests, fix fillers for test regeneration

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.4

Test format

Tests Fillers

GeneralStateTests Filled

Test Regeneration

Update all:

  • Documentation
    PR #915
    PR #925

  • Remove sender code in LegacyTests
    PR #905
    PR #911
    PR #914

  • More timestamp/difficulty formula tests
    PR #907

  • Prescise OOG on some operations
    Fix issue #909
    PR #913

  • SStore gas costs check
    PR #926

  • Update test fillers in a couple of tests
    PR #927
    PR #930

Upcoming changes:

==> Regenerate all the tests, file hash calculation change
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Assets 2
Loading

London update 9.0.4: Testnet's bugs

27 Jul 09:12
@winsvega winsvega
9.0.4
5d534e3
Compare
Choose a tag to compare
London update 9.0.4: Testnet's bugs

The recent tests changes:

Test RPC protocol documentation
Tests covering bugs discovered by testnets

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.3

Test format

Tests Fillers

Decimal values can be represented with _ for better readability. Example:

    "balance" : "100_000_000"

GeneralStateTests Filled

Test Regeneration

Update all:

  • Documentation
    PR #894
    PR #904

  • BaseFeePerGas computation bug discovered on besu
    PR #895

  • Intrinsic gas calculation bug in 1559 transaction
    Ropsten reported bug. where value was not included in intrinsic calculation caused an intrinsicly invalid transaction to pass on low baseFee
    PR #898
    PR #901
    PR #903

  • Timestamp difference test (WIP)
    A bit more coverage on timestamp diff/ difficulty formula, more tests to come
    PR #896

  • Split huge test into may small ones for better execution on threads (cosmetic)
    PR #897

  • Fix geth exception name (cosmetic)
    PR #902

Upcoming changes:

London EIP tests
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Assets 2
Loading

London update 9.0.3: Uncle tests

02 Jul 10:46
@winsvega winsvega
9.0.3
e20d7f3
Compare
Choose a tag to compare
London update 9.0.3: Uncle tests

The recent tests changes:

Template tests around new opcodes and badOpcodes
Uncle tests for London transition

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.2

Test format

GeneralStateTests Fillers

GeneralStateTests Filled

Test Regeneration

Update all:

  • Documentation
    PR #885
    PR #891
    PR #892

  • Remove code from sender account in StateTests
    PR #893

  • Template tests generator and template tests
    Creates tests that run 'X' code inside different scenarious.
    Template tests around basefee selfbalance and badopcodes
    PR #886

  • Uncle tests for London transition
    PR #887
    PR #890

  • Fix baseFee to baseFeePerGas in yml fillers
    PR #884

  • Fix baseFee field
    Cosmetic
    PR #888

Upcoming changes:

London EIP tests
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Assets 2
Loading

London update 9.0.2: Invalid transactions check

12 Jun 20:32
@winsvega winsvega
9.0.2
5d65138
Compare
Choose a tag to compare
London update 9.0.2: Invalid transactions check

The recent tests changes:

Check transaction rejection rules in Blockchain and StateTests
Rename baseFee to baseFeePerGas
Explicity baseFeePerGas in GeneralStateTests

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.1

Test format

GeneralStateTests Fillers

New field in state tests expect section expectException allows to check transaction rejection reason on different forks

"expect" : [
            {
                "indexes" : {   "data" : -1,   "gas" : 0,   "value" : -1  },
                "network" : [">=Berlin"],
                "expectException" : {
                    ">=Berlin" : "TR_IntrinsicGas"
                },
               ...
]

GeneralStateTests Filled

New field (optional) in post results section expectException indicating that transaction is expected to be rejected. The reason check is optional.

            "Berlin" : [
                {
                    "indexes" : { "data" : 0, "gas" : 0, "value" : 0 },
                    "hash" : "0x4c9c6cf002e6a88a5444662ca9ceb6a116b7b69ced38c470bf6e4a12a6313967",
                    "txbytes" : "0xf862...5",
                    "expectException" : "TR_IntrinsicGas",
                    "logs" : "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347"
                }
            ],

Test Regeneration

Update all:

Upcoming changes:

London EIP tests
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Assets 2
Loading

London update 9.0.1: Fixing issues

05 Jun 22:21
@winsvega winsvega
9.0.1
fa0ab11
Compare
Choose a tag to compare
London update 9.0.1: Fixing issues

The recent tests changes:

Fixing issues in the tests with EIP1559 formulas
More explicit transaction rejection reason from tools when filling the tests

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.0

Test format

BlockchainTests Fillers

"invalid" section in transaction replaced with "expectException" similar to the one in BlockHeader overwrite
It is now possible to set expected exception depending on the mining network for a specific transaction
This require test_mineBlocks RPC change in the test protocol and t8ntool response change as well.
The new return on test_mineBlocks is a json

{
   "valid" : true,
   "rejectedTransactions" : [
       {
            "hash" : "0x1122..11",
            "error" ; "Gas Limit reached"
       }
   ]
}

BlockchainTests Filled

If the block test filler had transactions marked "expectException" the filled test would have "transactionSequence" section with transaction order as its imported into state in the current block. if transaction import expected to fail it is marked so with exception. The new section allows clients to check transaction rejection in the blockchain tests. As previously invalid transaction been checked only during the test filling. Invalid transaction do not present in the valid block rlp. Now clients can try importing it on top of the previous block state and make sure the rejection exception is validated.

"transactionSequence" : [
 {
     "rawBytes" : "0xf863802b...4db0d72118",
     "valid" : "false",
     "exception" : "TR_NoFunds"
 },
 {
      "rawBytes" : "0xf863802...b4db0d72118",
     "valid" : "true"
 }
],

Test Regeneration

Update all:

  • Documentation
    PR #867

  • Fixes of pr's errors
    PR #864
    PR #866
    PR #870
    PR #871

  • GasLimit delta checks fix on London
    PR #860

  • EIP1559 transaction gasPrices
    PR #859

  • Update EIP1559 rules
    PR #862

  • Issue 854 Coinbase as hotload account
    #854
    PR #855

  • Correct tests with expected transaction failure reason
    PR #872

  • More transaction exception tests
    PR #873

Upcoming changes:

Transaction rejection check in StateTests
Deprecate "transaction" section in filled GeneralStateTests #828 Rejectd
London EIP tests
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Assets 2
Loading

London update 9.0.0: EIP1559

30 May 00:27
@winsvega winsvega
9.0.0
33c3174
Compare
Choose a tag to compare
London update 9.0.0: EIP1559

The recent tests changes:

Merge PRs with London EIP1559 tests.
Update test format to support EIP1559 transaction format (+change in test RPC json object)
Regenerate all tests using London rules.

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/8.0.5

Test format

State Tests

Treat legacy blockheader as 1559 with baseFee 10 on London
Tests are recalculated with transaction gasPrice set to 10

Blockchain Tests

Tests are recalculated with transaction gasPrice set to 1000
Tests with gasPrice 0 are not included in London (as minimum baseFee is 7)

State Tests | BlockchainTests | TransitionTests

  • the env section now could have
currentBaseFee: 0x3B9ACA00

indicating a eip1559 genesis.

  • transaction section eip1559 support could now have
  maxFeePerGas: '0x12A05F200'
  maxPriorityFeePerGas: '2'

instead of gasPrice
PR example: #826

BlockchainTests | TransitionTests

  • the blockHeader now could have
baseFee: 0x3B9ACA00

indicating a eip1559 block.
In blockchain fillers blocks section, the blockHeader (overwrite) section has a new field

               "blockHeader" : {
                   "gasLimit" : "600000",
                   "remove" : [ "baseFee" ]
               },

"remove" that indicates an array of fields to be removed from the client returned structure. making it possible to switch mined block into legacy and eip1559 back and forth. see transition test example.

Test Regeneration

Update all:
Regenerate all tests using London rules.
Treat legacy blockheader as 1559 with baseFee 10 on London rules
GStateTests transaction gasPrice set to 10
BCTests transaction gasPrice set to 1000
PR #857

  • Documentation update
    PR #838
    PR #849

  • EIP1559 coverage
    Blockchain tests, gasTarget, gasPrice, transaction rules
    PR #837

  • EIP1559 tests example
    PR #826

  • EIP 3541 coverage
    Check that contracts creation starting with 0xef is rejected
    PR #835
    PR #852

  • Enable BaseFee opcode support check in all opcodes test
    PR #841

  • EIP 3529 additional coverage
    refund gas reduction
    PR #850

Upcoming changes:

Deprecate "transaction" section in filled GeneralStateTests #828
London EIP tests
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)
OE wrapper support on retesteth.ethdevops.io. FAILED

Assets 2
Loading

Berlin update 8.0.5: Moving to London

18 May 14:32
@winsvega winsvega
8.0.5
5490db3
Compare
Choose a tag to compare
Berlin update 8.0.5: Moving to London

The recent tests changes:

Discussion that affects all custom test runners: #828 issue
save the space, deprecate general transaction vector parsing and signing (execution speed up).

- WARNING: it will break all the custom test executor tools!

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://github.com/ethereum/retesteth/wiki/Transition-Tool)
Previous release link: https://github.com/ethereum/tests/releases/tag/8.0.4

Test format

none

Test Regeneration

Update all:
none

  • Fixed a bug caused by the higher price of a call in Berlin
    PR #833

  • More tests around EIP2930 on "manual" account creation
    PR #834

  • Calling a contract which has an undefined opcode as first byte of code
    PR #836

  • Deploy contracts with CREATE1/CREATE2 with all possble first bytes
    PR #843

  • Check for correct rollback of init code with side effects
    This adds a test that checks that exceptional abort during init code correctly rolls back side effects.
    PR #844

  • Disagble python script checking tests format
    As the format check is done by retesteth already
    PR #840

Upcoming changes:

  • Fillers and Filled

State Tests | BlockchainTests | TransitionTests

  • the env section now could have
currentBaseFee: 0x3B9ACA00

indicating a eip1559 genesis.

  • transaction section eip1559 support could now have
  maxFeePerGas: '0x12A05F200'
  maxPriorityFeePerGas: '2'

instead of gasPrice
PR example: #826

BlockchainTests | TransitionTests

  • the blockHeader now could have
baseFee: 0x3B9ACA00

indicating a eip1559 block.
In blockchain fillers blocks section, the blockHeader (overwrite) section has a new field

               "blockHeader" : {
                   "gasLimit" : "600000",
                   "remove" : [ "baseFee" ]
               },

"remove" that indicates an array of fields to be removed from the client returned structure. making it possible to switch mined block into legacy and eip1559 back and forth. see transition test example.

Discuss #828 issue
London EIP tests
Besu retesteth RPC support (web vmtracing support)
OE wrapper support on retesteth.ethdevops.io. FAILED

Assets 2
Loading

Berlin update 8.0.4: Post fork tests

26 Apr 23:00
@winsvega winsvega
8.0.4
9b4c624
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Berlin update 8.0.4: Post fork tests

The recent tests changes:

Discussion that affects all custom test runners: #828 issue
save the space, deprecate general transaction vector parsing and signing.

Announcement

Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://github.com/ethereum/retesteth/wiki/Transition-Tool)
Previous release link: https://github.com/ethereum/tests/releases/tag/8.0.3

Test format

  • Fillers
    none

  • Filled
    none

Test Regeneration

Update all:
none

  • Documentation update
    PR #830

  • Check memory opcodes with and without enough gas
    PR #832

  • Precompiles and access list tests
    More tests around precompiles as the OE fork happened around it
    PR #829

  • Access list test texample
    PR #825

  • More coverage for eip2929 edge cases
    OOG for cold access with EIP2929
    PR #824

Upcoming changes:

Discuss #828 issue
EIP1559, typed transaction 02, according tests format change
London EIP tests
Besu retesteth RPC support (web vmtracing support)
OE wrapper support on retesteth.ethdevops.io

Assets 2
Loading

Berlin update 8.0.3: Consensus issues

10 Apr 13:27
@winsvega winsvega
8.0.3
644967e
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Berlin update 8.0.3: Consensus issues

The recent tests changes:

Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://github.com/ethereum/retesteth/wiki/Transition-Tool)
Previous release link: https://github.com/ethereum/tests/releases/tag/8.0.2

Test format

./BlockchainTests/ValidBlocks/VMTests removed
./src/VMTestsFiller moved to ./src/LegacyTests/Constantinople
./VMTests moved to ./LegacyTests/Constantinople/VMTests

  • Fillers
    none

  • Filled
    none

Test Regeneration

Update all:
none

  • Consensus issue random test 648
    This produces a consensus error, on nethermind, versus besu/geth/oe.
    PR #822

  • Consensus issue random test 649
    A crash in nethermind, where the codecopy produced an error when called with length 0. The same error was also in CALLDATACOPY, CODECOPY, EXTCODECOPY and RETURNDATACOPY
    PR #822

  • Consensus issue random test 650
    This one was a crash in besu, that crashed when calling the modexp precompile with base and mod length 0 and a big exponent.
    PR #822

  • Move original VMTests into LegacyTests
    PR #818

  • More test examples with amazing retesteth features
    PR #819
    PR #820

  • Cosmetic change requested by besu
    Typed transaction exported as 01 + tx.bytes instead of rlp prefix
    PR #821

Upcoming changes:

EIP1559, typed transaction 02, according tests format change
London EIP tests
Besu retesteth RPC support (web vmtracing support)

Assets 2
Loading

Berlin update 8.0.2: Precompile check

27 Mar 22:34
@winsvega winsvega
8.0.2
ff68495
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Berlin update 8.0.2: Precompile check

The recent tests changes:

Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://github.com/ethereum/retesteth/wiki/Transition-Tool)
Previous release link: https://github.com/ethereum/tests/releases/tag/8.0.1

Test format

New tests in GeneralStateTests/VMTests folder converted.

  • Fillers
    none

  • Filled
    none

Test Regeneration

Update all:
none

  • Precompile availability check
    Checks that a precompile address is enabled or disabled.
    PR #811

  • Migrate VMTests into StateTests format
    PR #806

  • Fix issue #459
    More tests around division operator
    PR #807

  • Fix issue #585
    More tests for RETURNDATASIZE after CALL/CREATE reverts
    PR #808

  • Fix issue #675
    Cosmetic change
    PR #810

  • Basic operations with non const arguments
    PR #813

  • Fix issue #364
    PR #814

Upcoming changes:

  • old VMTests and it's blockchain form are to be removed.
Assets 2
Loading