v2.0.0

🐍🏖️ Cancun and StateTest Format Release

Contains many important framework changes, including introduction of the StateTest format, and some additional Cancun and other test coverage.

Due to changes in the framework, there is a breaking change in the directory structure in the release tarball, please see the dedicated "💥 Breaking Changes" section below for more information.

🧪 Test Cases

• ✨ EIP-4844: Add test_sufficient_balance_blob_tx() and test_sufficient_balance_blob_tx_pre_fund_tx() (#379).
• ✨ EIP-6780: Add a reentrancy suicide revert test (#372).
• ✨ EIP-1153: Add test_run_until_out_of_gas() for transient storage opcodes (#401).
• ✨ EIP-198: Add tests for the MODEXP precompile (#364).
• ✨ Tests for nested CALL and CALLCODE gas consumption with a positive value transfer (previously lacking coverage) (#371).
• 🐞 EIP-4844: Fixed test_invalid_tx_max_fee_per_blob_gas() to account for extra gas required in the case where the account is incorrectly deduced the balance as if it had the correct block blob gas fee (#370).
• 🐞 EIP-4844: Fixed test_insufficient_balance_blob_tx() to correctly calculate the minimum balance required for the accounts (#379).
• 🐞 EIP-4844: Fix and enable test_invalid_blob_tx_contract_creation (#379).
• 🔀 Convert all eligible BlockchainTests to StateTests (and additionally generate corresponding BlockchainTests) (#368, #370).

🛠️ Framework

• ✨ Add StateTest fixture format generation; StateTests now generate a StateTest and a corresponding BlockchainTest test fixture, previously only BlockchainTest fixtures were generated (#368).
• ✨ Add StateTestOnly fixture format is now available and its only difference with StateTest is that it does not produce a BlockchainTest (#368).
• ✨ Add evm_bytes_to_python command-line utility which converts EVM bytecode to Python Opcodes (#357).
• ✨ Fork objects used to write tests can now be compared using the >, >=, <, <= operators, to check for a fork being newer than, newer than or equal, older than, older than or equal, respectively when compared against other fork (#367).
• ✨ Add solc 0.8.23 support (#373).
• ✨ Add framework unit tests for post state exception verification (#350).
• ✨ Add a helper class ethereum_test_tools.TestParameterGroup to define test parameters as dataclasses and auto-generate test IDs (#364).
• ✨ Add a --single-fixture-per-file flag to generate one fixture JSON file per test case (#331).
• 🐞 Storage type iterator is now fixed (#369).
• 🐞 Fix type coercion in FixtureHeader.join() (#398).
• 🔀 Locally calculate the transactions list's root instead of using the one returned by t8n when producing BlockchainTests (#353).
• 🔀 Change custom exception classes to dataclasses to improve testability (#386).
• 🔀 Update fork name from "Merge" to "Paris" used within the framework and tests (#363).
• 💥 Replace = with _ in pytest node ids and test fixture names (#342).
• 💥 The StateTest, spec format used to write tests, is now limited to a single transaction per test (#361).
• 💥 Tests must now use BlockException and TransactionException to define the expected exception of a given test, which can be used to test whether the client is hitting the proper exception when processing the block or transaction (#384).
• 💥 fill: Remove the --enable-hive flag; now all test types are generated by default (#358).
• 💥 Rename test fixtures names to match the corresponding pytest node ID as generated using fill (#342).

📋 Misc

• ✨ Docs: Add a "Consuming Tests" section to the docs, where each test fixture format is described, along with the steps to consume them, and the description of the structures used in each format (#375).
• 🔀 Docs: Update t8n tool branch to fill tests for development features in the readme (#338).
• 🔀 Filling tool: Updated the default filling tool (t8n) to go-ethereum@master (#368).
• 🐞 Docs: Fix error banner in online docs due to mermaid syntax error (#398).
• 🐞 Docs: Fix incorrectly formatted nested lists in online doc (#403).

💥 Breaking Changes

A concrete example of the test name renaming and change in directory structure is provided below.

1. Fixture output, including release tarballs, now contain subdirectories for different test types:

   1. blockchain_tests: Contains BlockchainTest formatted tests
   2. blockchain_tests_hive: Contains BlockchainTest with Engine API call directives for use in hive
   3. state_tests: Contains StateTest formatted tests

2. StateTest, spec format used to write tests, is now limited to a single transaction per test.

3. In this release the pytest node ID is now used for fixture names (previously only the test parameters were used), this should not be breaking. However, = in both node IDs (and therefore fixture names) have been replaced with _, which may break tooling that depends on the = character.

4. Produced blockchain_tests fixtures and their corresponding blockchain_tests_hive fixtures now contain the named exceptions BlockException and TransactionException as strings in the expectException and validationError fields, respectively. These exceptions can be used to test whether the client is hitting the proper exception when processing an invalid block.

   Blockchain test:

   "blocks": [
         {
            ...
            "expectException": "TransactionException.INSUFFICIENT_ACCOUNT_FUNDS",
            ...
         }
         ...
   ]

   Blockchain hive test:

   "engineNewPayloads": [
         {
            ...
            "validationError": "TransactionException.INSUFFICIENT_ACCOUNT_FUNDS",
            ...
         }
         ...
   ]

Renaming and Release Tarball Directory Structure Change Example

The fixture renaming provides a more consistent naming scheme between the pytest node ID and fixture name and allows the fixture name to be provided directly to pytest 5on the command line to execute individual tests in isolation, e.g. pytest tests/frontier/opcodes/test_dup.py::test_dup[fork_Frontier].

1. Pytest node ID example:

   1. Previous node ID: tests/frontier/opcodes/test_dup.py::test_dup[fork=Frontier].
   2. New node ID: tests/frontier/opcodes/test_dup.py::test_dup[fork_Frontier].

2. Fixture name example:

   1. Previous fixture name: 000-fork=Frontier
   2. New fixture name: tests/frontier/opcodes/test_dup.py::test_dup[fork_Frontier] (now the same as the pytest node ID).

3. Fixture JSON file name example (within the release tarball):

   1. Previous fixture file name: fixtures/frontier/opcodes/dup/dup.json (BlockChainTest format).
   2. New fixture file names (all present within the release tarball):
   • fixtures/state_tests/frontier/opcodes/dup/dup.json (StateTest format).
   • fixtures/blockchain_tests/frontier/opcodes/dup/dup.json (BlockChainTest format).
   • fixtures/blockchain_tests_hive/frontier/opcodes/dup/dup.json (a blockchain test in HiveFixture format).

New Contributors

• @jochem-brouwer made their first contribution in #364
• @MukulKolpe made their first contribution in #363
• @acolytec3 made their first contribution in #371
• @richardgreg made their first contribution in #373

Full Changelog: v1.0.6...v2.0.0