[email protected]

This is a minor release for Block-Level Access List tests following [email protected]. The changes here are mostly updates to test vectors that were flagged as not producing the correct vectors. Updates were made to the specs (EELS) for tracking opcode targets properly as well as implicit SLOAD within SSTORE.

The test_bal_sstore_and_oog test was expanded, hence a minor and not patch release, to include the boundaries with EIP-2200 stipend as a threshold for including reads. Since slot load does not happen if there is not enough gas for the stipend, if OOG occurs before this threshold there are no reads recorded.

What's Changed Related to Block-Level Access Lists

• 📄 docs: EIP-7928 BAL Remove redundant test by @raxhvl in #2292 (docs only)
• 📄 docs: EIP-7928 BAL coinbase test cases by @raxhvl in #2294 (docs only)
• fix(tests): Fix test expectations for BAL oog tests by @fselmo in #2293
• fix(tests): Fix and add cases for BALs tests for SSTORE OOG conditions by @fselmo in #2297

Full Changelog: https://github.com/ethereum/execution-spec-tests/compare/[email protected]@v1.3.0

Contributions from @jwasinger, @nerolation, @raxhvl, @fselmo

v5.3.0

🇯🇵 Summary

EEST v5.3.0 is a follow-up from our main v5.0.0 release, with updated BPO1 and BPO2 values aligning with the testnet parameters.

This release additionally includes fixes for tests in hive, as well as new test cases for EIP-7883, EIP-7934 and critical cases for EIP-7951 (added to EEST by @chfast following a coverage review of the test suite).

🔑 Key Changes

🛠️ Framework

• ✨ Add benchmark-specific test wrapper (benchmark_test) that supports EIP-7825 and create a benchmark code generator for common test pattern (#1945).

fill

• ✨ Added --optimize-gas, --optimize-gas-output and --optimize-gas-post-processing flags that allow to binary search the minimum gas limit value for a transaction in a test that still yields the same test result (#1979).
• ✨ Added --watch flag that monitors test files for changes and automatically re-runs the fill command when developing tests (#2173).
• 🔀 Upgraded ckzg version to 2.1.3 or newer for correct handling of points at infinity (#2171).
• 🔀 Move pytest marker registration for fill and execute-* from their respective ini files to the shared pytest_plugins.shared.execute_fill pytest plugin (#2110).

consume

• ✨ Add retry logic to RPC requests to fix flaky connection issues in Hive (#2205).
• 🛠️ Mark consume sync tests as flaky with 3 retires due to client sync inconsistencies (#2252).
• ✨ Add consume direct using evmone-statetest and evmone-blockchaintest (#2243).

📋 Misc

• ✨ Add tighter validation for EIP-7928 model coming from t8n when filling (#2138).
• ✨ Add flexible API for absence checks for EIP-7928 (BAL) tests (#2124).
• 🐞 Use engine_newPayloadV5 for >=Amsterdam forks in consume engine (#2170).
• 🔀 Refactor EIP-7928 (BAL) absence checks into a friendlier class-based DevEx (#2175).
• 🐞 Tighten up validation for empty lists on Block-Level Access List tests (#2118).
• ✨ Added the MemoryVariable EVM abstraction to generate more readable bytecode when there's heavy use of variables that are stored in memory (#1609).
• 🐞 Fix an issue with test_bal_block_rewards where the block base fee was wrongfully overridden (#2262).
• ✨ Complete EIP checklist for EIP-7934 and update the checklist template to include block-level constraint checks (#2282).

🧪 Test Cases

• ✨ Add safe EIP-6110 workaround to allow Geth/Reth to pass invalid deposit request tests even thought they are out of spec (#2177, #2233).
• ✨ Add an EIP-7928 test case targeting the SELFDESTRUCT opcode. (#2159).
• ✨ Add essential tests for coverage gaps in EIP-7951 (p256verify precompile) (#2179, #2203, #2215, #2216, #2217, #2218, #2221, #2229, #2230, #2237, #2238).
• ✨ Add EIP-7928 successful and OOG single-opcode tests (#2118).
• ✨ Add EIP-7928 tests for EIP-2930 interactions (#2167).
• ✨ Add EIP-7928 tests for NOOP operations (#2178).
• ✨ Add EIP-7928 tests for net-zero balance transfers (#2280).
• ✨ Add fork transition test cases for EIP-7934 (#2282).

New Contributors

• @holgerd77 made their first contribution in #2184

• @CPerezz made their first contribution in #2193
• @Khushi-Chaudhary04 made their first contribution in #2214
• @SDartayet made their first contribution in #2226
• @akafle01 made their first contribution in #1714
• @qu0b made their first contribution in #2257

Full Changelog: v5.0.0...v5.3.0

[email protected]

This is a new minor release following [email protected].

This release includes:

• A fix for test_bal_block_rewards test case where the base fee was not calculated properly
• Initial set of no-op test cases
• Initial set of single-opcode tests for success and OOG cases
• Initial set of EIP-2930 test cases
• Initial set of EIP-7702 test cases
• Net-zero balance test cases

PRs Merged

• ✨ feat(tests): EIP-7928 Single-Opcode Success and OOG test cases by @raxhvl in #2118
• ✨ feat(tests): EIP-7928 test cases for EIP-2930 transactions by @raxhvl in #2167
• fix(tests): BALs - set genesis environment to calculate base fee appropriately by @fselmo in #2262
• ✨ feat(tests): EIP-7928 No-op and adjacent test case by @raxhvl in #2178
• ✨ feat(tests): EIP-7928 test cases for EIP-7702 transactions by @raxhvl in #2212
• ✨ feat(tests): EIP-7928 net zero balance transactions by @raxhvl in #2280
• 📄 docs(tests): Update EIP-7928 test case document to include no-op and OOG by @raxhvl in #2116

[email protected]

This is a new minor release update for bal as it adds some new test cases related to self-destruct, account accesses, call opcodes and delegated calls, and beneficiary fees. This also includes some updates for some broken tests in the last release related to engine API versioning, invalid test cases, as well as better validation on ordering when filling the tests.

What's Changed

• feat(tests): Enforce EIP-7928 ordering rules by @fselmo in #2138
• feat(tests): Add flexibility to expected absent scenarios for BALs by @fselmo in #2124
• fix: use engine_newPayloadV5 for Glamsterdam by @fselmo in #2170
• refactor(bal): absence validators -> BalAccountAbsentValues by @fselmo in #2175
• ✨ feat(tests): EIP-7928 SELFDESTRUCT test by @raxhvl in #2159
• ✨ feat(tests): EIP-7928 account access test cases by @raxhvl in #2182

EIP-7928 Updates

EIP-7928: Block-Level Access Lists has been updated for self-destruct, precompile cases, and other updates. Not all of these have test cases yet in this release but these are relevant changes that have happened since the last tests release.

• ethereum/EIPs#10364
• ethereum/EIPs#10377
• ethereum/EIPs#10383
• ethereum/EIPs#10393
• ethereum/EIPs#10399
• ethereum/EIPs#10434
• ethereum/EIPs#10450

Updates to execution-specs

Updates have been made to account for some of the EIP updates above and the currently implemented tests on this release (thanks to @nerolation):

• self-destruct pre-execution balance edge cases
• fix call/delegate call tracking bug
• fix duplicated entries in BAL

v5.2.0

🇯🇵 Summary

EEST v5.2.0 is a patch release for hive failures.

Please see v5.0.0 for further release notes.

Full Changelog: v5.1.0...v5.2.0

v5.1.0

🇯🇵 Summary

EEST v5.1.0 is a patch release on top of our first full release for Osaka - v5.0.0.

This release includes fixes for test exceptions and additional tests for EIP-7883/7823/7825.

Please see v5.0.0 for further release notes.

Full Changelog: v5.0.0...v5.1.0

[email protected]

What's Changed

The first release ([email protected]) revealed some updates needed for EIP-7928: Block-Level Access Lists specifications and tests. This patch release includes fixes to both the specifications (EELS) and the tests (EEST), outlined below.

Fixes

EELS (execution-specs)

The specification was missing the state tracker on both the beacon roots and history storage pre-execution system contract calls in the t8n setup, so these were missing from the BAL. Some unit tests were also updated for consistency though they didn't affect the test vectors.

• fix(bal): Initialize the state tracker before system contract calls by @fselmo in ethereum/execution-specs@c730e6d
• correct system contract addresses by @nerolation in ethereum/execution-specs@70c0f9e

EEST (execution-spec-tests)

The testing side had all invalid test cases misconfigured due to some model changes in the repository. The valid test cases did not change in setup but now include the previously-missing contract calls from the updated spec changes above.

• fix(tests): Block-Level Access List invalid tests by @fselmo in #2121

[email protected]

EESTs Fusaka Devnet 5 release v2.1.0.

Summary

Small patch on top of the latest Fusaka Devnet-5 release including a fix for some hive test exceptions.

Included EIPs

The list below links the specific commit versions of the EIPs included in this release:

• EIP-7823: Set upper bounds for MODEXP ✋
• EIP-7883: ModExp Gas Cost Increase✋
• EIP-7918: Blob base fee bounded by execution cost ✋
• EIP-7825: Transaction Gas Limit Cap ✋
• EIP-7951: Precompile for secp256r1 Curve Support ✋
• EIP-7934: RLP Execution Block Size Limit ✋
• EIP-7939: Count leading zeros (CLZ) opcode ✋
• EIP-7892: Blob Parameter Only Hardforks ✋

Emoji Key

• ✋ - No change since last release
• 🆕 - Brand new EIP tests added
• 🆙 - Existing EIP tests updated

[email protected]

EEST feature release for EIP-7928: Block-Level Access Lists tests v1.0.0.

⚠️ Important: This release revealed updates needed on both the specifications and the testing side. Look for an updated bal release with corrected and updated test vectors.

Summary

This is the initial tests release for block-level access lists support for Glamsterdam.
Tests in this release are filled only for the initial test cases related to Block-Level Access list, which only exist in the Amsterdam fork.

Relevant changes

• feat(tests): Support for EIP-7928 - Block-Level Access Lists by @fselmo in #2067
• fix(tests): Restrict extra args being passed to BaseTest; fix valid BAL tests by @fselmo in #2102

Built on top of initial work done by @raxhvl and @nerolation.

v5.0.0

🇯🇵 Summary

EEST Fujisan is our first full release for Osaka, the first full release since Pectra!

In addition to the latest Osaka specific test cases, it includes re-filled GeneralStateTests from ethereum/tests (now fully maintained within EEST under tests/static) for Osaka adhering to the transaction gas limit cap from EIP-7825. Further framework changes include new simulators, test formats and test types.

⚔️ Future Weld with EELS

EEST will merge with EELS during Q4 2025, after which EEST becomes read-only for external contributors.

What this means?

• All EEST code moves to the EELS repository.
• New EEST framework location: execution-specs/src/ethereum_spec_tests/.
• New EEST tests location: execution-specs/tests/eest/.
• Future PRs go to EELS instead of EEST.

Important Notes

• All PRs for tests and framework changes should still be directed at EEST until further notice.
• There will be a brief freeze on EEST contributions during Q4 "The Switch", after which contributors can continue as before, but in EELS.
• Test releases will continue from EEST as normal before, during, and after this transition.

More information will be communicated accordingly through the normal communication channels.

❗Current Status Quo

Test Fixtures Overview

• fixtures_static.tar.gz has been deprecated.
• fixtures_stable.tar.gz & fixtures_develop.tar.gz now both contain re-filled static tests, GeneralStateTests from ethereum/tests, filled from Cancun.
• fixtures_stable.tar.gz contains tests filled for forks until Prague.
• fixtures_develop.tar.gz contains tests filled for forks until Osaka.
• fixtures_benchmark.tar.gz contains benchmark tests filled for only Prague.

EL Client Test Requirements

Prague Coverage (Mainnet):

• Run all state_test's & blockchain_test's from fixtures_stable.tar.gz.
• Run only BlockchainTests from the latest ethereum/tests release, filled until Prague.

Fusaka Coverage (Including Mainnet):

• Run all state_test's & blockchain_test's from fixtures_develop.tar.gz
• Run only BlockchainTests from the latest ethereum/tests release, filled until Prague.

Note: If you require GeneralStateTests from ethereum/tests filled for forks before Cancun then you must get these from the latest ethereum/tests release.

Benchmark Tests

For the most up-to-date benchmark tests, use fixtures_benchmark.tar.gz.

Note: Benchmark tests for Osaka (compliant with EIP-7825 transaction gas limit cap) will be added in a future feature release.

Test Fixture Formats

This release includes 2 new test formats designed primarily for Hive simulators:

• blockchain_tests_engine_x: An optimized version of blockchain_tests_engine where multiple tests share the same genesis state, allowing multiple tests to run on a single client instantiation within Hive's consume-engine. The standard format requires a fresh client startup for each test. Due its combined genesis state, this is additionally the primary format used by the Nethermind team for benchmarking.

• blockchain_tests_sync: A new format adjacent to the existing blockchain_tests_engine format. Used specifically for the upcoming consume-sync simulator, which delivers engine payloads from test fixtures to the client under test, then sync's a separate client to it. This test fixture is only marked to be filled for the EIP-7934 block RLP limit tests in Osaka.

Tooling & Simulators

Improved tooling and new Hive simulators are additionally included in this release:

• execute remote: this command now supports optional Engine RPC endpoints (--engine-endpoint) with JWT authentication #2070.
  • This allows manual control over block creation and transaction inclusion for more deterministic test execution on live networks. Previously, execute remote could only submit transactions and rely on the network's automatic block production, but now it can actively drive chain progression by creating blocks on-demand via the Engine API.
• consume sync: Adjacent to consume-engine, designed to work with the blockchain_tests_sync format for testing client sync scenarios.
• execute blobs: A new Hive specific simulator that uses the EEST execute pytest plugin. Sends blob transactions to the client under test and verifies its engine_getBlobsVX endpoint. Requires tests to be written with a new python test format blob_transaction_test. Primarily used to test PeerDAS from the EL perspective.
• execute eth config: A command used to test the eth_config endpoint from EIP-7910. Can be ran remotely or within Hive.

Filling For Stateless Clients

A witness-filler extension is included in this release, allowing for tests to be filled that include an executionWitness for each fixture #2066. This essentially calls an external executable written in rust, and hence must be installed for usage within fill using the --witness flag. The current approach is below:

cargo install --git https://github.com/kevaundray/reth.git --branch jsign-witness-filler witness-filler
uv run fill ... --output=fixtures-witness --witness --clean

Note: The witness-filler executable is not maintained by EEST so we cannot help with any issues.

💥 Breaking Changes

Important changes for EEST superusers

• EEST now requires uv>=0.7.0 (#1904). If your version of uv is too old.
• When filling fixtures transition forks are included within there respective "to" fork, where --fork Osaka will now include PragueToOsakaAtTime15k. Previously transitions fork would only be included when filling with --from Prague --until Osaka flags.
• Python 3.10 support was removed in this release (#1808).
• EEST no longer allows usage of Yul code in Python tests. From now on, please make use of our opcode wrapper. Yul code is now only allowed in the "static tests" located in ./tests/static/ (these are test cases defined by JSON and YAML files instead of Python test functions that were originally maintained in ethereum/tests).
• In order to fill the static tests (which is not the case by default), please ensure that solc is located in your PATH.
• The output behavior of fill has changed (#1608):
  • Before: fill wrote fixtures into the directory specified by the --output flag (default: fixtures). This could have many unintended consequences, including unexpected errors if old or invalid fixtures existed in the directory (for details see #1030).
  • Now: fill will exit without filling any tests if the specified directory exists and is not-empty. This may be overridden by adding the --clean flag, which will first remove the specified directory.
• Writing debugging information to the EVM "dump directory" by default has been disabled. To obtain debug output, the --evm-dump-dir flag must now be explicitly set. As a consequence, the now redundant --skip-evm-dump option was removed (#1874). This undoes functionality originally introduced in #999 and #1150.

Feature zkevm updated to benchmark

Due to the crossover between zkevm and benchmark tests, all instances of the former have been replaced with the latter nomenclature. Repository PR labels and titles are additionally updated to reflect this change.

This update renames the zkevm feature release to benchmark and further expands the latter for 1M, 10M, 30M, 45M, 60M, 90M, and 120M block gas limits in fixtures_benchmark.tar.gz.

To select a test for a given gas limit, the IDs of the tests have been expanded to contain benchmark-gas-value_XM, where X can be any of the aforementioned values.

The benchmark release also now includes BlockchainEngineX format that combines most of the tests into a minimal amount of genesis files. For more info see Blockchain Engine X Tests in the EEST documentation.

Users can select any of the artifacts depending on their benchmarking or testing needs for their provers.

🔑 Other Key Changes

🛠️ Framework

🔀 Refactoring

• 🔀 Move TransactionType enum from test file to proper module location in ethereum_test_types.transaction_types for better code organization ...