eest_tests sends blocks with too much gas

[siladu]

The eest_test files go up to 150M gas block tests, however for SELFDESTRUCT, SSTORE and SSLOAD we are seeing much bigger blocks.

I added timestamps to gas-benchmarks logs to cross reference with the Besu logs, which show the gas used by the block.

e.g. SELFDESTRUCT cases send up to 650MGas instead of 150MGas blocks:

< [2025-09-17 21:11:50.879] Running besu for the 1 time with test case eest_tests/tests_benchmark_test_worst_stateful_opcodes.py__test_worst_selfdestruct_existing[fork_Prague-benchmark-blockchain_test_engine_x-value_bearing_False]-
gas-value_100M.txt
> 2025-09-17 21:12:02.809+0000 | vert.x-worker-thread-0 | INFO  | AbstractEngineNewPayload | Imported #1  (210f3.....e1622)|    1 tx|  0 ws|  0 blobs|      7 wei bfee| 
434,787,335 (  0.0%) gas used| 11.658s exec|  37.30 Mgas/s|  0 peers

< [2025-09-17 21:12:02.923] Running besu for the 1 time with test case eest_tests/tests_benchmark_test_worst_stateful_opcodes.py__test_worst_selfdestruct_existing[fork_Prague-benchmark-blockchain_test_engine_x-value_bearing_False]-
gas-value_10M.txt
> 2025-09-17 21:12:03.432+0000 | vert.x-worker-thread-0 | INFO  | AbstractEngineNewPayload | Imported #1  (a0e84.....5aaaf)|    1 tx|  0 ws|  0 blobs|      7 wei bfee|  
43,393,301 (  0.0%) gas used| 0.243s exec| 178.57 Mgas/s|  0 peers

< [2025-09-17 21:12:03.544] Running besu for the 1 time with test case eest_tests/tests_benchmark_test_worst_stateful_opcodes.py__test_worst_selfdestruct_existing[fork_Prague-benchmark-
blockchain_test_engine_x-value_bearing_False]-
gas-value_150M.txt
> 2025-09-17 21:12:30.265+0000 | vert.x-worker-thread-0 | INFO  | AbstractEngineNewPayload | Imported #1  (6c673.....6006a)|    1 tx|  0 ws|  0 blobs|      7 wei bfee| 
652,239,656 (  0.1%) gas used| 26.448s exec|  24.66 Mgas/s|  0 peers

< [2025-09-17 21:12:30.380] Running besu for the 1 time with test case eest_tests/tests_benchmark_test_worst_stateful_opcodes.py__test_worst_selfdestruct_existing[fork_Prague-benchmark-blockchain_test_engine_x-value_bearing_False]-
gas-value_30M.txt
> 2025-09-17 21:12:32.194+0000 | vert.x-worker-thread-0 | INFO  | AbstractEngineNewPayload | Imported #1  (8e22d.....2362d)|    1 tx|  0 ws|  0 blobs|      7 wei bfee| 
130,380,944 (  0.0%) gas used| 1.544s exec|  84.44 Mgas/s|  0 peers

< [2025-09-17 21:12:32.304] Running besu for the 1 time with test case eest_tests/tests_benchmark_test_worst_stateful_opcodes.py__test_worst_selfdestruct_existing[fork_Prague-benchmark-blockchain_test_engine_x-value_bearing_False]-
gas-value_45M.txt
> 2025-09-17 21:12:35.432+0000 | vert.x-worker-thread-0 | INFO  | AbstractEngineNewPayload | Imported #1  (d5b18.....bdc7e)|    1 tx|  0 ws|  0 blobs|      7 wei bfee| 
195,613,283 (  0.0%) gas used| 2.851s exec|  68.61 Mgas/s|  0 peers

< [2025-09-17 21:12:35.542] Running besu for the 1 time with test case eest_tests/tests_benchmark_test_worst_stateful_opcodes.py__test_worst_selfdestruct_existing[fork_Prague-benchmark-blockchain_test_engine_x-value_bearing_False]-
gas-value_60M.txt
> 2025-09-17 21:12:40.478+0000 | vert.x-worker-thread-0 | INFO  | AbstractEngineNewPayload | Imported #1  (a10ed.....631f5)|    1 tx|  0 ws|  0 blobs|      7 wei bfee| 
260,845,622 (  0.0%) gas used| 4.670s exec|  55.86 Mgas/s|  0 peers

This appears to lead to a miscalculation in the final report, since in this case Besu is calculated as having ~5 MGas/s min throughput whereas the real min is 24.66 Mgas/s according to our logs.

[jochem-brouwer]

Hey! EEST will indeed send blocks with a higher gas limit for this specific test than the target attack gas limit we want to measure.

For this selfdestruct test it first deploys a lot of existing contracts which COINBASE SELFDESTRUCT as code. The deployment costs of the contracts we can reach given an attack gas limit X is higher than this attack gas limit. In the test this is all put into the block prior to the attack block which thus has a higher gas limit. It should thus be ignored for the test.

This tx: https://github.com/ethereum/execution-spec-tests/blob/10067fc08b0a8af2af5047f9f97dfd3b73eba453/tests/benchmark/test_worst_stateful_opcodes.py#L651 deploys the contracts and should be ignored for the bench

This tx (this is put in the block after the deployment tx above): https://github.com/ethereum/execution-spec-tests/blob/10067fc08b0a8af2af5047f9f97dfd3b73eba453/tests/benchmark/test_worst_stateful_opcodes.py#L679 should be benched.

It will also depend a bit how you calculate the MGas/s, are you taking the gas used from the block or are you assuming the target gas value from the test name? E.g. 150M while the predeploy block takes 600+M. This should still not run in lower numbers of course (5 MGas/s for predeployment is too low).

How do you calculate the MGas/s based on the report? Or are you using the numbers as reported from this tool?

[marcindsobczak]

We are using the number from the test name.
Actually, switching to real gas used will be the quickest workaround for now and also a more robust solution for the future.

Test cases are provided by STEEL team and we are using payloads delivered in the release: https://github.com/ethereum/execution-spec-tests/releases/tag/benchmark%40v0.0.4

[LouisTsai-Csie]

After reviewing the benchmark release, 4 cases would be affected:

• test_worst_bytecode_single_opcode
• test_worst_address_state_cold
• test_worst_storage_access_warm
• test_worst_selfdestruct_existing

The feature of these cases is, before the actual benchmark trasaction/block is executed, it will pre-deploy a huge amount of contracts. In EEST benchmark framework, gas_benchmark_value is the benchmark gas limit target, while the env.gas_limit is a relatively high value (much larger than gas_benchmark_value), which is a workaround for some of our framework settings.

What i would suggest now, is to filter the block for deployment in these cases, excluding them from the benchmark block/transaction target. From EEST's side, I am currently (1) working on refactoring these cases and (2) adding benchmark phases for you to filter non-benchmark operation.

[marioevz]

What i would suggest now, is to filter the block for deployment in these cases, excluding them from the benchmark block/transaction target.

In other words, the benchmark will always only be the last block of the test and prior blocks, including genesis, should not be taken into account during measurements.

[kamilchodola]

Will regenerate eest_tests soon from latest benchmark release and review the script as it may be taking not the proper payload