You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Benchmark tests aim to maximize the usage of a specific opcode, precompile, or operation within a transaction or block. They are located in the `./tests/benchmarks` folder and the available test cases are documented in [test case reference](../tests/benchmark/index.md).
4
+
5
+
To fill a benchmark test, in addition to the usual test flags, you must include the `-m benchmark` flag. This is necessary because benchmark tests are ignored by default; they must be manually selected via the `benchmark` pytest marker (="tag"). This marker is applied to all tests under `./tests/benchmark/` automatically by the framework.
6
+
7
+
## Setting the Gas Limit for Benchmarking
8
+
9
+
To consume the full benchmark gas limit, use the `gas_benchmark_value` fixture as the gas limit:
10
+
11
+
```py
12
+
deftest_benchmark(
13
+
blockchain_test: BlockchainTestFiller,
14
+
pre: Alloc,
15
+
gas_benchmark_value: int
16
+
):
17
+
...
18
+
```
19
+
20
+
You can specify the block gas limit used in benchmark tests by setting the `--gas-benchmark-values` flag. This flag accepts a comma-separated list of values (in millions of gas), e.g. `--gas-benchmark-values 1,10,45,60`. This example would run the test 4 times, using a `gas_benchmark_value` of 1M, 10M, 45M, and 60M respectively.
21
+
22
+
Do not configure the transaction/block gas limit to `env.gas_limit`. When running in benchmark mode, the test framework sets this value to a very large number (e.g., `1_000_000_000_000`), this setup allows the framework to reuse a single genesis file for all specified gas limits. I.e., the example below is invalid:
23
+
24
+
```py
25
+
deftest_benchmark(
26
+
blockchain_test: BlockchainTestFiller,
27
+
pre: Alloc,
28
+
env: Environment
29
+
):
30
+
...
31
+
tx = Transaction(
32
+
to=opcode_address,
33
+
gas_limit=env.gas_limit, # Do not set the gas_limit manually.
34
+
sender=pre.fund_eoa(),
35
+
)
36
+
...
37
+
```
38
+
39
+
## Expected Gas Usage
40
+
41
+
In benchmark mode, the developer should set the expected gas consumption using the `expected_benchmark_gas_used` field. Benchmark tests do not need to consume the full gas limit, instead, you could calculate and specify the expected usage. If `expected_benchmark_gas_used` is not set, the test will fall back to using `gas_benchmark_value` as the expected value.
42
+
43
+
```py
44
+
@pytest.mark.valid_from("Prague")
45
+
deftest_empty_block(
46
+
blockchain_test: BlockchainTestFiller,
47
+
pre: Alloc,
48
+
):
49
+
"""Test running an empty block as a baseline for fixed proving costs."""
50
+
blockchain_test(
51
+
pre=pre,
52
+
post={},
53
+
blocks=[Block(txs=[])],
54
+
expected_benchmark_gas_used=0,
55
+
)
56
+
```
57
+
58
+
This is a safety check to make sure the benchmark works as expected. For example, if a test uses the `JUMP` instruction but the jump destination is invalid, each transaction will stop early. That means it won't use as much gas as we expected.
59
+
60
+
This check helps catch such issues. As a result, the post-storage comparison method via `SSTORE` is no longer needed, thereby reducing the additional storage cost.
0 commit comments