Skip to content

Commit d197aaf

Browse files
committed
refactor(all): update lang from 'shared pre-alloc' to 'pre-allocation groups' 2/2
1 parent 0288797 commit d197aaf

File tree

13 files changed

+96
-96
lines changed

13 files changed

+96
-96
lines changed

docs/CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ Users can select any of the artifacts depending on their testing needs for their
3131

3232
- ✨ Add the `ported_from` test marker to track Python test cases that were converted from static fillers in [ethereum/tests](https://github.com/ethereum/tests) repository [#1590](https://github.com/ethereum/execution-spec-tests/pull/1590).
3333
- ✨ Add a new pytest plugin, `ported_tests`, that lists the static fillers and PRs from `ported_from` markers for use in the coverage Github Workflow [#1634](https://github.com/ethereum/execution-spec-tests/pull/1634).
34-
- ✨ Enable two-phase filling of fixtures with shared pre-allocation groups and add a `BlockchainEngineXFixture` format [#1606](https://github.com/ethereum/execution-spec-tests/pull/1706).
34+
- ✨ Enable two-phase filling of fixtures with pre-allocation groups and add a `BlockchainEngineXFixture` format [#1606](https://github.com/ethereum/execution-spec-tests/pull/1706).
3535
- 🔀 Refactor: Encapsulate `fill`'s fixture output options (`--output`, `--flat-output`, `--single-fixture-per-file`) into a `FixtureOutput` class ([#1471](https://github.com/ethereum/execution-spec-tests/pull/1471),[#1612](https://github.com/ethereum/execution-spec-tests/pull/1612)).
3636
- ✨ Don't warn about a "high Transaction gas_limit" for `zkevm` tests ([#1598](https://github.com/ethereum/execution-spec-tests/pull/1598)).
3737
- 🐞 `fill` no longer writes generated fixtures into an existing, non-empty output directory; it must now be empty or `--clean` must be used to delete it first ([#1608](https://github.com/ethereum/execution-spec-tests/pull/1608)).

docs/library/cli/extract_config.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ Since Hive doesn't directly expose container IDs, the tool uses a detection mech
120120
The tool supports:
121121

122122
- Individual fixture JSON files (BlockchainFixture format)
123-
- SharedPreStateGroup JSON files
123+
- PreAllocGroup JSON files
124124
- Directories containing multiple fixture files
125125

126126
## Troubleshooting

docs/running_tests/test_formats/blockchain_test_engine_x.md

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,30 @@
11
# Blockchain Engine X Tests <!-- markdownlint-disable MD051 (MD051=link-fragments "Link fragments should be valid") -->
22

3-
The Blockchain Engine X Test fixture format tests are included in the fixtures subdirectory `blockchain_tests_engine_x`, and use Engine API directives with optimized shared pre-allocation for improved execution performance.
3+
The Blockchain Engine X Test fixture format tests are included in the fixtures subdirectory `blockchain_tests_engine_x`, and use Engine API directives with optimized pre-allocation groups for improved execution performance.
44

5-
These are produced by the `StateTest` and `BlockchainTest` test specs when using the `--generate-shared-pre` and `--use-shared-pre` flags.
5+
These are produced by the `StateTest` and `BlockchainTest` test specs when using the `--generate-grouped-pre-allocs` and `--use-grouped-pre-allocs` flags.
66

77
## Description
88

99
The Blockchain Engine X Test fixture format is an optimized variant of the [Blockchain Engine Test](./blockchain_test_engine.md) format designed for large-scale test execution with performance optimizations.
1010

11-
It uses the Engine API to test block validation and consensus rules while leveraging **shared pre-allocation state** to significantly reduce test execution time and resource usage. Tests are grouped by their initial state (fork + environment + pre-allocation). Each groups are executed against the same client instance using a common genesis state.
11+
It uses the Engine API to test block validation and consensus rules while leveraging **pre-allocation groups** to significantly reduce test execution time and resource usage. Tests are grouped by their initial state (fork + environment + pre-allocation). Each group is executed against the same client instance using a common genesis state.
1212

1313
The key optimization is that **clients need only be started once per group** instead of once per test (as in the original engine fixture format), dramatically improving execution performance for large test suites.
1414

15-
Instead of including large pre-allocation state in each test fixture, this format references a shared pre-allocation folder (`pre_alloc`) which includes all different pre-allocation combinations used for any test fixture group.
15+
Instead of including large pre-allocation state in each test fixture, this format references a pre-allocation groups folder (`pre_alloc`) which contains all different pre-allocation combinations organized by group.
1616

1717
A single JSON fixture file is composed of a JSON object where each key-value pair is a different [`BlockchainTestEngineXFixture`](#BlockchainTestEngineXFixture) test object, with the key string representing the test name.
1818

1919
The JSON file path plus the test name are used as the unique test identifier.
2020

21-
## Shared Pre-Allocation File
21+
## Pre-Allocation Groups Folder
2222

23-
The `blockchain_tests_engine_x` directory contains a special directory `pre_alloc` that stores shared pre-allocation state file used by all tests in this format, one per pre-allocation group with the name of the pre-alloc hash. This folder is essential for test execution and must be present alongside the test fixtures.
23+
The `blockchain_tests_engine_x` directory contains a special directory `pre_alloc` that stores pre-allocation group files used by all tests in this format, one per pre-allocation group with the name of the pre-alloc hash. This folder is essential for test execution and must be present alongside the test fixtures.
2424

25-
### Pre-Allocation File Structure
25+
### Pre-Allocation Group File Structure
2626

27-
Each file in the `pre_alloc` folder corresponds to a pre-allocation hash to shared state groups:
27+
Each file in the `pre_alloc` folder corresponds to a pre-allocation group identified by a hash:
2828

2929
```json
3030
{
@@ -37,14 +37,14 @@ Each file in the `pre_alloc` folder corresponds to a pre-allocation hash to shar
3737
}
3838
```
3939

40-
#### SharedPreStateGroup Fields
40+
#### Pre-Allocation Group Fields
4141

42-
- **`test_count`**: Number of tests sharing this pre-allocation group
43-
- **`pre_account_count`**: Number of accounts in the shared pre-allocation state
44-
- **`testIds`**: Array of test identifiers that use this shared state
42+
- **`test_count`**: Number of tests in this pre-allocation group
43+
- **`pre_account_count`**: Number of accounts in the pre-allocation group
44+
- **`testIds`**: Array of test identifiers that belong to this group
4545
- **`network`**: Fork name (e.g., "Prague", "Cancun")
4646
- **`environment`**: Complete [`Environment`](./common_types.md#environment) object with execution context
47-
- **`pre`**: Shared [`Alloc`](./common_types.md#alloc-mappingaddressaccount) object containing initial account states
47+
- **`pre`**: Pre-allocation group [`Alloc`](./common_types.md#alloc-mappingaddressaccount) object containing initial account states
4848

4949
## Consumption
5050

@@ -70,7 +70,7 @@ For each [`BlockchainTestEngineXFixture`](#BlockchainTestEngineXFixture) test ob
7070
4. **Verify Final State**:
7171
- Compare the final chain head against [`lastblockhash`](#-lastblockhash-hash)
7272
- If [`postStateDiff`](#-poststatediff-optionalalloc) is present:
73-
- Apply the state differences to the shared pre-allocation
73+
- Apply the state differences to the pre-allocation group
7474
- Verify the resulting state matches the client's final state
7575
- If `post` field were present (not typical), verify it directly
7676

@@ -88,11 +88,11 @@ This field is going to be replaced by the value contained in `config.network`.
8888

8989
#### - `preHash`: `string`
9090

91-
Hash identifier referencing a shared pre-allocation group in the `pre_alloc` folder. This hash uniquely identifies the combination of fork, environment, and pre-allocation state shared by multiple tests.
91+
Hash identifier referencing a pre-allocation group in the `pre_alloc` folder. This hash uniquely identifies the combination of fork, environment, and pre-allocation state that defines the group.
9292

9393
#### - `genesisBlockHeader`: [`FixtureHeader`](./blockchain_test.md#fixtureheader)
9494

95-
Genesis block header. The state root in this header must match the state root calculated from the shared pre-allocation referenced by [`preHash`](#-prehash-string).
95+
Genesis block header. The state root in this header must match the state root calculated from the pre-allocation group referenced by [`preHash`](#-prehash-string).
9696

9797
#### - `engineNewPayloads`: [`List`](./common_types.md#list)`[`[`FixtureEngineNewPayload`](#fixtureenginenewpayload)`]`
9898

@@ -108,11 +108,11 @@ Hash of the last valid block after all payloads have been processed, or the gene
108108

109109
#### - `postStateDiff`: [`Optional`](./common_types.md#optional)`[`[`Alloc`](./common_types.md#alloc-mappingaddressaccount)`]`
110110

111-
State differences from the shared pre-allocation state after test execution. This optimization stores only the accounts that changed, were created, or were deleted during test execution, rather than the complete final state.
111+
State differences from the pre-allocation group after test execution. This optimization stores only the accounts that changed, were created, or were deleted during test execution, rather than the complete final state.
112112

113113
To reconstruct the final state:
114114

115-
1. Start with the shared pre-allocation from the `pre_alloc` folder
115+
1. Start with the pre-allocation group from the `pre_alloc` folder
116116
2. Apply the changes in `postStateDiff`:
117117
- **Modified accounts**: Replace existing accounts with new values
118118
- **New accounts**: Add accounts not present in pre-allocation
@@ -138,7 +138,7 @@ Engine API payload structure identical to the one defined in [Blockchain Engine
138138

139139
## Usage Notes
140140

141-
- This format is only generated when using `--generate-shared-pre` and `--use-shared-pre` flags
141+
- This format is only generated when using `--generate-grouped-pre-allocs` and `--use-grouped-pre-allocs` flags
142142
- The `pre_alloc` folder is essential and must be distributed with the test fixtures
143143
- Tests are grouped by identical (fork + environment + pre-allocation) combinations
144144
- The format is optimized for Engine API testing (post-Paris forks)

src/cli/extract_config.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
from ethereum_test_fixtures import BlockchainFixtureCommon
2222
from ethereum_test_fixtures.blockchain import FixtureHeader
2323
from ethereum_test_fixtures.file import Fixtures
24-
from ethereum_test_fixtures.shared_alloc import SharedPreStateGroup
24+
from ethereum_test_fixtures.pre_alloc_groups import PreAllocGroup
2525
from ethereum_test_forks import Fork
2626
from pytest_plugins.consume.simulators.helpers.ruleset import ruleset
2727

@@ -111,9 +111,9 @@ def create_genesis_from_fixture(fixture_path: Path) -> Tuple[FixtureHeader, Allo
111111
alloc = fixture.pre
112112
chain_id = int(fixture.config.chain_id)
113113
else:
114-
shared_alloc = SharedPreStateGroup.model_validate(fixture_json)
115-
genesis = shared_alloc.genesis # type: ignore
116-
alloc = shared_alloc.pre
114+
pre_alloc_group = PreAllocGroup.model_validate(fixture_json)
115+
genesis = pre_alloc_group.genesis # type: ignore
116+
alloc = pre_alloc_group.pre
117117

118118
return genesis, alloc, chain_id
119119

src/cli/show_pre_alloc_group_stats.py

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
from rich.table import Table
1111

1212
from ethereum_test_base_types import CamelModel
13-
from ethereum_test_fixtures import SharedPreState
13+
from ethereum_test_fixtures import PreAllocGroups
1414

1515

1616
def extract_test_module(test_id: str) -> str:
@@ -102,22 +102,22 @@ def calculate_size_distribution(
102102

103103
def analyze_pre_alloc_folder(folder: Path, verbose: int = 0) -> Dict:
104104
"""Analyze pre-allocation folder and return statistics."""
105-
pre_state = SharedPreState.from_folder(folder)
105+
pre_alloc_groups = PreAllocGroups.from_folder(folder)
106106

107107
# Basic stats
108-
total_groups = len(pre_state)
109-
total_tests = sum(group.test_count for group in pre_state.values())
110-
total_accounts = sum(group.pre_account_count for group in pre_state.values())
108+
total_groups = len(pre_alloc_groups)
109+
total_tests = sum(group.test_count for group in pre_alloc_groups.values())
110+
total_accounts = sum(group.pre_account_count for group in pre_alloc_groups.values())
111111

112112
# Group by fork
113113
fork_stats: Dict[str, Dict] = defaultdict(lambda: {"groups": 0, "tests": 0})
114-
for group in pre_state.values():
114+
for group in pre_alloc_groups.values():
115115
fork_stats[group.fork.name()]["groups"] += 1
116116
fork_stats[group.fork.name()]["tests"] += group.test_count
117117

118118
# Group by test module
119119
module_stats: Dict[str, Dict] = defaultdict(lambda: {"groups": set(), "tests": 0})
120-
for hash_key, group in pre_state.items():
120+
for hash_key, group in pre_alloc_groups.items():
121121
# Count tests per module in this group
122122
module_test_count: defaultdict = defaultdict(int)
123123
for test_id in group.test_ids:
@@ -135,7 +135,7 @@ def analyze_pre_alloc_folder(folder: Path, verbose: int = 0) -> Dict:
135135

136136
# Per-group details
137137
group_details = []
138-
for hash_key, group in pre_state.items():
138+
for hash_key, group in pre_alloc_groups.items():
139139
group_details.append(
140140
{
141141
"hash": hash_key[:8] + "...", # Shortened hash for display
@@ -158,7 +158,7 @@ class SplitTestFunction(CamelModel):
158158
split_test_functions: Dict[str, SplitTestFunction] = defaultdict(lambda: SplitTestFunction())
159159

160160
# Process all size-1 groups directly from pre_state
161-
for _hash_key, group_data in pre_state.items():
161+
for _hash_key, group_data in pre_alloc_groups.items():
162162
if group_data.test_count == 1: # Size-1 group
163163
test_id = group_data.test_ids[0]
164164
test_function = extract_test_function(test_id)

src/ethereum_test_fixtures/__init__.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
from .collector import FixtureCollector, TestInfo
1212
from .consume import FixtureConsumer
1313
from .eof import EOFFixture
14-
from .shared_alloc import SharedPreState, SharedPreStateGroup
14+
from .pre_alloc_groups import PreAllocGroup, PreAllocGroups
1515
from .state import StateFixture
1616
from .transaction import TransactionFixture
1717

@@ -27,8 +27,8 @@
2727
"FixtureConsumer",
2828
"FixtureFormat",
2929
"LabeledFixtureFormat",
30-
"SharedPreState",
31-
"SharedPreStateGroup",
30+
"PreAllocGroups",
31+
"PreAllocGroup",
3232
"StateFixture",
3333
"TestInfo",
3434
"TransactionFixture",

src/ethereum_test_fixtures/blockchain.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -523,7 +523,7 @@ class BlockchainEngineFixtureCommon(BaseFixture):
523523
Base blockchain test fixture model for Engine API based execution.
524524
525525
Similar to BlockchainFixtureCommon but excludes the 'pre' field to avoid
526-
duplicating large pre-allocations when using shared genesis approaches.
526+
duplicating large pre-allocations.
527527
"""
528528

529529
fork: Fork = Field(..., alias="network")
@@ -564,15 +564,15 @@ class BlockchainEngineXFixture(BlockchainEngineFixtureCommon):
564564
"""
565565
Engine X specific test fixture information.
566566
567-
Uses shared pre-allocations (and a shared client instance) for efficient
567+
Uses pre-allocation groups (and a single client instance) for efficient
568568
test execution without client restarts.
569569
"""
570570

571571
format_name: ClassVar[str] = "blockchain_test_engine_x"
572572
description: ClassVar[str] = "Tests that generate a Blockchain Test Engine X fixture."
573573

574574
pre_hash: str
575-
"""Hash of the shared pre-allocation group this test belongs to."""
575+
"""Hash of the pre-allocation group this test belongs to."""
576576

577577
post_state_diff: Alloc | None = None
578578
"""State difference from genesis after test execution (efficiency optimization)."""

src/ethereum_test_fixtures/shared_alloc.py renamed to src/ethereum_test_fixtures/pre_alloc_groups.py

Lines changed: 15 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
from .blockchain import FixtureHeader
1414

1515

16-
class SharedPreStateGroup(CamelModel):
16+
class PreAllocGroup(CamelModel):
1717
"""
1818
Pre-allocation group for tests with identical Environment and fork values.
1919
@@ -49,45 +49,43 @@ def genesis(self) -> FixtureHeader:
4949
)
5050

5151
def to_file(self, file: Path) -> None:
52-
"""Save SharedPreStateGroup to a file."""
52+
"""Save PreAllocGroup to a file."""
5353
lock_file_path = file.with_suffix(".lock")
5454
with FileLock(lock_file_path):
5555
if file.exists():
5656
with open(file, "r") as f:
57-
previous_shared_pre_state_group = SharedPreStateGroup.model_validate_json(
58-
f.read()
59-
)
60-
for account in previous_shared_pre_state_group.pre:
57+
previous_pre_alloc_group = PreAllocGroup.model_validate_json(f.read())
58+
for account in previous_pre_alloc_group.pre:
6159
if account not in self.pre:
62-
self.pre[account] = previous_shared_pre_state_group.pre[account]
63-
self.pre_account_count += previous_shared_pre_state_group.pre_account_count
64-
self.test_count += previous_shared_pre_state_group.test_count
65-
self.test_ids.extend(previous_shared_pre_state_group.test_ids)
60+
self.pre[account] = previous_pre_alloc_group.pre[account]
61+
self.pre_account_count += previous_pre_alloc_group.pre_account_count
62+
self.test_count += previous_pre_alloc_group.test_count
63+
self.test_ids.extend(previous_pre_alloc_group.test_ids)
6664

6765
with open(file, "w") as f:
6866
f.write(self.model_dump_json(by_alias=True, exclude_none=True, indent=2))
6967

7068

71-
class SharedPreState(EthereumTestRootModel):
72-
"""Root model mapping pre-state hashes to test groups."""
69+
class PreAllocGroups(EthereumTestRootModel):
70+
"""Root model mapping pre-allocation group hashes to test groups."""
7371

74-
root: Dict[str, SharedPreStateGroup]
72+
root: Dict[str, PreAllocGroup]
7573

7674
def __setitem__(self, key: str, value: Any):
7775
"""Set item in root dict."""
7876
self.root[key] = value
7977

8078
@classmethod
81-
def from_folder(cls, folder: Path) -> "SharedPreState":
82-
"""Create SharedPreState from a folder of pre-allocation files."""
79+
def from_folder(cls, folder: Path) -> "PreAllocGroups":
80+
"""Create PreAllocGroups from a folder of pre-allocation files."""
8381
data = {}
8482
for file in folder.glob("*.json"):
8583
with open(file) as f:
86-
data[file.stem] = SharedPreStateGroup.model_validate_json(f.read())
84+
data[file.stem] = PreAllocGroup.model_validate_json(f.read())
8785
return cls(root=data)
8886

8987
def to_folder(self, folder: Path) -> None:
90-
"""Save SharedPreState to a folder of pre-allocation files."""
88+
"""Save PreAllocGroups to a folder of pre-allocation files."""
9189
for key, value in self.root.items():
9290
value.to_file(folder / f"{key}.json")
9391

0 commit comments

Comments
 (0)