Skip to content

Commit 8ad785b

Browse files
fselmofselmo
committed
chore(docs): Document consume sync and add entry to CHANGELOG.md
1 parent 919fd3d commit 8ad785b

File tree

5 files changed

+262
-3
lines changed

5 files changed

+262
-3
lines changed

‎docs/CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@ Users can select any of the artifacts depending on their testing needs for their
8080
- ✨ Generate unique addresses with Python for compatible static tests, instead of using hard-coded addresses from legacy static test fillers ([#1781](https://github.com/ethereum/execution-spec-tests/pull/1781)).
8181
- ✨ Added support for the `--benchmark-gas-values` flag in the `fill` command, allowing a single genesis file to be used across different gas limit settings when generating fixtures. ([#1895](https://github.com/ethereum/execution-spec-tests/pull/1895)).
8282
- ✨ Static tests can now specify a maximum fork where they should be filled for ([#1977](https://github.com/ethereum/execution-spec-tests/pull/1977)).
83+
- ✨ Add support for `BlockchainEngineSyncFixture` format for tests with `verify_sync=True` to enable client synchronization testing via `consume sync` command ([#2007](https://github.com/ethereum/execution-spec-tests/pull/2007)).
8384

8485
#### `consume`
8586

@@ -88,6 +89,7 @@ Users can select any of the artifacts depending on their testing needs for their
8889
- 🔀 `consume` now automatically avoids GitHub API calls when using direct release URLs (better for CI environments), while release specifiers like `stable@latest` continue to use the API for version resolution ([#1788](https://github.com/ethereum/execution-spec-tests/pull/1788)).
8990
- 🔀 Refactor consume simulator architecture to use explicit pytest plugin structure with forward-looking architecture ([#1801](https://github.com/ethereum/execution-spec-tests/pull/1801)).
9091
- 🔀 Add exponential retry logic to initial fcu within consume engine ([#1815](https://github.com/ethereum/execution-spec-tests/pull/1815)).
92+
- ✨ Add `consume sync` command to test client synchronization capabilities by having one client sync from another via Engine API and P2P networking ([#2007](https://github.com/ethereum/execution-spec-tests/pull/2007)).
9193

9294
#### `execute`
9395

‎docs/running_tests/running.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ Both `consume` and `execute` provide sub-commands which correspond to different
1414
| [`consume direct`](#direct) | Client consume tests via a `statetest` interface | EVM | None | Module test |
1515
| [`consume direct`](#direct) | Client consume tests via a `blocktest` interface | EVM, block processing | None | Module test,</br>Integration test |
1616
| [`consume engine`](#engine) | Client imports blocks via Engine API `EngineNewPayload` in Hive | EVM, block processing, Engine API | Staging, Hive | System test |
17+
| [`consume sync`](#sync) | Client syncs from another client using Engine API in Hive | EVM, block processing, Engine API, P2P sync | Staging, Hive | System test |
1718
| [`consume rlp`](#rlp) | Client imports RLP-encoded blocks upon start-up in Hive | EVM, block processing, RLP import (sync\*) | Staging, Hive | System test |
1819
| [`execute hive`](./execute/hive.md) | Tests executed against a client via JSON RPC `eth_sendRawTransaction` in Hive | EVM, JSON RPC, mempool | Staging, Hive | System test |
1920
| [`execute remote`](./execute/remote.md) | Tests executed against a client via JSON RPC `eth_sendRawTransaction` on a live network | EVM, JSON RPC, mempool, EL-EL/EL-CL interaction (indirectly) | Production | System Test |
@@ -81,6 +82,25 @@ The `consume rlp` command:
8182

8283
This method simulates how clients import blocks during historical sync, testing the complete block validation and state transition pipeline, see below for more details and a comparison to consumption via the Engine API.
8384

85+
## Sync
86+
87+
| Nomenclature | |
88+
| -------------- |------------------------|
89+
| Command | `consume sync` |
90+
| Simulator | None |
91+
| Fixture format | `blockchain_test_sync` |
92+
93+
The consume sync method tests execution client synchronization capabilities by having one client sync from another via the Engine API and P2P networking. This method validates that clients can correctly synchronize state and blocks from peers, testing both the Engine API, sync triggering, and P2P block propagation mechanisms.
94+
95+
The `consume sync` command:
96+
97+
1. **Initializes the client under test** with genesis state and executes all test payloads.
98+
2. **Spins up a sync client** with the same genesis state.
99+
3. **Establishes P2P connection** between the two clients, utilizing ``admin_addPeer`` with enode url.
100+
4. **Triggers synchronization** by sending the target block to the sync client via `engine_newPayload` followed by `engine_forkchoiceUpdated` requests.
101+
5. **Monitors sync progress** and validates that the sync client reaches the same state.
102+
6. **Verifies final state** matches between both clients.
103+
84104
## Engine vs RLP Simulator
85105

86106
The RLP Simulator (`eest/consume-rlp`) and the Engine Simulator (`eest/consume-engine`) should be seen as complimentary to one another. Although they execute the same underlying EVM test cases, the block validation logic is executed via different client code paths (using different [fixture formats](./test_formats/index.md)). Therefore, ideally, **both simulators should be executed for full coverage**.

‎docs/running_tests/test_formats/blockchain_test_sync.md

Lines changed: 238 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,238 @@
1+
# Blockchain Engine Sync Tests <!-- markdownlint-disable MD051 (MD051=link-fragments "Link fragments should be valid") -->
2+
3+
The Blockchain Engine Sync Test fixture format tests are included in the fixtures subdirectory `blockchain_tests_sync`, and use Engine API directives to test client synchronization capabilities after fixtures are executed with valid payloads.
4+
5+
These are produced by the `BlockchainTest` test spec when `verify_sync=True` is specified.
6+
7+
## Description
8+
9+
The Blockchain Engine Sync Test fixture format is used to test execution client synchronization between peers. It validates that clients can correctly sync state and blocks from another client using the Engine API and P2P networking.
10+
11+
The test works by:
12+
1. Setting up a client under test, defining a pre-execution state, a series of `engine_newPayloadVX` directives, and a post-execution state, as in Blockchain Engine Test fixture formats.
13+
2. Starting a sync client with the same genesis and pre-execution state.
14+
3. Having the sync client synchronize from the client under test.
15+
4. Verifying that both clients reach the same final state.
16+
17+
A single JSON fixture file is composed of a JSON object where each key-value pair is a different [`SyncFixture`](#syncfixture) test object, with the key string representing the test name.
18+
19+
The JSON file path plus the test name are used as the unique test identifier, as well as a `{client under test}::sync_{sync client}` identifier.
20+
21+
## Consumption
22+
23+
For each [`HiveFixture`](#hivefixture) test object in the JSON fixture file, perform the following steps:
24+
25+
### Client Under Test Setup
26+
27+
1. Start the client under test using:
28+
- [`network`](#-network-fork) to configure the execution fork schedule according to the [`Fork`](./common_types.md#fork) type definition.
29+
- [`pre`](#-pre-alloc) as the starting state allocation of the execution environment for the test and calculate the genesis state root.
30+
- [`genesisBlockHeader`](#-genesisblockheader-fixtureheader) as the genesis block header.
31+
32+
2. Verify the head of the chain is the genesis block, and the state root matches the one calculated on step 1, otherwise fail the test.
33+
34+
3. Process all [`FixtureEngineNewPayload`](#fixtureenginenewpayload) objects in [`engineNewPayloads`](#-enginenewpayloads-listfixtureenginenewpayload) to build the complete chain on the client under test.
35+
36+
### Sync Client Setup and Synchronization
37+
38+
4. Start a sync client using the same genesis configuration:
39+
- Use the same [`network`](#-network-fork), [`pre`](#-pre-alloc), and [`genesisBlockHeader`](#-genesisblockheader-fixtureheader).
40+
41+
5. Establish P2P connection between the clients:
42+
- Get the enode URL from the client under test
43+
- Use `admin_addPeer` to connect the sync client to the client under test
44+
45+
6. Trigger synchronization on the sync client:
46+
- Send the [`syncPayload`](#-syncpayload-fixtureenginenewpayload) using `engine_newPayloadVX`
47+
- Send `engine_forkchoiceUpdatedVX` pointing to the last block hash
48+
49+
7. Monitor and verify synchronization:
50+
- Wait for the sync client to reach the [`lastblockhash`](#-lastblockhash-hash)
51+
- Verify the final state root matches between both clients
52+
- If [`post`](#-post-alloc) is provided, verify the final state matches
53+
54+
## Structures
55+
56+
### `HiveFixture`
57+
58+
#### - `network`: [`Fork`](./common_types.md#fork)
59+
60+
##### TO BE DEPRECATED
61+
62+
Fork configuration for the test.
63+
64+
This field is going to be replaced by the value contained in `config.network`.
65+
66+
#### - `genesisBlockHeader`: [`FixtureHeader`](./blockchain_test.md#fixtureheader)
67+
68+
Genesis block header.
69+
70+
#### - `engineNewPayloads`: [`List`](./common_types.md#list)`[`[`FixtureEngineNewPayload`](#fixtureenginenewpayload)`]`
71+
72+
List of `engine_newPayloadVX` directives to be processed by the client under test to build the complete chain.
73+
74+
#### - `syncPayload`: [`FixtureEngineNewPayload`](#fixtureenginenewpayload)
75+
76+
The final payload to be sent to the sync client to trigger synchronization. This is typically an empty block built on top of the last test block.
77+
78+
#### - `engineFcuVersion`: [`Number`](./common_types.md#number)
79+
80+
Version of the `engine_forkchoiceUpdatedVX` directive to use to set the head of the chain.
81+
82+
#### - `pre`: [`Alloc`](./common_types.md#alloc-mappingaddressaccount)
83+
84+
Starting account allocation for the test. State root calculated from this allocation must match the one in the genesis block.
85+
86+
#### - `lastblockhash`: [`Hash`](./common_types.md#hash)
87+
88+
Hash of the last valid block that the sync client should reach after successful synchronization.
89+
90+
#### - `post`: [`Alloc`](./common_types.md#alloc-mappingaddressaccount)
91+
92+
Account allocation for verification after synchronization is complete.
93+
94+
#### - `postStateHash`: [`Optional`](./common_types.md#optional)`[`[`Hash`](./common_types.md#hash)`]`
95+
96+
Optional state root hash for verification after synchronization is complete. Used when full post-state is not included.
97+
98+
#### - `config`: [`FixtureConfig`](#fixtureconfig)
99+
100+
Chain configuration object to be applied to both clients running the blockchain sync test.
101+
102+
### `FixtureConfig`
103+
104+
#### - `network`: [`Fork`](./common_types.md#fork)
105+
106+
Fork configuration for the test. It is guaranteed that this field contains the same value as the root field `network`.
107+
108+
#### - `chainId`: [`Number`](./common_types.md#number)
109+
110+
Chain ID configuration for the test network.
111+
112+
#### - `blobSchedule`: [`BlobSchedule`](./common_types.md#blobschedule-mappingforkforkblobschedule)
113+
114+
Optional; present from Cancun on. Maps forks to their blob schedule configurations as defined by [EIP-7840](https://eips.ethereum.org/EIPS/eip-7840).
115+
116+
### `FixtureEngineNewPayload`
117+
118+
#### - `executionPayload`: [`FixtureExecutionPayload`](#fixtureexecutionpayload)
119+
120+
Execution payload.
121+
122+
#### - `blob_versioned_hashes`: [`Optional`](./common_types.md#optional)`[`[`List`](./common_types.md#list)`[`[`Hash`](./common_types.md#hash)`]]` `(fork: Cancun)`
123+
124+
List of hashes of the versioned blobs that are part of the execution payload.
125+
126+
#### - `parentBeaconBlockRoot`: [`Optional`](./common_types.md#optional)`[`[`Hash`](./common_types.md#hash)`]` `(fork: Cancun)`
127+
128+
Hash of the parent beacon block root.
129+
130+
#### - `validationError`: [`Optional`](./common_types.md#optional)`[`[`TransactionException`](../../library/ethereum_test_exceptions.md#ethereum_test_exceptions.TransactionException)` | `[`BlockException`](../../library/ethereum_test_exceptions.md#ethereum_test_exceptions.BlockException)`]`
131+
132+
For sync tests, this field should not be present as sync tests only work with valid chains. Invalid blocks cannot be synced.
133+
134+
#### - `version`: [`Number`](./common_types.md#number)
135+
136+
Version of the `engine_newPayloadVX` directive to use to deliver the payload.
137+
138+
### `FixtureExecutionPayload`
139+
140+
#### - `parentHash`: [`Hash`](./common_types.md#hash)
141+
142+
Hash of the parent block.
143+
144+
#### - `feeRecipient`: [`Address`](./common_types.md#address)
145+
146+
Address of the account that will receive the rewards for building the block.
147+
148+
#### - `stateRoot`: [`Hash`](./common_types.md#hash)
149+
150+
Root hash of the state trie.
151+
152+
#### - `receiptsRoot`: [`Hash`](./common_types.md#hash)
153+
154+
Root hash of the receipts trie.
155+
156+
#### - `logsBloom`: [`Bloom`](./common_types.md#bloom)
157+
158+
Bloom filter composed of the logs of all the transactions in the block.
159+
160+
#### - `blockNumber`: [`HexNumber`](./common_types.md#hexnumber)
161+
162+
Number of the block.
163+
164+
#### - `gasLimit`: [`HexNumber`](./common_types.md#hexnumber)
165+
166+
Total gas limit of the block.
167+
168+
#### - `gasUsed`: [`HexNumber`](./common_types.md#hexnumber)
169+
170+
Total gas used by all the transactions in the block.
171+
172+
#### - `timestamp`: [`HexNumber`](./common_types.md#hexnumber)
173+
174+
Timestamp of the block.
175+
176+
#### - `extraData`: [`Bytes`](./common_types.md#bytes)
177+
178+
Extra data of the block.
179+
180+
#### - `prevRandao`: [`Hash`](./common_types.md#hash)
181+
182+
PrevRandao of the block.
183+
184+
#### - `blockHash`: [`Hash`](./common_types.md#hash)
185+
186+
Hash of the block.
187+
188+
#### - `transactions`: [`List`](./common_types.md#list)`[`[`Bytes`](./common_types.md#bytes)`]`
189+
190+
List of transactions in the block, in serialized format.
191+
192+
#### - `withdrawals`: [`List`](./common_types.md#list)`[`[`FixtureWithdrawal`](#fixturewithdrawal)`]`
193+
194+
List of withdrawals in the block.
195+
196+
#### - `baseFeePerGas`: [`HexNumber`](./common_types.md#hexnumber) `(fork: London)`
197+
198+
Base fee per gas of the block.
199+
200+
#### - `blobGasUsed`: [`HexNumber`](./common_types.md#hexnumber) `(fork: Cancun)`
201+
202+
Total blob gas used by all the transactions in the block.
203+
204+
#### - `excessBlobGas`: [`HexNumber`](./common_types.md#hexnumber) `(fork: Cancun)`
205+
206+
Excess blob gas of the block used to calculate the blob fee per gas for this block.
207+
208+
### `FixtureWithdrawal`
209+
210+
#### - `index`: [`HexNumber`](./common_types.md#hexnumber)
211+
212+
Index of the withdrawal
213+
214+
#### - `validatorIndex`: [`HexNumber`](./common_types.md#hexnumber)
215+
216+
Withdrawing validator index
217+
218+
#### - `address`: [`Address`](./common_types.md#address)
219+
220+
Address to withdraw to
221+
222+
#### - `amount`: [`HexNumber`](./common_types.md#hexnumber)
223+
224+
Amount of the withdrawal
225+
226+
## Differences from Blockchain Engine Tests
227+
228+
While the Blockchain Sync Test format is similar to the Blockchain Engine Test format, there are key differences:
229+
230+
1. **`syncPayload` field**: Contains the final block used to trigger synchronization on the sync client.
231+
2. **Multi-client testing**: Tests involve two clients (client under test and sync client) rather than a single client.
232+
3. **P2P networking**: Tests require P2P connection establishment between clients.
233+
4. **No invalid blocks**: Sync tests only work with valid chains as invalid blocks cannot be synced.
234+
5. **`postStateHash` field**: Optional field for state verification when full post-state is not included.
235+
236+
## Fork Support
237+
238+
Blockchain Sync Tests are only supported for post-merge forks (Paris and later) as they rely on the Engine API for synchronization triggering.

‎src/pytest_plugins/consume/simulators/base.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,7 @@ def check_live_port(test_suite_name: str) -> Literal[8545, 8551]:
2727
"""Port used by hive to check for liveness of the client."""
2828
if test_suite_name == "eest/consume-rlp":
2929
return 8545
30-
elif test_suite_name == "eest/consume-engine":
31-
return 8551
32-
elif test_suite_name == "eest/consume-sync":
30+
elif test_suite_name in {"eest/consume-engine", "eest/consume-sync"}:
3331
return 8551
3432
raise ValueError(
3533
f"Unexpected test suite name '{test_suite_name}' while setting HIVE_CHECK_LIVE_PORT."

‎src/pytest_plugins/consume/simulators/sync/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"""Consume Sync test functions."""

0 commit comments

Comments
 (0)