Skip to content

Commit 083244a

Browse files
bshastryclaude
andauthored
feat(cli): Blocktest builder (#2190)
* Initial commit * Some refactoring * Add batch mode * Support blob gas fields in genesis * Don't default to gasPrice=1 to not error on blob tx * Fix setcodetx translation * Parallelize * Per file progress meter * Fix auth translation * refactor(fuzzer-bridge): address PR feedback and improve architecture - Remove incorrect venv activation instructions from CLAUDE.md - Replace ASCII art with Mermaid diagram in README - Add fuzzer_bridge as CLI entry point in pyproject.toml - Create comprehensive documentation in docs/writing_tests/ - Add Pydantic models for type-safe fuzzer output parsing - Add BlockchainTest.from_fuzzer() classmethod for better integration - Simplify BlocktestBuilder to use new architecture - Fix README inconsistencies (punctuation, references, sections) This refactoring aligns the fuzzer bridge with EEST code standards and makes it more maintainable and future-proof. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * fix(fuzzer-bridge): apply linting and formatting fixes - Add ruff noqa comment for mixedCase variables in Pydantic models - Fix line length issues in performance_utils.py - Replace bare except with specific Exception handling - Apply automated formatting from ruff 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * fix: resolve CI errors for linting, formatting, and type checking - Fix markdown linting errors in fuzzer_bridge.md documentation - Apply ruff formatting to blockchain.py - Fix mypy type errors in fuzzer bridge modules: - Make orjson import optional with proper error handling - Fix TransitionTool instantiation to use GethTransitionTool - Correct EOA usage and add missing class attributes - Fix json.dump parameter usage to avoid kwargs issues - Fix docstring line lengths to comply with 79-char limit - Fix line length issues in function signatures All linting (ruff) and type checking (mypy) now pass successfully. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Fix docs link * feat(fuzzer_bridge): add EIP-7702 and EIP-4788 support Add support for two critical Ethereum features in the fuzzer bridge: 1. EIP-7702 Authorization Lists (Prague+) - Add FuzzerAuthorization model for authorization tuples - Parse authorizationList from transactions - Translate to AuthorizationTuple in from_fuzzer() - Enables testing of SetCode transactions (tx type 0x04) 2. EIP-4788 Parent Beacon Block Root (Cancun+) - Add parentBeaconBlockRoot as top-level field in FuzzerOutput - Pass to Block during creation in from_fuzzer() - Must be 32-byte hash from consensus layer - Enables testing of beacon root contract (EIP-4788) These changes close significant coverage gaps in geth core validation: - tx_setcode.go: Authorization validation paths - block_validator.go: Beacon root handling - state_transition.go: Beacon root system contract calls The implementation follows existing patterns (similar to blob sidecar handling) and maintains backwards compatibility through optional fields. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat(fuzzer-bridge): add multi-block blockchain test generation Add support for splitting fuzzer transactions across multiple blocks to enable testing of state transitions and block boundaries. New CLI Options: - --num-blocks: Number of blocks to generate (default: 1) - --block-strategy: Transaction distribution strategy - "distribute": Sequential chunks preserving nonce order - "first-block": All txs in first block, rest empty - --block-time: Seconds between blocks (default: 12) Implementation: - Sequential distribution maintains nonce ordering per account - Timestamps increment by block_time for each block - Only first block receives parent_beacon_block_root - Fully backward compatible (single-block default) Example Usage: fuzzer_bridge --num-blocks 3 --block-strategy distribute \ --fork Osaka input.json output/ This enables testing multi-block scenarios like: - State evolution across blocks - Transaction dependencies spanning blocks - Block time-sensitive operations - Cross-block state transitions 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat(fuzzer-bridge): add random block count selection Add --random-blocks flag to enable intelligent automatic selection of block counts for comprehensive testing coverage. New Feature: - --random-blocks: Randomly choose number of blocks (1 to min(num_txs, 10)) Implementation: - choose_random_num_blocks() helper with uniform distribution - Each file gets independent random selection in parallel mode - Edge case handling: empty blocks (0 txs → 1 block) - Cap at 10 blocks to prevent fixture bloat with large tx counts Algorithm Rationale: - Uniform distribution provides equal coverage for testing - Max of 10 blocks balances thoroughness with practicality - Independent per-file randomization maximizes corpus diversity Example Usage: # Random mode - each file gets random block count fuzzer_bridge --random-blocks --fork Osaka input/ output/ # Still works with fixed mode fuzzer_bridge --num-blocks 3 --fork Osaka input/ output/ Benefits: - Automated testing of various block configurations - Discovers edge cases in block boundary handling - Comprehensive multi-block scenario coverage - Zero configuration needed for random testing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * Add batch mode for improved perf * test(fuzzer-bridge): add comprehensive test suite with DTO pattern Implements complete testing infrastructure for fuzzer bridge with 25 comprehensive tests achieving 98% code coverage on core modules. Changes: - Add test_fuzzer_bridge.py with 25 tests covering DTO parsing, conversion, BlockchainTest generation, EIP features, and error handling - Refactor models.py to use DTO pattern with clean separation between external JSON-RPC format and internal EEST domain models - Create converter.py module with pure transformation functions (eliminates circular dependencies from blockchain.py) - Add 5 test vectors from real fuzzer outputs (Osaka fork, EIP-7702, EIP-4788) - Update README with comprehensive DTO architecture documentation including Mermaid diagram, field mappings, and design rationale - Simplify blocktest_builder.py by delegating conversion to converter module - Update examples with current fuzzer outputs and detailed README Architecture: - DTOs (models.py): Parse external format without side effects - Converters (converter.py): Explicit field transformations - Domain Models (EEST): Internal business logic with validation - Benefits: 98% test coverage, no circular dependencies, explicit mappings, prevents TestAddress pollution Test Coverage: - 25 tests pass in 0.46s - models.py: 100% coverage - converter.py: 97% coverage - Core modules combined: 98% coverage Key Features Tested: - EIP-7702 authorization lists - EIP-4788 parent beacon block root - Multi-block generation strategies - EOA creation from private keys - Field mapping (gas → gas_limit, from → sender) - Error handling and validation All quality checks pass: - Linting: ✓ (ruff check, format, W505) - Type checking: ✓ (mypy) - All CLI tests: ✓ (256/256 pass, no regressions) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> --------- Co-authored-by: Claude <[email protected]>
1 parent 7f32108 commit 083244a

20 files changed

+7166
-0
lines changed

cli/fuzzer_bridge/README.md

Lines changed: 301 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,301 @@
1+
# Fuzzer Bridge for Execution Spec Tests
2+
3+
This module provides a bridge between blocktest fuzzers (like `blocktest-fuzzer`) and the Ethereum execution-spec-tests framework, enabling automatic generation of valid blockchain test fixtures from fuzzer output.
4+
5+
## Overview
6+
7+
The fuzzer bridge solves a critical problem: fuzzers can generate transactions and pre-state, but creating valid blockchain tests requires complex calculations (state roots, RLP encoding, block headers, etc.). This bridge leverages the execution-spec-tests framework to handle all these complexities.
8+
9+
## Architecture
10+
11+
```mermaid
12+
graph LR
13+
A[Blocktest<br/>Fuzzer] -->|JSON<br/>v2 format| B[Fuzzer<br/>Bridge]
14+
B -->|Blockchain Test<br/>Fixtures| C[Ethereum<br/>Clients]
15+
```
16+
17+
## Fuzzer Output Format (v2)
18+
19+
The fuzzer must output JSON in the following format:
20+
21+
```json
22+
{
23+
"version": "2.0",
24+
"fork": "Prague",
25+
"chainId": 1,
26+
"accounts": {
27+
"0x7e5f4552091a69125d5dfcb7b8c2659029395bdf": {
28+
"balance": "0x1000000000000000000",
29+
"nonce": "0x0",
30+
"code": "",
31+
"storage": {},
32+
"privateKey": "0x0000000000000000000000000000000000000000000000000000000000000001"
33+
},
34+
"0x2b5ad5c4795c026514f8317c7a215e218dccd6cf": {
35+
"balance": "0x100000000000000000",
36+
"nonce": "0x0",
37+
"code": "0x600160005260206000f3",
38+
"storage": {}
39+
}
40+
},
41+
"transactions": [
42+
{
43+
"from": "0x7e5f4552091a69125d5dfcb7b8c2659029395bdf",
44+
"to": "0x2b5ad5c4795c026514f8317c7a215e218dccd6cf",
45+
"value": "0x100",
46+
"gas": "0x5208",
47+
"gasPrice": "0x7",
48+
"nonce": "0x0",
49+
"data": "0x"
50+
}
51+
],
52+
"env": {
53+
"currentCoinbase": "0xc014ba5e00000000000000000000000000000000",
54+
"currentDifficulty": "0x0",
55+
"currentGasLimit": "0x1000000",
56+
"currentNumber": "0x1",
57+
"currentTimestamp": "0x1000",
58+
"currentBaseFee": "0x7",
59+
"currentRandom": "0x0000000000000000000000000000000000000000000000000000000000000000"
60+
}
61+
}
62+
```
63+
64+
### Key Requirements
65+
66+
1. **Private Keys**: Any account that sends transactions MUST include a `privateKey` field.
67+
2. **Address-Key Match**: Private keys must generate the corresponding addresses.
68+
3. **Environment**: Describes the environment for block 1 (genesis is automatically derived).
69+
4. **Version**: Use "2.0" for this format.
70+
71+
## Architecture: DTO Pattern
72+
73+
The fuzzer bridge uses a **Data Transfer Object (DTO) pattern** for clean separation between external data format and internal domain logic.
74+
75+
### Design Flow
76+
77+
```mermaid
78+
graph TD
79+
A["Fuzzer JSON Output<br/>(JSON-RPC Format v2.0)"]
80+
81+
B["DTOs (Pydantic Models)<br/>models.py"]
82+
B1["FuzzerAccountInput<br/>Raw account JSON"]
83+
B2["FuzzerTransactionInput<br/>Raw transaction JSON (uses 'gas')"]
84+
B3["FuzzerAuthorizationInput<br/>Raw auth tuple (EIP-7702)"]
85+
B4["FuzzerOutput<br/>Complete fuzzer output"]
86+
87+
C["EEST Domain Models<br/>converter.py"]
88+
C1["Account<br/>With validation & defaults"]
89+
C2["Transaction<br/>With gas_limit, EOA sender"]
90+
C3["AuthorizationTuple<br/>EIP-7702 support"]
91+
C4["EOA<br/>Created from private keys"]
92+
93+
D["BlockchainTest"]
94+
E["Blockchain Fixture"]
95+
96+
A -->|parse| B
97+
B -.-> B1
98+
B -.-> B2
99+
B -.-> B3
100+
B -.-> B4
101+
102+
B -->|convert| C
103+
C -.-> C1
104+
C -.-> C2
105+
C -.-> C3
106+
C -.-> C4
107+
108+
C -->|generate| D
109+
D --> E
110+
111+
style A fill:#e1f5ff
112+
style B fill:#fff4e1
113+
style C fill:#e8f5e9
114+
style D fill:#f3e5f5
115+
style E fill:#fce4ec
116+
```
117+
118+
### Why DTOs?
119+
120+
1. **Separation of Concerns**
121+
- External JSON-RPC format ≠ EEST internal representation
122+
- Fuzzer format can change without affecting EEST domain models
123+
124+
2. **No Side Effects During Parsing**
125+
- DTOs don't trigger `model_post_init` validation logic
126+
- Parsing is purely data extraction, no business logic
127+
128+
3. **Explicit Field Mapping**
129+
- Clear visibility: `gas``gas_limit`, `from``sender` (EOA)
130+
- Type-safe conversions at boundary
131+
132+
4. **Prevents TestAddress Pollution**
133+
- EOA created in converter BEFORE Transaction instantiation
134+
- Transaction.model_post_init never injects TestAddress
135+
136+
### Key Field Mappings
137+
138+
| Fuzzer Field (JSON-RPC) | DTO Field | EEST Domain Field | Notes |
139+
|-------------------------|----------------------|----------------------|--------------------------------|
140+
| `from` | `from_` | `sender` (EOA) | Creates EOA from private key |
141+
| `gas` | `gas` | `gas_limit` | JSON-RPC vs internal naming |
142+
| `data` | `data` | `data` | Same field, explicit mapping |
143+
| `gasPrice` | `gas_price` | `gas_price` | CamelCase → snake_case |
144+
| `authorizationList` | `authorization_list` | `authorization_list` | EIP-7702 support |
145+
| `privateKey` | `private_key` | (used to create EOA) | Not stored in Account model |
146+
147+
### Module Responsibilities
148+
149+
#### `models.py` - Data Transfer Objects
150+
- Pure Pydantic models for fuzzer JSON format
151+
- No business logic, only data validation
152+
- Accepts JSON-RPC naming conventions (camelCase)
153+
- ~119 lines
154+
155+
#### `converter.py` - Transformation Logic
156+
- Pure functions: DTO → EEST domain models
157+
- All field mapping logic centralized here
158+
- Creates EOA objects from private keys
159+
- Builds BlockchainTest from validated data
160+
- ~305 lines
161+
162+
#### `blocktest_builder.py` - CLI Integration
163+
- Orchestrates conversion workflow
164+
- Handles file I/O and CLI options
165+
- Calls converter functions
166+
- ~90 lines
167+
168+
### Benefits
169+
170+
**Maintainability**: Changes to Account/Transaction propagate automatically
171+
**Testability**: Each layer tested independently
172+
**Type Safety**: Full type checking at DTO and domain layers
173+
**Clarity**: Field mappings are explicit and documented
174+
**No Circular Dependencies**: Clean module boundaries
175+
176+
### Alternative Design: Why Not Inheritance?
177+
178+
**Could have done**:
179+
```python
180+
class FuzzerAccount(Account):
181+
private_key: Hash | None = None
182+
```
183+
184+
**Why DTOs are better**:
185+
- Inheritance couples external format to domain model
186+
- model_post_init triggers during parsing (side effects)
187+
- Field name mismatches require complex aliasing
188+
- Harder to test layers independently
189+
190+
The DTO pattern provides cleaner separation and explicit control.
191+
192+
## Installation
193+
194+
See the [EEST installation guide](https://eest.ethereum.org/main/getting_started/installation/) for setting up the execution-spec-tests framework.
195+
196+
Once EEST is installed, the fuzzer bridge will be available as a command-line tool.
197+
198+
## Usage
199+
200+
### 1. Command Line Interface
201+
202+
```bash
203+
# Convert fuzzer output to blockchain test
204+
uv run fuzzer_bridge --input fuzzer_output.json --output blocktest.json
205+
206+
# With custom fork
207+
uv run fuzzer_bridge --input fuzzer_output.json --output blocktest.json --fork Shanghai
208+
209+
# Pretty print output
210+
uv run fuzzer_bridge --input fuzzer_output.json --output blocktest.json --pretty
211+
```
212+
213+
### 2. Python API
214+
215+
```python
216+
from fuzzer_bridge import FuzzerBridge
217+
218+
# Load fuzzer output
219+
with open("fuzzer_output.json") as f:
220+
fuzzer_data = json.load(f)
221+
222+
# Create bridge and convert
223+
bridge = FuzzerBridge()
224+
blocktest = bridge.convert(fuzzer_data)
225+
226+
# Save to file
227+
bridge.save(blocktest, "output.json")
228+
229+
# Or verify with geth directly
230+
result = bridge.verify_with_geth(blocktest, geth_path="../go-ethereum/build/bin/evm")
231+
print(f"Test passed: {result['pass']}")
232+
```
233+
234+
### 3. Integration with pytest
235+
236+
```python
237+
import pytest
238+
from fuzzer_bridge import create_test_from_fuzzer
239+
240+
def test_fuzzer_generated(blockchain_test):
241+
"""Test generated from fuzzer output."""
242+
test = create_test_from_fuzzer("fuzzer_output.json")
243+
blockchain_test(**test)
244+
```
245+
246+
## Key Insights
247+
248+
### Genesis Block Handling
249+
- The fuzzer describes the environment for block 1 (the block being tested)
250+
- Genesis (block 0) environment is automatically derived:
251+
- `number` = 0
252+
- `timestamp` = block1_timestamp - 12
253+
- Other values inherited or set to defaults
254+
255+
### System Contracts
256+
- The framework automatically adds system contracts (deposit, withdrawal, etc.)
257+
- These are included in the state root calculation
258+
- The fuzzer doesn't need to specify them
259+
260+
### Private Key Requirements
261+
- Every account that sends transactions needs a private key
262+
- The private key must generate the exact address specified
263+
- Without matching private keys, transactions cannot be signed
264+
265+
## Troubleshooting
266+
267+
### "Genesis block hash doesn't match"
268+
**Cause**: Usually means the environment is set incorrectly (e.g., block number = 1 instead of 0 for genesis)
269+
**Solution**: Ensure the fuzzer output follows the v2 format exactly
270+
271+
### "No private key for sender"
272+
**Cause**: Account sends transaction but no privateKey field provided
273+
**Solution**: Add privateKey to the account in the accounts section
274+
275+
### "Private key doesn't match address"
276+
**Cause**: The provided private key doesn't generate the specified address
277+
**Solution**: Use correct private key or generate address from private key
278+
279+
## Testing with Ethereum Clients
280+
281+
### Go-Ethereum (geth)
282+
```bash
283+
../go-ethereum/build/bin/evm blocktest generated_test.json
284+
```
285+
286+
### Besu
287+
```bash
288+
../besu/ethereum/evmtool/build/install/evmtool/bin/evmtool block-test generated_test.json
289+
```
290+
291+
### Nethermind
292+
```bash
293+
../nethermind/src/Nethermind/artifacts/bin/Nethermind.Test.Runner/release_linux-x64/nethtest -b -i generated_test.json
294+
```
295+
296+
## Contributing
297+
298+
When modifying the fuzzer bridge:
299+
1. Add tests for new features.
300+
2. Update this README.
301+
3. Ensure compatibility with latest execution-spec-tests.

cli/fuzzer_bridge/__init__.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
"""Fuzzer bridge for converting blocktest-fuzzer output to blocktests."""
2+
3+
from .blocktest_builder import BlocktestBuilder, build_blocktest_from_fuzzer
4+
5+
__all__ = ["BlocktestBuilder", "build_blocktest_from_fuzzer"]
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
"""Build valid blocktests from fuzzer-generated transactions and pre-state."""
2+
3+
import json
4+
import random
5+
from pathlib import Path
6+
from typing import Any, Dict, Optional
7+
8+
from ethereum_clis import GethTransitionTool, TransitionTool
9+
from ethereum_test_fixtures import BlockchainFixture
10+
11+
from .converter import blockchain_test_from_fuzzer
12+
from .models import FuzzerOutput
13+
14+
15+
def choose_random_num_blocks(num_txs: int, max_blocks: int = 10) -> int:
16+
"""
17+
Choose random number of blocks for given transaction count.
18+
19+
Selects a random number between 1 and min(num_txs, max_blocks) to enable
20+
testing of various block configurations.
21+
22+
Args:
23+
num_txs: Number of transactions to distribute
24+
max_blocks: Maximum number of blocks (default: 10)
25+
26+
Returns:
27+
Random integer between 1 and min(num_txs, max_blocks)
28+
29+
"""
30+
if num_txs == 0:
31+
return 1 # Allow empty block testing
32+
return random.randint(1, min(num_txs, max_blocks))
33+
34+
35+
class BlocktestBuilder:
36+
"""Build valid blocktests from fuzzer-generated transactions."""
37+
38+
def __init__(self, transition_tool: Optional[TransitionTool] = None):
39+
"""Initialize the builder with optional transition tool."""
40+
self.t8n = transition_tool or GethTransitionTool()
41+
42+
def build_blocktest(
43+
self,
44+
fuzzer_output: Dict[str, Any],
45+
num_blocks: int = 1,
46+
block_strategy: str = "distribute",
47+
block_time: int = 12,
48+
) -> Dict[str, Any]:
49+
"""Build a valid blocktest from fuzzer output."""
50+
# Parse and validate using Pydantic model
51+
fuzzer_data = FuzzerOutput(**fuzzer_output)
52+
53+
# Get fork
54+
fork = fuzzer_data.fork
55+
56+
# Create BlockchainTest using converter
57+
test = blockchain_test_from_fuzzer(
58+
fuzzer_data,
59+
fork,
60+
num_blocks=num_blocks,
61+
block_strategy=block_strategy,
62+
block_time=block_time,
63+
)
64+
65+
# Generate fixture
66+
fixture = test.generate(
67+
t8n=self.t8n,
68+
fork=fork,
69+
fixture_format=BlockchainFixture,
70+
)
71+
72+
return fixture.model_dump(exclude_none=True, by_alias=True, mode="json")
73+
74+
def build_and_save(self, fuzzer_output: Dict[str, Any], output_path: Path) -> Path:
75+
"""Build blocktest and save to file."""
76+
blocktest = self.build_blocktest(fuzzer_output)
77+
fixtures = {"fuzzer_generated_test": blocktest}
78+
79+
with open(output_path, "w") as f:
80+
json.dump(fixtures, f, indent=2)
81+
82+
return output_path
83+
84+
85+
def build_blocktest_from_fuzzer(
86+
fuzzer_data: Dict[str, Any], t8n: Optional[TransitionTool] = None
87+
) -> Dict[str, Any]:
88+
"""Build blocktest from fuzzer output."""
89+
builder = BlocktestBuilder(t8n)
90+
return builder.build_blocktest(fuzzer_data)

0 commit comments

Comments
 (0)