Skip to content

Commit 1138ec5

Browse files
bshastryclaude
andauthored
feat(cli): Blocktest builder (#2190)
* Initial commit * Some refactoring * Add batch mode * Support blob gas fields in genesis * Don't default to gasPrice=1 to not error on blob tx * Fix setcodetx translation * Parallelize * Per file progress meter * Fix auth translation * refactor(fuzzer-bridge): address PR feedback and improve architecture - Remove incorrect venv activation instructions from CLAUDE.md - Replace ASCII art with Mermaid diagram in README - Add fuzzer_bridge as CLI entry point in pyproject.toml - Create comprehensive documentation in docs/writing_tests/ - Add Pydantic models for type-safe fuzzer output parsing - Add BlockchainTest.from_fuzzer() classmethod for better integration - Simplify BlocktestBuilder to use new architecture - Fix README inconsistencies (punctuation, references, sections) This refactoring aligns the fuzzer bridge with EEST code standards and makes it more maintainable and future-proof. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * fix(fuzzer-bridge): apply linting and formatting fixes - Add ruff noqa comment for mixedCase variables in Pydantic models - Fix line length issues in performance_utils.py - Replace bare except with specific Exception handling - Apply automated formatting from ruff 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * fix: resolve CI errors for linting, formatting, and type checking - Fix markdown linting errors in fuzzer_bridge.md documentation - Apply ruff formatting to blockchain.py - Fix mypy type errors in fuzzer bridge modules: - Make orjson import optional with proper error handling - Fix TransitionTool instantiation to use GethTransitionTool - Correct EOA usage and add missing class attributes - Fix json.dump parameter usage to avoid kwargs issues - Fix docstring line lengths to comply with 79-char limit - Fix line length issues in function signatures All linting (ruff) and type checking (mypy) now pass successfully. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Fix docs link * feat(fuzzer_bridge): add EIP-7702 and EIP-4788 support Add support for two critical Ethereum features in the fuzzer bridge: 1. EIP-7702 Authorization Lists (Prague+) - Add FuzzerAuthorization model for authorization tuples - Parse authorizationList from transactions - Translate to AuthorizationTuple in from_fuzzer() - Enables testing of SetCode transactions (tx type 0x04) 2. EIP-4788 Parent Beacon Block Root (Cancun+) - Add parentBeaconBlockRoot as top-level field in FuzzerOutput - Pass to Block during creation in from_fuzzer() - Must be 32-byte hash from consensus layer - Enables testing of beacon root contract (EIP-4788) These changes close significant coverage gaps in geth core validation: - tx_setcode.go: Authorization validation paths - block_validator.go: Beacon root handling - state_transition.go: Beacon root system contract calls The implementation follows existing patterns (similar to blob sidecar handling) and maintains backwards compatibility through optional fields. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat(fuzzer-bridge): add multi-block blockchain test generation Add support for splitting fuzzer transactions across multiple blocks to enable testing of state transitions and block boundaries. New CLI Options: - --num-blocks: Number of blocks to generate (default: 1) - --block-strategy: Transaction distribution strategy - "distribute": Sequential chunks preserving nonce order - "first-block": All txs in first block, rest empty - --block-time: Seconds between blocks (default: 12) Implementation: - Sequential distribution maintains nonce ordering per account - Timestamps increment by block_time for each block - Only first block receives parent_beacon_block_root - Fully backward compatible (single-block default) Example Usage: fuzzer_bridge --num-blocks 3 --block-strategy distribute \ --fork Osaka input.json output/ This enables testing multi-block scenarios like: - State evolution across blocks - Transaction dependencies spanning blocks - Block time-sensitive operations - Cross-block state transitions 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat(fuzzer-bridge): add random block count selection Add --random-blocks flag to enable intelligent automatic selection of block counts for comprehensive testing coverage. New Feature: - --random-blocks: Randomly choose number of blocks (1 to min(num_txs, 10)) Implementation: - choose_random_num_blocks() helper with uniform distribution - Each file gets independent random selection in parallel mode - Edge case handling: empty blocks (0 txs → 1 block) - Cap at 10 blocks to prevent fixture bloat with large tx counts Algorithm Rationale: - Uniform distribution provides equal coverage for testing - Max of 10 blocks balances thoroughness with practicality - Independent per-file randomization maximizes corpus diversity Example Usage: # Random mode - each file gets random block count fuzzer_bridge --random-blocks --fork Osaka input/ output/ # Still works with fixed mode fuzzer_bridge --num-blocks 3 --fork Osaka input/ output/ Benefits: - Automated testing of various block configurations - Discovers edge cases in block boundary handling - Comprehensive multi-block scenario coverage - Zero configuration needed for random testing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * Add batch mode for improved perf * test(fuzzer-bridge): add comprehensive test suite with DTO pattern Implements complete testing infrastructure for fuzzer bridge with 25 comprehensive tests achieving 98% code coverage on core modules. Changes: - Add test_fuzzer_bridge.py with 25 tests covering DTO parsing, conversion, BlockchainTest generation, EIP features, and error handling - Refactor models.py to use DTO pattern with clean separation between external JSON-RPC format and internal EEST domain models - Create converter.py module with pure transformation functions (eliminates circular dependencies from blockchain.py) - Add 5 test vectors from real fuzzer outputs (Osaka fork, EIP-7702, EIP-4788) - Update README with comprehensive DTO architecture documentation including Mermaid diagram, field mappings, and design rationale - Simplify blocktest_builder.py by delegating conversion to converter module - Update examples with current fuzzer outputs and detailed README Architecture: - DTOs (models.py): Parse external format without side effects - Converters (converter.py): Explicit field transformations - Domain Models (EEST): Internal business logic with validation - Benefits: 98% test coverage, no circular dependencies, explicit mappings, prevents TestAddress pollution Test Coverage: - 25 tests pass in 0.46s - models.py: 100% coverage - converter.py: 97% coverage - Core modules combined: 98% coverage Key Features Tested: - EIP-7702 authorization lists - EIP-4788 parent beacon block root - Multi-block generation strategies - EOA creation from private keys - Field mapping (gas → gas_limit, from → sender) - Error handling and validation All quality checks pass: - Linting: ✓ (ruff check, format, W505) - Type checking: ✓ (mypy) - All CLI tests: ✓ (256/256 pass, no regressions) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> --------- Co-authored-by: Claude <[email protected]>
1 parent 18fc570 commit 1138ec5

24 files changed

+7432
-1
lines changed

CLAUDE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ Execute: Python Tests → execute → Live JSON-RPC Testing
2626

2727
## 🚀 Essential Commands
2828

29-
All commands use `uv run` prefix.
29+
All commands use `uv run` prefix. The `uv sync --all-extras` command (below) automatically creates and manages a virtual environment.
3030

3131
### Setup
3232

docs/navigation.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@
4141
* [Running Tests](running_tests/index.md)
4242
* [Methods of Running Tests](running_tests/running.md)
4343
* [EEST Fixture Releases](running_tests/releases.md)
44+
* [Fuzzer Bridge](writing_tests/fuzzer_bridge.md)
4445
* [Test Fixture Specifications](running_tests/test_formats/index.md)
4546
* [State Tests](running_tests/test_formats/state_test.md)
4647
* [Blockchain Tests](running_tests/test_formats/blockchain_test.md)
Lines changed: 263 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,263 @@
1+
# Fuzzer Bridge
2+
3+
The fuzzer bridge provides a seamless integration between blocktest fuzzers and the Ethereum execution-spec-tests framework, enabling automatic generation of valid blockchain test fixtures from fuzzer output.
4+
5+
## Overview
6+
7+
Fuzzers are excellent at generating test inputs to discover edge cases and bugs in Ethereum client implementations. However, creating valid blockchain tests from fuzzer-generated data requires complex calculations including:
8+
9+
- State root computations
10+
- RLP encoding of blocks and transactions
11+
- Proper block header generation
12+
- System contract interactions
13+
- Genesis block derivation
14+
15+
The fuzzer bridge handles all these complexities automatically by leveraging the execution-spec-tests framework.
16+
17+
## Architecture
18+
19+
```mermaid
20+
graph LR
21+
A[Blocktest<br/>Fuzzer] -->|JSON<br/>v2 format| B[Fuzzer<br/>Bridge]
22+
B -->|Blockchain Test<br/>Fixtures| C[Ethereum<br/>Clients]
23+
```
24+
25+
## Installation
26+
27+
The fuzzer bridge is included with the execution-spec-tests framework. Follow the [installation guide](../getting_started/installation.md) to set up EEST.
28+
29+
Once installed, the `fuzzer_bridge` command will be available through `uv run`.
30+
31+
## Fuzzer Output Format (v2)
32+
33+
The fuzzer must output JSON in the v2 format. Here's the structure:
34+
35+
```json
36+
{
37+
"version": "2.0",
38+
"fork": "Prague",
39+
"chainId": 1,
40+
"accounts": {
41+
"0x7e5f4552091a69125d5dfcb7b8c2659029395bdf": {
42+
"balance": "0x1000000000000000000",
43+
"nonce": "0x0",
44+
"code": "",
45+
"storage": {},
46+
"privateKey": "0x0000000000000000000000000000000000000000000000000000000000000001"
47+
}
48+
},
49+
"transactions": [
50+
{
51+
"from": "0x7e5f4552091a69125d5dfcb7b8c2659029395bdf",
52+
"to": "0x2b5ad5c4795c026514f8317c7a215e218dccd6cf",
53+
"value": "0x100",
54+
"gas": "0x5208",
55+
"gasPrice": "0x7",
56+
"nonce": "0x0",
57+
"data": "0x"
58+
}
59+
],
60+
"env": {
61+
"currentCoinbase": "0xc014ba5e00000000000000000000000000000000",
62+
"currentDifficulty": "0x0",
63+
"currentGasLimit": "0x1000000",
64+
"currentNumber": "0x1",
65+
"currentTimestamp": "0x1000",
66+
"currentBaseFee": "0x7",
67+
"currentRandom": "0x0000000000000000000000000000000000000000000000000000000000000000"
68+
}
69+
}
70+
```
71+
72+
### Key Requirements
73+
74+
1. **Private Keys**: Any account that sends transactions MUST include a `privateKey` field.
75+
2. **Address-Key Match**: Private keys must generate the corresponding addresses.
76+
3. **Environment**: Describes the environment for block 1 (genesis is automatically derived).
77+
4. **Version**: Must be "2.0" for this format.
78+
5. **Fork Name**: Use the standard fork name (e.g., "Prague", "Shanghai", "Cancun").
79+
80+
## Usage
81+
82+
### Command Line Interface
83+
84+
Convert fuzzer output to blockchain test fixtures:
85+
86+
```bash
87+
# Basic conversion
88+
uv run fuzzer_bridge --input fuzzer_output.json --output blocktest.json
89+
90+
# Specify a different fork
91+
uv run fuzzer_bridge --input fuzzer_output.json --output blocktest.json --fork Shanghai
92+
93+
# Pretty print output
94+
uv run fuzzer_bridge --input fuzzer_output.json --output blocktest.json --pretty
95+
96+
# Process multiple files in parallel
97+
uv run fuzzer_bridge --input fuzzer_outputs/ --output fixtures/ --parallel
98+
99+
# Merge multiple outputs into a single fixture file
100+
uv run fuzzer_bridge --input fuzzer_outputs/ --output fixtures/ --merge
101+
```
102+
103+
### Python API
104+
105+
You can also use the fuzzer bridge programmatically in your Python code:
106+
107+
```python
108+
from cli.fuzzer_bridge import FuzzerBridge
109+
import json
110+
111+
# Load fuzzer output
112+
with open("fuzzer_output.json") as f:
113+
fuzzer_data = json.load(f)
114+
115+
# Create bridge and convert
116+
bridge = FuzzerBridge()
117+
blocktest = bridge.convert(fuzzer_data)
118+
119+
# Save to file
120+
bridge.save(blocktest, "output.json")
121+
122+
# Verify with a client
123+
result = bridge.verify_with_geth(blocktest, geth_path="../go-ethereum/build/bin/evm")
124+
print(f"Test passed: {result['pass']}")
125+
```
126+
127+
### Integration with pytest
128+
129+
Generate tests dynamically from fuzzer output:
130+
131+
```python
132+
import pytest
133+
from cli.fuzzer_bridge import create_test_from_fuzzer
134+
135+
def test_fuzzer_generated(blockchain_test):
136+
"""Test generated from fuzzer output."""
137+
test = create_test_from_fuzzer("fuzzer_output.json")
138+
blockchain_test(**test)
139+
```
140+
141+
## How It Works
142+
143+
### Genesis Block Derivation
144+
145+
The fuzzer describes the environment for block 1 (the block containing the transactions). The genesis block (block 0) environment is automatically derived:
146+
147+
- `number` = 0
148+
- `timestamp` = block1_timestamp - 12 (assuming 12-second block time)
149+
- `baseFee` = calculated based on block 1's base fee
150+
- Other values are inherited or set to defaults
151+
152+
### System Contracts
153+
154+
The framework automatically includes system contracts required by the fork:
155+
156+
- Deposit contract (for proof-of-stake)
157+
- Withdrawal contract
158+
- Beacon roots contract
159+
- Other fork-specific contracts
160+
161+
These are included in the state root calculation without requiring fuzzer specification.
162+
163+
### Transaction Signing
164+
165+
All transactions are automatically signed using the provided private keys. The fuzzer bridge:
166+
167+
1. Validates that each sender has a corresponding private key
168+
2. Signs transactions with the appropriate signature type for the fork
169+
3. Handles EIP-1559 transactions when base fee is present
170+
4. Properly encodes legacy and typed transactions
171+
172+
## Troubleshooting
173+
174+
### Common Issues
175+
176+
#### "Genesis block hash doesn't match"
177+
178+
- **Cause**: Environment parameters are incorrect
179+
- **Solution**: Ensure the fuzzer output follows the v2 format exactly
180+
181+
#### "No private key for sender"
182+
183+
- **Cause**: Account sends transaction but no privateKey field provided
184+
- **Solution**: Add privateKey to the account in the accounts section
185+
186+
#### "Private key doesn't match address"
187+
188+
- **Cause**: The provided private key doesn't generate the specified address
189+
- **Solution**: Use correct private key or generate address from private key
190+
191+
#### "Transaction type not supported in fork"
192+
193+
- **Cause**: Using EIP-1559 transactions in pre-London forks
194+
- **Solution**: Ensure transaction types match the specified fork
195+
196+
## Testing with Clients
197+
198+
Once you've generated blockchain test fixtures, verify them with Ethereum clients:
199+
200+
### Go-Ethereum (geth)
201+
202+
```bash
203+
../go-ethereum/build/bin/evm blocktest generated_test.json
204+
```
205+
206+
### Besu
207+
208+
```bash
209+
../besu/ethereum/evmtool/build/install/evmtool/bin/evmtool block-test generated_test.json
210+
```
211+
212+
### Nethermind
213+
214+
```bash
215+
../nethermind/src/artifacts/bin/nethermind.test.runner test -b generated_test.json
216+
```
217+
218+
## Advanced Features
219+
220+
### Batch Processing
221+
222+
Process multiple fuzzer outputs efficiently:
223+
224+
```python
225+
from cli.fuzzer_bridge.cli import process_directory_parallel
226+
227+
# Process all JSON files in a directory
228+
process_directory_parallel(
229+
input_dir="fuzzer_outputs/",
230+
output_dir="fixtures/",
231+
fork="Prague",
232+
workers=8
233+
)
234+
```
235+
236+
### Custom Fork Configuration
237+
238+
Override default fork parameters:
239+
240+
```python
241+
from cli.fuzzer_bridge import BlocktestBuilder
242+
243+
builder = BlocktestBuilder(fork="Prague")
244+
# Custom configuration
245+
builder.env_overrides = {
246+
"currentRandom": "0x1234...",
247+
"currentExcessBlobGas": "0x0"
248+
}
249+
```
250+
251+
## Best Practices
252+
253+
1. **Validate Fuzzer Output**: Always validate that your fuzzer generates valid v2 format JSON
254+
2. **Test with Multiple Clients**: Verify generated fixtures with multiple client implementations
255+
3. **Use Parallel Processing**: For large batches, use `--parallel` flag for better performance
256+
4. **Version Control**: Track generated fixtures in version control for regression testing
257+
5. **Continuous Integration**: Integrate fuzzer bridge into CI pipelines for automated testing
258+
259+
## Further Resources
260+
261+
- [Blocktest Fuzzer Documentation](https://github.com/ethereum/blocktest-fuzzer)
262+
- [EEST Framework Documentation](../index.md)
263+
- [Ethereum Test Format Specifications](./reference_specification.md)

pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,7 @@ evm_bytes = "cli.evm_bytes:evm_bytes"
109109
hasher = "cli.hasher:main"
110110
eest = "cli.eest.cli:eest"
111111
fillerconvert = "cli.fillerconvert.fillerconvert:main"
112+
fuzzer_bridge = "cli.fuzzer_bridge.cli:main"
112113
groupstats = "cli.show_pre_alloc_group_stats:main"
113114
extract_config = "cli.extract_config:extract_config"
114115
compare_fixtures = "cli.compare_fixtures:main"

0 commit comments

Comments
 (0)