|
| 1 | +# Fuzzer Bridge for Execution Spec Tests |
| 2 | + |
| 3 | +This module provides a bridge between blocktest fuzzers (like `blocktest-fuzzer`) and the Ethereum execution-spec-tests framework, enabling automatic generation of valid blockchain test fixtures from fuzzer output. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +The fuzzer bridge solves a critical problem: fuzzers can generate transactions and pre-state, but creating valid blockchain tests requires complex calculations (state roots, RLP encoding, block headers, etc.). This bridge leverages the execution-spec-tests framework to handle all these complexities. |
| 8 | + |
| 9 | +## Architecture |
| 10 | + |
| 11 | +```mermaid |
| 12 | +graph LR |
| 13 | + A[Blocktest<br/>Fuzzer] -->|JSON<br/>v2 format| B[Fuzzer<br/>Bridge] |
| 14 | + B -->|Blockchain Test<br/>Fixtures| C[Ethereum<br/>Clients] |
| 15 | +``` |
| 16 | + |
| 17 | +## Fuzzer Output Format (v2) |
| 18 | + |
| 19 | +The fuzzer must output JSON in the following format: |
| 20 | + |
| 21 | +```json |
| 22 | +{ |
| 23 | + "version": "2.0", |
| 24 | + "fork": "Prague", |
| 25 | + "chainId": 1, |
| 26 | + "accounts": { |
| 27 | + "0x7e5f4552091a69125d5dfcb7b8c2659029395bdf": { |
| 28 | + "balance": "0x1000000000000000000", |
| 29 | + "nonce": "0x0", |
| 30 | + "code": "", |
| 31 | + "storage": {}, |
| 32 | + "privateKey": "0x0000000000000000000000000000000000000000000000000000000000000001" |
| 33 | + }, |
| 34 | + "0x2b5ad5c4795c026514f8317c7a215e218dccd6cf": { |
| 35 | + "balance": "0x100000000000000000", |
| 36 | + "nonce": "0x0", |
| 37 | + "code": "0x600160005260206000f3", |
| 38 | + "storage": {} |
| 39 | + } |
| 40 | + }, |
| 41 | + "transactions": [ |
| 42 | + { |
| 43 | + "from": "0x7e5f4552091a69125d5dfcb7b8c2659029395bdf", |
| 44 | + "to": "0x2b5ad5c4795c026514f8317c7a215e218dccd6cf", |
| 45 | + "value": "0x100", |
| 46 | + "gas": "0x5208", |
| 47 | + "gasPrice": "0x7", |
| 48 | + "nonce": "0x0", |
| 49 | + "data": "0x" |
| 50 | + } |
| 51 | + ], |
| 52 | + "env": { |
| 53 | + "currentCoinbase": "0xc014ba5e00000000000000000000000000000000", |
| 54 | + "currentDifficulty": "0x0", |
| 55 | + "currentGasLimit": "0x1000000", |
| 56 | + "currentNumber": "0x1", |
| 57 | + "currentTimestamp": "0x1000", |
| 58 | + "currentBaseFee": "0x7", |
| 59 | + "currentRandom": "0x0000000000000000000000000000000000000000000000000000000000000000" |
| 60 | + } |
| 61 | +} |
| 62 | +``` |
| 63 | + |
| 64 | +### Key Requirements |
| 65 | + |
| 66 | +1. **Private Keys**: Any account that sends transactions MUST include a `privateKey` field. |
| 67 | +2. **Address-Key Match**: Private keys must generate the corresponding addresses. |
| 68 | +3. **Environment**: Describes the environment for block 1 (genesis is automatically derived). |
| 69 | +4. **Version**: Use "2.0" for this format. |
| 70 | + |
| 71 | +## Architecture: DTO Pattern |
| 72 | + |
| 73 | +The fuzzer bridge uses a **Data Transfer Object (DTO) pattern** for clean separation between external data format and internal domain logic. |
| 74 | + |
| 75 | +### Design Flow |
| 76 | + |
| 77 | +```mermaid |
| 78 | +graph TD |
| 79 | + A["Fuzzer JSON Output<br/>(JSON-RPC Format v2.0)"] |
| 80 | +
|
| 81 | + B["DTOs (Pydantic Models)<br/>models.py"] |
| 82 | + B1["FuzzerAccountInput<br/>Raw account JSON"] |
| 83 | + B2["FuzzerTransactionInput<br/>Raw transaction JSON (uses 'gas')"] |
| 84 | + B3["FuzzerAuthorizationInput<br/>Raw auth tuple (EIP-7702)"] |
| 85 | + B4["FuzzerOutput<br/>Complete fuzzer output"] |
| 86 | +
|
| 87 | + C["EEST Domain Models<br/>converter.py"] |
| 88 | + C1["Account<br/>With validation & defaults"] |
| 89 | + C2["Transaction<br/>With gas_limit, EOA sender"] |
| 90 | + C3["AuthorizationTuple<br/>EIP-7702 support"] |
| 91 | + C4["EOA<br/>Created from private keys"] |
| 92 | +
|
| 93 | + D["BlockchainTest"] |
| 94 | + E["Blockchain Fixture"] |
| 95 | +
|
| 96 | + A -->|parse| B |
| 97 | + B -.-> B1 |
| 98 | + B -.-> B2 |
| 99 | + B -.-> B3 |
| 100 | + B -.-> B4 |
| 101 | +
|
| 102 | + B -->|convert| C |
| 103 | + C -.-> C1 |
| 104 | + C -.-> C2 |
| 105 | + C -.-> C3 |
| 106 | + C -.-> C4 |
| 107 | +
|
| 108 | + C -->|generate| D |
| 109 | + D --> E |
| 110 | +
|
| 111 | + style A fill:#e1f5ff |
| 112 | + style B fill:#fff4e1 |
| 113 | + style C fill:#e8f5e9 |
| 114 | + style D fill:#f3e5f5 |
| 115 | + style E fill:#fce4ec |
| 116 | +``` |
| 117 | + |
| 118 | +### Why DTOs? |
| 119 | + |
| 120 | +1. **Separation of Concerns** |
| 121 | + - External JSON-RPC format ≠ EEST internal representation |
| 122 | + - Fuzzer format can change without affecting EEST domain models |
| 123 | + |
| 124 | +2. **No Side Effects During Parsing** |
| 125 | + - DTOs don't trigger `model_post_init` validation logic |
| 126 | + - Parsing is purely data extraction, no business logic |
| 127 | + |
| 128 | +3. **Explicit Field Mapping** |
| 129 | + - Clear visibility: `gas` → `gas_limit`, `from` → `sender` (EOA) |
| 130 | + - Type-safe conversions at boundary |
| 131 | + |
| 132 | +4. **Prevents TestAddress Pollution** |
| 133 | + - EOA created in converter BEFORE Transaction instantiation |
| 134 | + - Transaction.model_post_init never injects TestAddress |
| 135 | + |
| 136 | +### Key Field Mappings |
| 137 | + |
| 138 | +| Fuzzer Field (JSON-RPC) | DTO Field | EEST Domain Field | Notes | |
| 139 | +|-------------------------|----------------------|----------------------|--------------------------------| |
| 140 | +| `from` | `from_` | `sender` (EOA) | Creates EOA from private key | |
| 141 | +| `gas` | `gas` | `gas_limit` | JSON-RPC vs internal naming | |
| 142 | +| `data` | `data` | `data` | Same field, explicit mapping | |
| 143 | +| `gasPrice` | `gas_price` | `gas_price` | CamelCase → snake_case | |
| 144 | +| `authorizationList` | `authorization_list` | `authorization_list` | EIP-7702 support | |
| 145 | +| `privateKey` | `private_key` | (used to create EOA) | Not stored in Account model | |
| 146 | + |
| 147 | +### Module Responsibilities |
| 148 | + |
| 149 | +#### `models.py` - Data Transfer Objects |
| 150 | +- Pure Pydantic models for fuzzer JSON format |
| 151 | +- No business logic, only data validation |
| 152 | +- Accepts JSON-RPC naming conventions (camelCase) |
| 153 | +- ~119 lines |
| 154 | + |
| 155 | +#### `converter.py` - Transformation Logic |
| 156 | +- Pure functions: DTO → EEST domain models |
| 157 | +- All field mapping logic centralized here |
| 158 | +- Creates EOA objects from private keys |
| 159 | +- Builds BlockchainTest from validated data |
| 160 | +- ~305 lines |
| 161 | + |
| 162 | +#### `blocktest_builder.py` - CLI Integration |
| 163 | +- Orchestrates conversion workflow |
| 164 | +- Handles file I/O and CLI options |
| 165 | +- Calls converter functions |
| 166 | +- ~90 lines |
| 167 | + |
| 168 | +### Benefits |
| 169 | + |
| 170 | +✅ **Maintainability**: Changes to Account/Transaction propagate automatically |
| 171 | +✅ **Testability**: Each layer tested independently |
| 172 | +✅ **Type Safety**: Full type checking at DTO and domain layers |
| 173 | +✅ **Clarity**: Field mappings are explicit and documented |
| 174 | +✅ **No Circular Dependencies**: Clean module boundaries |
| 175 | + |
| 176 | +### Alternative Design: Why Not Inheritance? |
| 177 | + |
| 178 | +**Could have done**: |
| 179 | +```python |
| 180 | +class FuzzerAccount(Account): |
| 181 | + private_key: Hash | None = None |
| 182 | +``` |
| 183 | + |
| 184 | +**Why DTOs are better**: |
| 185 | +- Inheritance couples external format to domain model |
| 186 | +- model_post_init triggers during parsing (side effects) |
| 187 | +- Field name mismatches require complex aliasing |
| 188 | +- Harder to test layers independently |
| 189 | + |
| 190 | +The DTO pattern provides cleaner separation and explicit control. |
| 191 | + |
| 192 | +## Installation |
| 193 | + |
| 194 | +See the [EEST installation guide](https://eest.ethereum.org/main/getting_started/installation/) for setting up the execution-spec-tests framework. |
| 195 | + |
| 196 | +Once EEST is installed, the fuzzer bridge will be available as a command-line tool. |
| 197 | + |
| 198 | +## Usage |
| 199 | + |
| 200 | +### 1. Command Line Interface |
| 201 | + |
| 202 | +```bash |
| 203 | +# Convert fuzzer output to blockchain test |
| 204 | +uv run fuzzer_bridge --input fuzzer_output.json --output blocktest.json |
| 205 | + |
| 206 | +# With custom fork |
| 207 | +uv run fuzzer_bridge --input fuzzer_output.json --output blocktest.json --fork Shanghai |
| 208 | + |
| 209 | +# Pretty print output |
| 210 | +uv run fuzzer_bridge --input fuzzer_output.json --output blocktest.json --pretty |
| 211 | +``` |
| 212 | + |
| 213 | +### 2. Python API |
| 214 | + |
| 215 | +```python |
| 216 | +from fuzzer_bridge import FuzzerBridge |
| 217 | + |
| 218 | +# Load fuzzer output |
| 219 | +with open("fuzzer_output.json") as f: |
| 220 | + fuzzer_data = json.load(f) |
| 221 | + |
| 222 | +# Create bridge and convert |
| 223 | +bridge = FuzzerBridge() |
| 224 | +blocktest = bridge.convert(fuzzer_data) |
| 225 | + |
| 226 | +# Save to file |
| 227 | +bridge.save(blocktest, "output.json") |
| 228 | + |
| 229 | +# Or verify with geth directly |
| 230 | +result = bridge.verify_with_geth(blocktest, geth_path="../go-ethereum/build/bin/evm") |
| 231 | +print(f"Test passed: {result['pass']}") |
| 232 | +``` |
| 233 | + |
| 234 | +### 3. Integration with pytest |
| 235 | + |
| 236 | +```python |
| 237 | +import pytest |
| 238 | +from fuzzer_bridge import create_test_from_fuzzer |
| 239 | + |
| 240 | +def test_fuzzer_generated(blockchain_test): |
| 241 | + """Test generated from fuzzer output.""" |
| 242 | + test = create_test_from_fuzzer("fuzzer_output.json") |
| 243 | + blockchain_test(**test) |
| 244 | +``` |
| 245 | + |
| 246 | +## Key Insights |
| 247 | + |
| 248 | +### Genesis Block Handling |
| 249 | +- The fuzzer describes the environment for block 1 (the block being tested) |
| 250 | +- Genesis (block 0) environment is automatically derived: |
| 251 | + - `number` = 0 |
| 252 | + - `timestamp` = block1_timestamp - 12 |
| 253 | + - Other values inherited or set to defaults |
| 254 | + |
| 255 | +### System Contracts |
| 256 | +- The framework automatically adds system contracts (deposit, withdrawal, etc.) |
| 257 | +- These are included in the state root calculation |
| 258 | +- The fuzzer doesn't need to specify them |
| 259 | + |
| 260 | +### Private Key Requirements |
| 261 | +- Every account that sends transactions needs a private key |
| 262 | +- The private key must generate the exact address specified |
| 263 | +- Without matching private keys, transactions cannot be signed |
| 264 | + |
| 265 | +## Troubleshooting |
| 266 | + |
| 267 | +### "Genesis block hash doesn't match" |
| 268 | +**Cause**: Usually means the environment is set incorrectly (e.g., block number = 1 instead of 0 for genesis) |
| 269 | +**Solution**: Ensure the fuzzer output follows the v2 format exactly |
| 270 | + |
| 271 | +### "No private key for sender" |
| 272 | +**Cause**: Account sends transaction but no privateKey field provided |
| 273 | +**Solution**: Add privateKey to the account in the accounts section |
| 274 | + |
| 275 | +### "Private key doesn't match address" |
| 276 | +**Cause**: The provided private key doesn't generate the specified address |
| 277 | +**Solution**: Use correct private key or generate address from private key |
| 278 | + |
| 279 | +## Testing with Ethereum Clients |
| 280 | + |
| 281 | +### Go-Ethereum (geth) |
| 282 | +```bash |
| 283 | +../go-ethereum/build/bin/evm blocktest generated_test.json |
| 284 | +``` |
| 285 | + |
| 286 | +### Besu |
| 287 | +```bash |
| 288 | +../besu/ethereum/evmtool/build/install/evmtool/bin/evmtool block-test generated_test.json |
| 289 | +``` |
| 290 | + |
| 291 | +### Nethermind |
| 292 | +```bash |
| 293 | +../nethermind/src/Nethermind/artifacts/bin/Nethermind.Test.Runner/release_linux-x64/nethtest -b -i generated_test.json |
| 294 | +``` |
| 295 | + |
| 296 | +## Contributing |
| 297 | + |
| 298 | +When modifying the fuzzer bridge: |
| 299 | +1. Add tests for new features. |
| 300 | +2. Update this README. |
| 301 | +3. Ensure compatibility with latest execution-spec-tests. |
0 commit comments