Commit 2e114da
authored
test: AVM opcode spammer (#18919)
# AVM Opcode Spammer
## Overview
The **Opcode Spammer** is a bytecode-building framework for spamming AVM
opcodes. It generates bytecode that repeatedly executes target opcodes
until the transaction runs out of gas, allowing us to measure worst-case
simulation and proving times, and confirm that all such transactions are
simulatable and provable.
## What's left after this
1. Spammer tests for external (regular/static) calls/returns/reverts,
internalcalls, min/max scaling factors for dynamic gas opcodes, all wire
formats (currently only doing smallest wire formats)
2. Scripts to run proving benchmarks with dedicated resources (and maybe
only a subset of the cases)
4. Use random numbers (preferably seeded) instead of consts like 42
5. Consider using calldatacopy for "unique" values (for EMITNULLIFIER
and SSTORE) (one big cdc instead of 1 add per target instr)
7. Make the tests for the following more meaningful:
- CALLDATACOPY
- RETURNDATACOPY
- RETURNDATASIZE
- GETENVVAR
- GETCONTRACTINSTANCE
## Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ SPAM_CONFIGS │
│ Record<Opcode, SpamConfig[]> │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ ADD_8 │ │ POSEIDON2 │ │EMITNULLIFIER│ ... │
│ │ [7 configs] │ │ [1 config] │ │ [1 config] │ │
│ │ (per type) │ │ │ │ (limit=63) │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ getSpamConfigsPerOpcode() │
│ Returns { opcodes, config[] } for test iteration │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ testOpcodeSpamCase() │
│ Routes to appropriate bytecode generator & executes test │
│ │
│ config.limit === undefined? │
│ YES → testStandardOpcodeSpam() │
│ NO → testSideEffectOpcodeSpam() │
└─────────────────────────────────────────────────────────────────┘
```
## Two Execution Strategies
### Strategy 1: Standard Opcodes (Gas-Limited)
For opcodes without per-TX limits (arithmetic, comparisons, memory ops,
etc.), we create a single contract with an infinite loop:
```
┌────────────────────────────────────────────────────────────────┐
│ SINGLE CONTRACT │
│ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ SETUP PHASE │ │
│ │ SET mem[0] = initial_value │ │
│ │ SET mem[1] = operand │ │
│ │ ... │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ LOOP (fills remaining bytecode space) ◄─────┐ │ │
│ │ TARGET_OPCODE ─┐ │ │ │
│ │ TARGET_OPCODE │ unrolled N times │ │ │
│ │ TARGET_OPCODE │ (N = available_bytes / instr_size)│ │ │
│ │ ... ─┘ │ │ │
│ │ JUMP back ──────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
│ Executes until: OUT OF GAS │
└────────────────────────────────────────────────────────────────┘
```
**Bytecode Layout:**
```
┌─────────────────────────────────────────────────────────────────┐
│ 0x00: SET instructions (setup) │
│ ... │
│ 0xNN: ┌─── LOOP START ◄──────────────────────────────────────┐ │
│ │ TARGET_OPCODE │ │
│ │ TARGET_OPCODE (unrolled to fill max bytecode size) │ │
│ │ TARGET_OPCODE │ │
│ │ ... │ │
│ └─► JUMP 0xNN ─────────────────────────────────────────┘ │
│ MAX_BYTECODE_BYTES │
└─────────────────────────────────────────────────────────────────┘
```
### Strategy 2: Side-Effect Limited Opcodes (Nested Call Pattern)
For opcodes with per-TX limits (EMITNOTEHASH, EMITNULLIFIER,
SENDL2TOL1MSG, etc.), we use a two-contract pattern where the inner
contract executes side effects up to the limit, then REVERTs to discard
them:
```
┌─────────────────────────────────────────────────────────────────┐
│ OUTER CONTRACT │
│ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ SETUP │ │
│ │ CALLDATACOPY inner_address from calldata[0] │ │
│ │ SET l2Gas = MAX_UINT32 │ │
│ │ SET daGas = MAX_UINT32 │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ LOOP ◄────┐ │ │
│ │ CALL inner_contract ──────────────────────┐ │ │ │
│ │ JUMP back ─────────────────────────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │ │
│ Executes until: OUT OF GAS │ │
└───────────────────────────────────────────────│─────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ INNER CONTRACT │
│ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ SETUP │ │
│ │ SET initial values for side-effect opcode │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ BODY (unrolled, NOT a loop) │ │
│ │ SIDE_EFFECT_OPCODE ─┐ │ │
│ │ SIDE_EFFECT_OPCODE │ repeated `limit` times │ │
│ │ SIDE_EFFECT_OPCODE │ (e.g., 64 for EMITNOTEHASH) │ │
│ │ ... ─┘ │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ CLEANUP │ │
│ │ REVERT (discards all side effects from this call) │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
**Why this pattern?**
Side-effect opcodes have per-TX limits:
- `EMITNOTEHASH`: max 64 per TX
- `EMITNULLIFIER`: max 63 per TX (one reserved for TX nullifier)
- `SENDL2TOL1MSG`: max 8 per TX
- `EMITUNENCRYPTEDLOG`: limited by total log payload size
By having the inner contract REVERT after emitting side effects, those
effects are discarded, allowing the outer contract to call it again.
This enables thousands of opcode executions per TX instead of just the
limit.
## SpamConfig Structure
```typescript
interface SpamConfig {
// Memory cells to initialize before the loop
setup: SetupItem[];
// Factory to create target instruction(s) to spam
targetInstructions: () => Bufferable[];
// Instructions to run after target spam (for side-effect opcodes)
cleanupInstructions?: () => Bufferable[];
// Per-TX limit (triggers nested call pattern if set)
limit?: number;
// Label for test display (e.g., "UINT32" for type variants)
label?: string;
}
```
## Type Variants
Many opcodes support multiple types. The spammer tests each type variant
separately:
```typescript
// ADD_8 has 7 configs - one per supported type
[Opcode.ADD_8]: ALL_TAGS.map(tag => ({
label: TypeTag[tag], // "FIELD", "UINT1", "UINT8", etc.
setup: [
{ offset: 0, value: withTag(1n, tag) },
{ offset: 1, value: withTag(1n, tag) },
],
targetInstructions: () => [new Add(0, 0, 1, 0).as(Opcode.ADD_8, Add.wireFormat8)],
})),
```
Test output hierarchy:
```
ADD_8
├── ADD_8/FIELD ✓
├── ADD_8/UINT1 ✓
├── ADD_8/UINT8 ✓
├── ADD_8/UINT16 ✓
├── ADD_8/UINT32 ✓
├── ADD_8/UINT64 ✓
└── ADD_8/UINT128 ✓
```
### In Tests
```typescript
import { getSpamConfigsPerOpcode, testOpcodeSpamCase } from '@aztec/simulator/public/fixtures';
const groupedSpamConfigs = getSpamConfigsPerOpcode();
describe.each(groupedSpamConfigs)('$opcode', ({ configs }) => {
it.each(configs)('$label', async config => {
await testOpcodeSpamCase(tester, config, expectToBeTrue);
});
});
```
## Test Suites
### Simulation Benchmarks
`yarn-project/simulator/src/public/public_tx_simulator/apps_tests/opcode_spam.test.ts`
Runs opcode spam through the C++ simulator (and optionally TS vs C++
comparison).
### Proving Benchmarks
`yarn-project/bb-prover/src/avm_proving_tests/avm_opcode_spam.test.ts`
Runs opcode spam through full AVM proving. Skipped in CI (meant for
local measurement).1 parent 07bfac8 commit 2e114da
File tree
19 files changed
+1564
-54
lines changed- barretenberg/cpp/src/barretenberg/vm2/testing
- yarn-project
- bb-prover/src/avm_proving_tests
- ivc-integration/src
- simulator
- artifacts
- src/public
- avm
- fixtures
- public_tx_simulator
- apps_tests
- stdlib/src
- abi
- avm
19 files changed
+1564
-54
lines changedBinary file not shown.
Lines changed: 69 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
37 | 46 | | |
38 | 47 | | |
39 | 48 | | |
| |||
107 | 116 | | |
108 | 117 | | |
109 | 118 | | |
110 | | - | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
111 | 126 | | |
112 | 127 | | |
113 | 128 | | |
| |||
Lines changed: 17 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
41 | 50 | | |
42 | 51 | | |
43 | 52 | | |
| |||
72 | 81 | | |
73 | 82 | | |
74 | 83 | | |
75 | | - | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
76 | 91 | | |
77 | 92 | | |
78 | 93 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
38 | 38 | | |
39 | | - | |
| 39 | + | |
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
| |||
856 | 856 | | |
857 | 857 | | |
858 | 858 | | |
859 | | - | |
| 859 | + | |
860 | 860 | | |
861 | 861 | | |
862 | 862 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
416 | 416 | | |
417 | 417 | | |
418 | 418 | | |
419 | | - | |
| 419 | + | |
420 | 420 | | |
421 | 421 | | |
422 | 422 | | |
| |||
426 | 426 | | |
427 | 427 | | |
428 | 428 | | |
429 | | - | |
| 429 | + | |
430 | 430 | | |
431 | 431 | | |
432 | 432 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
18 | 22 | | |
19 | 23 | | |
20 | 24 | | |
| |||
34 | 38 | | |
35 | 39 | | |
36 | 40 | | |
37 | | - | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
38 | 45 | | |
39 | 46 | | |
40 | 47 | | |
| |||
Lines changed: 53 additions & 19 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
8 | | - | |
9 | | - | |
| 9 | + | |
10 | 10 | | |
11 | | - | |
12 | | - | |
13 | | - | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
14 | 15 | | |
15 | | - | |
| 16 | + | |
16 | 17 | | |
17 | 18 | | |
18 | | - | |
19 | 19 | | |
20 | | - | |
21 | | - | |
22 | | - | |
| 20 | + | |
| 21 | + | |
23 | 22 | | |
24 | 23 | | |
25 | 24 | | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
26 | 28 | | |
27 | 29 | | |
28 | 30 | | |
29 | 31 | | |
30 | | - | |
| 32 | + | |
| 33 | + | |
31 | 34 | | |
32 | 35 | | |
33 | 36 | | |
34 | 37 | | |
| 38 | + | |
35 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
36 | 54 | | |
37 | 55 | | |
38 | 56 | | |
39 | | - | |
| 57 | + | |
40 | 58 | | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
| 59 | + | |
48 | 60 | | |
49 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
0 commit comments