Raw opcode benchmarks

[benaadams]

Changes

• Fix Benchmarks project so it runs and filters properly
• In Nethermind.Benchmark.Runner project dotnet run -c Release -- --filter "*EvmOpcodesBenchmark*"
• Add comprehensive EVM opcode benchmarks that execute real instruction handlers via function pointer dispatch, matching the production VirtualMachine execution path
• Every valid legacy opcode is benchmarked through the same GenerateOpCodes table used at runtime, parametrized via [ParamsSource] over all opcodes
• Three execution strategies to handle different opcode classes:
  • Stack-walk: most pure opcodes (ADD, SUB, AND, etc.) run 4096 iterations by walking the pre-filled stack
  • Per-run refresh: stateful opcodes (SLOAD, SSTORE, KECCAK256, CALL, etc.) get fresh inputs each iteration with dynamic storage keys and call targets to defeat caching
  • Independent binary inputs: MUL/DIV/MOD ops get fully independent operand pairs to avoid JIT/CPU pattern exploitation
• Custom BenchmarkDotNet columns display per-opcode Gas cost and MGas/s throughput derived from mean execution time
• Add CodeInfoRepository.Clear() to flush code cache between iterations, preventing warm-cache bias
• Modernize benchmark runner: consolidate assembly scanning with BenchmarkSwitcher, add --quick flag (ShortRun vs MediumRun), update to CoreRuntime.Core10_0
• Fix GasColumnProvider to gracefully handle non-precompile benchmarks in mixed runs

2:30mins to produce output for 140+ opcodes

Opcode	Mean	Error	Gas	MGas/s	Allocated
STOP	4.973 ns	0.0999 ns	0	0.00	-
ADD	32.689 ns	0.2424 ns	3	91.78	-
MUL	69.357 ns	0.3957 ns	5	72.09	-
SUB	32.185 ns	0.2999 ns	3	93.21	-
DIV	89.502 ns	1.4307 ns	5	55.86	-
SDIV	90.112 ns	0.8904 ns	5	55.49	-
MOD	91.771 ns	0.6138 ns	5	54.48	-
SMOD	91.674 ns	1.1894 ns	5	54.54	-
ADDMOD	96.055 ns	0.4057 ns	8	83.29	-
MULMOD	252.563 ns	4.1821 ns	8	31.68	-
EXP	791.937 ns	14.9318 ns	1610	2032.99	-
SIGNEXTEND	19.237 ns	0.3724 ns	5	259.91	-
LT	23.796 ns	0.4635 ns	3	126.07	-
....

Types of changes

What types of changes does your code introduce?

• New feature (a non-breaking change that adds functionality)
• Build-related changes

Testing

Requires testing

• Yes
• No

Documentation

Requires documentation update

• Yes
• No

Requires explanation in Release Notes

• Yes
• No

[Copilot]

Pull request overview

Adds a new BenchmarkDotNet suite to benchmark “raw” EVM opcode handlers (function-pointer dispatch, close to the VirtualMachine execution path) and extends benchmark output with opcode gas/throughput columns.

Changes:

• Introduce EvmOpcodesBenchmark plus a custom column provider to report per-opcode gas and MGas/s.
• Expose additional internals needed by the benchmark (InternalsVisibleTo, code cache clearing).
• Update benchmark runner/config and solution composition to include additional assemblies/projects.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/Nethermind/Nethermind.Precompiles.Benchmark/GasColumnProvider.cs	Makes gas column extraction safer and only available when the expected Input param type is present.
src/Nethermind/Nethermind.Evm/Instructions/EvmInstructions.cs	Grants Nethermind.Evm.Benchmark access to internal EvmInstructions for opcode table generation.
src/Nethermind/Nethermind.Evm/CodeInfoRepository.cs	Adds internal cache clearing APIs to support benchmark iteration isolation.
src/Nethermind/Nethermind.Evm.Benchmark/Nethermind.Evm.Benchmark.csproj	Enables unsafe blocks needed for function pointers in the new benchmark.
src/Nethermind/Nethermind.Evm.Benchmark/EvmOpcodesBenchmark.cs	New benchmark executing opcode handlers directly with per-opcode stack/environment setup.
src/Nethermind/Nethermind.Evm.Benchmark/EvmOpcodeGasColumnProvider.cs	New BenchmarkDotNet column provider computing per-opcode gas and derived throughput.
src/Nethermind/Nethermind.Benchmark.Runner/Program.cs	Adds --quick mode and switches to a BenchmarkSwitcher flow for running multiple assemblies.
src/Nethermind/Benchmarks.slnx	Adds additional projects to the benchmarks solution.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[smartprogrammer93]

Please make it produce the following columns:

Mean - Median - P90 - P95 - Min - Max - StdDev

Also ensure StdDev is low

look at perf/increase-benchmark-iterations branch. the BlockProcessingBenchmark.cs

[benaadams]

Please make it produce the following columns:

Mean - Median - P90 - P95 - Min - Max - StdDev

Also ensure StdDev is low

look at perf/increase-benchmark-iterations branch. the BlockProcessingBenchmark.cs

Alas it looks like a shared CI machine is very variable #10651 (comment)

[smartprogrammer93]

Alas it looks like a shared CI machine is very variable #10651 (comment)

this is why you should follow the same approach in that Branch:

1. increase iterations

2. introduce multiple operations per invoke: so innerCount needs to be high enough so each iteration is not less than 100ms, the higher the inner count the better i believe

Also, please add the columns as shown in the first picture.