Skip to content

feat(test-client-clis): add stream-mode opcode count support for geth#2170

Open
spencer-tb wants to merge 4 commits intoethereum:forks/amsterdamfrom
spencer-tb:feat/geth-opcode-count-stream
Open

feat(test-client-clis): add stream-mode opcode count support for geth#2170
spencer-tb wants to merge 4 commits intoethereum:forks/amsterdamfrom
spencer-tb:feat/geth-opcode-count-stream

Conversation

@spencer-tb
Copy link
Contributor

@spencer-tb spencer-tb commented Feb 9, 2026

🗒️ Description

Extends _evaluate_stream() in TransitionTool to read opcode count output from opcodes.json (mirrors existing _evaluate_filesystem() logic), updates safe_t8n_args() to set --output.basedir and --opcode.count flags when supports_opcode_count is true (even without --trace), and enables supports_opcode_count = True for GethTransitionTool.

The EELS benchmark framework uses opcode frequency counts to verify that benchmark tests exercise the intended opcodes at the expected iteration counts. This is critical for the gas repricing work where we need precise control over how many times each opcode executes.

Currently, benchmark tests can only be filled with opcode count verification using evmone or EELS. However, for Amsterdam (which evmone does not support currently) we need to fill benchmark tests with geth. As geth has this implemented the hope is to use geth as the default benchmark test filler moving forward. EELS is too slow.

Previously, opcode count support in EELS was only wired for filesystem-mode tools (evmone). Since geth uses stream mode (t8n_use_stream = True), the _evaluate_stream() path needed the same opcode count file reading that _evaluate_filesystem() already had. This PR adds that wiring so geth has full feature parity with evmone for benchmark fills.

🔗 Related Issues or PRs

✅ Checklist

  • All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    uvx tox -e static
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).

Cute Animal Picture

Put a link to a cute animal picture inside the parenthesis-->

@codecov
Copy link

codecov bot commented Feb 9, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.07%. Comparing base (87a414f) to head (548e36c).
⚠️ Report is 2 commits behind head on forks/amsterdam.

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #2170   +/-   ##
================================================
  Coverage            86.07%   86.07%           
================================================
  Files                  599      599           
  Lines                39472    39472           
  Branches              3780     3780           
================================================
  Hits                 33977    33977           
  Misses                4862     4862           
  Partials               633      633           
Flag Coverage Δ
unittests 86.07% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@spencer-tb spencer-tb added C-feat Category: an improvement or new feature A-test-client-clis Area: execution_testing.client_clis labels Feb 9, 2026
@spencer-tb spencer-tb changed the title feat(client_clis): add stream-mode opcode count support for geth feat(test-client-clis): add stream-mode opcode count support for geth Feb 9, 2026
Copy link
Member

@marioevz marioevz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition, just one comment, and we should wait until the Geth's PR is reviewed and eventually merged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@spencer-tb
Copy link
Contributor Author

Changes implemented and tested with fill again. Lets wait for the Geth PR. Thanks.

RazorClient pushed a commit to RazorClient/execution-specs that referenced this pull request Feb 11, 2026
* fix: use engine_newPayloadV5 for Amsterdam

* fix: update exception mapper for geth

* fix: We should update BAL hash in fixture for invalid tests

* Clean up extraneous BAL exceptions

* refactor: use fork check for BAL in engine payload
RazorClient pushed a commit to RazorClient/execution-specs that referenced this pull request Feb 11, 2026
* fix: use engine_newPayloadV5 for Amsterdam

* fix: update exception mapper for geth

* fix: We should update BAL hash in fixture for invalid tests

* Clean up extraneous BAL exceptions

* refactor: use fork check for BAL in engine payload
RazorClient pushed a commit to RazorClient/execution-specs that referenced this pull request Feb 11, 2026
* fix: use engine_newPayloadV5 for Amsterdam

* fix: update exception mapper for geth

* fix: We should update BAL hash in fixture for invalid tests

* Clean up extraneous BAL exceptions

* refactor: use fork check for BAL in engine payload
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-test-client-clis Area: execution_testing.client_clis C-feat Category: an improvement or new feature P-high

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants