Skip to content

Conversation

@ericnordelo
Copy link
Member

Fixes #1470

@ericnordelo ericnordelo requested review from Copilot and immrsd July 15, 2025 22:09
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces tooling to benchmark and track Cairo contract bytecode and class sizes.

  • Adds benchmark_diff.py to compare current and previous benchmarks with CLI color diff and markdown table outputs.
  • Adds benchmark.py to generate size metrics (felts and bytes) as JSON or formatted CLI output.
  • Updates Scarb.toml and CI workflows to build contracts, record baseline sizes, and post diffs on pull requests.

Reviewed Changes

Copilot reviewed 6 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
scripts/benchmark_diff.py New script for diffing benchmark outputs and printing colored/markdown summaries
scripts/benchmark.py New script for collecting contract size metrics
packages/test_common/Scarb.toml Enabled CASM compilation for benchmarking
benches/contract_sizes.json Initialized empty baseline for contract sizes
.github/workflows/benches/update-benches.yml Workflow to update baseline benchmarks after merges
.github/workflows/benches/show-diff.yml Workflow to comment benchmark diffs on new or updated PRs

TARGET_DIR = "target/release"


def try_get_name(filename):
Copy link

Copilot AI Jul 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The try_get_name function is duplicated in benchmark_diff.py; consider extracting it to a shared utility module to avoid code duplication.

Copilot uses AI. Check for mistakes.
@OpenZeppelin OpenZeppelin deleted a comment from github-actions bot Jul 15, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 15, 2025

🧪 Cairo Contract Size Benchmark Diff

🧪 BYTECODE SIZE (felts)

Contract Old New Δ Note
Attacker 138 +138 ✅ NEW
DualCaseAccessControlDefaultAdminRulesMock 8341 +8341 ✅ NEW
DualCaseAccessControlMock 4532 +4532 ✅ NEW
DualCaseAccountMock 2867 +2867 ✅ NEW
DualCaseERC1155Mock 9455 +9455 ✅ NEW
DualCaseERC1155ReceiverMock 2119 +2119 ✅ NEW
DualCaseERC20Mock 6203 +6203 ✅ NEW
DualCaseERC20PermitMock 7834 +7834 ✅ NEW
DualCaseERC721Mock 12768 +12768 ✅ NEW
DualCaseERC721ReceiverMock 1209 +1209 ✅ NEW
DualCaseEthAccountMock 4867 +4867 ✅ NEW
DualCaseOwnableMock 1358 +1358 ✅ NEW
DualCaseTwoStepOwnableMock 2361 +2361 ✅ NEW
ERC20BlockNumberVotesMock 12958 +12958 ✅ NEW
ERC20OptionalTransferPanicMock 5919 +5919 ✅ NEW
ERC20ReentrantMock 8450 +8450 ✅ NEW
ERC20TimestampVotesMock 12958 +12958 ✅ NEW
ERC2981AccessControlMock 6002 +6002 ✅ NEW
ERC2981Mock 2080 +2080 ✅ NEW
ERC2981OwnableMock 4604 +4604 ✅ NEW
ERC4626AssetsFeesMock 17492 +17492 ✅ NEW
ERC4626ExternalVaultMock 13546 +13546 ✅ NEW
ERC4626LimitsMock 13552 +13552 ✅ NEW
ERC4626Mock 12960 +12960 ✅ NEW
ERC4626MockWithHooks 12934 +12934 ✅ NEW
ERC4626OffsetMock 13464 +13464 ✅ NEW
ERC4626SharesFeesMock 17131 +17131 ✅ NEW
ERC721BlockNumberVotesMock 18786 +18786 ✅ NEW
ERC721EnumerableMock 10510 +10510 ✅ NEW
ERC721TimestampVotesMock 18786 +18786 ✅ NEW
GovernorMock 18513 +18513 ✅ NEW
GovernorQuorumFractionMock 23009 +23009 ✅ NEW
GovernorTimelockedMock 23507 +23507 ✅ NEW
InitializableMock 145 +145 ✅ NEW
LegacyERC20VotesMock 9272 +9272 ✅ NEW
LinearVestingMock 5198 +5198 ✅ NEW
MockContract 323 +323 ✅ NEW
MockTrace 4444 +4444 ✅ NEW
MultisigTargetMock 349 +349 ✅ NEW
MultisigWalletMock 10458 +10458 ✅ NEW
NonImplementingMock 96 +96 ✅ NEW
NoncesMock 215 +215 ✅ NEW
PausableMock 145 +145 ✅ NEW
ReentrancyMock 1473 +1473 ✅ NEW
SRC5Mock 235 +235 ✅ NEW
SRC9AccountMock 5577 +5577 ✅ NEW
SimpleMock 459 +459 ✅ NEW
SnakeAccountMock 3198 +3198 ✅ NEW
SnakeERC1155Mock 7667 +7667 ✅ NEW
SnakeERC1155MockWithHooks 7911 +7911 ✅ NEW
SnakeERC20Mock 5550 +5550 ✅ NEW
SnakeERC20MockWithHooks 5842 +5842 ✅ NEW
SnakeERC721Mock 10685 +10685 ✅ NEW
SnakeERC721MockWithHooks 10850 +10850 ✅ NEW
SnakeEthAccountMock 5221 +5221 ✅ NEW
StepsVestingMock 5408 +5408 ✅ NEW
TimelockAttackerMock 777 +777 ✅ NEW
TimelockControllerMock 10502 +10502 ✅ NEW
UpgradesV1 1074 +1074 ✅ NEW
UpgradesV2 1221 +1221 ✅ NEW

🧪 SIERRA CONTRACT CLASS SIZE (bytes)

Contract Old New Δ Note
Attacker 6085 +6085 ✅ NEW
DualCaseAccessControlDefaultAdminRulesMock 171622 +171622 ✅ NEW
DualCaseAccessControlMock 96706 +96706 ✅ NEW
DualCaseAccountMock 66378 +66378 ✅ NEW
DualCaseERC1155Mock 217375 +217375 ✅ NEW
DualCaseERC1155ReceiverMock 38147 +38147 ✅ NEW
DualCaseERC20Mock 128806 +128806 ✅ NEW
DualCaseERC20PermitMock 177200 +177200 ✅ NEW
DualCaseERC721Mock 274742 +274742 ✅ NEW
DualCaseERC721ReceiverMock 24342 +24342 ✅ NEW
DualCaseEthAccountMock 91233 +91233 ✅ NEW
DualCaseOwnableMock 30723 +30723 ✅ NEW
DualCaseTwoStepOwnableMock 47987 +47987 ✅ NEW
ERC20BlockNumberVotesMock 291969 +291969 ✅ NEW
ERC20OptionalTransferPanicMock 124793 +124793 ✅ NEW
ERC20ReentrantMock 189745 +189745 ✅ NEW
ERC20TimestampVotesMock 291932 +291932 ✅ NEW
ERC2981AccessControlMock 123006 +123006 ✅ NEW
ERC2981Mock 40853 +40853 ✅ NEW
ERC2981OwnableMock 93132 +93132 ✅ NEW
ERC4626AssetsFeesMock 379741 +379741 ✅ NEW
ERC4626ExternalVaultMock 293564 +293564 ✅ NEW
ERC4626LimitsMock 295692 +295692 ✅ NEW
ERC4626Mock 281128 +281128 ✅ NEW
ERC4626MockWithHooks 277901 +277901 ✅ NEW
ERC4626OffsetMock 293332 +293332 ✅ NEW
ERC4626SharesFeesMock 369762 +369762 ✅ NEW
ERC721BlockNumberVotesMock 432502 +432502 ✅ NEW
ERC721EnumerableMock 226427 +226427 ✅ NEW
ERC721TimestampVotesMock 432493 +432493 ✅ NEW
GovernorMock 457408 +457408 ✅ NEW
GovernorQuorumFractionMock 556705 +556705 ✅ NEW
GovernorTimelockedMock 579123 +579123 ✅ NEW
InitializableMock 6746 +6746 ✅ NEW
LegacyERC20VotesMock 210979 +210979 ✅ NEW
LinearVestingMock 103157 +103157 ✅ NEW
MockContract 8400 +8400 ✅ NEW
MockTrace 87974 +87974 ✅ NEW
MultisigTargetMock 9120 +9120 ✅ NEW
MultisigWalletMock 228078 +228078 ✅ NEW
NonImplementingMock 4165 +4165 ✅ NEW
NoncesMock 7019 +7019 ✅ NEW
PausableMock 7304 +7304 ✅ NEW
ReentrancyMock 32174 +32174 ✅ NEW
SRC5Mock 8385 +8385 ✅ NEW
SRC9AccountMock 129558 +129558 ✅ NEW
SimpleMock 11907 +11907 ✅ NEW
SnakeAccountMock 74308 +74308 ✅ NEW
SnakeERC1155Mock 181167 +181167 ✅ NEW
SnakeERC1155MockWithHooks 190917 +190917 ✅ NEW
SnakeERC20Mock 116477 +116477 ✅ NEW
SnakeERC20MockWithHooks 142850 +142850 ✅ NEW
SnakeERC721Mock 232720 +232720 ✅ NEW
SnakeERC721MockWithHooks 241047 +241047 ✅ NEW
SnakeEthAccountMock 100099 +100099 ✅ NEW
StepsVestingMock 107175 +107175 ✅ NEW
TimelockAttackerMock 22711 +22711 ✅ NEW
TimelockControllerMock 250722 +250722 ✅ NEW
UpgradesV1 25813 +25813 ✅ NEW
UpgradesV2 28303 +28303 ✅ NEW

This comment was generated automatically from benchmark diffs.

@codecov
Copy link

codecov bot commented Jul 15, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.40%. Comparing base (a03e01e) to head (9fa3442).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1471   +/-   ##
=======================================
  Coverage   92.40%   92.40%           
=======================================
  Files          82       82           
  Lines        2265     2265           
=======================================
  Hits         2093     2093           
  Misses        172      172           

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a03e01e...9fa3442. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ericnordelo ericnordelo reopened this Jul 16, 2025
Copy link
Collaborator

@immrsd immrsd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, just left a couple of minor suggestions

@ericnordelo ericnordelo requested a review from immrsd August 18, 2025 10:04
Copy link
Collaborator

@immrsd immrsd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job, LGTM!

@ericnordelo ericnordelo merged commit d99887d into main Aug 18, 2025
11 checks passed
@ericnordelo ericnordelo deleted the feat/benchmark-contract-size branch August 18, 2025 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmark contract sizes

3 participants