Skip to content

feat(bls): add benchmarks for napi bls batching apis#8979

Draft
spiral-ladder wants to merge 3 commits intocayman/bls-verifierfrom
bing/bls-batching-benchmarks
Draft

feat(bls): add benchmarks for napi bls batching apis#8979
spiral-ladder wants to merge 3 commits intocayman/bls-verifierfrom
bing/bls-batching-benchmarks

Conversation

@spiral-ladder
Copy link
Contributor

Add benchmarks for bls batching API.

Local run:

  BLS ops
    ✔ BLS verifyMultipleSignatures 3 - blst                               594.6279 ops/s    1.681724 ms/op        -        121 runs  0.706 s
    ✔ BLS verifyMultipleSignatures 8 - blst                               275.5645 ops/s    3.628914 ms/op        -         56 runs  0.713 s
    ✔ BLS verifyMultipleSignatures 32 - blst                              75.50866 ops/s    13.24351 ms/op        -         17 runs  0.727 s
    ✔ BLS verifyMultipleSignatures 64 - blst                              38.50739 ops/s    25.96904 ms/op        -         13 runs  0.843 s
    ✔ BLS verifyMultipleSignatures 128 - blst                             19.66558 ops/s    50.85027 ms/op        -         11 runs   1.07 s
    ✔ BLS verifyMultipleSignatures - same message - 3 - blst              1052.640 ops/s    949.9920 us/op        -        212 runs  0.706 s
    ✔ BLS verifyMultipleSignatures - same message - 8 - blst              921.3074 ops/s    1.085414 ms/op        -        186 runs  0.705 s
    ✔ BLS verifyMultipleSignatures - same message - 32 - blst             587.3160 ops/s    1.702661 ms/op        -        119 runs  0.727 s
    ✔ BLS verifyMultipleSignatures - same message - 64 - blst             398.4102 ops/s    2.509976 ms/op        -         81 runs  0.717 s
    ✔ BLS verifyMultipleSignatures - same message - 128 - blst            240.8998 ops/s    4.151104 ms/op        -         49 runs  0.736 s

  BLS napi batching bindings
    ✔ blsBatch.verify(indexed) 3                                          547.1447 ops/s    1.827670 ms/op        -         77 runs  0.949 s
    ✔ blsBatch.verify(indexed) 8                                          247.2610 ops/s    4.044309 ms/op        -         33 runs   1.00 s
    ✔ blsBatch.verify(indexed) 32                                         68.42452 ops/s    14.61464 ms/op        -         11 runs   1.08 s
    ✔ blsBatch.verify(indexed) 64                                         34.28697 ops/s    29.16560 ms/op        -         10 runs   1.33 s
    ✔ blsBatch.verify(indexed) 128                                        17.56316 ops/s    56.93736 ms/op        -         11 runs   1.90 s
    ✔ blsBatch.verify(single) 3                                           500.5496 ops/s    1.997804 ms/op        -         66 runs  0.994 s
    ✔ blsBatch.verify(single) 8                                           223.3085 ops/s    4.478110 ms/op        -         29 runs   1.06 s
    ✔ blsBatch.verify(single) 32                                          62.10694 ops/s    16.10126 ms/op        -         13 runs   1.24 s
    ✔ blsBatch.verify(single) 64                                          31.24975 ops/s    32.00025 ms/op        -         11 runs   1.51 s
    ✔ blsBatch.verify(single) 128                                         15.72774 ops/s    63.58192 ms/op        -         10 runs   2.01 s
    ✔ blsBatch.asyncVerify(indexed) 3                                     547.1971 ops/s    1.827495 ms/op        -         77 runs  0.828 s
    ✔ blsBatch.asyncVerify(indexed) 8                                     247.2244 ops/s    4.044909 ms/op        -         32 runs   1.00 s
    ✔ blsBatch.asyncVerify(indexed) 32                                    68.08356 ops/s    14.68783 ms/op        -         11 runs   1.11 s
    ✔ blsBatch.asyncVerify(indexed) 64                                    35.13980 ops/s    28.45776 ms/op        -         10 runs   1.32 s
    ✔ blsBatch.asyncVerify(indexed) 128                                   17.62230 ops/s    56.74628 ms/op        -         11 runs   1.88 s
    ✔ blsBatch.asyncVerify(single) 3                                      2795.834 ops/s    357.6750 us/op        -        134 runs   1.71 s
    ✔ blsBatch.asyncVerify(single) 8                                      1076.646 ops/s    928.8100 us/op        -         51 runs   2.35 s
    ✔ blsBatch.asyncVerify(single) 32                                     271.1158 ops/s    3.688461 ms/op        -         15 runs   2.42 s
    ✔ blsBatch.asyncVerify(single) 64                                     138.8553 ops/s    7.201740 ms/op        -         13 runs   2.59 s
    ✔ blsBatch.asyncVerify(single) 128                                    67.72748 ops/s    14.76506 ms/op        -         11 runs   2.92 s
    ✔ blsBatch.asyncVerifySameMessage 3                                   836.4876 ops/s    1.195475 ms/op        -         98 runs   1.07 s
    ✔ blsBatch.asyncVerifySameMessage 8                                   579.9115 ops/s    1.724401 ms/op        -         51 runs   1.40 s
    ✔ blsBatch.asyncVerifySameMessage 32                                  260.1900 ops/s    3.843345 ms/op        -         17 runs   1.96 s
    ✔ blsBatch.asyncVerifySameMessage 64                                  159.4168 ops/s    6.272863 ms/op        -         11 runs   2.27 s
    ✔ blsBatch.asyncVerifySameMessage 128                                 87.91540 ops/s    11.37457 ms/op        -         14 runs   2.87 s

@spiral-ladder spiral-ladder self-assigned this Mar 4, 2026
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces new performance benchmarks for the BLS batching NAPI bindings. The primary goal is to measure and evaluate the efficiency of batch BLS signature verification operations, providing crucial data for performance optimization and understanding the overheads associated with different verification strategies.

Highlights

  • New Benchmarks: Added a comprehensive suite of benchmarks for the BLS batching NAPI bindings, covering various verification scenarios including indexed, single, and same-message verification for both synchronous and asynchronous operations.
  • Performance Evaluation: The new benchmarks allow for performance evaluation of the blsBatch.verify and blsBatch.asyncVerify APIs across different signature counts (3, 8, 32, 64, 128), providing insights into their efficiency.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • packages/beacon-node/test/perf/bls/bls.test.ts
    • Added a new test suite for 'BLS napi batching bindings'.
    • Implemented benchmarks for blsBatch.verify with indexed and single public key sets.
    • Implemented benchmarks for blsBatch.asyncVerify with indexed, single, and same-message public key sets.
    • Imported blsBatch from @chainsafe/lodestar-z/bls-batch.
    • Imported pubkeyCache from @chainsafe/lodestar-z/pubkeys.
    • Updated the import path for linspace from .js to .ts.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces benchmarks for the BLS batching NAPI APIs, which is a valuable addition for performance monitoring. The new benchmarks cover various scenarios including indexed, single, and same-message verifications, both synchronously and asynchronously. The changes are well-structured and integrate smoothly with the existing benchmark suite.


describe("BLS napi batching bindings", () => {
const maxKeys = 256;
blsBatch.init(40_000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The number 40_000 used in blsBatch.init appears to be a magic number. It would improve readability and maintainability to define this value as a named constant, clearly indicating its purpose (e.g., MAX_BATCH_SIZE or INITIAL_CAPACITY).

Suggested change
blsBatch.init(40_000);
const maxKeys = 256;
const BLS_BATCH_INITIAL_CAPACITY = 40_000;
blsBatch.init(BLS_BATCH_INITIAL_CAPACITY);

beforeEach: () => linspace(0, count - 1).map((i) => getIndexedSet(i)),
fn: (sets) => {
const isValid = blsBatch.verify(blsBatch.indexed, sets);
if (!isValid) throw Error("Invalid");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The error message "Invalid" is very generic. If a verification fails in a benchmark, a more descriptive error message would be beneficial for debugging. Consider including details like the id of the benchmark or the count of items being verified.

        const isValid = blsBatch.verify(blsBatch.indexed, sets);
        if (!isValid) throw Error(`Invalid verification for blsBatch.verify(indexed) with count ${count}`);

beforeEach: () => linspace(0, count - 1).map((i) => getSingleSet(i)),
fn: (sets) => {
const isValid = blsBatch.verify(blsBatch.single, sets);
if (!isValid) throw Error("Invalid");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The error message "Invalid" is very generic. If a verification fails in a benchmark, a more descriptive error message would be beneficial for debugging. Consider including details like the id of the benchmark or the count of items being verified.

        const isValid = blsBatch.verify(blsBatch.single, sets);
        if (!isValid) throw Error(`Invalid verification for blsBatch.verify(single) with count ${count}`);

beforeEach: () => linspace(0, count - 1).map((i) => getIndexedSet(i)),
fn: async (sets) => {
const isValid = await blsBatch.asyncVerify(blsBatch.indexed, sets);
if (!isValid) throw Error("Invalid");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The error message "Invalid" is very generic. If a verification fails in a benchmark, a more descriptive error message would be beneficial for debugging. Consider including details like the id of the benchmark or the count of items being verified.

        const isValid = await blsBatch.asyncVerify(blsBatch.indexed, sets);
        if (!isValid) throw Error(`Invalid verification for blsBatch.asyncVerify(indexed) with count ${count}`);

beforeEach: () => linspace(0, count - 1).map((i) => getSingleSet(i)),
fn: (sets) => {
const isValid = blsBatch.asyncVerify(blsBatch.single, sets);
if (!isValid) throw Error("Invalid");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The error message "Invalid" is very generic. If a verification fails in a benchmark, a more descriptive error message would be beneficial for debugging. Consider including details like the id of the benchmark or the count of items being verified.

        const isValid = blsBatch.asyncVerify(blsBatch.single, sets);
        if (!isValid) throw Error(`Invalid verification for blsBatch.asyncVerify(single) with count ${count}`);

beforeEach: () => linspace(0, count - 1).map((i) => getSameMessageSet(i)),
fn: async (sets) => {
const isValid = await blsBatch.asyncVerifySameMessage(sets, seedMessage);
if (!isValid) throw Error("Invalid");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The error message "Invalid" is very generic. If a verification fails in a benchmark, a more descriptive error message would be beneficial for debugging. Consider including details like the id of the benchmark or the count of items being verified.

        const isValid = await blsBatch.asyncVerifySameMessage(sets, seedMessage);
        if (!isValid) throw Error(`Invalid verification for blsBatch.asyncVerifySameMessage with count ${count}`);

@spiral-ladder spiral-ladder force-pushed the bing/bls-batching-benchmarks branch from 1ca773f to 928f48d Compare March 5, 2026 05:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant