fix(merkle): use checkpoint root when local tree is ahead of contract

[niconiconi]

Problem

AmortizedMerkleTree commits to the main tree in 32-element batches. When totalElements=64, the contract has committed merkleRoots[1] (after the first 32 elements). But the local tree keeps merging all received leaves eagerly — so after ingesting all 64 leaves, tree.root represents the root after both batches, which would become merkleRoots[2] only when element 65 triggers the next commit.

pickMerkleRootIndex searches merkleRoots[0..9] for the proof's Merkle root and fails:

Error: Remote merkle root not found on-chain

Fix

src/merkle/merkleEngine.ts

1. Checkpoint storage (ingestEntryMemos): after each 32-element batch merge, store a checkpoint-{N} node with the committed root at exactly N merged elements:

   const checkpointNode = { id: `checkpoint-${state.mergedElements}`, hash: state.root, level: -1, ... };

2. Checkpoint lookup (getProofByCids): when tree.totalElements > contractTreeElements, look up checkpoint-{contractTreeElements} and use its hash as the proof's merkle_root instead of the ahead-of-contract tree.root:

   if (tree.totalElements > contractTreeElements && contractTreeElements > 0) {
     const checkpoint = await this.storage!.getMerkleNode!(chainId, `checkpoint-${contractTreeElements}`);
     if (checkpoint) effectiveRoot = checkpoint.hash;
   }

src/ops/ops.ts: pass payIncludesFee through prepareTransfer.

Verified

Tested on Sepolia testnet with totalElements=64 (merkleRoots[1] active). Transfer-to-self loop runs continuously without errors.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical synchronization problem between the local Merkle tree state and the on-chain contract's committed Merkle root. By implementing a checkpointing system, the application can now correctly generate proofs even when its local tree has processed more elements than the contract has officially acknowledged. Additionally, a minor enhancement was made to the transfer preparation logic to better handle fee inclusion.

Highlights

• Merkle Tree Root Mismatch Resolution: Addressed an issue where the local Merkle tree's root could get ahead of the contract's committed root, leading to proof generation failures. This was resolved by implementing a checkpointing mechanism.
• Checkpoint Root Storage: Introduced the storage of 'checkpoint' Merkle roots after each 32-element batch merge in MerkleEngine, keyed by the exact number of merged elements at that point.
• Dynamic Merkle Root Selection for Proofs: Modified the getProofByCids function to dynamically select the correct Merkle root for proof generation. If the local tree is ahead of the contract, it now retrieves and uses the appropriate checkpoint root instead of the current local tree root.
• Transfer Fee Handling: Updated the prepareTransfer function in ops.ts to accept and pass through a payIncludesFee parameter, enhancing flexibility in transfer operations.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• src/merkle/merkleEngine.ts
  • Added logic to store checkpoint Merkle roots after each 32-element batch merge, ensuring historical roots are accessible.
  • Modified the getProofByCids method to retrieve and utilize the correct checkpoint root when the local tree's total elements exceed the contract's committed elements, preventing proof failures.
• src/ops/ops.ts
  • Updated the prepareTransfer function signature to include an optional payIncludesFee boolean parameter.
  • Passed the new payIncludesFee parameter to the internal transfer planning logic.

Activity

• The changes were verified on the Sepolia testnet with totalElements=64, confirming that transfer-to-self loops run continuously without errors, indicating the fix is effective.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request effectively addresses the issue where the local Merkle tree state can be ahead of the on-chain contract state, leading to proof validation failures. The proposed solution of creating and using checkpoints for historical Merkle roots is well-implemented. The changes in merkleEngine.ts correctly store these checkpoints during leaf ingestion and look them up when generating proofs.

I have one suggestion in src/merkle/merkleEngine.ts to make the error handling more robust when a checkpoint is expected but not found.

The change in src/ops/ops.ts to pass payIncludesFee through prepareTransfer is a straightforward and correct addition.

[gemini-code-assist]

The logic to use a checkpoint root is correct. However, if the checkpoint is not found, the current implementation silently falls back to using tree.root. This will likely cause the original Remote merkle root not found on-chain error downstream, making debugging difficult.

Since a checkpoint is expected to exist if tree.totalElements > contractTreeElements, its absence indicates a data inconsistency. It would be more robust to throw an error in this case to provide a clear signal that the local Merkle tree state is corrupted.

            if (tree.totalElements > contractTreeElements && contractTreeElements > 0) {
              const checkpoint = await this.storage!.getMerkleNode!(input.chainId, `checkpoint-${contractTreeElements}`);
              if (!checkpoint) {
                throw new SdkError('MERKLE', `Local merkle checkpoint not found for ${contractTreeElements} elements`, {
                  chainId: input.chainId,
                  contractTreeElements,
                });
              }
              effectiveRoot = checkpoint.hash;
            }