[feat] Improve eth_estimateGas performance further using optimistic gas based on the gasUsed + gasRefunded in the first run

[Unique-Divine]

Inspiration: Optimistic Gas Limit Pre-Check

Geth implementation (lines 143-159 in gasestimator.go):

// There's a fairly high chance for the transaction to execute successfully
// with gasLimit set to the first execution's usedGas + gasRefund. Explicitly
// check that gas amount and use as a limit for the binary search.
optimisticGasLimit := (result.UsedGas + result.RefundedGas + params.CallStipend) * 64 / 63
if optimisticGasLimit < hi {
    failed, _, err = execute(ctx, call, opts, optimisticGasLimit)
    if err != nil {
        log.Error("Execution error in estimate gas", "err", err)
        return 0, nil, err
    }
    if failed {
        lo = optimisticGasLimit
    } else {
        hi = optimisticGasLimit
    }
}

The Ask

Implement analogous behavior for Nibiru's EstimateGas keeper method, which
powers the RPC method.

[Unique-Divine]

Rationale: Why an Additional Optimistic Gas Limit Pre‑Check Is Not Needed on Nibiru

This note explains why Nibiru's current EstimateGas implementation already achieves the performance goals behind Geth's "optimistic gas limit" heuristic from issue #2426, and why adding an extra optimistic pre‑check would provide only marginal benefit in practice.

1. Background: What the issue requests

Geth's gas estimator contains an "optimistic" step after the first successful execution:

• Run the transaction once with a high gas limit.

• From that run, compute an optimistic gas limit:

  $$ \text{optimisticGasLimit} = (\text{UsedGas} + \text{RefundedGas} + \text{CallStipend}) \cdot \frac{64}{63} $$

• If this optimistic limit is below the current upper bound (hi), execute the transaction again with gasLimit = optimisticGasLimit to quickly shrink the search interval.

• Then do a binary search between the updated [lo, hi] to find a minimal working gas limit.

Issue #2426 proposes adding analogous behavior to Nibiru's EstimateGas keeper method (which powers the eth_estimateGas JSON‑RPC).

2. Nibiru's existing EstimateGas flow

On Nibiru, eth_estimateGas is implemented via the keeper's EstimateGas method in x/evm/evmstate/grpc_query.go. Its core behavior is:

1. Derive a conservative upper bound hi

   // Start from block max gas
   params := rootCtx.ConsensusParams()
   if params != nil && params.Block != nil && params.Block.MaxGas > 0 {
       hi = uint64(params.Block.MaxGas)
   } else {
       // Fallback to gasCap if block params not available
       hi = req.GasCap
   }
   
   // Override with user-provided gas limit if valid
   if args.Gas != nil && uint64(*args.Gas) >= gethparams.TxGas {
       hi = uint64(*args.Gas)
   }
   
   // Cap by gasCap from the request
   if req.GasCap != 0 && hi > req.GasCap {
       hi = req.GasCap
   }

   This ensures:

   • hi never exceeds the block's MaxGas (so we never estimate a tx that cannot fit in a block).
   • A user‑supplied gas (when valid) is honored as the first upper bound.
   • gasCap acts as a final safety cap.

2. Define an executable(gas) probe with robust panic handling

   The estimator defines a reusable function:

   executable := func(gas uint64) (vmError bool, rsp *evm.MsgEthereumTxResponse, err error) {
       defer func() {
           // Classify panics: distinguish out-of-gas from unexpected panics
           panicInfo := recover()
           if panicInfo != nil {
               oog, perr := evm.ParseOOGPanic(...)
               if oog {
                   vmError, rsp, err = true, nil, nil
                   return
               } else if perr != nil {
                   err = perr // abort search on unexpected panic
                   return
               }
           }
       }()
   
       // Rebuild the core.Message with GasLimit = gas
       evmMsg = core.Message{
           To:               evmMsg.To,
           From:             evmMsg.From,
           Nonce:            evmMsg.Nonce,
           Value:            evmMsg.Value,
           GasLimit:         gas,
           GasPrice:         evmMsg.GasPrice,
           GasFeeCap:        evmMsg.GasFeeCap,
           GasTipCap:        evmMsg.GasTipCap,
           Data:             evmMsg.Data,
           AccessList:       evmMsg.AccessList,
           BlobGasFeeCap:    evmMsg.BlobGasFeeCap,
           BlobHashes:       evmMsg.BlobHashes,
           SkipNonceChecks:  evmMsg.SkipNonceChecks,
           SkipFromEOACheck: evmMsg.SkipFromEOACheck,
       }
   
       // New SDB on a context with:
       // - infinite gas meter bounded by gas
       // - zero KV/transient KV gas costs
       sdb := k.NewSDB(rootCtx, k.TxConfig(rootCtx, rootCtx.EvmTxHash()))
       sdb.SetCtx(
           sdb.Ctx().
               WithGasMeter(eth.NewInfiniteGasMeterWithLimit(evmMsg.GasLimit)).
               WithKVGasConfig(storetypes.GasConfig{}).
               WithTransientKVGasConfig(storetypes.GasConfig{}),
       )
   
       // Ensure an account exists with a realistic nonce
       acct := k.GetAccount(sdb.Ctx(), evmMsg.From)
       if acct == nil {
           acc := k.accountKeeper.NewAccountWithAddress(sdb.Ctx(), from[:])
           k.accountKeeper.SetAccount(sdb.Ctx(), acc)
           acct = NewEmptyAccount()
       }
       acct.Nonce = nonce + 1
       if err := k.SetAccount(sdb.Ctx(), from, *acct); err != nil {
           return true, nil, err
       }
   
       // Execute without committing state
       evmObj := k.NewEVM(sdb.Ctx(), evmMsg, evmCfg, nil /* tracer */, sdb)
       rsp, err = k.ApplyEvmMsg(evmMsg, evmObj, false /* commit */)
       if err != nil {
           if strings.Contains(err.Error(), core.ErrIntrinsicGas.Error()) {
               return true, nil, nil // special case: increase gas
           }
           return true, nil, fmt.Errorf("error applying EVM message to StateDB: %w", err)
       }
       return len(rsp.VmError) > 0, rsp, nil
   }

   Key properties:

   • All probes run on a fresh SDB with txn‑style gas metering (infinite gas meter capped by gas, zero KV gas costs).
   • Out-of-gas panics are converted into vmError=true so the search can safely adjust lo/hi.
   • Any unexpected panic or consensus error aborts the search with a descriptive error.

3. Base case: single execution at hi to validate and set a smart lower bound

   failed, result, err := executable(hi)
   if err != nil {
       return nil, fmt.Errorf("eth call exec error: %w", err)
   }
   
   // If base case fails for non-gas reasons, bubble the error
   if failed {
       if result != nil && result.VmError != "" && result.VmError != vm.ErrOutOfGas.Error() {
           if result.VmError == vm.ErrExecutionReverted.Error() {
               return nil, fmt.Errorf("estimate gas VMError: %w", evm.NewRevertError(result.Ret))
           }
           return nil, fmt.Errorf("estimate gas VMError: %s", result.VmError)
       }
       return nil, fmt.Errorf("gas required exceeds allowance (%d)", hi)
   }
   
   // Smart lower bound from actual gas used
   if result.GasUsed > 0 {
       lo = result.GasUsed - 1
   } else {
       lo = 0
   }

   This step:

   • Guarantees feasibility at hi (or returns an error if the tx cannot succeed at all under the constraints).
   • Uses the actual GasUsed from a successful execution to set a tight lower bound lo ≈ GasUsed − 1, which is typically very close to the true minimal required gas.

   Note: in ApplyEvmMsg, the refund processing is already folded into GasUsed:

   gasUsed := msg.GasLimit - gasRemaining
   refundAmount := gasToRefund(sdb.GetRefund(), gasUsed)
   gasRemaining += refundAmount
   gasUsed -= refundAmount
   evmResp = &evm.MsgEthereumTxResponse{
       GasUsed: gasUsed,
       ...
   }

   So GasUsed is effectively "usedGas − refundedGas" already. There is no separate RefundedGas counter exposed, because the final GasUsed returned from the EVM is itself the post‑refund amount.

4. Biased, tolerance‑based binary search around lo and hi

   After the base case, Nibiru calls a custom BinSearch helper in x/evm/msg.go:

   estimateTolerance := evm.GasEstimateErrorRatioTolerance
   if rootCtx.Value(evm.CtxKeyGasEstimateZeroTolerance) == true {
       estimateTolerance = 0.00
   }
   hi, err = evm.BinSearch(lo, hi, executable, estimateTolerance)

   The implementation is:

   // GasEstimateErrorRatioTolerance is the amount of overestimation eth_estimateGas is
   // allowed to produce in order to speed up calculations.
   const GasEstimateErrorRatioTolerance = 0.015
   
   // BinSearch executes the binary search and hone in on an executable gas limit
   func BinSearch(
       lo uint64,
       hi uint64,
       executable func(uint64) (bool, *MsgEthereumTxResponse, error),
       estimateTolerance float64,
   ) (uint64, error) {
       for lo+1 < hi {
           // It is a bit pointless to return a perfect estimation, as changing
           // network conditions require the caller to bump it up anyway. Since
           // wallets tend to use 20-25% bump, allowing a small approximation
           // error is fine (as long as it's upwards).
           if float64(hi-lo)/float64(hi) < estimateTolerance {
               break
           }
   
           mid := (hi + lo) / 2
           if mid > lo*2 {
               // Skew the binary search toward lower gas values. Since most transactions
               // only need slightly more gas than they actually consume (lo ≈ gasUsed),
               // testing values closer to lo converges faster than standard bisection.
               mid = lo * 2
           }
           failed, _, err := executable(mid)
           if err != nil {
               return 0, err
           }
           if failed {
               lo = mid
           } else {
               hi = mid
           }
       }
       return hi, nil
   }

   Important characteristics:

   • Tolerance‑based stopping: as soon as the interval width satisfies

   $$
   \frac{hi - lo}{hi} < \text{estimateTolerance},
   $$

   the search terminates. With the default estimateTolerance = 0.015, this yields at most ~1.5% overestimation, which is negligible compared to the 20–25% safety bump that most wallets add on top of eth_estimateGas.

   • Bias toward lower gas: the mid > lo*2 adjustment ensures early probes are focused near lo, not the geometrical center of [lo, hi]. Since lo was derived from a successful run's GasUsed, in typical cases we have lo ≈ true minimum. This greatly reduces the number of extra EVM executions needed to converge.

   • Well‑behaved failure handling: any consensus or unexpected error aborts the search; only VM‑level failures (vmError including out‑of‑gas) drive lo upward in the usual way.

3. Comparison with Geth's optimistic pre‑check

We can now compare:

• Geth's heuristic (high level):

  1. Choose an initial [lo, hi] and execute once at a high gas limit.
  2. From that run, compute optimisticGasLimit = (usedGas + refundedGas + CallStipend) * 64 / 63.
  3. If optimisticGasLimit < hi, execute again at optimisticGasLimit to aggressively shrink the search interval in one shot.
  4. Perform binary search between the final [lo, hi].

• Nibiru's current approach:

  1. Execute once at hi, where hi is derived from block MaxGas, user gas, and gasCap.

  2. Set lo from the observed GasUsed:

     • lo = GasUsed - 1 if GasUsed > 0;
     • lo = 0 otherwise.

     Because GasUsed is already post‑refund, this is directly tied to the effective minimum gas required.

  3. Run BinSearch(lo, hi, executable, tolerance):

     • stops when (hi - lo) / hi < tolerance;
     • biases midpoints towards 2 * lo to quickly test values near the lower bound.

Algorithmically:

• Both designs use a first successful run at a high gas limit to derive information about the minimal required gas.
• Geth turns that information into a single extra "optimistic" probe near usedGas + refundedGas, then binary searches.
• Nibiru turns that information into an aggressively tight lo plus a skewed, tolerance‑based search that focuses probes around lo.

This means that in typical scenarios where GasUsed is not extremely close to hi:

• Geth does: base run at hi → optimistic run near usedGas + refundedGas → a few binary search steps.
• Nibiru does: base run at hi → set lo ≈ GasUsed - 1 → biased binary search that immediately probes very near lo and stops once within 1.5%.

In terms of the number of EVM executions, the difference between:

• "base run + optimistic probe + 2–3 binary search steps" and
• "base run + 3–4 biased binary search steps from a lo that is already close to minimal"

is at most one or two extra calls to executable in common cases. Meanwhile, Nibiru's estimator is already:

• ignoring KV store gas,
• running on an infinite gas meter with a hard cap equal to the probe gas limit,
• and short‑circuiting on consensus errors.

So the marginal effect on wall‑clock latency is small.

4. Why an additional optimistic pre‑check is only marginally useful

Given the existing design, an extra Geth‑style optimistic pre‑check would have, at best, a small impact:

1. lo is already very tight

   The first run at hi yields a successful GasUsed, and Nibiru sets:

   lo = max(result.GasUsed - 1, 0)

   Since GasUsed is post‑refund and derived from a successful run, lo is already a near‑minimal bound. There is limited room for a separate optimistic heuristic to improve on that lower bound.

2. Binary search is biased toward lo

   The mid > lo*2 rule ensures that, when lo is much smaller than hi, the search rapidly tests gas limits very close to lo rather than expending many steps on the "far" side of the interval. This recovers a large part of what Geth's optimistic probe is trying to do (i.e., get quickly from "safe but high" down to "tight but safe").

3. Tolerance‑based stopping matches client behavior

   With GasEstimateErrorRatioTolerance = 0.015, the estimator is allowed up to ~1.5% overestimation. Wallets and clients typically add a 20–25% bump on top of eth_estimateGas for safety under changing network conditions. In that context:

   • Shaving the estimator's error from "≤ 1.5%" to, say, "≤ 0.5%" using an extra optimistic probe has negligible real‑world effect.
   • The main practical concern is avoiding severe underestimation and excessive runtime; Nibiru's current design already satisfies both.

4. Refund information is already folded into GasUsed

   The Geth heuristic explicitly uses usedGas + refundedGas to form its optimistic limit. On Nibiru, refund logic is:

   gasUsed := msg.GasLimit - gasRemaining
   refundAmount := gasToRefund(sdb.GetRefund(), gasUsed)
   gasRemaining += refundAmount

   I.e., the final GasUsed already accounts for refunds. To reproduce Geth's exact behavior, we would need to track and expose both usedGas and refundedGas separately from GasUsed, and then recreate a similar formula. That adds extra surface area and complexity for a performance improvement that is, based on the above, at most marginal.

5. When might it matter?

   The optimistic pre‑check might offer slightly more value in very specific conditions:

   • Extremely large hi (e.g., very high MaxGas or user‑supplied gas) combined with
   • CtxKeyGasEstimateZeroTolerance = true (i.e., disabled error tolerance) and
   • Contracts whose required gas is a tricky function of the input such that the biased binary search must perform many exact steps.

   Even in those edge cases, the biased search plus smart lo ensure reasonable convergence, and the difference between "with" and "without" optimistic probe is primarily in constant factors, not in the overall algorithmic behavior.

5. Conclusion

Nibiru's EstimateGas implementation already incorporates:

• a validated high bound hi tied to consensus MaxGas, user gas, and gasCap,
• a single base‑case execution at hi to ensure feasibility,
• a tight lower bound lo ≈ GasUsed − 1 from that execution,
• and a biased, tolerance‑based binary search that focuses probes near lo and exits early with at most ~1.5% overestimation.

Together, these elements capture the core performance benefits that Geth's optimistic gas limit pre‑check is designed to provide. Adding an additional optimistic probe of the form

$$ (usedGas + refundedGas + CallStipend) \cdot \frac{64}{63} $$

would at best reduce the number of EVM executions by one or a few steps in niche scenarios, while requiring extra plumbing to separately expose RefundedGas.

Given the current design and typical client behavior (which already adds substantial safety margins above eth_estimateGas), the optimistic pre‑check is not necessary for performance on Nibiru today. It could still be implemented in the future for strict behavioral parity with Geth if profiling ever indicates that EstimateGas remains a material bottleneck, but from a cost–benefit standpoint, the existing Nibiru estimator is already sufficiently optimized to satisfy the intent of issue #2426.