Skip to content

Comments

fix(jsbn2): prevent modInverse hangs on zero and negative inputs#648

Merged
kjur merged 2 commits intokjur:masterfrom
Kr0emer:fix/bug-004-modinverse-dos
Feb 20, 2026
Merged

fix(jsbn2): prevent modInverse hangs on zero and negative inputs#648
kjur merged 2 commits intokjur:masterfrom
Kr0emer:fix/bug-004-modinverse-dos

Conversation

@Kr0emer
Copy link
Contributor

@Kr0emer Kr0emer commented Feb 20, 2026

Summary

This PR fixes an infinite-loop DoS in BigInteger.modInverse (ext/jsbn2.js) for:

  • zero input (a = 0)
  • negative input (a < 0)

Root Cause

bnModInverse assumes a positive, invertible operand and directly clones this:

  • a = 0 can make the loop non-terminating
  • negative values are not normalized before the binary-GCD loop, which can prevent convergence

Fix

In bnModInverse(m):

  • return early when m == 0
  • normalize input with x = this.mod(m)
  • return early when normalized x == 0
  • run the existing algorithm with x (instead of raw this) in state updates

Tests

Added regression tests in test/qunit-do-crypto.html:

  • BigInteger.modInverse returns quickly for zero input
  • BigInteger.modInverse normalizes negative input

Validation

  • Reproduced previous hang behavior on old code with timeout harness:
    • modInverse(0,3), modInverse(0,97), modInverse(-1,7), modInverse(-13,97) timed out
  • After patch:
    • modInverse(0,3) -> 0
    • modInverse(0,97) -> 0
    • modInverse(-1,7) -> 6
    • modInverse(-13,97) -> 82
    • no timeout/hang
  • Existing npm tests still pass in this environment.

@kjur kjur merged commit ca5b027 into kjur:master Feb 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants