Add some tests covering edge cases in the ECDSA r <=> x comparison

[davidben]

This upstreams some tests from BoringSSL, though I've taken another pass at the generation script to cover all the curves in Wycheproof. (NB: I've only run the test vectors against an implementation for BoringSSL's curves. It is possible I've gotten the obscure curves wrong.)

ECDSA verification involves computing some point (x, y) and then checking if x mod n = r. x is only reduced mod p, so this leads to some edge cases right when x does and does not need a reduction. The existing test vectors covered some of these, but add some missing ones, so we catch both sides of the boundary condition.

Additionally, there is an optimized variant which leads to another boundary condition. EC points are typically implemented with Jacobian coordinates, (X:Y:Z) => (x, y) = (X/Z², Y/Z³). This avoids expensive inversions in the point addition formulae. Only when extracting the final affine coordinates do you invert a field element, to divide by Z.

Naively, ECDSA verification requires one such inversion. However, this inversion can be avoided in ECDSA verification by observing that comparing X/Z² to r is the same as comparing X to r*Z². However, this works mod p instead of mod n and we must deal with this mismatch. In most cases, n < p, where we must check two cases: r == x and r == x - n.

0 <= r < n < p and 0 <= x < p, so the first case is equivalent to checking x = r (mod p), where we can apply our optimization.

The second is equivalent to x == r + n, which in turn is equivalent to checking that r + n < p and then x == r + n (mod p), where we can also apply our optimization.

This leads to some other edge cases where we don't check the second case when we should, or when we check it when we shouldn't. The latter could happen if we reduce oversized r + n mod p, or forget a carry bit. Add some tests for these, getting as close to the boundary conditions as we can. Tested by simulating some bugs in BoringSSL's implementation of this optimization and ensuring Wycheproof noticed.

(A similar optimization works for the less common p < n curves, though it's a bit simpler.)

Test vectors generated with this tool

import base64
import hashlib
import json
import random

# Go's crypto/elliptic would normally be easier for this, but it only supports
# a = -3. We may as well get all the curves. No optimizations, except Jacobian
# coordinates because the inversion is too expensive.

# Too lazy to write % all the time.
class ModP:
    def __init__(self, value, p):
        assert 0 <= value < p
        assert p & 1
        self.value = value
        self.p = p

    def inv(self):
        if self.value == 0:
            raise ZeroDivisionError()
        return self ** (self.p - 2)

    def _unpack(self, other):
        assert other.p == self.p
        return other.value

    def __add__(self, other):
        return ModP((self.value + self._unpack(other)) % self.p, self.p)

    def __sub__(self, other):
        return ModP((self.value - self._unpack(other)) % self.p, self.p)

    def __mul__(self, other):
        return ModP((self.value * self._unpack(other)) % self.p, self.p)

    def __truediv__(self, other):
        assert self.p == other.p
        return self * other.inv()

    def __neg__(self):
        return ModP((-self.value) % self.p, self.p)

    def __pow__(self, exp):
        return ModP(pow(self.value, exp, self.p), self.p)

    def __eq__(self, other):
        assert self.p == other.p
        return self.value == other.value

    def has_sqrt(self):
        if self.value == 0:
            return True
        # Euler's criterion
        return pow(self.value, (self.p - 1) // 2, self.p) == 1

    def sqrt(self):
        assert self.has_sqrt()
        # Tonelli–Shanks. For reasonable curves, square roots are much more
        # straightforward, but P-224 is (uniquely) horrible, so just use the
        # generic algorithm.
        # https://en.wikipedia.org/wiki/Tonelli%E2%80%93Shanks_algorithm#The_algorithm
        q, s = self.p - 1, 0
        while q & 1 == 0:
            q >>= 1
            s += 1
        # Find a quadratic non-residue.
        while True:
            z = ModP(random.randint(1, self.p - 1), self.p)
            if not z.has_sqrt():
                break
        m = s
        c = z ** q
        t = self ** q
        r = self ** ((q + 1) // 2)
        while True:
            if t.value == 0:
                assert self.value == 0
                return ModP(0, self.p)
            if t.value == 1:
                assert r ** 2 == self
                return r
            # Find smallest i such that t**(2**i) == 1.
            tmp = t
            for i in range(1, m):
                tmp *= tmp
                if tmp.value == 1:
                    break
            assert tmp.value == 1
            # b = c**(2**(m-i-1)), or square it m - i - 1 times.
            b = c
            for _ in range(m - i - 1):
                b *= b
            m = i
            c = b * b
            t *= c
            r *= b

class InvalidCompressedPoint(Exception): pass

class Curve:
    def __init__(self, name, p, a, b, gx, gy, n):
        self.name = name
        self.p = p
        self.a = self.felem(a)
        self.b = self.felem(b)
        self.g = self.affine(gx, gy)
        self.n = n

    def felem(self, x):
        return ModP(x, self.p)

    def scalar(self, x):
        return ModP(x, self.n)

    def infinity(self):
        return Point(self, self.felem(0), self.felem(0), self.felem(0))

    def affine(self, x, y):
        return Point(self, self.felem(x), self.felem(y), self.felem(1))

    def compressed(self, x, y_bit=0):
        assert y_bit in (0, 1)
        x = self.felem(x)
        y2 = (x**3 + self.a * x + self.b)
        if not y2.has_sqrt():
            raise InvalidCompressedPoint()
        y = y2.sqrt()
        if y_bit != (y.value & 1):
            y = -y
        return Point(self, x, y, self.felem(1))

class Point:
    def __init__(self, curve, x, y, z):
        # (Y/Z^3)^2 = (X/Z^2)^3 + a(X/Z^2) + b
        # Y^2 / Z^6 = X^3 / Z^6 + a X / Z^2 + b
        # Y^2 = X^3 + a X Z^4 + b Z^6
        assert z.value == 0 or y**2 == x**3 + curve.a * x * z**4 + curve.b * z**6
        self.curve = curve
        self.x = x
        self.y = y
        self.z = z

    def __repr__(self):
        if self.is_infinity():
            return f"{self.curve.name}.infinity()"
        x, y = self.affine()
        return f"{self.curve.name}.affine(0x{x.value:x}, 0x{y.value:x})"

    def is_infinity(self):
        return self.z.value == 0

    def affine(self):
        assert not self.is_infinity()
        return self.x / self.z**2, self.y / self.z**3

    def __add__(self, other):
        assert isinstance(other, Point)
        assert self.curve == other.curve
        if self.is_infinity():
            return other
        if other.is_infinity():
            return self
        a = self.curve.a
        # Point formulas are transcribed a little sloppily and should have some
        # strength reductions. E.g. the constant multiplications should be
        # additions. But this is fast enough for these purposes.
        x1, y1, z1 = self.x, self.y, self.z
        x2, y2, z2 = other.x, other.y, other.z
        if x1 * z2 * z2 == x2 * z1 * z1:
            # https://www.hyperelliptic.org/EFD/g1p/auto-shortw-jacobian.html#doubling-dbl-2007-bl
            two = self.curve.felem(2)
            three = self.curve.felem(3)
            eight = self.curve.felem(8)
            xx = x1*x1
            yy = y1*y1
            yyyy = yy*yy
            zz = z1*z1
            s = two*((x1+yy)**2-xx-yyyy)
            m = three*xx+a*zz*zz
            t = m*m-two*s
            x3 = t
            y3 = m*(s-t)-eight*yyyy
            z3 = (y1+z1)**2-yy-zz
            return Point(self.curve, x3, y3, z3)

        # https://www.hyperelliptic.org/EFD/g1p/auto-shortw-jacobian.html#addition-add-2007-bl
        two = self.curve.felem(2)
        z1z1 = z1*z1
        z2z2 = z2*z2
        u1 = x1*z2z2
        u2 = x2*z1z1
        s1 = y1*z2*z2z2
        s2 = y2*z1*z1z1
        h = u2-u1
        i = (two*h)**2
        j = h*i
        r = two*(s2-s1)
        v = u1*i
        x3 = r*r-j-two*v
        y3 = r*(v-x3)-two*s1*j
        z3 = ((z1+z2)**2-z1z1-z2z2)*h
        return Point(self.curve, x3, y3, z3)

    def __sub__(self, other):
        return self + (-other)

    def __neg__(self):
        if self.is_infinity():
            return self
        return Point(self.curve, self.x, -self.y, self.z)

    def __mul__(self, scalar):
        assert scalar.p == self.curve.n
        scalar = scalar.value
        ret = self.curve.infinity()
        for i in reversed(range(scalar.bit_length())):
            ret += ret
            if scalar & (1 << i):
                ret += self
        return ret

    def __eq__(self, other):
        assert self.curve == other.curve
        if self.is_infinity():
            return other.is_infinity()
        if other.is_infinity():
            return False
        x1, y1, z1 = self.x, self.y, self.z
        x2, y2, z2 = other.x, other.y, other.z
        z1z1 = z1 * z1
        z2z2 = z2 * z2
        return x1 * z2z2 == x2 * z1z1 and y1 * z2z2 * z2 == y2 * z1z1 * z1

def ecdsa_digest_to_scalar(curve, digest):
    e = int.from_bytes(digest, 'big')
    n_bits = curve.n.bit_length()
    if n_bits < len(digest) * 8:
        # ECDSA takes the leftmost bits of the hash. Within a byte, the leftmost
        # bit is the most-significant bit.
        e >>= len(digest) * 8 - n_bits
    assert e.bit_length() <= n_bits
    # e may still need to be reduced.
    return ModP(e % curve.n, curve.n)

def ecdsa_pubkey_from_signature(point, digest, r, s):
    curve = point.curve
    r = ModP(r % curve.n, curve.n)
    s = ModP(s % curve.n, curve.n)
    e = ecdsa_digest_to_scalar(curve, digest)
    u, v_inv = e / s, s / r
    pub = (point - curve.g * u) * v_inv
    return pub

def ecdsa_verify(pub, digest, r, s):
    curve = pub.curve
    if not 0 < r < curve.n or not 0 < s < curve.n:
        return False
    r, s = ModP(r, curve.n), ModP(s, curve.n)
    e = ecdsa_digest_to_scalar(curve, digest)
    s_inv = s.inv()
    u, v = e * s_inv, r * s_inv
    point = curve.g * u + pub * v
    if point.is_infinity():
        return False
    r1, _ = point.affine()
    return r.value == r1.value % curve.n

SEQUENCE = 0x30
INTEGER = 0x02

def encode_der(tag, body):
    ret = bytearray()
    ret.append(tag)
    if len(body) < 0x80:
        ret.append(len(body))
    else:
        assert len(body) <= 0xff
        ret.append(0x81)
        ret.append(len(body))
    ret.extend(body)
    return bytes(ret)

def encode_der_integer(v):
    l = (v.bit_length() + 7) // 8
    b = v.to_bytes(l, 'big')
    if b[0] & 0x80:
        b = b"\x00" + b
    return encode_der(INTEGER, b)

def encode_ecdsa_signature(r, s):
    return encode_der(SEQUENCE, encode_der_integer(r) + encode_der_integer(s))

# All curves supported by Wycheproof.
brainpoolP224r1 = Curve(
    name="brainpoolP224r1",
    p= 0xd7c134aa264366862a18302575d1d787b09f075797da89f57ec8c0ff,
    a= 0x68a5e62ca9ce6c1c299803a6c1530b514e182ad8b0042a59cad29f43,
    b= 0x2580f63ccfe44138870713b1a92369e33e2135d266dbb372386c400b,
    gx=0x0d9029ad2c7e5cf4340823b2a87dc68c9e4ce3174c1e6efdee12c07d,
    gy=0x58aa56f772c0726f24c6b89e4ecdac24354b9e99caa3f6d3761402cd,
    n= 0xd7c134aa264366862a18302575d0fb98d116bc4b6ddebca3a5a7939f,
)
brainpoolP256r1 = Curve(
    name="brainpoolP256r1",
    p= 0xa9fb57dba1eea9bc3e660a909d838d726e3bf623d52620282013481d1f6e5377,
    a= 0x7d5a0975fc2c3057eef67530417affe7fb8055c126dc5c6ce94a4b44f330b5d9,
    b= 0x26dc5c6ce94a4b44f330b5d9bbd77cbf958416295cf7e1ce6bccdc18ff8c07b6,
    gx=0x8bd2aeb9cb7e57cb2c4b482ffc81b7afb9de27e1e3bd23c23a4453bd9ace3262,
    gy=0x547ef835c3dac4fd97f8461a14611dc9c27745132ded8e545c1d54c72f046997,
    n= 0xa9fb57dba1eea9bc3e660a909d838d718c397aa3b561a6f7901e0e82974856a7,
)
brainpoolP320r1 = Curve(
    name="brainpoolP320r1",
    p= 0xd35e472036bc4fb7e13c785ed201e065f98fcfa6f6f40def4f92b9ec7893ec28fcd412b1f1b32e27,
    a= 0x3ee30b568fbab0f883ccebd46d3f3bb8a2a73513f5eb79da66190eb085ffa9f492f375a97d860eb4,
    b= 0x520883949dfdbc42d3ad198640688a6fe13f41349554b49acc31dccd884539816f5eb4ac8fb1f1a6,
    gx=0x43bd7e9afb53d8b85289bcc48ee5bfe6f20137d10a087eb6e7871e2a10a599c710af8d0d39e20611,
    gy=0x14fdd05545ec1cc8ab4093247f77275e0743ffed117182eaa9c77877aaac6ac7d35245d1692e8ee1,
    n= 0xd35e472036bc4fb7e13c785ed201e065f98fcfa5b68f12a32d482ec7ee8658e98691555b44c59311,
)
brainpoolP384r1 = Curve(
    name="brainpoolP384r1",
    p= 0x8cb91e82a3386d280f5d6f7e50e641df152f7109ed5456b412b1da197fb71123acd3a729901d1a71874700133107ec53,
    a= 0x7bc382c63d8c150c3c72080ace05afa0c2bea28e4fb22787139165efba91f90f8aa5814a503ad4eb04a8c7dd22ce2826,
    b= 0x04a8c7dd22ce28268b39b55416f0447c2fb77de107dcd2a62e880ea53eeb62d57cb4390295dbc9943ab78696fa504c11,
    gx=0x1d1c64f068cf45ffa2a63a81b7c13f6b8847a3e77ef14fe3db7fcafe0cbd10e8e826e03436d646aaef87b2e247d4af1e,
    gy=0x8abe1d7520f9c2a45cb1eb8e95cfd55262b70b29feec5864e19c054ff99129280e4646217791811142820341263c5315,
    n= 0x8cb91e82a3386d280f5d6f7e50e641df152f7109ed5456b31f166e6cac0425a7cf3ab6af6b7fc3103b883202e9046565,
)
brainpoolP512r1 = Curve(
    name="brainpoolP512r1",
    p= 0xaadd9db8dbe9c48b3fd4e6ae33c9fc07cb308db3b3c9d20ed6639cca703308717d4d9b009bc66842aecda12ae6a380e62881ff2f2d82c68528aa6056583a48f3,
    a= 0x7830a3318b603b89e2327145ac234cc594cbdd8d3df91610a83441caea9863bc2ded5d5aa8253aa10a2ef1c98b9ac8b57f1117a72bf2c7b9e7c1ac4d77fc94ca,
    b= 0x3df91610a83441caea9863bc2ded5d5aa8253aa10a2ef1c98b9ac8b57f1117a72bf2c7b9e7c1ac4d77fc94cadc083e67984050b75ebae5dd2809bd638016f723,
    gx=0x81aee4bdd82ed9645a21322e9c4c6a9385ed9f70b5d916c1b43b62eef4d0098eff3b1f78e2d0d48d50d1687b93b97d5f7c6d5047406a5e688b352209bcb9f822,
    gy=0x7dde385d566332ecc0eabfa9cf7822fdf209f70024a57b1aa000c55b881f8111b2dcde494a5f485e5bca4bd88a2763aed1ca2b2fa8f0540678cd1e0f3ad80892,
    n= 0xaadd9db8dbe9c48b3fd4e6ae33c9fc07cb308db3b3c9d20ed6639cca70330870553e5c414ca92619418661197fac10471db1d381085ddaddb58796829ca90069,
)
secp160k1 = Curve(
    name="secp160k1",
    p= 0x00fffffffffffffffffffffffffffffffeffffac73,
    a= 0x000000000000000000000000000000000000000000,
    b= 0x000000000000000000000000000000000000000007,
    gx=0x003b4c382ce37aa192a4019e763036f4f5dd4d7ebb,
    gy=0x00938cf935318fdced6bc28286531733c3f03c4fee,
    n= 0x0100000000000000000001b8fa16dfab9aca16b6b3,
)
secp160r1 = Curve(
    name="secp160r1",
    p= 0x00ffffffffffffffffffffffffffffffff7fffffff,
    a= 0x00ffffffffffffffffffffffffffffffff7ffffffc,
    b= 0x001c97befc54bd7a8b65acf89f81d4d4adc565fa45,
    gx=0x004a96b5688ef573284664698968c38bb913cbfc82,
    gy=0x0023a628553168947d59dcc912042351377ac5fb32,
    n= 0x0100000000000000000001f4c8f927aed3ca752257,
)
secp160r2 = Curve(
    name="secp160r2",
    p= 0x00fffffffffffffffffffffffffffffffeffffac73,
    a= 0x00fffffffffffffffffffffffffffffffeffffac70,
    b= 0x00b4e134d3fb59eb8bab57274904664d5af50388ba,
    gx=0x0052dcb034293a117e1f4ff11b30f7199d3144ce6d,
    gy=0x00feaffef2e331f296e071fa0df9982cfea7d43f2e,
    n= 0x0100000000000000000000351ee786a818f3a1a16b,
)
secp192k1 = Curve(
    name="secp192k1",
    p= 0xfffffffffffffffffffffffffffffffffffffffeffffee37,
    a= 0x000000000000000000000000000000000000000000000000,
    b= 0x000000000000000000000000000000000000000000000003,
    gx=0xdb4ff10ec057e9ae26b07d0280b7f4341da5d1b1eae06c7d,
    gy=0x9b2f2f6d9c5628a7844163d015be86344082aa88d95e2f9d,
    n= 0xfffffffffffffffffffffffe26f2fc170f69466a74defd8d,
)
secp192r1 = Curve(
    name="secp192r1",
    p= 0xfffffffffffffffffffffffffffffffeffffffffffffffff,
    a= 0xfffffffffffffffffffffffffffffffefffffffffffffffc,
    b= 0x64210519e59c80e70fa7e9ab72243049feb8deecc146b9b1,
    gx=0x188da80eb03090f67cbf20eb43a18800f4ff0afd82ff1012,
    gy=0x07192b95ffc8da78631011ed6b24cdd573f977a11e794811,
    n= 0xffffffffffffffffffffffff99def836146bc9b1b4d22831,
)
secp224k1 = Curve(
    name="secp224k1",
    p= 0x00fffffffffffffffffffffffffffffffffffffffffffffffeffffe56d,
    a= 0x0000000000000000000000000000000000000000000000000000000000,
    b= 0x0000000000000000000000000000000000000000000000000000000005,
    gx=0x00a1455b334df099df30fc28a169a467e9e47075a90f7e650eb6b7a45c,
    gy=0x007e089fed7fba344282cafbd6f7e319f7c0b0bd59e2ca4bdb556d61a5,
    n= 0x10000000000000000000000000001dce8d2ec6184caf0a971769fb1f7,
)
secp224r1 = Curve(
    name="secp224r1",
    p= 0xffffffffffffffffffffffffffffffff000000000000000000000001,
    a= 0xfffffffffffffffffffffffffffffffefffffffffffffffffffffffe,
    b= 0xb4050a850c04b3abf54132565044b0b7d7bfd8ba270b39432355ffb4,
    gx=0xb70e0cbd6bb4bf7f321390b94a03c1d356c21122343280d6115c1d21,
    gy=0xbd376388b5f723fb4c22dfe6cd4375a05a07476444d5819985007e34,
    n= 0xffffffffffffffffffffffffffff16a2e0b8f03e13dd29455c5c2a3d,
)
secp256k1 = Curve(
    name="secp256k1",
    p= 0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f,
    a= 0x0000000000000000000000000000000000000000000000000000000000000000,
    b= 0x0000000000000000000000000000000000000000000000000000000000000007,
    gx=0x79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798,
    gy=0x483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8,
    n= 0xfffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141,
)
secp256r1 = Curve(
    name="secp256r1",
    p= 0xffffffff00000001000000000000000000000000ffffffffffffffffffffffff,
    a= 0xffffffff00000001000000000000000000000000fffffffffffffffffffffffc,
    b= 0x5ac635d8aa3a93e7b3ebbd55769886bc651d06b0cc53b0f63bce3c3e27d2604b,
    gx=0x6b17d1f2e12c4247f8bce6e563a440f277037d812deb33a0f4a13945d898c296,
    gy=0x4fe342e2fe1a7f9b8ee7eb4a7c0f9e162bce33576b315ececbb6406837bf51f5,
    n= 0xffffffff00000000ffffffffffffffffbce6faada7179e84f3b9cac2fc632551,
)
secp384r1 = Curve(
    name="secp384r1",
    p= 0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeffffffff0000000000000000ffffffff,
    a= 0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeffffffff0000000000000000fffffffc,
    b= 0xb3312fa7e23ee7e4988e056be3f82d19181d9c6efe8141120314088f5013875ac656398d8a2ed19d2a85c8edd3ec2aef,
    gx=0xaa87ca22be8b05378eb1c71ef320ad746e1d3b628ba79b9859f741e082542a385502f25dbf55296c3a545e3872760ab7,
    gy=0x3617de4a96262c6f5d9e98bf9292dc29f8f41dbd289a147ce9da3113b5f0b8c00a60b1ce1d7e819d7a431d7c90ea0e5f,
    n= 0xffffffffffffffffffffffffffffffffffffffffffffffffc7634d81f4372ddf581a0db248b0a77aecec196accc52973,
)
secp521r1 = Curve(
    name="secp521r1",
    p= 0x01ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff,
    a= 0x01fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc,
    b= 0x0051953eb9618e1c9a1f929a21a0b68540eea2da725b99b315f3b8b489918ef109e156193951ec7e937b1652c0bd3bb1bf073573df883d2c34f1ef451fd46b503f00,
    gx=0x00c6858e06b70404e9cd9e3ecb662395b4429c648139053fb521f828af606b4d3dbaa14b5e77efe75928fe1dc127a2ffa8de3348b3c1856a429bf97e7e31c2e5bd66,
    gy=0x011839296a789a3bc0045c8a5fb42c7d1bd998f54449579b446817afbd17273e662c97ee72995ef42640c550b9013fad0761353c7086a272c24088be94769fd16650,
    n= 0x01fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffa51868783bf2f966b7fcc0148f709a5d03bb5c9b8899c47aebb6fb71e91386409,
)

# Curve/hash combinations from Wycheproof.
TESTS = [
    ("ecdsa_brainpoolP224r1_sha224_test.json", brainpoolP224r1, "SHA-224"),
    ("ecdsa_brainpoolP256r1_sha256_test.json", brainpoolP256r1, "SHA-256"),
    ("ecdsa_brainpoolP320r1_sha384_test.json", brainpoolP320r1, "SHA-384"),
    ("ecdsa_brainpoolP384r1_sha384_test.json", brainpoolP384r1, "SHA-384"),
    ("ecdsa_brainpoolP512r1_sha512_test.json", brainpoolP512r1, "SHA-512"),
    ("ecdsa_secp160k1_sha256_test.json", secp160k1, "SHA-256"),
    ("ecdsa_secp160r1_sha256_test.json", secp160r1, "SHA-256"),
    ("ecdsa_secp160r2_sha256_test.json", secp160r2, "SHA-256"),
    ("ecdsa_secp192k1_sha256_test.json", secp192k1, "SHA-256"),
    ("ecdsa_secp192r1_sha256_test.json", secp192r1, "SHA-256"),
    ("ecdsa_secp224k1_sha224_test.json", secp224k1, "SHA-224"),
    ("ecdsa_secp224k1_sha256_test.json", secp224k1, "SHA-256"),
    ("ecdsa_secp224r1_sha224_test.json", secp224r1, "SHA-224"),
    ("ecdsa_secp224r1_sha256_test.json", secp224r1, "SHA-256"),
    ("ecdsa_secp224r1_sha512_test.json", secp224r1, "SHA-512"),
    ("ecdsa_secp256k1_sha256_test.json", secp256k1, "SHA-256"),
    ("ecdsa_secp256k1_sha512_test.json", secp256k1, "SHA-512"),
    ("ecdsa_secp256r1_sha256_test.json", secp256r1, "SHA-256"),
    ("ecdsa_secp256r1_sha512_test.json", secp256r1, "SHA-512"),
    ("ecdsa_secp384r1_sha256_test.json", secp384r1, "SHA-256"),
    ("ecdsa_secp384r1_sha384_test.json", secp384r1, "SHA-384"),
    ("ecdsa_secp384r1_sha512_test.json", secp384r1, "SHA-512"),
    ("ecdsa_secp521r1_sha512_test.json", secp521r1, "SHA-512"),
]

# Sanity-check
x = 0xf3033d1e548d245b5e45ff1147db8cd44db8a1f2823c3c164125be88f9a982c2
y = 0x3c078f6cee2f50e95e8916aa9c4e93de3fdf9b045abac6f707cfcb22d065638e
pub = secp256r1.affine(x, y)
assert pub == secp256r1.compressed(x, y & 1)
digest = bytes.fromhex("e8d38e4c6a905a814b04c2841d898ed6da023c34")
r = 0xd4255db86a416a5a688de4e238071ef16e5f2a20e31b9490c03dee9ae6164c34
s = 0x4e0ac1e1a6725bf7c6bd207439b2d370c5f2dea1ff4decf1650ab84c7769efc0
assert ecdsa_verify(pub, digest, r, s)

x = 0x4a6f1e7f7268174d23993b8b58aa60c2a87b18de79b36a750ec86dd6f9e12227
y = 0x572df22bd6487a863a51ca544b8c5de2b47f801372a881cb996a97d9a98aa825
pub = secp256r1.affine(x, y)
assert pub == secp256r1.compressed(x, y & 1)
digest = bytes.fromhex("54e9a048559f370425e9c8e54a460ec91bcc930a")
r = 0x4a800e24de65e5c57d4cab4dd1ef7b6c38a2f0aa5cfd3a571a4b552fb1993e69
s = 0xd9c89fb983640a7e65edf632cacd1de0823b7efbc798fc1f7bbfacdda7398955
assert not ecdsa_verify(pub, digest, r, s)

# These tests exercise the final r == x comparison, where x is the x-coordinate
# of uG + vQ. When testing boundary conditions, we would ideally use cases like
# r = 1, x = 1 or r = 1 + n, x = 1. However, not all x-coordinates are on the
# curve, so these helpers find a target x = ε or x = n - ε, for some ε > 0.

def plus_epsilon(curve, x):
    epsilon = 1
    while True:
        try:
            return curve.compressed(x + epsilon), epsilon
        except InvalidCompressedPoint:
            epsilon += 1

def minus_epsilon(curve, x):
    epsilon = 1
    while True:
        try:
            return curve.compressed(x - epsilon), epsilon
        except InvalidCompressedPoint:
            epsilon += 1

def key_to_pem(key: bytes) -> str:
    ret = "-----BEGIN PUBLIC KEY-----\n"
    b64 = base64.b64encode(key).decode("ascii")
    while b64:
        l = min(len(b64), 64)
        ret += b64[:l]
        ret += "\n"
        b64 = b64[l:]
    ret += "-----END PUBLIC KEY-----\n"
    return ret

INPUT = b"hello, world"
for (path, curve, hash_name) in TESTS:
    print(path)
    with open(path) as f:
        data = json.load(f)

    def add_test(comment, pub, digest, r, s, valid):
        felem_bytes = (pub.curve.p.bit_length() + 7) // 8
        x, y = pub.affine()
        x_bytes = x.value.to_bytes(felem_bytes, "big")
        y_bytes = y.value.to_bytes(felem_bytes, "big")
        uncompressed = b"\x04" + x_bytes + y_bytes
        if valid:
            flags = ["ValidSignature"]
            result = "valid"
        else:
            flags = ["ArithmeticError"]
            result = "invalid"

        # Reconstruct the SPKI encoding from the existing examples, rather than
        # encoding an OID and everything.
        sample_spki = bytes.fromhex(data["testGroups"][0]["publicKeyDer"])
        sample_uncompressed = bytes.fromhex(data["testGroups"][0]["publicKey"]["uncompressed"])
        assert sample_spki.endswith(sample_uncompressed)
        assert key_to_pem(sample_spki) == data["testGroups"][0]["publicKeyPem"]
        spki_prefix = sample_spki[:-len(sample_uncompressed)]

        spki = spki_prefix + uncompressed

        group = {
            "type": "EcdsaVerify",
            "source": {
                "name" : "github/davidben/ecdsa-r-s-edge-cases",
                "version" : "0.1"
            },
            "publicKey": {
                "type": "EcPublicKey",
                "curve": pub.curve.name,
                "keySize": pub.curve.p.bit_length(),
                "uncompressed": uncompressed.hex(),
                "wx": x_bytes.hex(),
                "wy": y_bytes.hex(),
            },
            "publicKeyDer": spki.hex(),
            "publicKeyPem": key_to_pem(spki),
            "sha": hash_name,
            "tests": [
                {
                    "tcId": data["numberOfTests"] + 1,
                    "comment": comment,
                    "flags": flags,
                    "msg": INPUT.hex(),
                    "sig": encode_ecdsa_signature(r, s).hex(),
                    "result": result,
                },
            ],
        }

        data["testGroups"].append(group)
        data["numberOfTests"] += 1

    def valid_test(comment, point, digest, r, s):
        pub = ecdsa_pubkey_from_signature(point, digest, r, s)
        assert ecdsa_verify(pub, digest, r, s)
        add_test(comment, pub, digest, r, s, True)

    def invalid_test(comment, point, digest, r, s):
        pub = ecdsa_pubkey_from_signature(point, digest, r, s)
        assert not ecdsa_verify(pub, digest, r, s)
        add_test(comment, pub, digest, r, s, False)

    h = hashlib.new(hash_name.replace("-", ""))
    h.update(INPUT)
    digest = h.digest()

    # Arbitrarily use s = n - 3.
    s = curve.n - 3

    point, epsilon = plus_epsilon(curve, 0)
    r = epsilon
    valid_test(f"r = {epsilon}, x = {epsilon} is valid", point, digest, r, s)

    r = epsilon + 1
    invalid_test(f"r = {epsilon+1}, x = {epsilon} is invalid", point, digest, r, s)

    r = epsilon + curve.n
    invalid_test(f"r = {epsilon} + n, x = {epsilon} is invalid; r was not reduced mod n", point, digest, r, s)

    if curve.p > curve.n:
        print("  p > n")
        # This is redundant with the existing "r,s are large" test.
        # point, epsilon = minus_epsilon(curve, curve.n)
        # r = curve.n - epsilon
        # valid_test(f"r = n - {epsilon}, x = n - {epsilon} is the largest x without a reduction", point, digest, r, s)

        point, epsilon = minus_epsilon(curve, curve.n)
        r = curve.n - epsilon - 1
        invalid_test(f"r = n - {epsilon+1}, x = n - {epsilon} is invalid", point, digest, r, s)

        point, epsilon = plus_epsilon(curve, curve.n)
        r = epsilon
        valid_test(f"r = {epsilon}, x = n + {epsilon} is the smallest possible x with a reduction", point, digest, r, s)

        r = epsilon + 1
        invalid_test(f"r = {epsilon+1}, x = n + {epsilon} is invalid", point, digest, r, s)

        # This is redundant with the existing "k*G has a large x-coordinate" test.
        # point, epsilon = minus_epsilon(curve, curve.p)
        # r = curve.p - epsilon - curve.n
        # valid_test(f"r = p - {epsilon} - n, x = p - {epsilon} is the largest valid x", point, digest, r, s)

        point, epsilon = plus_epsilon(curve, 0)
        r = curve.p - curve.n + epsilon
        invalid_test(f"r = p - n + {epsilon}, x = {epsilon} is invalid; r is too large to compare r + n with x", point, digest, r, s)

        b = curve.n.bit_length()
        r = (1<<b) - curve.n + epsilon
        invalid_test(f"r = 2^{b} - n + {epsilon}, x = {epsilon} is invalid; r + n is too large to compare r + n with x, and overflows 2^{b} bits", point, digest, r, s)

    else:
        print("  n > p")
        # Not as many cases to test. x is already reduced mod n.

        # This is redundant with the existing "k*G has a large x-coordinate" test.
        # point, epsilon = minus_epsilon(curve, curve.p)
        # r = curve.p - epsilon
        # valid_test(f"r = p - {epsilon}, x = p - {epsilon} is the largest valid r and x", point, digest, r, s)

        point, epsilon = minus_epsilon(curve, curve.p)
        r = curve.p - epsilon - 1
        invalid_test(f"r = p - {epsilon+1}, x = p - {epsilon} is invalid", point, digest, r, s)

        point, epsilon = plus_epsilon(curve, 0)
        r = epsilon + curve.p
        invalid_test(f"r = {epsilon} + p, x = {epsilon} is invalid; values only match mod p", point, digest, r, s)

    with open(path, "w") as f:
        json.dump(data, f, indent=2, separators=(',', ': '), ensure_ascii=False)
        f.write("\n")

[botovq]

This looks good to me. I tested this against LibreSSL's ECDSA verifier (which covers all the curves except `secp160* and secp192*) and we agree on the results. That's not too surprising since we're not trying to be smart.

Nit: your commit message has "the generation script to cover all the covers in Wycheproof"

[davidben]

Nit: your commit message has "the generation script to cover all the covers in Wycheproof"

Oops, fixed.

[cpu]

Thanks!

[cpu]

Going to merge this now, but happy to do further revisions if anything comes up!