Revert ts5.9 changes for Uint8Array

[paulmillr]

⚙ Compilation target

es2022

⚙ Library

es2022, dom

Missing / Incorrect Definition

ts5.7 change which made uint8arrays generic was okay.

ts5.9 introduced absolutely horrific change which breaks Uint8Array-heavy projects.

It seems like some changes made types too strict. It's unclear at which phase - perhaps during inference (i'm using infer). I'm now constantly getting those errors and it's really unhelpful - it's unclear what to do. Blog post recommendations (https://www.typescriptlang.org/docs/handbook/release-notes/typescript-5-9.html#libdts-changes) make arrays subtype-specific. It seems like this is a wrong way to fix things. Users should not be forced to always consume ArrayBuffer-based type arrays. It's perfectly fine to use either ArrayBuffer or SharedArrayBuffer based arrays, the changes are minor. The defaults should be able to consume both types!

// type Poly = Uint16Array
TS2345: Argument of type 'Poly' is not assignable to parameter of type 'Uint16Array<ArrayBuffer>'.
  Type 'ArrayBufferLike' is not assignable to type 'ArrayBuffer'.
    Type 'SharedArrayBuffer' is not assignable to type 'ArrayBuffer'.
      Types of property '[Symbol.toStringTag]' are incompatible.
        Type '"SharedArrayBuffer"' is not assignable to type '"ArrayBuffer"'.

Sample Code

I've spent a lot of time but was not able to produce a standalone example which reproduces the issue. There are a bunch of types at play and it's unclear which exact type causes the issue.

The code which produces errors is in commit cb76058 of https://github.com/paulmillr/noble-post-quantum

git clone https://github.com/paulmillr/noble-post-quantum
cd noble-post-quantum
git checkout cb76058
npm install
npm run build

[MartinJohns]

This is intentional and documented. They mention a way to fix it, which you didn't use.

https://www.typescriptlang.org/docs/handbook/release-notes/typescript-5-9.html#libdts-changes

[paulmillr]

@MartinJohns Nope. As I mentioned above:

The "fix" mentioned in blog posts makes arrays subtype-specific. The goal is for my code to consume any typed array - both ArrayBuffer and SharedArrayBuffer based. I am not using any "specific" methods. It seems like type inference screws up somewhere and makes stuff strict.

Changing this line from type Poly = Int32Array to type Poly = Int32Array<any> fixes things but it seems to be wrong. Shouldn't Int32Array<any> be the default?

https://github.com/paulmillr/noble-post-quantum/blob/cb76058c3328cb5ec09c007824ecd8de3f345dc4/src/ml-dsa.ts#L77

[Renegade334]

The problem, as you've identified, is where in-types and out-types collide. The array buffer view constructors almost always return a T<ArrayBuffer> by design; if that type is inferred and is then carried forward to type, say, a function parameter, then the change in variance can lead to issues.

These cases now require explicit annotations in certain places, to avoid inferring a narrowed type from a newly constructed array buffer view where narrowing is not desirable. In this instance, two changes are all that's needed for the project to build:

--- a/src/ml-dsa.ts
+++ b/src/ml-dsa.ts
@@ -75,7 +75,7 @@ export const PARAMS: Record<string, DSAParam> = {
 
 // NOTE: there is a lot cases where negative numbers used (with smod instead of mod).
 type Poly = Int32Array;
-const newPoly = (n: number) => new Int32Array(n);
+const newPoly = (n: number): Int32Array => new Int32Array(n);
 
 const { mod, smod, NTT, bitsCoder } = genCrystals({
   N,
diff --git a/src/ml-kem.ts b/src/ml-kem.ts
index fbd9a6a..18c8994 100644
--- a/src/ml-kem.ts
+++ b/src/ml-kem.ts
@@ -46,7 +46,7 @@ const { mod, nttZetas, NTT, bitsCoder } = genCrystals({
   Q,
   F,
   ROOT_OF_UNITY,
-  newPoly: (n: number) => new Uint16Array(n),
+  newPoly: (n: number): Uint16Array => new Uint16Array(n),
   brvBits: 7,
   isKyber: true,
 });

ts5.9 introduced absolutely horrific change which breaks Uint8Array-heavy projects.

This is only semi-accurate; these changes already existed from TS 5.7, but only for es2024 targets, which is where the changes to ArrayBuffer and SharedArrayBuffer made those interfaces diverge (#59417). Reverting the change which made this apply to all targets in TS 5.9 would only kick this can down the road until users start adopting es2024, and reverting the original change altogether isn't an option for the reasons detailed there.

[paulmillr]

The fix is reasonable, simple, and makes sense. Thanks.

However, it's completely non-obvious. If it's non-obvious to me (a lib author), then to my users it could be even worse. They would just slip on Uint8Array and call it a day. Or they would slip on Uint8Array and get more type errors in some cases.

Is there any way to make this kind of fix more documented? The blog post really does a bad job in this regard. Something like "Writing fn = () => new Uint8Array() would always return specific Uint8Array since ts5.9" would work.

[paulmillr]

Applied this fix in other libs -- just for the record, those are the diffs of changes that I thought are correct vs what's correct as per comment above

• paulmillr/noble-curves@878621b
• paulmillr/noble-hashes@46bf893
• paulmillr/noble-secp256k1@c00b81b