Feature Request: Type Inference

[hmtrent]

Today, borsher does not perform type inference using the structure of schema:

    const schema = BorshSchema.Struct({
      myU64: BorshSchema.u64,
      myBool: BorshSchema.bool,
    });

    // x is typed as "unknown"
    const x = borshDeserialize(schema, bytes);

It's certainly possible for TypeScript to infer the type of x as { myU64: bigint; myBool: boolean }

[hanakannzashi]

Thanks for your issue and PR, currently borshDeserialize has a generic type T which you can use to specify your return type manually. Nesting return types within the schema seems to add a lot of complexity, and I'm not sure if it's better to do so

[hmtrent]

There's a considerable difference between having the user supply their own type T, and having the library infer the type.

• The existing implementation is exactly the same as the following code, except we're performing the cast on the caller's behalf:

// Consider this non-generic signature...
function borshDeserialize(schema: BorshSchema, bytes: Uint8Array): unknown { /* ... */ }

// The user can easily cast to their own type if they want to:
const foo = borshDeserialize(schema, bytes) as MyStruct;

• Type inference makes it so that the library tells the caller exactly what type it will be parsing.
  • This makes it harder for the user to make mistakes, since the source-of-truth for the type information is held in the BorshSchema instances.
  • This is all around much safer -- the user doesn't have to guess, or look up the documentation as to which types that the library will parse to (e.g. will a u64 deserialize to a bigint? a string? or worse -- a number?). Type inference gets rid of this guesswork, because the return type has this information in it.

I'd argue that as a library, borsher provides a better user experience when inferring types.

There's a tradeoff here though, as borsher becomes slightly more complex, but this should be very useful to pretty much anybody using this library.

[hanakannzashi]

There's a considerable difference between having the user supply their own type T, and having the library infer the type.

• The existing implementation is exactly the same as the following code, except we're performing the cast on the caller's behalf:

// Consider this non-generic signature...
function borshDeserialize(schema: BorshSchema, bytes: Uint8Array): unknown { /* ... */ }

// The user can easily cast to their own type if they want to:
const foo = borshDeserialize(schema, bytes) as MyStruct;

• Type inference makes it so that the library tells the caller exactly what type it will be parsing.

  • This makes it harder for the user to make mistakes, since the source-of-truth for the type information is held in the BorshSchema instances.
  • This is all around much safer -- the user doesn't have to guess, or look up the documentation as to which types that the library will parse to (e.g. will a u64 deserialize to a bigint? a string? or worse -- a number?). Type inference gets rid of this guesswork, because the return type has this information in it.

I'd argue that as a library, borsher provides a better user experience when inferring types.

There's a tradeoff here though, as borsher becomes slightly more complex, but this should be very useful to pretty much anybody using this library.

But there is another problem, the serialized type and deserialized type can be different. e.g.

const schema = BorshSchema.u128;

const value = 100; // number
// const value = 100n; // bigint
// const value = '100'; // string

const raw = borshSerialize(schema, value);

const returnedValue = borshDeserialize(schema, raw); // bigint

If we nest the bigint in BorshSchema, we can not serialize number and string. The solution is define 2 generic in BorshSchema, i.e. BorshSchema<Ser, De>, more complex lol

[hmtrent]

Ah I see, when serializing we might want to be more flexible about the types that borsh can handle.

This doesn't require an extra type parameter -- all of the type information is still in the schema, after all -- we can add another type function like this one¹ which can add more flexibility in type-checking the value to serialize. From a user's point of view, the API will stay the same, and this should still add more flexibility and safety.

Footnotes

1. https://github.com/nameskyteam/borsher/pull/4/files#diff-979795a24936e5bc1530c0d85ef9b1fb081be69dec1c24ac6199f84facb352acR358-R410 ↩

[r-near]

But there is another problem, the serialized type and deserialized type can be different. e.g.

So initially I wrote my library to support unique inputs/outputs, but after taking a closer look, I feel like for a type-safe implementation, the Borsh schema should be the source of truth, and that supporting separate inputs like in your example, should be out of scope. That's validation, but this library should be concerned with serialization and deserialization.

For example

const schema = BorshSchema.u128

const value: number = 100 // This should throw an error, because a u128 would map to a BigInt
const value: string = '100' // This should also throw an error

TypeScript is not expressive enough to discriminate between various numeric types. So while a TypeScript number could serialize to unique Borsh entities (u8, u16, u32) - I think it's also clear that those Borsh entities would deserialize to a TypeScript number, and not a string, or a bigint, for example.

So, in some way, we have a 1:1 mapping.

Also, after practical testing, I realized that I ended up writing methods to coerce the types into different types. For example, coercing a numeric string into a bigint, before serializing.

I think if this library is scoped purely to serialization, it makes this a lot easier, and we can get really far with being strongly-typed.

[hanakannzashi]

But there is another problem, the serialized type and deserialized type can be different. e.g.

So initially I wrote my library to support unique inputs/outputs, but after taking a closer look, I feel like for a type-safe implementation, the Borsh schema should be the source of truth, and that supporting separate inputs like in your example, should be out of scope. That's validation, but this library should be concerned with serialization and deserialization.

For example

const schema = BorshSchema.u128

const value: number = 100 // This should throw an error, because a u128 would map to a BigInt
const value: string = '100' // This should also throw an error
TypeScript is not expressive enough to discriminate between various numeric types. So while a TypeScript number could serialize to unique Borsh entities (u8, u16, u32) - I think it's also clear that those Borsh entities would deserialize to a TypeScript number, and not a string, or a bigint, for example.

So, in some way, we have a 1:1 mapping.

Also, after practical testing, I realized that I ended up writing methods to coerce the types into different types. For example, coercing a numeric string into a bigint, before serializing.

I think if this library is scoped purely to serialization, it makes this a lot easier, and we can get really far with being strongly-typed.

I agree that 1:1 mapping is good