Make TypeScript FFI Typechecking easier

[daniellionel01]

Hello!

I've recently experimented with getting tsc to work in a compiled gleam project with target = javascript and typescript_declarations = true, given that with runtimes like bun, deno and recently even node (https://nodejs.org/en/learn/typescript/run-natively) you can run *.ts files natively without having to compile them to js.

Since I mostly work in typescript codebases, where everything is typed, the thought of having to write FFI bindings that are not type checked is a bit unsettling. Although I would love to replace typescript packages inside of my codebases with gleam packages.

My first idea was to write a generator for gleam that would automatically be able to map typescript types to gleam, so I could always make sure my FFI bindings are in sync with my typescript code. For example:

// app.ts
export function sum(a: number, b: number) {
  return a + b
}

I could automatically generate this:

// app.gleam
@external(javascript, "./app.ts", "sum)
pub fn sum(a: Int, b: Int) -> Int

I actually have a very small prototype with https://ts-morph.com/ that already works for scalar types, but would need further work and testing for more complex types. So this idea is still something I am considering.

I also had the following idea: maybe we can just run tsc inside of the build/dev/javascript directory, since it should be able to catch a mismatch between the types inside of the .ts file and the function call inside of gleam. However this has not worked out in practice for me. There were some issues I ran into when working on this:

• the function call is in a .mjs file, so it is not typechecked
• gleam creates a .d.mts file if you enable typescript_declarations which basically overrides the types from the .ts file, so the typescript compiler is blindsighted

Another idea I had was to use JSDoc (https://jsdoc.app/) since I then would not have to generate typescript declarations. However I was unable to get it working as of today.

A couple of talking points I am aware of and are fair to discuss:

• How relevant is this problem really in production? I myself have not shipped a large gleam application with a javascript target and lots of bindings
• Is this outside of the scope of gleam of "just" compiling to javascript, should we have to bend to make a tool like tsc work?

But maybe there are modifications to how gleam produces javascript and typescript declarations that would make typechecking easier while keeping the changes on gleam's end minimal.

Perhaps right now there are ways to get it working, that I am just not aware of and this is just a skill issue.

Happy to discuss further,
Cheers!

[ghivert]

As a huge user of Gleam + JavaScript + TypeScript in production in both frontend and backend (on Node), I have probably some food for thoughts for you. 🙂

Is this outside of the scope of gleam of "just" compiling to javascript, should we have to bend to make a tool like tsc work?

you can run *.ts files natively without having to compile them to js.

Unfortunately, that's not universally right. JavaScript as a target is used by runtimes but also (and especially in Gleam with Lustre) in browsers. And as of today, browsers do not support TypeScript out-of-the-box. You could argue everyone will use a bundler, like ESBuild, Vite or Rollup, but not everyone want to go that process.

Also, a lot of people are targetting specific environments, like Node 20 or 22, CloudFlare workers, etc. While TypeScript is not in the ECMA specification, I think it's bad to output TypeScript just like this. In my opinion, Gleam should as much as possible stick with JS standards, and not try to force any tool on top of the developer stack. Everyone use the tool they like, and sticking with standard is the good thing to do to let people bring their tooling. Some will like yarn, some npm, others pnpm, and that's OK: Gleam let you use what you prefer because in the end, it's just plain JS.

That being said, Gleam adds the TS superset over JS, by emitting .d.ts files (which is what TypeScript recommends when publishing a package), and is compliant with TS. So that's the best of both world: you get JS for every users, and TS for those who use TS (and some people dislike TS, so that makes sense to provide an option to escape TS for them).

The only part that won't really be type-checked is the FFI, but why not writing your own .d.ts for the FFI file?

How relevant is this problem really in production? I myself have not shipped a large gleam application with a javascript target and lots of bindings

Gleam works well in the browser and JS runtimes, but I can say interoperability between JS and Gleam can be hard sometimes, because API are not built the same way in Gleam and in JS. Builder patterns in Gleam are nice, while they are a bit clumsy in pure JS. Builders in JS are built around objects (because there's no pipe), and it makes a huge difference between the two.

What we do at Steerlab, is applying some simple patterns: we use pure JS in FFI in Gleam, and we don't have functions in the FFI that are longer than 10 lines. We also manipulate only simple data structure, and we try to avoid at most to open/use the data structure in the FFI. The FFI should stay minimal, and all the logic is written in Gleam. When we need something really specific in JS/TS, we write a proper JS/TS package instead of bending the language (from experience, when what you need is 90% in JS, write it directly in JS instead of going in the FFI too much, that's a huge gain of time).
However, we use Gleam packages from TS: typescript_declarations are working great, and every functions are type-checked, thanks to *.d.ts files. So doing a breaking change in our Gleam code will trigger an error on the TS side (except for Dict, but that's another story). That's usually what we do: write everything in Gleam, then reuse parts here and there.

To give some numbers: we had 0 bugs due to FFI in production, because the FFI code is probably like 1% of the codebase at most. Every bugs are coming from something else (business logic most of the time, let assert the rest of the time), and type-checking the FFI would probably be more time consuming than really useful. We isolate the FFI in their own module, and it's a code that never change. So we have some equivalent of plinth specific to our needs for example, and that code never changes once it's written and tested.

My first idea was to write a generator for gleam that would automatically be able to map typescript types to gleam, so I could always make sure my FFI bindings are in sync with my typescript code.

I'm not sure I get you here. What you want to do is generate your Gleam code FFI directly from your TS code? That sounds good, but isn't it a bit using a flamethrower to kill a fly? I mean, that's a few lines of code to write as FFI, and that would let you design the FFI as you like. However if you have a lot of code, I understand why you would do this. 🙂 In that case, I would advise to write your TS files as *.ffi.ts instead of *.ts, so it wont interfere with your Gleam code! That's what we do for every FFI file!

---

Maybe a solution could be for you to run tsc on your src folder for every *.ffi.ts file (with emitting in place JS files), and git ignore *.ffi.js? That way, you would have your *.ffi.ts files along *.ffi.js files compatible with Gleam, and everything could be type-checked?

[daniellionel01]

Great input, thank you! Yeah all valid points, I think I should first collect some experience with gleam in large typescript codebases before looking at what problems to solve. Having a couple of tests for the FFI stuff is probably going to do the trick for starters.

My idea or "vision" with the FFI code generation was to at some point be able to autogenerate whole gleam wrappers for certain TS libraries. So if I wanted to use playwright from gleam I could just generate a whole bunch of FFI functions that I wanted to use from that package if it's a fairly large one. But yeah probably overkill for most things.

[ghivert]

But yeah probably overkill for most things.

More than overkill, I'd say especially extra hard to do, because API from TS are not the same as API in Gleam. For example, in TS you often write things like

new Data()
  .myFunction()
  .myOtherFunction()

And you'd like to have something like:

data.new()
|> my_function()
|> my_other_function()

But what happens when the TypeScript module is

export class Data {
  myFunction() {}
  myOtherFunction() {}
}

export function myFunction(data) {
  return data.myFunction()
}

export function myOtherFunction(data) {
  return data.myOtherFunction()
}

What do you generate here? What is my_other_function here? And that's just an example from other specific cases really hard to convert as is. What about variadic arguments? You can pass them as a List, but they have to be the same type. How do you handle enums? Something like 'a' | 'b' | 'c'. Probably with

type MyType { 
  A 
  B 
  C
}

And if that type is not named, because it's implicit, thanks to as const of TypeScript?

There's a lot of questions and issues to address, and you'll have to make choices. A converter from TS to Gleam, that would do like 90% of the job with codegen would be awesome, but such a thing does not come for free, and you'll have to spend time figuring out how to solve issues one by one. I think starting simple with some packages where you'll write the API by hand would be better to start, and figure out what to do, and how.

My honest answer is rather than if your project relies like 90% on Playwright or other NPM packages, you probably should start with TS. TS is now a strong language, and you'll spend an infinite amount of time to create the bindings you need otherwise, especially if you need to maintain it over time, because the team behind Playwright or other NPM stuff most of the time does not care about bindings in Gleam, unfortunately. 😭

[daniellionel01]

unfortunately

yeah I've started a project where I need to control a browser with playwright and for now I'm writing a sort of playwright remote with typescript and then use bun to compile it to an executable so I can run it side by side with my gleam service. should be fun