|
| 1 | +--- |
| 2 | +author: rescript-team |
| 3 | +date: "2023-04-17" |
| 4 | +title: Better interop with customizable variants |
| 5 | +badge: roadmap |
| 6 | +description: | |
| 7 | + A tour of new capabilities coming in ReScript v11 |
| 8 | +--- |
| 9 | + |
| 10 | +ReScript v11 is around the corner, and it comes packed with new features that will improve interop with JavaScript/TypeScript. Recently we've made some changes to the runtime representation of variants that'll allow you to use variants for a large number of new interop scenarios, zero cost. This is important, because variants are _the_ feature of ReScript, enabling great data modeling, pattern matching and more. |
| 11 | + |
| 12 | +- **Customizable runtime representation.** We're making the runtime representation of variants customizable. This will allow you to cleanly map variants to external data and APIs in many more cases than before. |
| 13 | +- **Zero cost bindings to discriminated unions.** Variants with inline records will map cleanly to JavaScript/TypeScript [discriminated unions](https://www.typescriptlang.org/docs/handbook/2/narrowing.html#discriminated-unions). |
| 14 | +- **Unboxed (untagged) variants.** We also introduce untagged variants - variants where the underlying runtime representation can be a primitive, without a specific discriminator. This will let you cleanly map to things like heterogenous array items, nullable values, and more. |
| 15 | + |
| 16 | +Let's dive into the details. |
| 17 | + |
| 18 | +## Tagged variants |
| 19 | + |
| 20 | +Variants with payloads have always been represented as a tagged union at runtime. Here's an example: |
| 21 | + |
| 22 | +```rescript |
| 23 | +type entity = User({name: string}) | Group({workingName: string}) |
| 24 | +
|
| 25 | +let user = User({name: "Hello"}) |
| 26 | +``` |
| 27 | + |
| 28 | +This is represented as: |
| 29 | + |
| 30 | +```javascript |
| 31 | +var user = { |
| 32 | + TAG: /* User */ 0, |
| 33 | + name: "Hello", |
| 34 | +}; |
| 35 | +``` |
| 36 | + |
| 37 | +However, this has been problematic when binding to external data because there has been no way to customize the discriminator (the `TAG` property) or how its value is represented for each variant case (`0` representing `User` here). This means that unless your external data is modeled the exact same way as above, which is unlikely, you'd be forced to convert to the structure ReScript expects at runtime. |
| 38 | + |
| 39 | +To illustrate this, let's imagine we're binding to an external union that looks like this in TypeScript: |
| 40 | + |
| 41 | +```typescript |
| 42 | +type LoadingState = |
| 43 | + | { state: "loading"; ready: boolean } |
| 44 | + | { state: "error"; message: string } |
| 45 | + | { state: "done"; data: Data }; |
| 46 | +``` |
| 47 | + |
| 48 | +Currently, there's no good way to use a ReScript variant to represent this type without resorting to manual and error-prone runtime conversion. However, with the new functionality, binding to the above with no additional runtime cost is easy: |
| 49 | + |
| 50 | +```rescript |
| 51 | +@tag("state") |
| 52 | +type loadingState = | @as("loading") Loading({ready: bool}) | @as("error") Error({message: string}) | @as("done") Done({data: data}) |
| 53 | +
|
| 54 | +let state = Error({message: "Something went wrong!"}) |
| 55 | +``` |
| 56 | + |
| 57 | +This will compile to: |
| 58 | + |
| 59 | +```javascript |
| 60 | +var state = { |
| 61 | + state: "error", |
| 62 | + message: "Something went wrong!", |
| 63 | +}; |
| 64 | +``` |
| 65 | + |
| 66 | +Let's break down what we've done to make this work: |
| 67 | + |
| 68 | +- The `@tag` attribute lets you customize the discriminator (default: `TAG`). We're setting that to `"state"` so we map to what the external data looks like. |
| 69 | +- Each variant case has an `@as` attribute. That controls what each variant case is discriminated on (default: the variant case name as string). We're setting all of the cases to their lowercase equivalent, because that's what the external data looks like. |
| 70 | + |
| 71 | +The end result is clean and zero cost bindings to the external data, in a way that previously would require manual runtime conversion. |
| 72 | + |
| 73 | +Now, let's look at a few more real-world examples. |
| 74 | + |
| 75 | +### Binding to TypeScript enums |
| 76 | + |
| 77 | +```typescript |
| 78 | +// direction.ts |
| 79 | +/** Direction of the action. */ |
| 80 | +enum Direction { |
| 81 | + /** The direction is up. */ |
| 82 | + Up = "UP", |
| 83 | + |
| 84 | + /** The direction is down. */ |
| 85 | + Down = "DOWN", |
| 86 | + |
| 87 | + /** The direction is left. */ |
| 88 | + Left = "LEFT", |
| 89 | + |
| 90 | + /** The direction is right. */ |
| 91 | + Right = "RIGHT", |
| 92 | +} |
| 93 | + |
| 94 | +export const myDirection = Direction.Up; |
| 95 | +``` |
| 96 | + |
| 97 | +Previously, you'd be forced to use a polymorphic variant for this if you wanted clean, zero-cost interop: |
| 98 | + |
| 99 | +```rescript |
| 100 | +type direction = [#UP | #DOWN | #LEFT | #RIGHT] |
| 101 | +@module("./direction.js") external myDirection: direction = "myDirection" |
| 102 | +``` |
| 103 | + |
| 104 | +Notice a few things: |
| 105 | + |
| 106 | +- We're forced to use the names of the enum payload, meaning it won't fully map to what you'd use in TypeScript |
| 107 | +- There's no way to bring over the documentation strings, because polymorphic variants are structural, so there's no one source definition for them to look for docstrings on. This is true _even_ if you annotate with your explicitly written out polymorphic variant definition. |
| 108 | + |
| 109 | +With the new runtime representation, this is how you'd bind to the above enum instead: |
| 110 | + |
| 111 | +```rescript |
| 112 | +/** Direction of the action. */ |
| 113 | +type direction = |
| 114 | + | /** The direction is up. */ |
| 115 | + @as("UP") |
| 116 | + Up |
| 117 | +
|
| 118 | + | /** The direction is down. */ |
| 119 | + @as("DOWN") |
| 120 | + Down |
| 121 | +
|
| 122 | + | /** The direction is left. */ |
| 123 | + @as("LEFT") |
| 124 | + Left |
| 125 | +
|
| 126 | + | /** The direction is right. */ |
| 127 | + @as("RIGHT") |
| 128 | + Right |
| 129 | +
|
| 130 | +@module("./direction.js") external myDirection: direction = "myDirection" |
| 131 | +``` |
| 132 | + |
| 133 | +Now, this maps 100% to the TypeScript code, including letting us bring over the documentation strings so we get a nice editor experience. |
| 134 | + |
| 135 | +### String literals |
| 136 | + |
| 137 | +The same logic is easily applied to string literals from TypeScript, only here the benefit is even larger, because string literals have the same limitations in TypeScript that polymorphic variants have in ReScript. |
| 138 | + |
| 139 | +```typescript |
| 140 | +// direction.ts |
| 141 | +type direction = "UP" | "DOWN" | "LEFT" | "RIGHT"; |
| 142 | +``` |
| 143 | + |
| 144 | +There's no way to attach documentation strings to string literals in TypeScript, and you only get the actual value to interact with. |
| 145 | + |
| 146 | +With the new customizable variants, you could bind to the above string literal type easily, but add documentation, and change the name you interact with in ReScript. And there's no runtime cost. |
| 147 | + |
| 148 | +### Untagged variants |
| 149 | + |
| 150 | +We've also implemented support for _untagged variants_. This will let you use variants to represent values that are primitives and literals in a way that hasn't been possible before. |
| 151 | + |
| 152 | +We'll explain what this is and why it's useful by showing a number of real world examples. Let's start with a simple one on how we can now represent a heterogenous array. |
| 153 | + |
| 154 | +```rescript |
| 155 | +@unboxed type listItemValue = String(string) | Boolean(bool) | Number(float) |
| 156 | +
|
| 157 | +let myArray = [String("Hello"), Boolean(true), Boolean(false), Number(13.37)] |
| 158 | +``` |
| 159 | + |
| 160 | +Here, each value will be _unboxed_ at runtime. That means that the variant payload will be all that's left, the variant case name wrapping the payload itself will be stripped out and the payload will be all that remains. |
| 161 | + |
| 162 | +It, therefore, compiles to this JS: |
| 163 | + |
| 164 | +```javascript |
| 165 | +var myArray = ["hello", true, false, 13.37]; |
| 166 | +``` |
| 167 | + |
| 168 | +This was previously possible to do, leveraging a few tricks, when you didn't need to potentially read the values from the array again in ReScript. But, if you wanted to read back the values, you'd have to do a number of manual steps. |
| 169 | + |
| 170 | +In the above example, reaching back into the values is as simple as pattern matching on them. |
| 171 | + |
| 172 | +Let's look at a few more examples of what untagged variants enable. |
| 173 | + |
| 174 | +### Pattern matching on nullable values |
| 175 | + |
| 176 | +Previously, any value that might be `null` would need to be explicitly converted to an option by using for example `Null.toOption` before you could use pattern matching on it. Here's a typical example of how that could look: |
| 177 | + |
| 178 | +```rescript |
| 179 | +type userAge = {ageNum: Null.t<int>} |
| 180 | +
|
| 181 | +type rec user = { |
| 182 | + name: string, |
| 183 | + age: Null.t<userAge>, |
| 184 | + bestFriend: Null.t<user>, |
| 185 | +} |
| 186 | +
|
| 187 | +let getBestFriendsAge = user => |
| 188 | + switch user.bestFriend->Null.toOption { |
| 189 | + | Some({age}) => |
| 190 | + switch age->Null.toOption { |
| 191 | + | None => None |
| 192 | + | Some({ageNum}) => ageNum->Null.toOption |
| 193 | + } |
| 194 | + | None => None |
| 195 | + } |
| 196 | +``` |
| 197 | + |
| 198 | +As you can see, you need to convert each level of nullables explicitly, which makes it hard to fully utilize pattern matching. With the new unboxed variant representation, we'll instead be able to do this: |
| 199 | + |
| 200 | +```rescript |
| 201 | +// The type definition below is inlined here to examplify, but this definition will live in [Core](https://github.com/rescript-association/rescript-core) and be easily accessible |
| 202 | +module Null = { |
| 203 | + @unboxed type t<'a> = Present('a) | @as(null) Null |
| 204 | +} |
| 205 | +
|
| 206 | +type userAge = {ageNum: Null.t<int>} |
| 207 | +
|
| 208 | +type rec user = { |
| 209 | + name: string, |
| 210 | + age: Null.t<userAge>, |
| 211 | + bestFriend: Null.t<user>, |
| 212 | +} |
| 213 | +
|
| 214 | +let getBestFriendsAge = user => |
| 215 | + switch user.bestFriend { |
| 216 | + | Present({age: Present({ageNum: Present(ageNum)})}) => Some(ageNum) |
| 217 | + | _ => None |
| 218 | + } |
| 219 | +``` |
| 220 | + |
| 221 | +> Notice how `@as` now allows us to say that an unboxed variant case should map to a specific underlying _primitive_. `Present` has a type variable, so it can hold any type. And since it's an unboxed type, only the payloads `'a` or `null` will be kept at runtime. That's where the magic comes from. |
| 222 | +
|
| 223 | +We can now utilize pattern matching fully without needing to do any conversion. |
| 224 | + |
| 225 | +This has a few implications: |
| 226 | + |
| 227 | +- Dealing with external data, that is often nullable and seldom guaranteed to map cleanly to `option` without needing conversion, becomes much easier and zero cost. |
| 228 | +- Special handling like [@return(nullable)](https://rescript-lang.org/syntax-lookup#return-decorator) becomes redundant. This is good also because the current functionality does not work in all cases. The new functionality will work anywhere. |
| 229 | + |
| 230 | +### Decoding and encoding JSON idiomatically |
| 231 | + |
| 232 | +With unboxed variants, we have everything we need to define a JSON type: |
| 233 | + |
| 234 | +```rescript |
| 235 | +@unboxed |
| 236 | +type rec json = |
| 237 | + | @as(false) False |
| 238 | + | @as(true) True |
| 239 | + | @as(null) Null |
| 240 | + | String(string) |
| 241 | + | Number(float) |
| 242 | + | Object(Js.Dict.t<json>) |
| 243 | + | Array(array<json>) |
| 244 | +
|
| 245 | +let myValidJsonValue = Array([String("Hi"), Number(123.)]) |
| 246 | +``` |
| 247 | + |
| 248 | +Here's an example of how you could write your own JSON decoders easily using the above, leveraging pattern matching: |
| 249 | + |
| 250 | +```rescript |
| 251 | +@unboxed |
| 252 | +type rec json = |
| 253 | + | @as(false) False |
| 254 | + | @as(true) True |
| 255 | + | @as(null) Null |
| 256 | + | String(string) |
| 257 | + | Number(float) |
| 258 | + | Object(Js.Dict.t<json>) |
| 259 | + | Array(array<json>) |
| 260 | +
|
| 261 | +type rec user = { |
| 262 | + name: string, |
| 263 | + age: int, |
| 264 | + bestFriend: option<user>, |
| 265 | +} |
| 266 | +
|
| 267 | +let rec decodeUser = json => |
| 268 | + switch json { |
| 269 | + | Object(userDict) => |
| 270 | + switch ( |
| 271 | + userDict->Dict.get("name"), |
| 272 | + userDict->Dict.get("age"), |
| 273 | + userDict->Dict.get("bestFriend"), |
| 274 | + ) { |
| 275 | + | (Some(String(name)), Some(Number(age)), Some(maybeBestFriend)) => |
| 276 | + Some({ |
| 277 | + name, |
| 278 | + age: age->Float.toInt, |
| 279 | + bestFriend: maybeBestFriend->decodeUser, |
| 280 | + }) |
| 281 | + | _ => None |
| 282 | + } |
| 283 | + | _ => None |
| 284 | + } |
| 285 | +
|
| 286 | +let decodeUsers = json => |
| 287 | + switch json { |
| 288 | + | Array(array) => array->Array.map(decodeUser)->Array.keepSome |
| 289 | + | _ => [] |
| 290 | + } |
| 291 | +``` |
| 292 | + |
| 293 | +Encoding that same structure back into JSON is also easy: |
| 294 | + |
| 295 | +```rescript |
| 296 | +let rec userToJson = user => Object( |
| 297 | + Dict.fromArray([ |
| 298 | + ("name", String(user.name)), |
| 299 | + ("age", Number(user.age->Int.toFloat)), |
| 300 | + ( |
| 301 | + "bestFriend", |
| 302 | + switch user.bestFriend { |
| 303 | + | None => Null |
| 304 | + | Some(friend) => userToJson(friend) |
| 305 | + }, |
| 306 | + ), |
| 307 | + ]), |
| 308 | +) |
| 309 | +
|
| 310 | +let usersToJson = users => Array(users->Array.map(userToJson)) |
| 311 | +``` |
| 312 | + |
| 313 | +This can be extrapolated to many more cases. |
| 314 | + |
| 315 | +## Wrapping up |
| 316 | + |
| 317 | +We hope you'll enjoy using these new capabilities. Some of them are a big leap forward for ReScript's interop with JavaScript and TypeScript, and we hope they will simplify many scenarios and open up a few new doors. |
| 318 | + |
| 319 | +And last but not least, you can try ReScript v11 today by installing `npm i rescript@next`. |
0 commit comments