feat(typescript): refactor serialization pipeline

[thebarndog]

Description

Linear ticket: Closes FER-4068.

Note: The majority of the lines of code come from:

• generated tests
• documentation comments
• code reorganization and refactors
• unit tests

Problem Statement

Currently, end users are limited to adding schema serialization and validation via our own proprietary tool, zurg. Unfortunately zurg and the code it produces suffers from performance issues, the inability to tree shake, and bundle size increases. Customers typically have to disable the option which is not ideal because they're not getting fully what they want and it will over time cause feature and api drift.

Proposal

Introduce a new SerializationPipeline and SerializationFormat abstraction to make the serialization system more extensible and unlock the ability to start targeting performance optimizations. I also propose in a future PR to move zurg to be a public npm package because it will:

• increase company and brand visibility
• allow for better performance optimizations like tree-shaking
• allow open source peer review + contributions
• importing zurg will be no different than zod; serialization code will have a similar and unified interface
• removes bundled code from SDK

Changes Made

• Added a SerializationPipeline class
• Added a SerializationFormat type
• Added an entire slew of namespaced types to SerializationFormat i.e. Schema etc
• Implemented serialization output formats for zurg, zod, and passthrough (none or turned off)

How It Works

The SDK and Express configs both now take a new config option, serializationFormat. Users can supply zurg, zod, or none (equivalent of noSerdeLayer true). Internally, the generator will take that format type and construct a full concrete formatter from it and use that to transform the internal Fern representation into generated ASTs. There's a fair amount of complex AST generation code happening but at the top level, it's relatively simple. There's the pipeline, the different formats, and thats about it.

Future Work

This PR is already quite large and there's a lot more work that could have scoped creeped but I believe this is an excellent first step. The next objective would be:

• extract zurg as a public npm package
• do targeted performance optimizations (Zod for example doesn't support bi-directional transforms b/c its just a serialization library so we have to perform the reverse ourselves which currently runnings in O(N) where n is the depth of the recursive object being serialized). That can be improved to O(log n) but would require even further refactors and thought to go into it.
• add Avj, Yup, and Typebox support. Now that the system is extensible, more formats can be easily add
• helper classes for AST generation code to simplify and unify the format implementations right now.

Testing

• Unit tests added/updated
• Performance benchmarks added
• Manual testing completed (via visual spot checks on disk)

[github-actions]

🌱 Seed Test Selector

Select languages to run seed tests for:

• Python
• TypeScript
• Java
• Go
• Ruby
• C#
• PHP
• Swift
• Rust
• OpenAPI
• Postman

---

How to use: Click the ⋯ menu above → "Edit" → check the boxes you want → click "Update comment". Tests will run automatically and snapshots will be committed to this PR.

[thebarndog]

Note that tests will fail until seed tests are regenerated.

[fern-support]

There's a lot of casting, which makes me nervous that at runtime, the result is actually different than what is typed.
We could do something like this: Add a compile-time check using satisfies:

type _Check = z.infer<typeof UserSchema> satisfies User;

This gives you a tsc error if the schema's inferred type drifts from User.