chore: add vbare blog

Author: NathanFlurry

site/src/posts/2025-09-24-vbare-simple-schema-evolution-with-maximum-performance/image.png

@@ -0,0 +1,248 @@
+export const author = "nathan-flurry"
+export const published = "2025-09-24"
+export const category = "technical"
+export const keywords = ["bare","serialization","internals"]
+
+# VBARE: A simple alternative to Protobuf & Cap'n Proto for schema evolution
+
+**At Rivet, we're building an open-source alternative to Cloudflare Durable Objects — a tool for running stateful compute workloads. VBARE is a small but crucial component in meeting the demanding performance requirements of Rivet. If you enjoy this article, [consider giving Rivet a star on GitHub](https://github.com/rivet-dev/engine).**
+
+## Growing pains with Protobuf
+
+We decided to adopt Protocol Buffers for any internal communications & data stored at rest. We've always believed it's a flawed technology, but it was the **only mainstream and portable tool** that provided binary serialization and schema evolution — or what we _thought_ would be sufficient schema evolution.
+
+### The problem
+
+Issues started arriving after our v1 launched and we started adding new features that **required significant changes to the datastructures in our Protobufs**. Eventually, all of our Protobuf files started generating bitrot of sorts. They became tedious to read, understand defaults, and understand migration paths. We frequently ended up with a handful of common issues:
+
+- **All new properties need to be optional**: We had to litter our application logic with defaults if a property was none
+- **Properties cannot be moved or restructured**: Say we want to change a bool to an enum or a struct to a union — good luck
+- **Datastructures need to be reshaped**: The most simple example is switching from a list to a map or switching from a bool to a union
+
+More importantly, the more we changed in Protobuf, **more and more cruft had to be added to our application logic** that made adding features a tedious process.
+
+### The crude solution
+
+To solve this, we started following this pattern to allow us to clean up our application logic:
+
+1. Copy and paste the entire Protobuf spec to a v2
+2. Update the v2 to the ideal schema
+3. Write a migration function between v1 and v2
+4. Feed the final v2 version into the application logic
+
+This became such a standard practice that most of our API saw a turnover to a new version once every 3-6 months.
+
+### The breaking point
+
+While this process helped simplify our application logic significantly, it required a lot of manual effort to implement the Protobuf migrations. So we started evaluating other options that might provide better schema evolution.
+
+## Evaluating existing options
+
+### Goals
+
+- **Simple** — Minimal features, can be easily forked and modified if needed
+- **Portable** — Cross-language support with a well-defined standard
+- **Fast** — Self-contained binary encoding (think: binary data without keys), ideally with zero-copy reads
+- **Clean SDK** — Ser/de code has a huge impact on the legibility & friction of working with application logic
+
+### Non-Goals
+
+- **Data compactness** — That's what gzip is for
+- **RPC layer** — This is trivial to implement yourself based on your specific requirements
+
+### Exploring existing options
+
+We evaluated numerous serialization protocols before deciding to build VBARE. Here's a brief overview of why each fell short:
+
+- **Protocol Buffers**: Makes migrations your problem at runtime by making everything optional, with dangerous default values that lead to subtle bugs
+- **Flatbuffers**: Similar issues to Protocol Buffers with field indexing requirements, awful codegen
+- **Cap'n Proto**: Provides powerful schema migrations at the cost of complexity, focused on C++, poor generated clients for non-C++ languages
+- **CBOR/MessagePack/BSON**: Self-describing formats, unsuitable for our performance needs
+- **Bebop/Borsh**: Provides cross-language, self-contained binary encoding. We almost chose one of these instead of BARE, but BARE is simpler.
+- **Rust-specific options (postcard, bincode, etc.)**: Not cross-language, only supports Rust
+
+(_For a detailed comparison, see our [full evaluation document](https://github.com/rivet-dev/vbare/blob/e08ca552b3e3703cf23e4764342be772ddc43879/docs/COMPARISON.md)._)
+
+In the end, none of these solutions provided a compelling schema evolution mechanism.
+
+## VBARE overview
+
+Enter, VBARE: a **tiny extension of BARE to provide a version header** and handle version migrations.
+
+Instead of using an off-the-shelf solution, we opted to build a simple evolution system similar to the pattern we were already using: **manually writing code to migrate between schemas**. We would pair this technique with the BARE encoding to create VBARE.
+
+<Info>
+	[BARE (Binary Application Record Encoding)](https://baremessages.org/) — which VBARE extends — is a simple binary serialization format designed for efficiency and simplicity. Unlike self-describing formats like JSON or CBOR, BARE requires a schema to encode and decode data, similar to Protocol Buffers but with a much simpler design philosophy.
+</Info>
+
+### Schema evolution in VBARE
+
+VBARE's design philosophy centers on four core beliefs:
+
+1. **Make smaller, incremental schema changes** instead of massive v1 to v2 overhauls. Build tools that make this easy and your schema will be easier to work with.
+
+2. **Manual evolution simplifies application logic** by isolating schema evolution logic to a separate module before passing it to the application logic.
+
+3. **Real-world schema evolutions require more than simple property renaming**: it involves complex reshaping and fetching data from remote sources, which automatic migration systems can't handle.
+
+4. **Manual evolution is less error prone** by forcing developers to explicitly handle edge cases of migrations and breaking changes, trading verbosity for safety.
+
+### Versions
+
+VBARE operates by declaring a schema file for every version of your protocol, then writing explicit conversion functions between adjacent versions.
+
+Each message has an associated version number (unsigned 16-bit integer) that increases monotonically from 1. Versions are specified in the filename: `my-schema/v1.bare`, `my-schema/v2.bare`, etc.
+
+### Converters
+
+Servers define conversion code that transforms between versions bidirectionally:
+- **Upgrade** converters for deserialization (old → new)
+- **Downgrade** converters for serialization (new → old)
+
+There are no evolution semantics in the schema language itself. To create a new version, you simply copy the previous schema and make your changes.
+
+### Version Negotiation
+
+Every message has an associated version, which can be either:
+
+- **Embedded** in the message itself (first 2 bytes)
+- **Pre-negotiated** via HTTP paths (`POST /v3/users`), query parameters, or handshakes
+
+### Servers vs Clients
+
+- **Servers** must include converters between all versions to handle any client version.
+
+- **Clients** only need to include a single version since the server handles all version conversion.
+
+## Use cases
+
+Common use cases include:
+
+- **Network protocols** — Allow the server to cleanly evolve the protocol version without breaking old clients
+- **Data at rest** — Upgrade your file format without breaking old files
+
+For examples, VBARE powers almost all of Rivet:
+
+- **[Rivet Engine](https://github.com/rivet-dev/engine)**
+  - [Data at rest](https://github.com/rivet-dev/engine/tree/f62142df1fa538499692deeb44f79b68f6e3c3c0/sdks/schemas/data)
+  - Internal network protocols ([Epoxy](https://github.com/rivet-dev/engine/tree/f62142df1fa538499692deeb44f79b68f6e3c3c0/sdks/schemas/epoxy-protocol), [UPS](https://github.com/rivet-dev/engine/tree/f62142df1fa538499692deeb44f79b68f6e3c3c0/sdks/schemas/ups-protocol))
+  - [Public network protocols](https://github.com/rivet-dev/engine/tree/f62142df1fa538499692deeb44f79b68f6e3c3c0/sdks/schemas/runner-protocol)
+
+- **[RivetKit](https://github.com/rivet-dev/rivetkit)**
+  - [Client protocol](https://github.com/rivet-dev/rivetkit/tree/b81d9536ba7ccad4449639dd83a770eb7c353617/packages/rivetkit/schemas/client-protocol)
+  - [Persisted state](https://github.com/rivet-dev/rivetkit/tree/b81d9536ba7ccad4449639dd83a770eb7c353617/packages/rivetkit/schemas/actor-persist)
+  - [File system driver](https://github.com/rivet-dev/rivetkit/tree/b81d9536ba7ccad4449639dd83a770eb7c353617/packages/rivetkit/schemas/file-system-driver)
+
+
+## Example Code
+
+Here's a simple example demonstrating how VBARE handles a complex schema migration that is not possible with any existing tools:
+
+```text {{"title":"schema/v1.bare"}}
+type User struct {
+  id: string
+  name: string
+}
+```
+
+```text {{"title":"schema/v2.bare"}}
+type User struct {
+  id: string
+  // Split `name` in to 2 properties
+  firstName: string
+  lastName: string
+}
+```
+
+```typescript
+import * as V1 from "./v1";
+import * as V2 from "./v2";
+import { createVersionedDataHandler } from "vbare";
+
+// Converter from v1 to v2
+function upgradeUserV1ToV2(v1: V1.User): V2.User {
+  const [firstName, ...rest] = v1.name.split(' ');
+  return {
+    ...v1,
+    firstName,
+    lastName: rest.join(' ') || ''
+  };
+}
+
+// Converter from v2 to v1
+function downgradeUserV2ToV1(v2: V2.User): V1.User {
+  return {
+    ...v2,
+    name: `${v2.firstName} ${v2.lastName}`.trim()
+  };
+}
+
+// Create versioned data handler
+export const USER_VERSIONED = createVersionedDataHandler<V2.User>({
+  deserializeVersion: (bytes: Uint8Array, version: number): any => {
+    switch (version) {
+      case 1:
+        return V1.decodeUser(bytes);
+      case 2:
+        return V2.decodeUser(bytes);
+      default:
+        throw new Error(`invalid version: ${version}`);
+    }
+  },
+  serializeVersion: (data: any, version: number): Uint8Array => {
+    switch (version) {
+      case 1:
+        return V1.encodeUser(data);
+      case 2:
+        return V2.encodeUser(data);
+      default:
+        throw new Error(`invalid version: ${version}`);
+    }
+  },
+  deserializeConverters: () => [upgradeUserV1ToV2],
+  serializeConverters: () => [downgradeUserV2ToV1],
+});
+
+```
+
+## Implementations
+
+- **[TypeScript](https://github.com/rivet-dev/vbare/tree/main/typescript)** — [Example Code](https://github.com/rivet-dev/vbare/blob/main/typescript/examples/basic/src/index.ts)
+- **[Rust](https://github.com/rivet-dev/vbare/tree/main/rust)** — [Example Code](https://github.com/rivet-dev/vbare/blob/main/rust/examples/basic/src/lib.rs)
+
+For a full list of BARE implementations, visit [baremessages.org](https://baremessages.org/).
+
+## FAQ
+
+### Why is copying the entire schema for every version better than using decorators for gradual migrations?
+
+- Decorators are limited and become very complicated over time
+- It's unclear at what version of the protocol a decorator takes effect — explicit versions help clarify this
+- Generated SDKs become more and more bloated with every change
+- You need a validation build step for your validators
+- Manual migrations provide more flexibility for complex transformations
+
+### Why not include RPC?
+
+RPC interfaces are trivial to implement yourself. Libraries that provide RPC interfaces tend to add extra bloat and cognitive load through things like abstracting transports, compatibility with the language's async runtime, and complex codegen to implement handlers.
+
+Usually, you just want a `ToServer` and `ToClient` union that looks like this: 
+- [ToClient example](https://github.com/rivet-dev/rivetkit/blob/b81d9536ba7ccad4449639dd83a770eb7c353617/packages/rivetkit/schemas/client-protocol/v1.bare#L34)
+- [ToServer example](https://github.com/rivet-dev/rivetkit/blob/b81d9536ba7ccad4449639dd83a770eb7c353617/packages/rivetkit/schemas/client-protocol/v1.bare#L56)
+
+### Don't migration steps get repetitive?
+
+Migration steps are fairly minimal to write. The most verbose migration steps will be for deeply nested structures that changed, but even that is relatively straightforward.
+
+### What are the downsides?
+
+- More verbose migration code — but this is usually because VBARE forces you to handle all edge cases you wouldn't otherwise bother with
+- The older the version, the more migration steps that need to run to bring it to the latest version — though migration steps are usually negligible in cost
+- Migration steps are not portable across languages, but only the server needs the migration steps, so this is usually only implemented once
+
+## Getting started & source code
+
+VBARE is open source and [available on GitHub](https://github.com/rivet-dev/vbare).
+
+See the guides for getting started with [TypeScript](https://github.com/rivet-dev/vbare/tree/main/typescript) and [Rust](https://github.com/rivet-dev/vbare/tree/main/rust).
+