Add binary trace format documentation

Author: nickysn

README.md

@@ -9,16 +9,21 @@ It's self contained, so one can debug a record of a version of a program from a
 The record data consists of a stream of objects, each of which describes a certain program event: call, step, return etc.
 
 A trace contains several files:
-* trace.json : the record data
-* trace_metadata.json : metadata for the recorded program
-* trace_paths.json : a list of the recorded files
-* `files/`: a folder including all the source/repository files copied into the trace.
+* `trace.json` – the event data in JSON format
+* `trace.bin` – the same event stream encoded in a binary format (see
+  [docs/trace_binary_spec.md](docs/trace_binary_spec.md))
+* `trace_metadata.json` – metadata for the recorded program
+* `trace_paths.json` – a list of the recorded files
+* `files/` – a folder including all the source/repository files copied into the trace.
 
-The record data format is currently json which matches the record event rust types from `src/types.rs`. 
+The event stream can be stored either as JSON (`trace.json`) or in the
+binary format (`trace.bin`). Both representations correspond to the Rust
+types in `src/types.rs`.
 
 We plan on 
 * defining a more precise document or a list of examples or a specification.
-* producing a more optimized next version of this format, probably based on a binary format.
+* producing a more optimized version of the format. A binary variant is now
+  available as `trace.bin`.
 
 A future goal of this format is to make it possible to stream traces: to be able to replay them while they're still being recorded. 
 This is one of the reasons for the decision to maintain a single "stream" of events currently. 

docs/trace_binary_spec.md

@@ -0,0 +1,40 @@
+# CodeTracer Trace Binary Format
+
+This document describes the binary format stored in `trace.bin` files produced by the `runtime_tracing` crate. The binary format contains the same information as the JSON trace but encoded using [Cap'n Proto](https://capnproto.org/) for efficiency.
+
+## File Layout
+
+A binary trace begins with an 8‑byte header:
+
+```
+C0 DE 72 AC E2 00 00 00
+```
+
+* The first five bytes (`C0 DE 72 AC E2`) identify the file as a CodeTracer trace.
+* The remaining three bytes are reserved for versioning. They are zero for the initial version. Non‑zero values indicate an incompatible future format.
+
+After the header comes a Cap'n Proto message serialized with the packed encoding. The schema for this message is defined in [`runtime_tracing/src/trace.capnp`](../runtime_tracing/src/trace.capnp). The root object is `Trace`, which contains an array of `TraceLowLevelEvent` values.
+
+The mapping between the Rust data structures and the Cap'n Proto schema is implemented in `capnptrace.rs`. Helper functions `write_trace` and `read_trace` write and read the binary format.
+
+## Usage
+
+To write a binary trace:
+
+```rust
+use runtime_tracing::{Tracer, TraceEventsFileFormat};
+# let mut tracer = Tracer::new("prog", &[]);
+# // record events
+tracer.store_trace_events(Path::new("trace.bin"), TraceEventsFileFormat::Binary)?;
+```
+
+To read it back:
+
+```rust
+let mut tracer = Tracer::new("prog", &[]);
+tracer.load_trace_events(Path::new("trace.bin"), TraceEventsFileFormat::Binary)?;
+```
+
+## Summary
+
+`trace.bin` provides a compact representation of the same event stream described in [Trace JSON Format](trace_json_spec.md). It starts with the 8‑byte magic header followed by a packed Cap'n Proto `Trace` message.