Skip to content

Commit 0f704a6

Browse files
committed
Merge origin/main into perf/scheduler
- Resolve README.md, execution-plan.md, echo-total.md conflicts. - Keep scheduler lint-only changes; no runtime logic edits. - Regenerate docs/echo-total.md per Docs Guard.
2 parents 61499dd + bf6d289 commit 0f704a6

File tree

6 files changed

+274
-90
lines changed

6 files changed

+274
-90
lines changed

README.md

Lines changed: 173 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -13,30 +13,24 @@
1313
//! (Recursively, in the Metaverse)
1414
```
1515

16-
<img src="https://github.com/user-attachments/assets/d31abba2-276e-4740-b370-b4a9c80b30de" height="500" align="right" />
16+
## **tl;dr:**
1717

18+
> Echo is a recursive metagraph (RMG) simulation engine that treats _everything_–code, data, and time itself—as one big living graph.
19+
> It’s built so every change can branch, merge, and replay perfectly.
1820
19-
> _Echo is a recursive metagraph (RMG) simulation engine that executes and rewrites typed graphs deterministically across branching timelines and merges them through confluence._
21+
<img src="https://github.com/user-attachments/assets/d31abba2-276e-4740-b370-b4a9c80b30de" height="400" align="right" />
2022

2123
### Say what??
2224

2325
**Echo is an ambitious, mind-bending, radically different computational model for game engines and other interactive simulations.** The RMG is a powerful mathematical tool that brings the full weight of textbook category theory to interactive computational experiences.
2426

2527
Most game engines are object-oriented state machines. Unity, Unreal, Godot all maintain mutable object hierarchies that update every frame. Echo says: "No, everything is a graph, and the engine rewrites that graph deterministically using typed transformation rules."
2628

27-
## Snapshot Hashes
28-
29-
Echo records two hashes during a commit:
30-
- `state_root`: deterministic hash of the reachable graph state under the current root.
31-
- `commit hash` (commit_id): hash of a canonical header including `state_root`, parents, and deterministic digests for plan/decisions/rewrites.
32-
33-
See `docs/spec-merkle-commit.md` for the precise encoding and invariants.
34-
3529
Echo is fundamentally **built different**.
3630

37-
RMG provides atomic, in-place edits of recursive meta-graphs with deterministic local scheduling and snapshot isolation.
31+
RMG provides atomic, in-place edits of recursive meta-graphs with deterministic local scheduling and snapshot isolation. It’s the core of the Echo engine: runtime, assets, networking, and tools all operate on the same living graph of graphs.
3832

39-
It’s the core of the Echo engine: runtime, assets, networking, and tools all operate on the same living graph of graphs.
33+
Echo is a mathematically rigorous game engine that replaces traditional OOP with deterministic graph rewriting, enabling time-travel debugging, perfect replay, and Git-like branching for game states.
4034

4135
## Developer: Running Benchmarks
4236

@@ -62,20 +56,13 @@ It’s the core of the Echo engine: runtime, assets, networking, and tools all o
6256

6357
---
6458

65-
## **tl;dr:**
59+
### What's Echo?
6660

67-
> ECHO is a game engine that treats _everything_—code, data, and time itself—as one big living graph.
68-
> It’s built so every change can branch, merge, and replay perfectly.
69-
70-
---
71-
72-
### The short pitch
73-
74-
ECHO runs on something called an **RMG (Recursive Meta-Graph)**. Think of it as a graph-based operating system. Everything in the engine (worlds, entities, physics, shaders, even the tools) lives inside that graph.
61+
Echo runs on something called an **RMG (Recursive Meta-Graph)**. Think of it as a graph-based operating system. Everything in the engine (worlds, entities, physics, shaders, even the tools) lives inside that graph.
7562

7663
Echo doesn’t “update objects.” It _rewrites_ parts of the graph using a set of deterministic rules. That’s what “graph rewriting” means.
7764

78-
### Why this is cool
65+
### Why Echo's Cool
7966

8067
- **Deterministic:** same inputs = same world every time.
8168
- **Branching:** you can fork reality, change it, and merge it back without chaos.
@@ -92,7 +79,7 @@ You can pause time, fork a copy of reality, try out a new idea, and merge the ti
9279

9380
## Advantages
9481

95-
> *"Things are only impossible until they're not."* — Jean-Luc Picard
82+
> _"Things are only impossible until they're not." — Jean-Luc Picard_
9683
9784
Can your game engine do...
9885

@@ -107,7 +94,7 @@ Same input graph + same rules = same output, always. This is huge for:
10794

10895
### Branching Timelines
10996

110-
> “All we have to decide is what to do with the time that is given to us.” — _Gandalf, The Lord of the Rings_
97+
> _“All we have to decide is what to do with the time that is given to us.” — Gandalf, The Lord of the Rings_
11198
11299
The Git metaphor is accurate. Fork reality, try something, merge back. This enables:
113100

@@ -134,61 +121,176 @@ Rules are graphs. Systems are graphs. The whole runtime is a graph. This gives y
134121

135122
---
136123

137-
## Current Status
124+
| Principle | Vision | Implementation |
125+
| :--- | :--- | :--- |
126+
| **Determinism (The "Replay")** | Same input graph + same rules = same output graph. Always. This is huge for networked multiplayer (no desync), perfect replays, and reproducible bug testing. | Achieved via an $O(n)$ deterministic scheduler. Pending graph rewrites are sorted using a stable radix sort (not a comparison-based sort) based on their scope, rule ID, and nonce. Combined with a deterministic math module (Vec3, Quat, PRNG), this ensures identical inputs always produce identical execution order and final state. |
127+
| **Branching & Confluence (The "Time Travel")** | Fork reality, try something, and merge it back like a Git branch. Independent, non-conflicting changes converge to the same canonical state, guaranteed. | The engine's Timeline Tree (modeling Chronos, Kairos, and Aion) allows for branching realities. The engine's core transaction model (begin, apply, commit) and footprint-based independence checks (MWMR) allow for safe, parallel execution and deterministic, conflict-free merges. |
128+
| **Snapshot Isolation (The "Commit")** | Snapshots are emitted from the live graph; append-only history is optional. This enables save/load, time-travel debugging, and collaborative editing. | Each commit produces two Merkle hashes derived from 256-bit BLAKE3: <ul><li><code>state_root</code>: deterministic hash of the reachable graph state under the current root.</li><li><code>commit hash</code> (commit_id): hash of a canonical header including <code>state_root</code>, parents, and deterministic digests for plan/decisions/rewrites.</li></ul> See <code>docs/spec-merkle-commit.md</code> for the precise encoding and invariants. |
129+
| **Everything-is-a-Graph (The "Substrate")** | Nodes, edges, systems, assets, and even rewrite rules are all graphs. Graphs can contain subgraphs recursively. | The engine operates on typed, directed graphs. All identifiers (NodeId, TypeId, EdgeId) are domain-separated BLAKE3 hashes. Rewrite Rules are defined with a matcher, executor, and compute_footprint function, allowing the engine to deterministically transform the graph. |
130+
| **Hexagonal Architecture (The "Ports")** | A core engine that is pure logic, completely decoupled from the outside world (rendering, input, networking). | Echo uses a Ports & Adapters design. The core engine (rmg-core) knows nothing of pixels, sockets, or key presses. It exposes narrow interfaces ("Ports") that external crates ("Adapters") implement. This allows swapping renderers (e.g., WebGPU, SDL) or physics engines without changing the core simulation logic. |
138131

139-
Echo is in **active development**. We're currently:
132+
---
140133

141-
- ✅ Formal proofs of confluence (tick-level determinism proven)
142-
-[C implementation](http://github.com/meta-graph/core) of independence checks and footprint calculus
143-
- ✅ 200-iteration property tests validating commutativity
144-
- 🚧 Performance optimization (subgraph matching, spatial indexing)
145-
- 🚧 Rust rewrite of core runtime
146-
- ❌ Lua scripting integration (not started)
147-
- ❌ Rendering backend (not started)
134+
## Architecture
148135

149-
---
136+
> _“Roads? Where we’re going, we don’t need roads.” — Doc Brown, Back to the Future_
137+
138+
Echo is a Rust workspace organized into a multi-crate setup. The core engine is pure, dependency-free Rust (#![no_std] capable) with I/O isolated to adapter crates.
150139

151-
### The Math Checks Out
140+
```bash
141+
echo/
142+
├── crates/
143+
│ ├── rmg-core/ (Core engine: RMG, scheduler, transaction model, snapshotting)
144+
│ ├── rmg-geom/ (Geometry primitives: AABB, transforms, broad-phase)
145+
│ ├── rmg-benches/ (Criterion microbenchmarks: snapshot_hash, scheduler_drain)
146+
│ ├── rmg-wasm/ (WebAssembly bindings for tools and web)
147+
│ ├── rmg-ffi/ (C ABI for Lua/host integration)
148+
│ └── rmg-cli/ (Command-line interface, demos launcher)
149+
├── docs/ (Comprehensive specifications and diagrams)
150+
└── scripts/ (Build automation, benchmarking)
151+
```
152152

153-
The mathematical properties of RMGs offer:
153+
### Core Architectural Layers
154154

155-
- **Folds (catamorphisms)**: there is a guaranteed, one-true way to “walk” the graph.
156-
- That’s how rendering, physics, and serialization all stay consistent: they’re just different folds over the same data.
157-
- **Double-Pushout (DPO) rewriting**: a safe, proven way to modify graphs.
158-
- Instead of ad-hoc mutation, every change is a rewrite rule with an explicit match and replacement, so the engine can reason about merges, rollbacks, and conflicts.
159-
- **Confluence** – when two people or two threads make compatible edits, they deterministically converge to the same state.
160-
- That’s the key to multiplayer sync, time-travel debugging, and collaborative editing.
155+
1. ECS (Entity-Component-System): Type-safe components with archetype-based storage
156+
2. Scheduler: Deterministic DAG ordering via O(n) radix sort
157+
3. Event Bus: Command buffering for deterministic event handling
158+
4. Timeline Tree: Branching/merging with temporal mechanics (Chronos, Kairos, Aion)
159+
5. Ports & Adapters: Renderer, Input, Physics, Networking, Audio, Persistence
160+
6. Deterministic Math: Vec3, Mat4, Quat, PRNG with reproducible operations
161161

162-
There's a ton of other advanced reasons why it's cool, but that's nerd stuff. Let's just say that the RMG is weird, and **extremely powerful.**
162+
### Key Technical Concepts
163163

164-
---
164+
#### Recursive Meta Graph Core
165+
166+
The engine operates on typed, directed graphs:
167+
168+
- Nodes = typed entities with component data
169+
- Edges = typed relationships between nodes
170+
- Rules = deterministic transformations that match patterns and rewrite subgraphs
171+
172+
All identifiers are 256-bit BLAKE3 hashes with domain separation:
173+
174+
```rust
175+
pub type Hash = [u8; 32];
176+
pub struct NodeId(pub Hash); // Entities
177+
pub struct TypeId(pub Hash); // Type descriptors
178+
pub struct EdgeId(pub Hash); // Relationships
179+
```
180+
181+
#### Deterministic Rewriting
182+
183+
Each tick follows a transaction model:
184+
185+
1. begin() → Create new transaction
186+
2. apply(tx, rule) → Enqueue pending rewrites
187+
3. commit(tx) → Execute in deterministic order, emit snapshot
188+
189+
#### $O(n)$ Deterministic Scheduler
190+
191+
Rewrites are ordered using stable radix sort (not comparison-based):
192+
193+
- Order: (`scope_hash`, `rule_id`, `nonce`) lexicographically
194+
- Time: $O(n)$ with 20 passes of 16-bit radix digits
195+
196+
This ensures identical initial state + rules = identical execution order and final snapshot.
165197

166-
### Learning the Vision
198+
#### Snapshot Hashing (Merkle Commits)
167199

168-
> *“Roads? Where we’re going, we don’t need roads.” — Doc Brown, Back to the Future*
200+
Two hashes per commit:
169201

170-
- Read [`docs/architecture-outline.md`](docs/architecture-outline.md) for the full spec (storage, scheduler, ports, timelines).
171-
- Explore [`docs/diagrams.md`](docs/diagrams.md) for Mermaid visuals of system constellations and the Chronos loop.
172-
- Honor Caverns with [`docs/memorial.md`](docs/memorial.md)—we carry the torch forward.
173-
- Peek at [`docs/legacy-excavation.md`](docs/legacy-excavation.md) to see which ideas survived the archaeological roast.
174-
- Track active work in [`docs/execution-plan.md`](docs/execution-plan.md); update it every session.
202+
- `state_root`: BLAKE3 of canonical graph encoding (sorted nodes/edges)
203+
- `commit_id`: BLAKE3 of commit header (`state_root` + parent + plan + decisions + rewrites)
175204

176-
### Docs
205+
### Footprints & Independence (MWMR)
177206

178-
- Dev server: `make docs` (opens browser; uses `PORT` env var, default 5173)
179-
- Static build: `make docs-build`
180-
- Single-file rollup: `make echo-total` (generates `docs/echo-total.md` from top‑level docs; commit the result if it changes)
207+
For parallel rewriting:
181208

182-
### CI Tips
209+
```rust
210+
struct Footprint {
211+
n_read, n_write: IdSet, // Node reads/writes
212+
e_read, e_write: IdSet, // Edge reads/writes
213+
b_in, b_out: PortSet, // Boundary ports
214+
factor_mask: u64, // Spatial partitioning hint
215+
}
216+
```
217+
218+
Disjoint footprints = independent rewrites = safe parallel execution.
219+
220+
### Component Interaction
221+
222+
#### `rmg-core` (`crates/rmg-core/src/`)
223+
224+
- `engine_impl.rs`: Transaction lifecycle, rewrite application
225+
- `scheduler.rs`: $O(n)$ radix drain, conflict detection
226+
- `graph.rs`: BTreeMap-based node/edge storage
227+
- `snapshot.rs`: State root and commit ID computation
228+
- `rule.rs`: Rewrite rule definitions with pattern matching
229+
- `footprint.rs`: Independence checks for concurrent execution
230+
- `math/`: Deterministic Vec3, Mat4, Quat, PRNG
231+
232+
## Execution Flow
233+
234+
```c
235+
loop {
236+
let tx = engine.begin();
237+
238+
// Application phase
239+
for rule in rules_to_apply {
240+
engine.apply(tx, rule, &scope)?;
241+
}
242+
243+
// Deterministic execution
244+
let snapshot = engine.commit(tx)?;
245+
246+
// Emit to networking, tools, etc.
247+
publish_snapshot(snapshot);
248+
}
249+
```
250+
251+
## Design Principles
252+
253+
1. Determinism as Foundation: Every operation must produce identical results given identical input
254+
2. Snapshot Isolation: State captured as immutable graph hashes (not event logs)
255+
3. Hexagonal Architecture: Core never touches I/O directly; all flows through ports
256+
4. Dependency Injection: Services wired at bootstrap for hot-reload support
257+
5. Property-Based Testing: Extensive use of proptest for mathematical invariants
258+
259+
## Current Status
260+
261+
Phase 1 MVP (active development on echo/pr-12-snapshot-bench):
262+
263+
✅ Completed:
264+
- Formal confluence proofs (tick-level determinism proven)
265+
- Rust core runtime with transaction model
266+
- 200+ property tests validating commutativity
267+
- Benchmark infrastructure with D3 dashboard
268+
269+
🚧 In Progress:
270+
- Performance optimization (subgraph matching, spatial indexing)
271+
- Temporal mechanics integration
272+
273+
## Key Files to Explore
274+
275+
### Documentation
276+
277+
- `README.md` — Project vision
278+
- `docs/architecture-outline.md` — Full system design
279+
- `docs/spec-rmg-core.md` — RMG Core spec v2
280+
- `docs/spec-merkle-commit.md` — Snapshot hashing spec
281+
- `docs/spec-scheduler.md` — Deterministic scheduler design
282+
283+
### Core Implementation
183284

184-
- Manual macOS run: trigger the "CI (macOS — manual)" workflow from the Actions tab to run fmt/clippy/tests on macOS on demand.
185-
- Reproduce CI locally:
186-
- Format: `cargo fmt --all -- --check`
187-
- Clippy: `cargo clippy --all-targets -- -D warnings -D missing_docs`
188-
- Tests: `cargo test --workspace`
189-
- Rustdoc (warnings as errors): `RUSTDOCFLAGS="-D warnings" cargo doc -p rmg-core --no-deps` (repeat for other crates)
190-
- Security: `cargo install cargo-audit --locked && cargo audit --deny warnings`
191-
- Dependency policy: `cargo deny check` (requires `cargo-deny`)
285+
- `crates/rmg-core/src/engine_impl.rs` — Engine core
286+
- `crates/rmg-core/src/scheduler.rs` — O(n) scheduler
287+
- `crates/rmg-core/src/snapshot.rs` — Merkle hashing
288+
- `crates/rmg-core/src/demo/motion.rs` — Example rewrite rule
289+
290+
### Tests & Benchmarks:
291+
292+
- `crates/rmg-core/tests/permutation_commute_tests.rs` — Determinism proofs
293+
- `crates/rmg-benches/benches/snapshot_hash.rs` — Hashing throughput
192294

193295
---
194296

@@ -201,8 +303,8 @@ There's a ton of other advanced reasons why it's cool, but that's nerd stuff. Le
201303
202304
- Start each task by verifying a clean git state and branching (`echo/<feature>` recommended).
203305
- Tests go in `packages/echo-core/test/` (fixtures in `test/fixtures/`). End-to-end scenarios will eventually live under `apps/playground`.
204-
- Use expressive commits (`subject` / `body` / optional `trailer`)—tell future us the *why*, not just the *what*.
205-
- Treat determinism as sacred: prefer Echo’s PRNG, avoid non-deterministic APIs without wrapping them.
306+
- Use expressive commits (`subject` / `body` / optional `trailer`). Tell future us the *why*, not just the *what*.
307+
- Treat determinism as sacred: use Echo’s PRNG, avoid non-deterministic APIs without wrapping them.
206308

207309
### Git Hooks
208310

@@ -212,15 +314,6 @@ Install the repo’s hooks so formatting and quick checks run before commits:
212314
make hooks
213315
```
214316

215-
- The pre-commit hook auto-fixes formatting by default (runs `cargo fmt --all`).
216-
- To switch to check-only mode for a commit, set `ECHO_AUTO_FMT=0`:
217-
218-
```
219-
ECHO_AUTO_FMT=0 git commit -m "your message"
220-
```
221-
222-
You can also export `ECHO_AUTO_FMT=0` in your shell rc if you prefer check-only always.
223-
224317
### Development Principles
225318

226319
1. **Tests First** – Write failing unit/integration/branch tests before new engine work.
@@ -230,12 +323,12 @@ You can also export `ECHO_AUTO_FMT=0` in your shell rc if you prefer check-only
230323

231324
### Roadmap Highlights
232325

233-
- [x] **Phase 0** – Finalize specs and design.
234-
- [ ] **Phase 1** – Ship Echo Core MVP with tests and headless harness.
235-
- [ ] **Phase 2 “Double-Jump”** – Deliver reference render/input adapters and the playground.
236-
- [ ] **Phase 3+** – Physics, WebGPU, audio, inspector, and full temporal tooling.
326+
**Phase 0** – Finalize specs and design.
327+
**Phase 1** – Ship Echo Core MVP with tests and headless harness.
328+
☑️ **Phase 2** – Deliver reference render/input adapters and **the playground**.
329+
☑️ **Phase 3+** – Physics, WebGPU, audio, inspector, and full temporal tooling.
237330

238-
Chrononauts welcome. Strap in, branch responsibly, and leave the timeline cleaner than you found it.
331+
**Chrononauts welcome.** Strap in, branch responsibly, and leave the timeline cleaner than you found it.
239332

240333
---
241334

crates/rmg-core/src/math/mod.rs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ use std::f32::consts::TAU;
88
mod mat4;
99
mod prng;
1010
mod quat;
11+
mod scalar;
1112
mod vec3;
1213

1314
#[doc(inline)]
@@ -17,6 +18,8 @@ pub use prng::Prng;
1718
#[doc(inline)]
1819
pub use quat::Quat;
1920
#[doc(inline)]
21+
pub use scalar::Scalar;
22+
#[doc(inline)]
2023
pub use vec3::Vec3;
2124

2225
/// Degeneracy threshold used by math routines to detect near-zero magnitudes.

0 commit comments

Comments
 (0)