Skip to content

Commit a469149

Browse files
zxch3nzxch3n
committed
feat: init repo ✨
0 parents  commit a469149

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

85 files changed

+13506
-0
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
jobs:
10+
build-test:
11+
runs-on: ubuntu-latest
12+
13+
strategy:
14+
matrix:
15+
node-version: [20.x, 22.x]
16+
17+
steps:
18+
- name: Checkout repository
19+
uses: actions/checkout@v4
20+
21+
- name: Install pnpm
22+
uses: pnpm/action-setup@v4
23+
with:
24+
version: 9
25+
26+
- name: Setup Node.js ${{ matrix.node-version }}
27+
uses: actions/setup-node@v4
28+
with:
29+
node-version: ${{ matrix.node-version }}
30+
cache: 'pnpm'
31+
32+
- name: Install dependencies
33+
run: pnpm install --frozen-lockfile
34+
35+
- name: Build packages
36+
run: pnpm build
37+
38+
- name: Run tests
39+
run: pnpm test
40+
41+
- name: Type checking
42+
run: pnpm typecheck
43+
44+
- name: Lint
45+
run: pnpm lint

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
node_modules
2+
dist
3+
coverage
4+
.DS_Store
5+

‎.oxlintrc.json‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"categories": {
3+
"correctness": "error",
4+
"suspicious": "warn"
5+
},
6+
"rules": {
7+
"typescript/no-explicit-any": "warn",
8+
"typescript/no-unsafe-type-assertion": "off"
9+
},
10+
"overrides": [
11+
{
12+
"files": ["*.test.ts"],
13+
"rules": {
14+
"typescript/no-explicit-any": "off"
15+
}
16+
}
17+
]
18+
}

‎.prettierrc‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"semi": true,
3+
"trailingComma": "es5",
4+
"singleQuote": false,
5+
"printWidth": 80,
6+
"tabWidth": 2,
7+
"useTabs": false,
8+
"bracketSpacing": true,
9+
"arrowParens": "avoid"
10+
}

‎AGENTS.md‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
# AGENTS.md
2+
3+
Guidance for code agents working in this repository.
4+
5+
## Project Overview
6+
7+
Loro Protocol is a transport‑agnostic synchronization protocol for collaborative real‑time data structures (CRDTs). This monorepo contains:
8+
9+
- TypeScript protocol encoder/decoder (`packages/loro-protocol`)
10+
- WebSocket client and a minimal Node server (`packages/loro-websocket`)
11+
- Adaptors that bridge the protocol to the Loro CRDT (`packages/loro-adaptors`)
12+
- A Rust workspace with protocol/client/server equivalents under `rust/`
13+
14+
No Cloudflare/DO code lives in this repo.
15+
16+
## Common Development Commands
17+
18+
```bash
19+
# Install dependencies
20+
pnpm install
21+
22+
# Build / test / lint across packages
23+
pnpm -r build
24+
pnpm -r test
25+
pnpm -r typecheck
26+
pnpm -r lint
27+
28+
# Dev/watch (where supported)
29+
pnpm -r dev
30+
31+
# Clean build artifacts
32+
pnpm -r clean
33+
```
34+
35+
## Architecture
36+
37+
### Monorepo Structure
38+
39+
- `packages/loro-protocol`: Core wire encoding/decoding logic (TS)
40+
- `packages/loro-websocket`: WebSocket client and `SimpleServer` (TS)
41+
- `packages/loro-adaptors`: Adaptors for `loro-crdt` docs and ephemeral state (TS)
42+
- `examples/excalidraw-example`: React demo using `SimpleServer`
43+
- `rust/`: Rust workspace with `loro-protocol`, `loro-websocket-client`, `loro-websocket-server`
44+
45+
### Protocol Essentials
46+
47+
See `/protocol.md`.
48+
49+
- Message envelope: 4‑byte CRDT magic, varBytes roomId (≤128B), 1‑byte type, payload
50+
- Types: JoinRequest/JoinResponseOk/JoinError, DocUpdate, DocUpdateFragmentHeader/Fragment, UpdateError, Leave
51+
- Limit: 256 KiB max per message; large payloads must be fragmented
52+
- Keepalive: connection‑scoped text frames "ping"/"pong" (out‑of‑band)
53+
54+
Key TS files:
55+
56+
- `packages/loro-protocol/src/{bytes,encoding,protocol}.ts`
57+
58+
## Build & Test
59+
60+
- TypeScript: `vitest` tests are co‑located with sources (`*.test.ts`). The websocket package includes e2e tests spinning up `SimpleServer`.
61+
- Rust: run with `cargo` under `rust/` (tests in crates and `tests/`).
62+
63+
Useful references:
64+
65+
- TS E2E: `packages/loro-websocket/src/e2e.test.ts`
66+
- Rust server example: `rust/loro-websocket-server/examples/simple-server.rs`
67+
68+
## Implementation Notes
69+
70+
- Fragmentation: servers/clients reassemble using `DocUpdateFragmentHeader` + `DocUpdateFragment` with an 8‑byte batch id
71+
- Rooms: one WS connection can join multiple rooms (CRDT+roomId pair)
72+
- CRDTs: Loro (`%LOR`), Loro Ephemeral (`%EPH`), Yjs (`%YJS`), Yjs Awareness (`%YAW`)
73+
- Errors: explicit small codes for join/update failures
74+
- Auth: `SimpleServer` exposes `authenticate` and snapshot load/save hooks
75+
76+
## Development Guidelines
77+
78+
Principles:
79+
80+
- Incremental, boring, and obvious code
81+
- Single responsibility; avoid premature abstractions
82+
- Tests for new behavior; don’t disable existing tests
83+
84+
Process:
85+
86+
1. Understand existing patterns
87+
2. Add/adjust tests
88+
3. Implement minimally to pass
89+
4. Refactor with tests passing
90+
91+
Quality gates:
92+
93+
- All packages build and tests pass
94+
- Lint/format clean
95+
- Clear commit messages (explain “why”)
96+
97+
When stuck (≤3 attempts): document failures, explore alternatives, question assumptions, try a simpler angle.

‎CLAUDE.md‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with this repository.
4+
5+
## Project Overview
6+
7+
Transport‑agnostic sync protocol and tooling for collaborative CRDTs (Loro, Yjs, etc.). The repo contains TypeScript packages (protocol, websocket client/server, adaptors) and Rust equivalents under `rust/`.
8+
9+
## Common Development Commands
10+
11+
```bash
12+
pnpm install
13+
pnpm -r build
14+
pnpm -r test
15+
pnpm -r typecheck
16+
pnpm -r lint
17+
pnpm -r clean
18+
```
19+
20+
## Architecture
21+
22+
### Structure
23+
24+
- `packages/loro-protocol`: protocol types + encoder/decoder (TS)
25+
- `packages/loro-websocket`: WS client and `SimpleServer` (TS)
26+
- `packages/loro-adaptors`: adaptors for `loro-crdt` doc/ephemeral (TS)
27+
- `examples/excalidraw-example`: demo using `SimpleServer`
28+
- `rust/`: Rust workspace with protocol/client/server crates
29+
30+
### Protocol
31+
32+
Source of truth: `/protocol.md`.
33+
34+
- Envelope: 4‑byte CRDT magic, varBytes roomId (≤128B), 1‑byte type, payload
35+
- Types: JoinRequest/JoinResponseOk/JoinError, DocUpdate, DocUpdateFragmentHeader/Fragment, UpdateError, Leave
36+
- Limits: 256 KiB per message; fragment large updates
37+
- Keepalive: connection‑scoped text frames "ping"/"pong"
38+
39+
Key TS files: `packages/loro-protocol/src/{bytes,encoding,protocol}.ts`.
40+
41+
## Testing
42+
43+
- TS: unit and e2e tests with `vitest` (e.g. `packages/loro-websocket/src/e2e.test.ts`).
44+
- Rust: `cargo test` in `rust/` crates.
45+
46+
## Notes for Implementers
47+
48+
- Fragmentation: reassemble fragments by batch header + index; timeout triggers UpdateError.FragmentTimeout
49+
- Rooms: a WS connection can join multiple CRDT+room pairs
50+
- Auth/persistence hooks exposed by `SimpleServer` and Rust server
51+
52+
## Development Guidelines
53+
54+
Keep changes small and test‑backed. Prefer clarity over cleverness. Follow existing patterns and formatting; ensure all tests pass. When stuck, document attempts and try simpler alternatives.

‎README.md‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# Loro Protocol Monorepo
2+
3+
loro-protocol is a small, transport-agnostic syncing protocol for collaborative CRDT documents. This repo hosts the protocol implementation, a WebSocket client, and minimal servers for local testing or self‑hosting.
4+
5+
- Protocol: multiplex multiple rooms on one connection, 256 KiB max per message, large update fragmentation supported
6+
- CRDTs: Loro document, Loro ephemeral store; extensible (e.g., Yjs, Yjs Awareness)
7+
- Transports: WebSocket or any integrity-preserving transport (e.g., WebRTC)
8+
9+
See `protocol.md` for the full wire spec.
10+
11+
## Packages
12+
13+
- `packages/loro-protocol` (MIT): Core TypeScript definitions, encoders/decoders for the wire protocol
14+
- `packages/loro-websocket` (MIT): WebSocket client + a SimpleServer for local testing
15+
- `packages/loro-adaptors` (MIT): Shared CRDT adaptors for Loro documents and ephemeral state
16+
17+
Rust workspace (AGPL):
18+
19+
- `rust/loro-protocol`: Rust encoder/decoder mirroring the TS implementation
20+
- `rust/loro-websocket-client`: Async WS client for the protocol
21+
- `rust/loro-websocket-server`: Minimal async WS server with optional SQLite snapshotting
22+
23+
## Quick Start (Local)
24+
25+
Use the minimal WebSocket server for local development and tests.
26+
27+
1. Install dependencies
28+
29+
```bash
30+
pnpm install
31+
pnpm -r build
32+
```
33+
34+
2. Start a SimpleServer (Node.js)
35+
36+
Create `examples/simple-server.ts`:
37+
38+
```ts
39+
import { SimpleServer } from "loro-websocket/server";
40+
41+
const server = new SimpleServer({ port: 8787 });
42+
server.start().then(() => {
43+
// eslint-disable-next-line no-console
44+
console.log("SimpleServer listening on ws://localhost:8787");
45+
});
46+
```
47+
48+
Run it (Node 18+):
49+
50+
```bash
51+
node --loader ts-node/esm examples/simple-server.ts
52+
# or compile to JS first with your preferred setup
53+
```
54+
55+
3. Connect a client and sync a Loro document
56+
57+
```ts
58+
// examples/client.ts
59+
import { LoroWebsocketClient, createLoroAdaptor } from "loro-websocket";
60+
61+
// In Node, provide a WebSocket implementation
62+
import { WebSocket } from "ws";
63+
(globalThis as any).WebSocket = WebSocket;
64+
65+
const client = new LoroWebsocketClient({ url: "ws://localhost:8787" });
66+
await client.waitConnected();
67+
68+
const adaptor = createLoroAdaptor({ peerId: 1 });
69+
const room = await client.join({ roomId: "demo-room", crdtAdaptor: adaptor });
70+
71+
// Edit the shared doc
72+
const text = adaptor.getDoc().getText("content");
73+
text.insert(0, "Hello, Loro!");
74+
adaptor.getDoc().commit();
75+
76+
// Later…
77+
await room.destroy();
78+
```
79+
80+
Tip: For a working reference, see `packages/loro-websocket/src/e2e.test.ts` which spins up `SimpleServer` and syncs two clients end‑to‑end.
81+
82+
### Optional: SimpleServer hooks
83+
84+
`SimpleServer` accepts optional hooks for basic auth and persistence:
85+
86+
```ts
87+
const server = new SimpleServer({
88+
port: 8787,
89+
authenticate: async (roomId, crdt, auth) => {
90+
// return 'read' | 'write' | null to deny
91+
return "write";
92+
},
93+
onLoadDocument: async (roomId, crdt) => null, // return snapshot bytes
94+
onSaveDocument: async (roomId, crdt, data) => {
95+
// persist snapshot bytes somewhere (e.g., filesystem/db)
96+
},
97+
saveInterval: 60_000, // ms
98+
});
99+
```
100+
101+
### Alternative: Rust server
102+
103+
The Rust workspace contains a minimal async WebSocket server (`loro-websocket-server`) with optional SQLite persistence. See `rust/loro-websocket-server/examples/simple-server.rs` for a CLI example.
104+
105+
## Protocol Highlights
106+
107+
- Magic bytes per CRDT: "%LOR" (Loro doc), "%EPH" (Loro ephemeral), "%YJS", "%YAW", …
108+
- Messages: JoinRequest/JoinResponseOk/JoinError, DocUpdate, DocUpdateFragmentHeader/Fragment, UpdateError, Leave
109+
- Limits: 256 KiB per message; large updates must be fragmented; default reassembly timeout 10s
110+
- Multi‑room: room ID is part of every message; one connection can join multiple rooms
111+
112+
See `protocol.md` for the full description and error codes.
113+
114+
## Monorepo Dev
115+
116+
- Build all: `pnpm -r build`
117+
- Test all: `pnpm -r test`
118+
- Typecheck: `pnpm -r typecheck`
119+
- Lint: `pnpm -r lint`
120+
121+
Node 18+ is required for local development.
122+
123+
## Licensing
124+
125+
- `loro-protocol`: MIT
126+
- `loro-websocket`: MIT
127+
- `loro-adaptors`: MIT
128+
- Rust workspace crates under `rust/`: AGPL-3.0-only
129+
130+
## Project Structure
131+
132+
```
133+
.
134+
├── protocol.md # Wire protocol spec
135+
├── packages/
136+
│ ├── loro-protocol/ # Core encoders/decoders (MIT)
137+
│ ├── loro-websocket/ # Client + SimpleServer (MIT)
138+
│ ├── loro-adaptors/ # Shared CRDT adaptors (MIT)
139+
├── rust/ # Rust implementations (AGPL)
140+
│ ├── loro-protocol/
141+
│ ├── loro-websocket-client/
142+
│ └── loro-websocket-server/
143+
└── pnpm-workspace.yaml
144+
```
145+
146+
## FAQ
147+
148+
- How do I test locally? Use `SimpleServer` in `loro-websocket` or the Rust server.
149+
- Can I bring my own auth/storage? Yes — `SimpleServer` and the Rust server provide hooks for auth and persistence.

0 commit comments

Comments
 (0)