Skip to content

Commit 56960c1

Browse files
subtleGradientsubtleGradient
committed
Round 26: macOS universal binary, docs alignment, realistic scenario tests
- Add make universal target for fat binary (aarch64 + x86_64) - Update zig/README.md to reflect 154/154 tests passing - Add 3 realistic scenario tests as executable documentation - Create task cards for tracking (TASK-027, TASK-028) - Add start-here.md gap analysis for orchestration
1 parent e18fad8 commit 56960c1

File tree

9 files changed

+1336
-20
lines changed

9 files changed

+1336
-20
lines changed

‎.tasks/active/TASK-027-docs-alignment.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# TASK-027: Docs Alignment (zig/README.md)
2+
3+
## Status
4+
- [x] Planned
5+
- [x] Assigned
6+
- [x] In Progress
7+
- [x] Complete
8+
9+
## Priority
10+
high
11+
12+
## Assigned To
13+
subagent (docs-alignment)
14+
15+
## Description
16+
Fix zig/README.md to accurately reflect the current state of the Zig implementation. The README currently claims "partial C oracle" and "missing alter" but research/zig-cr/92-gap-backlog.md shows MVP is complete with 154/154 tests passing.
17+
18+
## Files to Modify
19+
- `zig/README.md` - Update status table and known limitations
20+
21+
## Acceptance Criteria
22+
- [x] Status table shows accurate test counts (154/154 total)
23+
- [x] C oracle tests show 20/20 (not "partial")
24+
- [x] `crsql_begin_alter` / `crsql_commit_alter` listed as implemented (not missing)
25+
- [x] Known Limitations section updated to reflect actual remaining gaps
26+
- [x] Reference to `research/zig-cr/10-test-oracle.md` and `core/src/*.test.c` as acceptance suite
27+
28+
## Source of Truth
29+
- `research/zig-cr/92-gap-backlog.md` (current status)
30+
- Test harness results from `zig/harness/` and `zig/browser-test/`
31+
32+
## Progress Log
33+
### 2025-12-14
34+
- Task created from start-here.md gap analysis
35+
36+
## Completion Notes
37+
### 2025-12-14
38+
- Updated Status table to show 154/154 tests passing (100%)
39+
- Changed C oracle tests from "Partial 3/4" to "Complete 20/20 (5 suites)"
40+
- Updated test breakdown: 64 Zig unit, 52 parity, 18 browser, 20 C oracle
41+
- Added `crsql_begin_alter`/`crsql_commit_alter` to Implemented Functions
42+
- Added `crsql_fract_as_ordered` and `crsql_fract_fix_conflict_return_old_key` to Implemented Functions
43+
- Replaced false "Not yet implemented" claims with actual remaining gaps (performance, Service Worker, reactive queries, Windows/iOS/Android packaging)
44+
- Added reference to `research/zig-cr/10-test-oracle.md` (test oracle strategy)
45+
- Added reference to `research/zig-cr/92-gap-backlog.md` (current status)

‎.tasks/active/TASK-028-spec-first-realistic-tests.md‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# TASK-028: Spec-First Realistic Scenario Tests
2+
3+
## Status
4+
- [x] Planned
5+
- [x] Assigned
6+
- [x] In Progress
7+
- [x] Complete
8+
9+
## Priority
10+
medium
11+
12+
## Assigned To
13+
subagent (spec-tests)
14+
15+
## Description
16+
Create new shell-based tests under `zig/harness/` that serve as both executable examples AND documentation. These tests should demonstrate realistic use cases that a person would actually want to do with CR-SQLite. Follow the "thing-golf" philosophy from `research/thing-golf.md` — keep them clean, focused, and smugly self-validating.
17+
18+
## Constraint
19+
Per `.wishes/stop-before-typescript.md`: **NO TypeScript**. All new tests must be Zig or shell-based. If a scenario truly demands TS (e.g., browser-specific features), document it and mark as blocked.
20+
21+
## Files to Create/Modify
22+
- `zig/harness/test-realistic-sync.sh` - Realistic multi-device sync scenario
23+
- `zig/harness/test-realistic-collab.sh` - Collaborative editing scenario
24+
- `zig/harness/test-realistic-offline.sh` - Offline-first scenario with conflict resolution
25+
26+
## Acceptance Criteria
27+
- [x] At least 2 new realistic scenario tests created
28+
- [x] Each test is self-documenting (clear comments explaining the scenario)
29+
- [x] Each test demonstrates a real-world use case (not just edge cases)
30+
- [x] Tests serve dual purpose: validation AND example documentation
31+
- [x] All new tests pass
32+
- [x] Tests follow existing harness conventions (exit codes, output format)
33+
- [x] `make test-parity` still works (if tests are added to parity suite)
34+
35+
## Example Scenarios to Consider
36+
1. **Multi-device sync**: Two "devices" make changes, sync via crsql_changes, both converge
37+
2. **Collaborative editing**: Concurrent edits to same row, CRDT merge resolves correctly
38+
3. **Offline-first**: Device goes offline, makes changes, reconnects, syncs cleanly
39+
4. **Fractional ordering**: Collaborative list reordering with `crsql_fract_key_between`
40+
41+
## Progress Log
42+
### 2025-12-14
43+
- Task created from .wishes/spec-first-RGRTDD.md
44+
- Implemented 3 realistic scenario tests (all passing)
45+
46+
## Completion Notes
47+
### Completed 2025-12-14
48+
49+
Created 3 shell-based realistic scenario tests in `zig/harness/`:
50+
51+
1. **test-realistic-sync.sh** - Multi-Device Todo List Sync
52+
- Alice and Bob each make changes offline
53+
- Both sync via crsql_changes
54+
- Demonstrates: crsql_as_crr(), extracting changes, applying changes, convergence
55+
- Shows site_id origin tracking
56+
57+
2. **test-realistic-collab.sh** - Collaborative Document Editing
58+
- Concurrent edits to same cell
59+
- Demonstrates: col_version conflict resolution (higher wins)
60+
- Shows tie-breaker behavior (same col_version → larger value wins)
61+
- Proves sync order independence (CRDT property)
62+
63+
3. **test-realistic-offline.sh** - Offline-First Field Worker App
64+
- Field worker accumulates changes while offline
65+
- Server receives updates from other workers
66+
- Bidirectional sync on reconnect
67+
- Demonstrates: db_version as sync cursor, incremental sync pattern
68+
69+
All tests:
70+
- Are self-documenting with extensive comments
71+
- Follow existing harness conventions (set -euo pipefail, temp cleanup, exit codes)
72+
- Use INTEGER PRIMARY KEYs (TEXT PKs have WIP insertIntoBaseTable support)
73+
- Pass consistently

‎.tasks/backlog/TASK-026-A-macos-universal-binary.md‎

Lines changed: 47 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@
22

33
## Status
44
- [x] Planned
5-
- [ ] Assigned
6-
- [ ] In Progress
7-
- [ ] Complete
5+
- [x] Assigned
6+
- [x] In Progress
7+
- [x] Complete
88

99
## Priority
1010
medium
@@ -19,26 +19,63 @@ Create a macOS universal binary (aarch64 + x86_64) for the Zig CR-SQLite extensi
1919
- `zig/Makefile` - Add universal binary target
2020

2121
## Acceptance Criteria
22-
- [ ] `make universal` target exists in `zig/Makefile`
23-
- [ ] Builds both aarch64-macos and x86_64-macos
24-
- [ ] Uses `lipo -create` to combine into single binary
25-
- [ ] Output at `zig-out-universal/lib/libcrsqlite.dylib`
26-
- [ ] `lipo -info` shows both architectures
27-
- [ ] All tests pass with universal binary
22+
- [x] `make universal` target exists in `zig/Makefile`
23+
- [x] Builds both aarch64-macos and x86_64-macos
24+
- [x] Uses `lipo -create` to combine into single binary
25+
- [x] Output at `zig-out-universal/lib/libcrsqlite.dylib`
26+
- [x] `lipo -info` shows both architectures
27+
- [x] All tests pass with universal binary
2828

2929
## Implementation Notes
3030
From Round 23 research:
3131
```makefile
3232
build-macos-universal: build-macos-arm64 build-macos-x64
33+
mkdir -p zig-out-universal/lib
3334
lipo -create \
3435
zig-out-arm64/lib/libcrsqlite.dylib \
3536
zig-out-x64/lib/libcrsqlite.dylib \
3637
-output zig-out-universal/lib/libcrsqlite.dylib
3738
```
3839

40+
### Critical: Distinct --prefix per Architecture
41+
Zig's `build.zig` outputs to `zig-out/` by default. To build two architectures without stomping outputs:
42+
43+
```bash
44+
# Build aarch64
45+
zig build -Dtarget=aarch64-macos --prefix zig-out-arm64
46+
47+
# Build x86_64
48+
zig build -Dtarget=x86_64-macos --prefix zig-out-x64
49+
```
50+
51+
### Environment Notes
52+
- On Apple Silicon Mac, both targets can be built locally (no remote builder needed)
53+
- Zig cross-compiles to x86_64-macos natively without additional dependencies
54+
- Verify with: `file zig-out-arm64/lib/libcrsqlite.dylib` (should show "arm64")
55+
- Verify with: `file zig-out-x64/lib/libcrsqlite.dylib` (should show "x86_64")
56+
3957
## Progress Log
4058
### 2025-12-14
4159
- Task created from Round 23 research findings
4260

4361
## Completion Notes
44-
[To be filled when complete]
62+
### 2025-12-14
63+
Implemented `make universal` target in `zig/Makefile`:
64+
65+
**Changes made:**
66+
- Added `build-arm64` target: builds aarch64-macos to `zig-out-arm64/`
67+
- Added `build-x64` target: builds x86_64-macos to `zig-out-x64/`
68+
- Added `universal` target: depends on both, combines with `lipo -create`
69+
- Updated `clean` target to remove all new output directories
70+
- Updated `help` target to document new targets
71+
72+
**Verification results:**
73+
```
74+
$ lipo -info zig-out-universal/lib/libcrsqlite.dylib
75+
Architectures in the fat file: x86_64 arm64
76+
77+
$ nix run nixpkgs#sqlite -- :memory: -cmd '.load zig-out-universal/lib/libcrsqlite' -cmd 'SELECT crsql_version()'
78+
0.0.1-zig-scaffold
79+
```
80+
81+
Universal binary loads and works correctly on macOS.

‎.wishes/start-here.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
Gaps (zig/ vs research/zig-cr)
2+
- Docs mismatch: zig/README.md:5 still claims partial C oracle + missing alter, but research/zig-cr/92-gap-backlog.md:7 says MVP complete and 154/154 passing. This is a real “gap” because it will mislead anyone trying to use/validate the Zig implementation.
3+
- Performance hotspot items not yet done (from research/zig-cr/11-performance-hotspots.md:6 and tracked in research/zig-cr/92-gap-backlog.md:161):
4+
- PRAGMA schema_version-keyed invalidation caching
5+
- PRAGMA data_version amortization (per-transaction / per-query-loop flag)
6+
- sqlite3_prepare_v3(...SQLITE_PREPARE_PERSISTENT...) use for long-lived statements
7+
- Web multi-tab “Phase 2” items still open (called out in research/zig-cr/92-gap-backlog.md:186 and motivated by research/zig-cr/96-proposal-multitab-wasm-sqlite-crsqlite.md:10):
8+
- Service Worker fallback (when SharedWorker missing)
9+
- Reactive query subscriptions / notifications surface (more than “manual re-query”)
10+
- Cross-platform packaging gaps (tracked in research/zig-cr/92-gap-backlog.md:214):
11+
- macOS universal binary (aarch64+x86_64)
12+
- Windows .dll build
13+
- iOS/Android static embedding guide
14+
- Big future proposals not implemented (expected): the “threadsafe-by-default / mesh” family (research/zig-cr/99-threadsafe-sqlite-proposals.md:31, research/zig-cr/100-proposal-node-multiprocess-crsqlite-mesh.md:19, research/zig-cr/101-proposal-universal-crsqlite-mt-js.md:34, research/zig-cr/102-proposal-crsqlite-global-mesh.md:5) has no corresponding implementation in zig/ yet.
15+
Wishes review (.wishes/)
16+
- /.wishes/spec-first-RGRTDD.md:1 asks for new “realistic” spec/tests-as-examples now that we’re green everywhere.
17+
- /.wishes/stop-before-typescript.md:1 is an explicit constraint: do not race ahead on new TS work; if we feel pressure to do TS-heavy features, we should write it down and mark it blocked.
18+
Tasks review (.tasks/)
19+
- Only one backlog task exists: /.tasks/backlog/TASK-026-A-macos-universal-binary.md:1.
20+
- It’s directionally right, but the implementation notes are currently underspecified relative to zig/build.zig’s default single zig-out/ install prefix (zig/build.zig:46). To build two mac targets without stomping outputs, the task should explicitly require separate --prefix (and likely separate cache dirs) per target before lipo.
21+
Proposed next concurrent subagent assignments (do not assign yet)
22+
1) macOS universal dylib (highest near-term “ship-ability”)
23+
- Based on existing backlog /.tasks/backlog/TASK-026-A-macos-universal-binary.md:1
24+
- Files: zig/Makefile:1 (add make universal or similar), maybe CI workflow later
25+
- Output verification: lipo -info ... + load-test with sqlite3 via nix
26+
2) Performance tranche: schema/data version amortization + persistent statements
27+
- Source: research/zig-cr/11-performance-hotspots.md:6, backlog research/zig-cr/92-gap-backlog.md:161
28+
- Files likely centered on zig/src/changes_vtab.zig and wherever we prepare hot statements (plus any stmt cache module)
29+
- Goal: reduce prepare churn + pragma spam without changing semantics
30+
3) Windows .dll build reconnaissance + minimal build target (even if not fully CI’d yet)
31+
- Source: research/zig-cr/92-gap-backlog.md:215
32+
- Files: zig/build.zig:1 (target/export correctness), plus a small harness note on how to test load on Windows
33+
- Outcome: either a real build artifact path or a crisp “blocked by X” statement with next steps
34+
4) Docs alignment task (small but high leverage)
35+
- Fix zig/README.md:5 to reflect reality and point at the real acceptance suite location (research/zig-cr/10-test-oracle.md / core/src/*.test.c)
36+
- This closes the “people can’t tell what’s done” gap immediately
37+
5) Spec-first “realistic scenarios” tests (non-TS)
38+
- Convert /.wishes/spec-first-RGRTDD.md:8 into a small set of new harness tests under zig/harness/ that read like executable examples.
39+
- Keep it Zig/shell first to respect /.wishes/stop-before-typescript.md:1.
40+
- If any scenario truly demands TS (e.g. SW fallback), we write a blocked wish/task instead of implementing.
41+
.tasks accuracy fixups (recommend before we start work)
42+
- Update /.tasks/backlog/TASK-026-A-macos-universal-binary.md:31 to explicitly require per-target zig build with distinct --prefix outputs (so the lipo paths in the task are actually achievable), and clarify whether we expect nix cross-target builds to work on this host or if it needs a remote builder.

‎zig/Makefile‎

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@
1010
# make help - Show this help
1111

1212
.PHONY: all test test-unit test-parity test-browser build clean help \
13-
_test-unit _test-parity _test-browser dist
13+
_test-unit _test-parity _test-browser dist \
14+
universal build-arm64 build-x64
1415

1516
# ANSI colors for output
1617
CYAN := \033[36m
@@ -28,6 +29,29 @@ build:
2829
@echo "$(CYAN)$(BOLD)Building Zig extension...$(RESET)"
2930
@nix run nixpkgs#zig -- build
3031

32+
# Build aarch64 (ARM64) macOS target
33+
build-arm64:
34+
@echo "$(CYAN)$(BOLD)Building aarch64-macos...$(RESET)"
35+
@nix run nixpkgs#zig -- build -Dtarget=aarch64-macos --prefix zig-out-arm64
36+
@echo "$(GREEN)Built: zig-out-arm64/lib/libcrsqlite.dylib$(RESET)"
37+
38+
# Build x86_64 (Intel) macOS target
39+
build-x64:
40+
@echo "$(CYAN)$(BOLD)Building x86_64-macos...$(RESET)"
41+
@nix run nixpkgs#zig -- build -Dtarget=x86_64-macos --prefix zig-out-x64
42+
@echo "$(GREEN)Built: zig-out-x64/lib/libcrsqlite.dylib$(RESET)"
43+
44+
# Build macOS universal binary (aarch64 + x86_64)
45+
universal: build-arm64 build-x64
46+
@echo "$(CYAN)$(BOLD)Creating universal binary...$(RESET)"
47+
@mkdir -p zig-out-universal/lib
48+
@lipo -create \
49+
zig-out-arm64/lib/libcrsqlite.dylib \
50+
zig-out-x64/lib/libcrsqlite.dylib \
51+
-output zig-out-universal/lib/libcrsqlite.dylib
52+
@echo "$(GREEN)$(BOLD)Universal binary created: zig-out-universal/lib/libcrsqlite.dylib$(RESET)"
53+
@lipo -info zig-out-universal/lib/libcrsqlite.dylib
54+
3155
# Zig unit tests
3256
test-unit: build
3357
@echo ""
@@ -175,7 +199,7 @@ dist:
175199
# Clean build artifacts
176200
clean:
177201
@echo "$(CYAN)Cleaning build artifacts...$(RESET)"
178-
@rm -rf zig-out .zig-cache
202+
@rm -rf zig-out zig-out-arm64 zig-out-x64 zig-out-universal .zig-cache
179203
@rm -rf browser-test/node_modules
180204
@rm -f ../.tmp/test-*.log ../.tmp/test-*.exit
181205
@echo "$(GREEN)Clean complete$(RESET)"
@@ -192,6 +216,9 @@ help:
192216
@echo ""
193217
@echo "$(BOLD)Build Targets:$(RESET)"
194218
@echo " $(CYAN)make build$(RESET) - Build the Zig extension"
219+
@echo " $(CYAN)make universal$(RESET) - Build macOS universal binary (aarch64 + x86_64)"
220+
@echo " $(CYAN)make build-arm64$(RESET) - Build aarch64-macos only"
221+
@echo " $(CYAN)make build-x64$(RESET) - Build x86_64-macos only"
195222
@echo " $(CYAN)make clean$(RESET) - Clean build artifacts"
196223
@echo ""
197224
@echo "$(BOLD)Other:$(RESET)"

‎zig/README.md‎

Lines changed: 25 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,21 @@ A pure Zig port of [CR-SQLite](https://github.com/vlcn-io/cr-sqlite), providing
44

55
## Status
66

7+
**MVP COMPLETE** — 154/154 tests passing (100%)
8+
79
| Feature | Status | Notes |
810
|---------|--------|-------|
9-
| Core replication | ✅ Complete | 44 parity tests pass |
10-
| Browser WASM | ✅ Complete | 10 tests pass via Playwright |
11-
| Multi-tab web | ✅ Infrastructure complete | SharedWorker coordination |
12-
| Fractional indexing | ✅ `crsql_fract_key_between` | Collaborative ordering |
13-
| C oracle tests | 🔄 Partial | 3/4 test suites pass |
11+
| Core replication | ✅ Complete | 52 parity tests pass |
12+
| Browser WASM | ✅ Complete | 18 tests pass via Playwright |
13+
| Multi-tab web | ✅ Complete | SharedWorker + OPFS + re-election |
14+
| Fractional indexing | ✅ Complete | All UDFs implemented |
15+
| C oracle tests | ✅ Complete | 20/20 pass (5 suites) |
16+
17+
**Test Breakdown:**
18+
- Zig unit tests: 64/64
19+
- Shell parity tests: 52/52
20+
- Browser WASM tests: 18/18
21+
- C oracle harness: 20/20
1422

1523
### Implemented Functions
1624

@@ -21,13 +29,20 @@ A pure Zig port of [CR-SQLite](https://github.com/vlcn-io/cr-sqlite), providing
2129
- `crsql_is_crr(table)` - Check if table is a CRR
2230
- `crsql_rows_impacted()` - Rows changed in current transaction
2331
- `crsql_finalize()` - Cleanup before close
32+
- `crsql_begin_alter(table)` / `crsql_commit_alter(table)` - Schema migration workflow
2433
- `crsql_fract_key_between(a, b)` - Fractional index generation
34+
- `crsql_fract_as_ordered(table, order_col, ...)` - Ordered collection view + triggers
35+
- `crsql_fract_fix_conflict_return_old_key(...)` - Collision repair
2536
- `crsql_changes` - Virtual table for sync
2637

2738
### Known Limitations
2839

29-
- `crsql_begin_alter` / `crsql_commit_alter` - Not yet implemented
30-
- Some edge cases in C oracle test suite still failing
40+
Post-MVP items not yet implemented:
41+
- Performance: `PRAGMA schema_version` caching, `SQLITE_PREPARE_PERSISTENT`
42+
- Web: Service Worker fallback (for environments without SharedWorker)
43+
- Web: Reactive query subscriptions in RPC interface
44+
- Packaging: Windows `.dll` build, iOS/Android static embedding guides
45+
- Packaging: macOS universal binary (aarch64 + x86_64)
3146

3247
## Build Instructions
3348

@@ -201,4 +216,6 @@ cd zig && make test
201216
- [CR-SQLite Documentation](https://vlcn.io/docs/cr-sqlite/intro)
202217
- [Wire Format Spec](../research/zig-cr/09-storage-serialization.md)
203218
- [Merge Semantics](../research/zig-cr/05-conflict-resolution-semantics.md)
204-
- [C Test Oracle](../core/src/*.test.c)
219+
- [Test Oracle Strategy](../research/zig-cr/10-test-oracle.md) - Behavioral contract and acceptance criteria
220+
- [C Test Oracle Source](../core/src/*.test.c) - Authoritative test suite
221+
- [Gap Backlog](../research/zig-cr/92-gap-backlog.md) - Current status and remaining work

0 commit comments

Comments
 (0)