delegate round 48: merge atomicity verify, WAL concurrency tests, sqlite-cr policy (TASK-088, TASK-106, TASK-107)

Author: subtleGradient

.tasks/DELEGATE_WORK_HANDOFF.md

@@ -68,6 +68,109 @@ Artifacts:
 
 ---
 
+## Round 2025-12-20 (48) — Merge atomicity verify + WAL concurrency tests + sqlite-cr policy (3 tasks)
+
+**Tasks executed**
+- `.tasks/done/TASK-088-impl-merge-atomicity.md`
+- `.tasks/done/TASK-106-zig-wal-concurrency-persistence-test.md`
+- `.tasks/done/TASK-107-clarify-sqlite-cr-wrapper-for-zig-tests.md`
+
+**Commits**
+- `0bbec043` — delegate round 48: merge atomicity verify, WAL concurrency tests, sqlite-cr policy (TASK-088, TASK-106, TASK-107)
+
+**Environment**
+- OS: darwin (macOS ARM64)
+- Tooling: nix, bash, zig (via nix)
+
+**Commands run (exact)**
+```bash
+bash zig/harness/test-merge-atomicity.sh
+bash zig/harness/test-wal-concurrency.sh
+```
+
+**Outputs (paste)**
+
+<details>
+<summary>TASK-088: test-merge-atomicity.sh (8/8 pass)</summary>
+
+```text
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+Merge Atomicity Tests Summary: 8 passed, 0 failed, 0 skipped
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+All merge atomicity tests passed!
+```
+
+**Key finding:** No implementation was needed. SQLite's native statement/transaction semantics already provide the required atomicity guarantees. The Zig changes vtab sets `xBegin`, `xCommit`, `xRollback` to `null`, and SQLite's built-in statement atomicity handles the rest.
+</details>
+
+<details>
+<summary>TASK-106: test-wal-concurrency.sh (10/10 pass)</summary>
+
+```text
+╔═══════════════════════════════════════════════════════════════════════╗
+║              WAL CONCURRENCY TEST SUMMARY                            ║
+╠═══════════════════════════════════════════════════════════════════════╣
+║  PASSED:  10                                                         ║
+║  FAILED:  0                                                          ║
+║  SKIPPED: 0                                                          ║
+╚═══════════════════════════════════════════════════════════════════════╝
+
+All WAL concurrency tests PASSED
+```
+
+**Test cases created:**
+1. WAL mode setup and basic isolation
+2. Uncommitted changes NOT visible to other connections (snapshot isolation)
+3. Concurrent readers do not block (3 parallel readers)
+4. Writer does not block readers
+5. CRR changes tracked correctly with multiple connections
+6. db_version consistency across multiple connections
+7. site_id consistency across concurrent connections
+
+**Key findings:**
+- SQLite WAL serializes writes (one writer at a time)
+- Readers don't block writers; writers don't block readers
+- `PRAGMA read_uncommitted` only applies to shared-cache mode
+- CR-SQLite `crsql_site_id()`, `crsql_db_version()`, clock tables work correctly under WAL
+</details>
+
+<details>
+<summary>TASK-107: AGENTS.md policy update</summary>
+
+**Policy decision:** The sqlite-cr wrapper can be used, but ONLY for testing the Rust/C oracle (reference implementation), NEVER for testing the Zig extension.
+
+**Audit results:** All 39 test scripts in `zig/harness/test-*.sh` are compliant:
+- 38 scripts use clean `nix run nixpkgs#sqlite` + explicit `.load $ZIG_EXT`
+- 1 script (`test-alter-parity.sh`) uses sqlite-cr correctly — only for Rust/C oracle
+
+**AGENTS.md updated** with detailed Zig testing policy including:
+- Core rule (no double-loading)
+- sqlite-cr wrapper usage guidelines (ALLOWED vs FORBIDDEN)
+- Test script pattern examples
+- Explanation of conflicts from double-loading
+</details>
+
+**Files created/modified:**
+- `zig/harness/test-wal-concurrency.sh` (new, ~200 lines)
+- `zig/harness/test-parity.sh` (wired in WAL concurrency test)
+- `AGENTS.md` (expanded Zig testing policy section)
+- `.tasks/done/TASK-088-impl-merge-atomicity.md` (completion notes)
+- `.tasks/done/TASK-106-zig-wal-concurrency-persistence-test.md` (completion notes)
+- `.tasks/done/TASK-107-clarify-sqlite-cr-wrapper-for-zig-tests.md` (completion notes)
+- `research/zig-cr/92-gap-backlog.md` (status update)
+
+**Reproduction steps (clean checkout)**
+1. `git clone <repo> && cd cr-sqlite`
+2. `bash zig/harness/test-merge-atomicity.sh` — verify 8/8 pass
+3. `bash zig/harness/test-wal-concurrency.sh` — verify 10/10 pass
+4. Review `AGENTS.md` "Zig testing (detailed policy)" section
+
+**Known gaps / unverified claims**
+- No coverage captured
+- CI integration not verified this round (local runs only)
+
+---
+
 ## Round 2025-12-20 (45) — clset impl + merge atomicity spec + unpack_columns spec (3 tasks)
 
 **Tasks executed**

.tasks/backlog/TASK-088-impl-merge-atomicity.md

@@ -0,0 +1,73 @@
+# TASK-088: Implement (RGRTDD) — Savepoint-backed atomic batch merge in Zig
+
+## Status
+- [ ] Planned
+- [ ] Assigned
+- [ ] In Progress
+- [ ] Blocked (reason: ...)
+- [x] Complete
+
+## Priority
+high
+
+## Assigned To
+(unassigned)
+
+## Parent Docs / Cross-links
+- Spec task: `.tasks/backlog/TASK-087-spec-merge-atomicity.md`
+- Rust reference: `core/rs/core/src/changes_vtab_write.rs`
+- Zig merge entrypoint: `zig/src/changes_vtab.zig`
+- Zig vtab plumbing: `zig/src/sqlite/vtab.zig` (if xBegin/xCommit/xRollback hooks are used)
+- Gap backlog: `research/zig-cr/92-gap-backlog.md`
+
+## Description
+Implement atomicity guarantees for batch apply.
+
+Potential strategies (choose the one that matches SQLite vtab semantics best):
+- Use vtab `xBegin/xCommit/xRollback` to wrap the *statement* in a savepoint.
+- Or: detect statement boundaries and manage a savepoint per statement.
+
+Must satisfy the tests defined in TASK-087.
+
+## Files to Modify
+- `zig/src/changes_vtab.zig`
+- `zig/src/sqlite/vtab.zig` (only if needed)
+- `research/zig-cr/92-gap-backlog.md`
+
+## Acceptance Criteria
+- [x] `zig/harness/test-merge-atomicity.sh` passes.
+- [x] No regression in `make -C zig test-parity`.
+
+## Progress Log
+### 2025-12-18
+- Task created from Rust↔Zig gap analysis.
+
+### 2025-12-20
+- Ran `zig/harness/test-merge-atomicity.sh`: **8/8 tests passed**
+- Verified parity tests: all passing (filter tests, rowid slab, alter, noops, fract)
+- Analyzed Zig implementation: no explicit savepoint logic needed
+
+## Completion Notes
+
+**No implementation was needed.** SQLite's native statement/transaction semantics already provide the required atomicity guarantees for `crsql_changes` batch operations.
+
+### Evidence
+1. `changes_vtab.zig` lines 1997-2005 set `xBegin`, `xCommit`, `xRollback` to **null**
+2. SQLite guarantees single-statement atomicity - a multi-row INSERT either fully commits or fully rolls back
+3. All 8 atomicity tests pass:
+   - Test 1: Multi-row INSERT applies all rows atomically ✓
+   - Test 2: Invalid column in batch causes entire statement to fail ✓
+   - Test 3: rows_impacted resets to 0 after commit ✓
+   - Test 4: Failed transaction commits nothing ✓
+   - Test 5: Explicit savepoints allow partial rollback ✓
+   - Test 6: Duplicate PKs handled correctly (LWW) ✓
+   - Test 7: Base table integrity after failed batch ✓
+   - Test 8: rows_impacted accumulates within transaction ✓
+
+### Why Rust uses explicit savepoints but Zig doesn't need them
+The Rust implementation (`core/rs/core/src/changes_vtab_write.rs`) uses explicit savepoints for additional safety during complex merge operations. The Zig implementation relies on SQLite's native semantics which achieve the same effect for our use case because:
+- Each INSERT INTO crsql_changes is a single statement
+- SQLite atomic statement guarantee applies automatically
+- Transaction boundaries (BEGIN/COMMIT) provide outer atomicity
+
+This is a case where "less code = correct behavior" - SQLite's design already handles the atomicity requirements.