Skip to content

Commit 5fda98a

Browse files
subtleGradientsubtleGradient
committed
++
1 parent 72ab370 commit 5fda98a

13 files changed

+2033
-160
lines changed

‎.tasks/active/TASK-089-api-surface-completeness.md‎

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
# TASK-089: Oracle Parity — API surface completeness
2+
3+
## Status
4+
- [ ] Planned
5+
- [ ] Assigned
6+
- [ ] In Progress
7+
- [ ] Blocked (reason: ...)
8+
- [x] Complete
9+
10+
## Priority
11+
high
12+
13+
## Assigned To
14+
Claude (completed 2025-12-17)
15+
16+
## Parent Docs / Cross-links
17+
- Rust extension entry: `core/rs/core/src/lib.rs`
18+
- Zig extension entry: `zig/src/crsqlite.zig`
19+
- Gap backlog: `research/zig-cr/92-gap-backlog.md`
20+
21+
## Description
22+
Verify that Zig exposes the same SQL API surface as Rust/C. Use `pragma_function_list` and `pragma_module_list` to enumerate all registered functions and virtual table modules, then compare.
23+
24+
This is an **oracle test**: Rust/C is the golden master. Any function or module present in Rust/C but missing from Zig is a gap.
25+
26+
## Files to Modify
27+
- `zig/harness/test-api-surface.sh` (new)
28+
- `zig/harness/test-parity.sh` (wire into suite)
29+
- `research/zig-cr/92-gap-backlog.md`
30+
31+
## Acceptance Criteria
32+
- [x] Test extracts function list from Rust/C extension: `SELECT name FROM pragma_function_list WHERE name LIKE 'crsql%' ORDER BY name`
33+
- [x] Test extracts function list from Zig extension using same query.
34+
- [x] Test extracts module list from both: `SELECT name FROM pragma_module_list WHERE name LIKE 'crsql%' OR name = 'clset' ORDER BY name`
35+
- [x] Test fails if Rust/C has functions/modules not present in Zig.
36+
- [x] Test documents which functions are intentionally excluded (if any) with rationale.
37+
- [x] Functions to verify include (at minimum):
38+
- `crsql_as_crr`, `crsql_as_table` ✓ (both present in Zig)
39+
- `crsql_begin_alter`, `crsql_commit_alter` ✓ (both present in Zig)
40+
- `crsql_changes`, `crsql_tracked_peers` ✓ (crsql_changes present; crsql_tracked_peers not in Rust/C either - it's a different concept)
41+
- `crsql_db_version`, `crsql_next_db_version` ✓ (both present in Zig)
42+
- `crsql_site_id`, `crsql_siteid` (alias) ✓ (crsql_site_id present; alias not found in Rust/C pragma_function_list)
43+
- `crsql_finalize` ✓ (present in Zig)
44+
- `crsql_fract_key_between` ✓ (present in Zig)
45+
- `crsql_pack_columns`, `crsql_rows_impacted` ✓ (both present in Zig)
46+
- `crsql_automigrate` (if implemented) ✗ MISSING - in Rust/C, not in Zig
47+
- `crsql_config_get`, `crsql_config_set` (if implemented) ✗ MISSING - in Rust/C, not in Zig
48+
49+
## Progress Log
50+
### 2025-12-18
51+
- Task created from oracle-based parity test suite.
52+
53+
### 2025-12-17 (execution)
54+
- Created `zig/harness/test-api-surface.sh` - oracle parity test comparing Rust/C vs Zig
55+
- Wired into `zig/harness/test-parity.sh`
56+
- Ran `bash zig/harness/test-parity.sh` - API surface test integrated successfully
57+
- Documented all gaps discovered
58+
59+
## Commands Run
60+
```bash
61+
# Standalone API surface test
62+
bash /Users/tom/Developer/effect-native/cr-sqlite/zig/harness/test-api-surface.sh
63+
64+
# Full parity suite
65+
bash /Users/tom/Developer/effect-native/cr-sqlite/zig/harness/test-parity.sh
66+
```
67+
68+
## Full Test Output
69+
```
70+
╔═══════════════════════════════════════════════════════════════════════╗
71+
║ API Surface Parity Test (Oracle: Rust/C) ║
72+
╚═══════════════════════════════════════════════════════════════════════╝
73+
74+
Rust/C extension: /Users/tom/Developer/effect-native/cr-sqlite/lib/crsqlite.dylib
75+
Zig extension: /Users/tom/Developer/effect-native/cr-sqlite/lib/crsqlite-zig-darwin-aarch64.dylib
76+
77+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
78+
Extracting function lists...
79+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
80+
81+
Rust/C functions (23):
82+
crsql_after_delete
83+
crsql_after_insert
84+
crsql_after_update
85+
crsql_as_crr
86+
crsql_as_table
87+
crsql_automigrate
88+
crsql_begin_alter
89+
crsql_commit_alter
90+
crsql_config_get
91+
crsql_config_set
92+
crsql_db_version
93+
crsql_finalize
94+
crsql_fract_as_ordered
95+
crsql_fract_fix_conflict_return_old_key
96+
crsql_fract_key_between
97+
crsql_get_seq
98+
crsql_increment_and_get_seq
99+
crsql_internal_sync_bit
100+
crsql_next_db_version
101+
crsql_pack_columns
102+
crsql_rows_impacted
103+
crsql_sha
104+
crsql_site_id
105+
106+
Zig functions (18):
107+
crsql_as_crr
108+
crsql_as_table
109+
crsql_begin_alter
110+
crsql_commit_alter
111+
crsql_db_version
112+
crsql_finalize
113+
crsql_fract_as_ordered
114+
crsql_fract_fix_conflict_return_old_key
115+
crsql_fract_key_between
116+
crsql_increment_and_get_seq
117+
crsql_internal_sync_bit
118+
crsql_is_crr
119+
crsql_next_db_version
120+
crsql_pack_columns
121+
crsql_rows_impacted
122+
crsql_site_id
123+
crsql_version
124+
crsql_zig_version
125+
126+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
127+
Extracting module lists...
128+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
129+
130+
Rust/C modules (3):
131+
clset
132+
crsql_changes
133+
crsql_unpack_columns
134+
135+
Zig modules (1):
136+
crsql_changes
137+
138+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
139+
Comparing API surfaces...
140+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
141+
142+
FAIL: 8 functions in Rust/C but missing from Zig:
143+
- crsql_after_delete
144+
- crsql_after_insert
145+
- crsql_after_update
146+
- crsql_automigrate
147+
- crsql_config_get
148+
- crsql_config_set
149+
- crsql_get_seq
150+
- crsql_sha
151+
152+
INFO: 3 functions in Zig but not in Rust/C (Zig-specific):
153+
+ crsql_is_crr
154+
+ crsql_version
155+
+ crsql_zig_version
156+
157+
FAIL: 2 modules in Rust/C but missing from Zig:
158+
- clset
159+
- crsql_unpack_columns
160+
161+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
162+
Intentional Exclusions (documented rationale):
163+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
164+
165+
crsql_after_delete, crsql_after_insert, crsql_after_update:
166+
Internal trigger functions - not part of public API.
167+
These are registered for use by auto-generated triggers only.
168+
169+
crsql_sha:
170+
Debug/utility function - not essential for CRDT operations.
171+
172+
crsql_siteid (alias):
173+
Legacy alias - crsql_site_id is the canonical function.
174+
175+
176+
╔═══════════════════════════════════════════════════════════════════════╗
177+
║ API SURFACE SUMMARY ║
178+
╠═══════════════════════════════════════════════════════════════════════╣
179+
║ Rust/C Functions: 23 ║
180+
║ Zig Functions: 18 ║
181+
║ Rust/C Modules: 3 ║
182+
║ Zig Modules: 1 ║
183+
╠═══════════════════════════════════════════════════════════════════════╣
184+
║ Missing from Zig: 10 ║
185+
╚═══════════════════════════════════════════════════════════════════════╝
186+
187+
✗ API surface parity: FAIL (10 gaps found)
188+
```
189+
190+
## Gaps Discovered
191+
192+
### Functions Missing from Zig (require implementation)
193+
| Function | Priority | Notes |
194+
|----------|----------|-------|
195+
| `crsql_automigrate` | Medium | Schema migration support |
196+
| `crsql_config_get` | Medium | Configuration API |
197+
| `crsql_config_set` | Medium | Configuration API |
198+
| `crsql_get_seq` | Low | Sequence getter (crsql_increment_and_get_seq exists) |
199+
200+
### Functions Intentionally Excluded (internal/debug)
201+
| Function | Rationale |
202+
|----------|-----------|
203+
| `crsql_after_delete` | Internal trigger function |
204+
| `crsql_after_insert` | Internal trigger function |
205+
| `crsql_after_update` | Internal trigger function |
206+
| `crsql_sha` | Debug/utility function |
207+
208+
### Modules Missing from Zig
209+
| Module | Priority | Notes |
210+
|--------|----------|-------|
211+
| `clset` | Medium | Changeset virtual table |
212+
| `crsql_unpack_columns` | Low | Column unpacking vtab |
213+
214+
### Zig-Specific Additions (not in Rust/C)
215+
| Function | Notes |
216+
|----------|-------|
217+
| `crsql_is_crr` | Utility to check if table is a CRR |
218+
| `crsql_version` | Version string |
219+
| `crsql_zig_version` | Zig implementation version |
220+
221+
## Completion Notes
222+
Task completed. Created `zig/harness/test-api-surface.sh` that:
223+
1. Loads both extensions into clean sqlite3 processes
224+
2. Extracts function and module lists via pragma queries
225+
3. Compares using `comm` to find gaps
226+
4. Reports missing items with clear categorization
227+
5. Documents intentional exclusions with rationale
228+
229+
The test is wired into `test-parity.sh` and runs as part of `make -C zig test-parity`.
230+
231+
**10 total gaps identified** (8 functions + 2 modules), of which 4 functions are intentionally excluded (internal trigger functions + debug utility).

.tasks/backlog/TASK-091-fract-index-algorithm-parity.md renamed to .tasks/active/TASK-091-fract-index-algorithm-parity.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
## Status
44
- [ ] Planned
55
- [ ] Assigned
6-
- [ ] In Progress
6+
- [x] In Progress
77
- [ ] Blocked (reason: ...)
88
- [ ] Complete
99

‎.tasks/active/TASK-092-db-version-advancement-parity.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# TASK-092: Oracle Parity — db_version advancement timing
2+
3+
## Status
4+
- [ ] Planned
5+
- [ ] Assigned
6+
- [ ] In Progress
7+
- [ ] Blocked (reason: ...)
8+
- [x] Complete
9+
10+
## Priority
11+
high
12+
13+
## Assigned To
14+
(unassigned)
15+
16+
## Parent Docs / Cross-links
17+
- Rust db_version logic: `core/rs/core/src/db_version.rs`
18+
- Zig db_version logic: `zig/src/crsqlite.zig` (crsql_db_version, crsql_next_db_version)
19+
- Gap backlog: `research/zig-cr/92-gap-backlog.md`
20+
21+
## Description
22+
Verify that `crsql_db_version()` and `crsql_next_db_version()` increment at exactly the same moments in both implementations.
23+
24+
This is an **oracle test**: The db_version is critical for sync protocols. If Zig and Rust/C increment it at different times (e.g., per-statement vs per-transaction), sync will break.
25+
26+
## Files to Modify
27+
- `zig/harness/test-db-version-parity.sh` (new)
28+
- `zig/harness/test-parity.sh` (wire into suite)
29+
- `research/zig-cr/92-gap-backlog.md`
30+
31+
## Acceptance Criteria
32+
- [x] Test performs identical operations in both implementations and records db_version after each.
33+
- [x] Operations tested:
34+
1. Initial state (should be 0 or 1)
35+
2. Single INSERT → record db_version
36+
3. Single UPDATE → record db_version
37+
4. Multiple INSERTs in one transaction → record db_version at COMMIT
38+
5. DELETE → record db_version
39+
6. No-op UPDATE (same value) → db_version should NOT change
40+
7. Merge from remote (crsql_changes INSERT) → record db_version
41+
8. No-op merge (lower col_version) → record db_version
42+
- [ ] All db_version values match exactly between implementations. **DIVERGENCE FOUND - see below**
43+
- [x] `crsql_next_db_version()` returns `db_version + 1` in both.
44+
- [x] Test fails if any db_version diverges.
45+
46+
## Progress Log
47+
### 2025-12-18
48+
- Task created from oracle-based parity test suite.
49+
50+
### 2025-12-18 (work session)
51+
- Created `zig/harness/test-db-version-parity.sh` with 8 test cases
52+
- Wired into `zig/harness/test-parity.sh`
53+
- Ran tests via `make -C zig test-parity`
54+
55+
#### Commands Run
56+
```bash
57+
bash /Users/tom/Developer/effect-native/cr-sqlite/zig/harness/test-db-version-parity.sh
58+
make -C zig test-parity
59+
```
60+
61+
#### Test Results Summary
62+
```
63+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
64+
db_version Parity Test Summary
65+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
66+
PASSED: 12
67+
FAILED: 1
68+
DIVERGENCES: 1 (critical - will break sync)
69+
```
70+
71+
#### Full Test Output (db_version values at each step)
72+
```
73+
Test 1: Initial db_version state
74+
Rust/C: DB_VERSION=0 NEXT_DB_VERSION=1
75+
Zig: DB_VERSION=0 NEXT_DB_VERSION=1
76+
PASS: Initial db_version matches
77+
PASS: next_db_version = db_version + 1 (Rust/C)
78+
PASS: next_db_version = db_version + 1 (Zig)
79+
80+
Test 2: Single INSERT -> db_version
81+
Rust/C: BEFORE_INSERT_VERSION=0 AFTER_INSERT_VERSION=1 NEXT_DB_VERSION=2
82+
Zig: BEFORE_INSERT_VERSION=0 AFTER_INSERT_VERSION=1 NEXT_DB_VERSION=2
83+
PASS: Single INSERT db_version matches
84+
85+
Test 3: Single UPDATE -> db_version
86+
Rust/C: BEFORE_UPDATE_VERSION=1 AFTER_UPDATE_VERSION=2
87+
Zig: BEFORE_UPDATE_VERSION=1 AFTER_UPDATE_VERSION=2
88+
PASS: Single UPDATE db_version matches
89+
90+
Test 4: Multiple INSERTs in transaction -> db_version at COMMIT
91+
Rust/C: BEFORE_TX_VERSION=0 AFTER_INSERT_1_VERSION=0 AFTER_INSERT_2_VERSION=0 AFTER_INSERT_3_VERSION=0 AFTER_COMMIT_VERSION=1
92+
Zig: BEFORE_TX_VERSION=0 AFTER_INSERT_1_VERSION=0 AFTER_INSERT_2_VERSION=0 AFTER_INSERT_3_VERSION=0 AFTER_COMMIT_VERSION=1
93+
PASS: Multiple INSERTs in TX db_version matches
94+
95+
Test 5: DELETE -> db_version
96+
Rust/C: BEFORE_DELETE_VERSION=1 AFTER_DELETE_VERSION=2
97+
Zig: BEFORE_DELETE_VERSION=1 AFTER_DELETE_VERSION=2
98+
PASS: DELETE db_version matches
99+
100+
Test 6: No-op UPDATE (same value) -> db_version should NOT change
101+
Rust/C: BEFORE_NOOP_VERSION=1 AFTER_NOOP_VERSION=2
102+
Zig: BEFORE_NOOP_VERSION=1 AFTER_NOOP_VERSION=1
103+
DIVERGENCE DETECTED:
104+
Rust/C (oracle):
105+
BEFORE_NOOP_VERSION=1
106+
AFTER_NOOP_VERSION=2
107+
Zig (candidate):
108+
BEFORE_NOOP_VERSION=1
109+
AFTER_NOOP_VERSION=1
110+
FAIL: No-op UPDATE db_version diverges
111+
112+
Test 7: Merge from remote (crsql_changes INSERT) -> db_version
113+
Rust/C: BEFORE_MERGE_VERSION=1 AFTER_MERGE_VERSION=2
114+
Zig: BEFORE_MERGE_VERSION=1 AFTER_MERGE_VERSION=2
115+
PASS: Merge from remote db_version matches
116+
PASS: Merge correctly advanced db_version (Rust/C: 1 -> 2)
117+
PASS: Merge correctly advanced db_version (Zig: 1 -> 2)
118+
119+
Test 8: No-op merge (lower col_version) -> db_version should NOT change
120+
Rust/C: BEFORE_NOOP_MERGE_VERSION=3 AFTER_NOOP_MERGE_VERSION=3
121+
Zig: BEFORE_NOOP_MERGE_VERSION=3 AFTER_NOOP_MERGE_VERSION=3
122+
PASS: No-op merge db_version matches
123+
PASS: No-op merge correctly did not advance db_version
124+
```
125+
126+
## Divergence Discovered
127+
128+
**Test 6: No-op UPDATE (same value)**
129+
130+
| Implementation | Before | After |
131+
|---------------|--------|-------|
132+
| Rust/C (oracle) | 1 | 2 (advances) |
133+
| Zig (candidate) | 1 | 1 (no change) |
134+
135+
**Analysis:**
136+
- Rust/C implementation advances db_version even when UPDATE sets a column to its existing value
137+
- Zig implementation correctly detects no change occurred and does NOT advance db_version
138+
- This is a semantic divergence that could affect sync protocols expecting version advancement
139+
140+
**Sync Impact:**
141+
- If sync protocol uses db_version to detect "any change happened", Zig may not signal no-op updates
142+
- However, since the data hasn't actually changed, this might be intentionally more efficient in Zig
143+
- Need clarification: Is this a bug in Zig (should match Rust) or an intentional optimization?
144+
145+
## Completion Notes
146+
- Test script created and wired into suite
147+
- Critical divergence discovered in no-op UPDATE handling
148+
- Filed for follow-up: Determine if Zig behavior should match Rust/C or if Rust/C has a bug

0 commit comments

Comments
 (0)