Skip to content

Commit b91e3ad

Browse files
subtleGradientsubtleGradient
committed
Complete 3 parallel tasks: perf hotspots, Windows DLL, hosted WASM proposal
TASK-029: Performance hotspot closure - Add prepare_v3 wrapper with SQLITE_PREPARE_PERSISTENT flag - Add schema_version keyed invalidation caching - Add data_version check amortization (once per transaction) - Wire cached table discovery into changes_vtab TASK-030: Windows DLL build verification - Confirmed zig build -Dtarget=x86_64-windows-gnu works out of box - Verified sqlite3_crsqlite_init export via llvm-objdump - Updated README with Windows build instructions TASK-035: Hosted browser ESM proposal - Comprehensive proposal in zig/browser-dist/README.md - Documents files to host, versioning, API surface, examples - Lists browser requirements and cross-origin considerations All e2e sync tests pass. Moved completed tasks to .tasks/done/
1 parent 7414c3b commit b91e3ad

14 files changed

+916
-183
lines changed

‎.tasks/backlog/TASK-029-performance-hotspot-closure.md‎

Lines changed: 0 additions & 48 deletions
This file was deleted.

‎.tasks/backlog/TASK-030-windows-dll-build.md‎

Lines changed: 0 additions & 44 deletions
This file was deleted.

‎.tasks/backlog/TASK-035-hosted-wasm-proposal.md‎

Lines changed: 0 additions & 49 deletions
This file was deleted.

‎.tasks/done/TASK-029-performance-hotspot-closure.md‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# TASK-029: Performance Hotspot Closure (schema/data version + persistent prepares)
2+
3+
## Status
4+
- [ ] Planned
5+
- [ ] Assigned
6+
- [ ] In Progress
7+
- [ ] Blocked (reason: ...)
8+
- [x] Complete
9+
10+
## Priority
11+
high
12+
13+
## Assigned To
14+
subagent (general)
15+
16+
## Parent Docs / Cross-links
17+
- Gap backlog: `research/zig-cr/92-gap-backlog.md` ("Remaining Gaps → Performance Optimizations")
18+
- Hotspot analysis: `research/zig-cr/11-performance-hotspots.md`
19+
- Likely code hotspots: `zig/src/stmt_cache.zig`, `zig/src/changes_vtab.zig`, `zig/src/ffi/api.zig`
20+
21+
## Description
22+
Close the remaining post-MVP performance gaps that are explicitly tracked but not yet implemented:
23+
24+
1) `PRAGMA schema_version` keyed invalidation caching (avoid unnecessary union rebuild / stmt recreation)
25+
2) `PRAGMA data_version` check amortization (avoid polling/pragma spam on hot loops)
26+
3) Prefer `sqlite3_prepare_v3(..., SQLITE_PREPARE_PERSISTENT)` for long-lived statements
27+
28+
This task is intentionally narrow: improve hot-path behavior without changing query semantics.
29+
30+
## Files to Modify
31+
- `zig/src/stmt_cache.zig`
32+
- `zig/src/changes_vtab.zig`
33+
- `zig/src/ffi/api.zig` (if prepare_v3 / flags wiring needed)
34+
- `research/zig-cr/92-gap-backlog.md` (check off items + add brief notes)
35+
36+
## Acceptance Criteria
37+
- [x] `stmt_cache` exposes an explicit "schema version changed" signal that callers can use to invalidate cached derived artifacts.
38+
- [x] `changes_vtab` avoids rebuilding or repreparing union statements when schema_version unchanged.
39+
- [x] `PRAGMA data_version` checks are amortized (e.g. once per transaction / once per cursor scan loop) with no correctness regressions.
40+
- [x] If supported by the SQLite target, long-lived statements are prepared using `sqlite3_prepare_v3` with `SQLITE_PREPARE_PERSISTENT`.
41+
- [x] `make test-unit` and `make test-parity` pass (pre-existing rowid slab failures excluded).
42+
43+
## Progress Log
44+
### 2025-12-14
45+
- Task created during gap review; not yet started
46+
- Task started - implementing 3 performance optimizations
47+
- Added `prepare_v3` wrapper with `SQLITE_PREPARE_PERSISTENT` flag to `api.zig`
48+
- Added `schemaVersionChanged()` and `getSchemaVersion()` to `stmt_cache.zig` for invalidation signaling
49+
- Added `checkDataVersionAmortized()` and `resetDataVersionCheck()` to `stmt_cache.zig` for per-transaction amortization
50+
- Added `getOrPreparePersistent()` and `prepareOncePersistent()` for long-lived statements
51+
- Added schema-version keyed table name cache to `ChangesVTab` in `changes_vtab.zig`
52+
- Updated `changesFilter` to use schema-version keyed cache, avoiding re-query of sqlite_master when schema unchanged
53+
- Updated `discoverTablesCached` to use `prepareOncePersistent` for persistent statement preparation
54+
- All unit tests pass (64/64)
55+
- Parity tests pass except for pre-existing rowid slab failures (48/52)
56+
57+
## Completion Notes
58+
### 2025-12-14
59+
60+
**Implemented:**
61+
62+
1. **`PRAGMA schema_version` keyed invalidation caching** (`stmt_cache.zig`, `changes_vtab.zig`)
63+
- Added `schema_changed_flag` field to `StmtCache` that is set when schema version changes
64+
- Added `schemaVersionChanged()` method to check and consume the flag
65+
- Added `getSchemaVersion()` to get cached version without re-checking
66+
- Added schema-version keyed table name cache to `ChangesVTab` struct
67+
- `changesFilter` now reuses cached table names when schema_version is unchanged
68+
- New helper functions: `getCachedTableNames()`, `setCachedTableNames()`, `freeCachedTableNames()`, `copyTableNamesToCursor()`, `discoverTablesCachedWithInvalidation()`
69+
70+
2. **`PRAGMA data_version` check amortization** (`stmt_cache.zig`)
71+
- Added `data_version_checked_this_txn` flag to `StmtCache`
72+
- Added `checkDataVersionAmortized()` that only performs actual PRAGMA query once per transaction
73+
- Added `resetDataVersionCheck()` for clearing the flag at transaction boundaries
74+
- Added `getDataVersion()` to get cached value
75+
76+
3. **`sqlite3_prepare_v3` with `SQLITE_PREPARE_PERSISTENT`** (`api.zig`, `stmt_cache.zig`)
77+
- Added `prepare_v3()` wrapper to `api.zig` with fallback to `prepare_v2` if not available
78+
- Added `SQLITE_PREPARE_PERSISTENT`, `SQLITE_PREPARE_NORMALIZE`, `SQLITE_PREPARE_NO_VTAB` constants
79+
- Added `getOrPreparePersistent()` method to `StmtCache`
80+
- Added standalone `prepareOncePersistent()` helper function
81+
- Updated `checkSchemaVersion()`, `checkDataVersion()`, and `discoverTablesCached()` to use persistent prepares
82+
83+
**Test Results:**
84+
- Unit tests: 64/64 PASS
85+
- Parity tests: 48/52 (4 pre-existing rowid slab failures, not related to this task)

‎.tasks/done/TASK-030-windows-dll-build.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# TASK-030: Windows `.dll` Build + Loadability Recon
2+
3+
## Status
4+
- [ ] Planned
5+
- [ ] Assigned
6+
- [ ] In Progress
7+
- [ ] Blocked (reason: ...)
8+
- [x] Complete
9+
10+
## Priority
11+
high
12+
13+
## Assigned To
14+
subagent (general)
15+
16+
## Parent Docs / Cross-links
17+
- Gap backlog: `research/zig-cr/92-gap-backlog.md` ("Cross-platform Packaging & CI")
18+
- Build config: `zig/build.zig`
19+
- FFI/exports: `zig/src/ffi/init.zig`, `zig/src/ffi/root.zig`
20+
21+
## Description
22+
Make the Windows packaging story crisp and testable.
23+
24+
Goal is not "full Windows CI" in one go; it is to either:
25+
- produce a `.dll` artifact that exports `sqlite3_crsqlite_init` correctly, OR
26+
- decisively document why it is blocked (missing toolchain, missing zig target support in CI, missing import lib, etc.)
27+
28+
## Files to Modify
29+
- `zig/build.zig`
30+
- `zig/README.md` (if usage/testing instructions need update)
31+
- `research/zig-cr/92-gap-backlog.md` (status notes)
32+
- (optional) `.github/workflows/*` (only if extremely small matrix tweak is safe)
33+
34+
## Acceptance Criteria
35+
- [x] `zig build -Dtarget=x86_64-windows-gnu` (or `-msvc` if chosen) produces `crsqlite.dll` (or documents why not).
36+
- [x] Exported symbol `sqlite3_crsqlite_init` present (document the command used to verify, e.g. `llvm-objdump` / `dumpbin`).
37+
- [x] `research/zig-cr/92-gap-backlog.md` updated with current Windows status and next concrete step.
38+
39+
## Progress Log
40+
### 2025-12-14
41+
- Task created during gap review; not yet started
42+
- Started work on Windows DLL build
43+
44+
### 2025-12-14 (completion)
45+
- Attempted `zig build -Dtarget=x86_64-windows-gnu` — **SUCCESS**
46+
- Attempted `zig build -Dtarget=x86_64-windows-msvc` — **SUCCESS**
47+
- Both targets produce valid PE32+ DLL at `zig-out/bin/crsqlite.dll`
48+
49+
## Completion Notes
50+
### Date: 2025-12-14
51+
52+
**Result: SUCCESS** — Windows DLL builds work out of the box with no changes needed.
53+
54+
### Build Commands
55+
```bash
56+
# GNU target (MinGW ABI) - builds successfully
57+
cd zig && nix run nixpkgs#zig -- build -Dtarget=x86_64-windows-gnu
58+
59+
# MSVC target (Windows SDK ABI) - builds successfully
60+
cd zig && nix run nixpkgs#zig -- build -Dtarget=x86_64-windows-msvc
61+
```
62+
63+
### Output
64+
- `zig-out/bin/crsqlite.dll` (1.4MB for GNU, 1.4MB for MSVC)
65+
- `zig-out/bin/crsqlite.pdb` (debug symbols)
66+
- `zig-out/lib/crsqlite.lib` (import library)
67+
68+
### Symbol Verification
69+
Command used:
70+
```bash
71+
nix-shell -p llvmPackages.bintools --run "llvm-objdump --private-headers zig-out/bin/crsqlite.dll" | grep -A 15 "Export Table:"
72+
```
73+
74+
Output (both targets):
75+
```
76+
Export Table:
77+
DLL name: crsqlite.dll
78+
Ordinal base: 1
79+
Ordinal RVA Name
80+
...
81+
8 0x1030 sqlite3_crsqlite_init <-- PRESENT ✅
82+
```
83+
84+
### Why It Works
85+
Zig's cross-compilation handles Windows targets natively. The build system:
86+
1. Already excluded WASM-specific paths for non-WASM targets
87+
2. Links against Windows kernel32/ntdll/advapi32 automatically
88+
3. No platform-specific code changes were needed
89+
90+
### Next Steps
91+
- CI integration: Add Windows matrix entry to `.github/workflows/zig-tests.yaml` (optional, low priority)
92+
- Runtime testing: Validate DLL loads in actual Windows SQLite (requires Windows environment)

0 commit comments

Comments
 (0)