|
| 1 | +# Wish: Decide behavior for batch apply when some incoming changes are ignorable |
| 2 | + |
| 3 | +## Context |
| 4 | +We recently decided **unknown columns during sync are ignored** (lenient behavior) to support rolling upgrades. |
| 5 | + |
| 6 | +That decision interacts with our existing **merge atomicity** expectations. |
| 7 | + |
| 8 | +Today: |
| 9 | +- `zig/harness/test-merge-atomicity.sh` reports **2 failing checks** |
| 10 | +- Those failures are caused by using “unknown column” as the error injection mechanism, but unknown columns are now **ignored**, so the batch doesn’t fail and the first row legitimately persists. |
| 11 | + |
| 12 | +## Repro |
| 13 | +```bash |
| 14 | +cd /Users/tom/Developer/effect-native/cr-sqlite |
| 15 | +bash zig/harness/test-merge-atomicity.sh |
| 16 | +``` |
| 17 | + |
| 18 | +Failing checks: |
| 19 | +- Test 2: “Invalid column in batch causes entire statement to fail” |
| 20 | +- Test 7: “Base table integrity after failed batch” |
| 21 | + |
| 22 | +## What’s the actual decision point? |
| 23 | +When applying a batch of incoming changes (often shipped in a single SQL statement with multiple VALUES rows): |
| 24 | + |
| 25 | +If some rows are **ignored by policy** (unknown column), do we want: |
| 26 | + |
| 27 | +1) **Apply the valid subset** (current behavior) |
| 28 | + - “best effort apply” |
| 29 | + - matches lenient schema mismatch policy |
| 30 | + |
| 31 | +2) **Fail the statement / rollback entire batch** if *any* row is unapplicable |
| 32 | + - stricter atomicity semantics |
| 33 | + - but conflicts with “ignore unknown columns” unless we special-case |
| 34 | + |
| 35 | +## Recommendation |
| 36 | +**Keep applying the valid subset** when the “failure” is an ignorable policy case (unknown column). |
| 37 | + |
| 38 | +Then update `test-merge-atomicity.sh` to inject errors using something that remains a hard error even under lenient schema mismatch, e.g. |
| 39 | +- invalid table name |
| 40 | +- invalid PK blob / malformed pk encoding |
| 41 | +- invalid site_id length (if we decide to add validation) |
| 42 | + |
| 43 | +This maintains a useful atomicity guarantee: |
| 44 | +- real errors rollback |
| 45 | +- intentionally ignored changes don’t poison the whole batch |
| 46 | + |
| 47 | +## Likely follow-up work (if approved) |
| 48 | +- Update `zig/harness/test-merge-atomicity.sh` to align with the chosen policy |
| 49 | +- Possibly add a new test that explicitly validates “unknown column rows are ignored but known columns still apply” (this is now part of the contract) |
| 50 | + |
| 51 | +## Cross-links |
| 52 | +- Related decision already implemented: `.tasks/done/TASK-186-schema-mismatch-unknown-column-behavior.md` |
| 53 | +- Existing spec task: `.tasks/done/TASK-087-spec-merge-atomicity.md` |
| 54 | +- Test: `zig/harness/test-merge-atomicity.sh` |
| 55 | + |
| 56 | +## Decision requested from Tom |
| 57 | +- Confirm that “unknown columns ignored” implies “best effort apply” within a batch (recommended) |
| 58 | +- Or require strict all-or-nothing batch failure semantics even for unknown columns |
0 commit comments