test: add Clotho bug reproduction tests for Rust bindings (#33)

Add 5 integration tests mirroring exact query patterns reported as
broken from Clotho (count aggregates, property-match syntax, OPTIONAL
MATCH with nulls, undirected match, pattern predicates). All pass —
confirms the issues are Clotho-side, not in graphqlite or its Rust
bindings.

Also adds Metis tickets GQLITE-T-0139 (EXISTS subquery syntax) and
GQLITE-T-0140 (Clotho investigation).

Co-authored-by: Dylan Bobby Storey <dstorey@dstorey-personal-m3.local>

Author: dylanbstorey

.metis/backlog/bugs/GQLITE-T-0140.md

@@ -0,0 +1,96 @@
+---
+id: investigate-clotho-integration
+level: task
+title: "Investigate Clotho integration: count(), OPTIONAL MATCH, property-match reported as broken"
+short_code: "GQLITE-T-0140"
+created_at: 2026-03-22T00:45:57.695444+00:00
+updated_at: 2026-03-22T00:45:57.695444+00:00
+parent: 
+blocked_by: []
+archived: false
+
+tags:
+  - "#task"
+  - "#phase/backlog"
+  - "#bug"
+
+
+exit_criteria_met: false
+initiative_id: NULL
+---
+
+# Investigate Clotho integration: count(), OPTIONAL MATCH, property-match reported as broken
+
+## Objective
+
+Investigate why Clotho (colliery-io/clotho) reported `count()`, `OPTIONAL MATCH`, and `{key: value}` property-match as broken, when all three work correctly at the C extension level on both v0.3.7 and current main.
+
+## Background
+
+**Reported from Clotho integration sessions (v0.3.7, rusqlite 0.31).** The report claims 8 of 33 queries failed (24% error rate), including:
+
+1. **`count()` aggregate** — "returns column headers but no rows." Cannot reproduce at C level on v0.3.7 or current; returns `[{"total":3}]` correctly.
+2. **`{key: 'value'}` property-match** — "syntax error on `:`". Cannot reproduce at C level on v0.3.7 or current; parses and executes correctly.
+3. **`OPTIONAL MATCH`** — "returns empty result set." Cannot reproduce at C level; correctly returns rows with `null` for unmatched optionals.
+
+### Reproduction Results
+
+Tested all three on v0.3.7 (via `git checkout v0.3.7 && make extension`) and current main:
+
+| Bug | v0.3.7 | v0.3.10 (current) |
+|-----|--------|-------------------|
+| count() empty | Works correctly | Works correctly |
+| {key: value} syntax error | Works correctly | Works correctly |
+| OPTIONAL MATCH empty | Works correctly | Works correctly |
+
+### Suspected Root Cause: Rust Bindings
+
+Since the C extension works, the issue likely lives in the Rust binding layer or Clotho's usage of it. Analysis of `bindings/rust/src/` found several potential data-loss paths:
+
+- **`connection.rs:131`** — If SQLite returns `NULL`, `query_row` gets `None` → `CypherResult::empty()` silently. Could happen if Clotho uses a different execution path.
+- **`result.rs:424-425`** — Empty string from extension → silent empty result.
+- **`result.rs:437-444`** — Non-JSON strings (e.g., malformed error messages) are silently wrapped as `{"result": "..."}` instead of raising an error.
+- **`result.rs:450-451`** — Empty JSON arrays → silent empty result, no way to distinguish from "query returned no matches."
+
+Clotho may also be using `conn.query_row` or `conn.execute` directly instead of the `cypher()` wrapper, which could bypass result parsing.
+
+## Acceptance Criteria
+
+- [ ] Reproduce the failures in the Rust integration test harness, or confirm they are Clotho-side
+- [ ] If Rust binding issue: fix silent empty results — return errors instead of empty `CypherResult`
+- [ ] If Clotho-side: document correct usage patterns and close
+- [ ] Add Rust integration tests for count(), OPTIONAL MATCH with nulls, and property-match syntax
+
+## Implementation Notes
+
+### Investigation Steps
+
+1. Write Rust integration tests that mirror the exact Clotho queries
+2. Check if Clotho pins an older graphqlite-rs version that might have bugs since fixed
+3. Check if Clotho uses `execute()` instead of `query_row()` for read queries (would discard results)
+4. Review Clotho's `Cargo.lock` for graphqlite version
+
+### Risk Considerations
+- May require changes in the Clotho repo rather than graphqlite
+- Silent empty results are a usability hazard regardless — worth hardening even if not the root cause
+
+## Status Updates
+
+### 2026-03-21: Rust binding path verified — not the cause
+
+**Investigation complete.** Added 5 Rust integration tests mirroring exact Clotho query patterns (`test_clotho_bug1` through `test_clotho_bug5` + `test_clotho_pattern_predicate_in_where`). All pass through the full Rust binding path (connection → query_row → JSON parsing → CypherResult).
+
+**Key findings:**
+- `count()` with WHERE filter: works — returns `{"total":3}` correctly
+- `{key: 'value'}` property-match: works — parses and executes correctly
+- `OPTIONAL MATCH` with null: works — returns rows with `null` for unmatched optionals, `Option<String>` extracts correctly
+- `(a)--(b)` undirected: works on current (was fixed in v0.3.8)
+- Pattern predicates: works with updated extension binary
+
+**Stale cache finding:** The bundled extension binary caching in `platform.rs` (versioned filename + size-only check) can serve stale binaries if the binary is rebuilt without a version bump. This caused a false-negative during testing. Not likely the Clotho root cause, but a potential footgun.
+
+**Verdict:** The issue is Clotho-side, not in graphqlite or its Rust bindings. Recommend:
+1. Check Clotho's Cargo.lock for pinned graphqlite version
+2. Check if Clotho constructs queries with incorrect escaping
+3. Check if Clotho uses rusqlite `execute()` for reads (would discard results)
+4. Close this ticket once Clotho is investigated

.metis/backlog/features/GQLITE-T-0139.md

@@ -0,0 +1,78 @@
+---
+id: existential-subquery-syntax-exists
+level: task
+title: "Existential subquery syntax: EXISTS { MATCH ... WHERE ... }"
+short_code: "GQLITE-T-0139"
+created_at: 2026-03-22T00:45:57.656842+00:00
+updated_at: 2026-03-22T00:45:57.656842+00:00
+parent: 
+blocked_by: []
+archived: false
+
+tags:
+  - "#task"
+  - "#phase/backlog"
+  - "#feature"
+
+
+exit_criteria_met: false
+initiative_id: NULL
+---
+
+# Existential subquery syntax: EXISTS { MATCH ... WHERE ... }
+
+## Objective
+
+Implement full existential subquery syntax `EXISTS { MATCH ... WHERE ... }` per CIP2015-05-13-EXISTS. Currently only `EXISTS((pattern))` and `EXISTS(property)` are supported. The full subquery form allows filtering within the existence check.
+
+## Background
+
+**Reported from Clotho integration (v0.3.7).** Query:
+```cypher
+MATCH (t) WHERE t.entity_type = 'Transcript'
+AND NOT EXISTS { MATCH (e)-[:EXTRACTED_FROM]->(t) }
+RETURN t.title
+```
+Fails with: `syntax error, unexpected '{'`
+
+**Note:** The reporter also used a double-WHERE variant (`WHERE ... WHERE NOT EXISTS ...`) which is invalid Cypher regardless. The single-WHERE + AND form is the correct syntax.
+
+**Workarounds that already work:**
+- `EXISTS((t)<-[:EXTRACTED_FROM]-())` — simple pattern form
+- `NOT (t)<-[:EXTRACTED_FROM]-()` — pattern predicate (added in v0.3.10)
+
+**Spec basis:** CIP2015-05-13-EXISTS defines three levels:
+1. `EXISTS(property)` — property existence (supported)
+2. `EXISTS((pattern))` — pattern existence (supported)
+3. `EXISTS { MATCH pattern [WHERE filter] }` — full subquery (NOT supported)
+
+## Acceptance Criteria
+
+- [ ] `EXISTS { MATCH (a)-[:REL]->(b) }` parses and evaluates as existence check
+- [ ] `NOT EXISTS { MATCH (a)-[:REL]->(b) }` works for non-existence
+- [ ] `EXISTS { MATCH (a)-[:REL]->(b) WHERE b.prop = value }` supports filtering within the subquery
+- [ ] Variables from outer scope are correctly correlated (e.g., `t` from outer MATCH)
+- [ ] Unit tests and functional SQL tests added
+- [ ] Existing `EXISTS((pattern))` and `EXISTS(property)` behavior unaffected
+
+## Implementation Notes
+
+### Technical Approach
+
+**Grammar (`cypher_gram.y`):** Add production for `EXISTS '{' clause_list '}'` or a specialized `EXISTS '{' MATCH pattern_list where_opt '}'` rule. The braces `{` `}` already tokenize correctly (used for map literals). The main challenge is disambiguating `EXISTS { ... }` from `EXISTS(...)`.
+
+**AST:** May need a new AST node type (`AST_NODE_EXISTS_SUBQUERY`) or extend `cypher_exists_expr` with a subquery variant that holds a full `MATCH` + optional `WHERE`.
+
+**Transform:** The subquery needs its own variable scope that can reference outer variables. Generate a correlated `EXISTS (SELECT 1 FROM ... WHERE ...)` SQL subquery, similar to the pattern form but with additional WHERE conditions from the subquery body.
+
+### Dependencies
+- Pattern predicate support (GQLITE-T-0138) — completed
+- CALL subquery (GQLITE-T-0133) may share scoping infrastructure
+
+### Risk Considerations
+- Variable scoping: inner variables must not leak to outer scope, but outer variables must be accessible in the subquery
+- Grammar conflicts with `EXISTS(...)` form — need careful disambiguation of `{` vs `(`
+
+## Status Updates
+
+*To be added during implementation*

bindings/rust/Cargo.lock

@@ -3268,3 +3268,111 @@ fn test_negative_epoch() {
     assert!(val.contains("1969-12-31"));
 }
 
+// =============================================================================
+// Clotho Bug Report Reproduction Tests
+// These mirror the exact query patterns reported as broken from the Clotho
+// integration (v0.3.7, rusqlite 0.31). All should pass — if any fail, the
+// issue is in the Rust binding layer, not the C extension.
+// =============================================================================
+
+#[test]
+fn test_clotho_bug1_count_aggregate_with_where_filter() {
+    // Reported: count() returns column headers but no rows
+    let conn = test_connection();
+    conn.cypher("CREATE (:Decision {entity_type: 'Decision', title: 'D1'})").unwrap();
+    conn.cypher("CREATE (:Decision {entity_type: 'Decision', title: 'D2'})").unwrap();
+    conn.cypher("CREATE (:Decision {entity_type: 'Decision', title: 'D3'})").unwrap();
+    conn.cypher("CREATE (:Other {entity_type: 'Other', title: 'O1'})").unwrap();
+
+    let results = conn
+        .cypher("MATCH (n) WHERE n.entity_type = 'Decision' RETURN count(n) AS total")
+        .unwrap();
+    assert_eq!(results.len(), 1, "count() should return exactly one row");
+    assert_eq!(results[0].get::<i64>("total").unwrap(), 3);
+}
+
+#[test]
+fn test_clotho_bug2_property_match_syntax() {
+    // Reported: {key: 'value'} causes "syntax error, unexpected ':'"
+    let conn = test_connection();
+    conn.cypher("CREATE (:Decision {entity_type: 'Decision', id: 'd1', title: 'First'})").unwrap();
+    conn.cypher("CREATE (:Decision {entity_type: 'Decision', id: 'd2', title: 'Second'})").unwrap();
+
+    let results = conn
+        .cypher("MATCH (n {entity_type: 'Decision'}) RETURN n.id, n.title")
+        .unwrap();
+    assert_eq!(results.len(), 2, "property-match should find 2 decisions");
+}
+
+#[test]
+fn test_clotho_bug3_optional_match_with_where_filter() {
+    // Reported: OPTIONAL MATCH produces no results at all
+    let conn = test_connection();
+    conn.cypher("CREATE (:ClothoDecision {entity_type: 'Decision', title: 'Connected'})").unwrap();
+    conn.cypher("CREATE (:ClothoDecision {entity_type: 'Decision', title: 'Orphan'})").unwrap();
+    conn.cypher("CREATE (:ClothoProgram {entity_type: 'Program', title: 'Alpha'})").unwrap();
+    conn.cypher("MATCH (d:ClothoDecision {title: 'Connected'}), (p:ClothoProgram {title: 'Alpha'}) CREATE (d)-[:BELONGS_TO]->(p)").unwrap();
+
+    let results = conn
+        .cypher("MATCH (d) WHERE d.entity_type = 'Decision' OPTIONAL MATCH (d)-[:BELONGS_TO]->(p) RETURN d.title, p.title AS program")
+        .unwrap();
+    assert_eq!(results.len(), 2, "OPTIONAL MATCH should return all decisions, including unmatched");
+
+    // Find the connected and orphan rows
+    let mut found_connected = false;
+    let mut found_orphan = false;
+    for row in results.iter() {
+        let title: String = row.get("d.title").unwrap();
+        if title == "Connected" {
+            assert_eq!(row.get::<String>("program").unwrap(), "Alpha");
+            found_connected = true;
+        } else if title == "Orphan" {
+            assert!(row.get::<Option<String>>("program").unwrap().is_none());
+            found_orphan = true;
+        }
+    }
+    assert!(found_connected, "Should find the connected decision");
+    assert!(found_orphan, "Should find the orphan decision with null program");
+}
+
+#[test]
+fn test_clotho_bug5_undirected_match_bare() {
+    // Reported: (a)--(b) not supported
+    let conn = test_connection();
+    conn.cypher("CREATE (:UndirA {name: 'Alice'})").unwrap();
+    conn.cypher("CREATE (:UndirB {name: 'Bob'})").unwrap();
+    conn.cypher("MATCH (a:UndirA), (b:UndirB) CREATE (a)-[:KNOWS]->(b)").unwrap();
+
+    // Bare -- syntax
+    let results = conn
+        .cypher("MATCH (a)--(b) WHERE a.name = 'Alice' RETURN b.name")
+        .unwrap();
+    assert_eq!(results.len(), 1);
+    assert_eq!(results[0].get::<String>("b.name").unwrap(), "Bob");
+
+    // -[]- syntax
+    let results2 = conn
+        .cypher("MATCH (a)-[]-(b) WHERE a.name = 'Alice' RETURN b.name")
+        .unwrap();
+    assert_eq!(results2.len(), 1);
+    assert_eq!(results2[0].get::<String>("b.name").unwrap(), "Bob");
+}
+
+#[test]
+fn test_clotho_pattern_predicate_in_where() {
+    // The original trigger for this investigation:
+    // MATCH (n {entity_type: 'Note'}) WHERE NOT (n)-[:BELONGS_TO]->() RETURN n.id, n.title
+    let conn = test_connection();
+    conn.cypher("CREATE (:Note {entity_type: 'Note', id: 'n1', title: 'Orphan'})").unwrap();
+    conn.cypher("CREATE (:Note {entity_type: 'Note', id: 'n2', title: 'Connected'})").unwrap();
+    conn.cypher("CREATE (:Group {id: 'g1'})").unwrap();
+    conn.cypher("MATCH (n:Note {id: 'n2'}), (g:Group {id: 'g1'}) CREATE (n)-[:BELONGS_TO]->(g)").unwrap();
+
+    let results = conn
+        .cypher("MATCH (n {entity_type: 'Note'}) WHERE NOT (n)-[:BELONGS_TO]->() AND NOT (n)-[:RELATES_TO]->() RETURN n.id, n.title")
+        .unwrap();
+    assert_eq!(results.len(), 1);
+    assert_eq!(results[0].get::<String>("n.id").unwrap(), "n1");
+    assert_eq!(results[0].get::<String>("n.title").unwrap(), "Orphan");
+}
+