Changed schema for Value -> Term, majorly breaking change (#622)

* Changed schema for Value -> Term, majorly breaking change

* Following the schema change, Value -> Term into all processing

* Updated Cassandra for g, p, s, o index patterns (7 indexes)

* Reviewed and updated all tests

* Neo4j, Memgraph and FalkorDB remain broken, will look at once settled down

Author: cybermaggedon

Makefile

@@ -70,8 +70,8 @@ some-containers:
 	    -t ${CONTAINER_BASE}/trustgraph-base:${VERSION} .
 	${DOCKER} build -f containers/Containerfile.flow \
 	    -t ${CONTAINER_BASE}/trustgraph-flow:${VERSION} .
-	${DOCKER} build -f containers/Containerfile.vertexai \
-	    -t ${CONTAINER_BASE}/trustgraph-vertexai:${VERSION} .
+#	${DOCKER} build -f containers/Containerfile.vertexai \
+#	    -t ${CONTAINER_BASE}/trustgraph-vertexai:${VERSION} .
 #	${DOCKER} build -f containers/Containerfile.mcp \
 #	    -t ${CONTAINER_BASE}/trustgraph-mcp:${VERSION} .
 #	${DOCKER} build -f containers/Containerfile.vertexai \

docs/tech-specs/graph-contexts.md

@@ -228,10 +228,15 @@ Following SPARQL conventions for backward compatibility:
 
 - **`g` omitted / None**: Query the default graph only
 - **`g` = specific IRI**: Query that named graph only
-- **`g` = wildcard / `*`**: Query across all graphs
+- **`g` = wildcard / `*`**: Query across all graphs (equivalent to SPARQL
+  `GRAPH ?g { ... }`)
 
 This keeps simple queries simple and makes named graph queries opt-in.
 
+Cross-graph queries (g=wildcard) are fully supported. The Cassandra schema
+includes dedicated tables (SPOG, POSG, OSPG) where g is a clustering column
+rather than a partition key, enabling efficient queries across all graphs.
+
 #### Temporal Queries
 
 **Find all facts discovered after a given date:**
@@ -388,12 +393,78 @@ will proceed in phases:
 Cassandra requires multiple tables to support different query access patterns
 (each table efficiently queries by its partition key + clustering columns).
 
-**Challenge: Quads**
+##### Query Patterns
+
+With quads (g, s, p, o), each position can be specified or wildcard, giving
+16 possible query patterns:
+
+| # | g | s | p | o | Description |
+|---|---|---|---|---|-------------|
+| 1 | ? | ? | ? | ? | All quads |
+| 2 | ? | ? | ? | o | By object |
+| 3 | ? | ? | p | ? | By predicate |
+| 4 | ? | ? | p | o | By predicate + object |
+| 5 | ? | s | ? | ? | By subject |
+| 6 | ? | s | ? | o | By subject + object |
+| 7 | ? | s | p | ? | By subject + predicate |
+| 8 | ? | s | p | o | Full triple (which graphs?) |
+| 9 | g | ? | ? | ? | By graph |
+| 10 | g | ? | ? | o | By graph + object |
+| 11 | g | ? | p | ? | By graph + predicate |
+| 12 | g | ? | p | o | By graph + predicate + object |
+| 13 | g | s | ? | ? | By graph + subject |
+| 14 | g | s | ? | o | By graph + subject + object |
+| 15 | g | s | p | ? | By graph + subject + predicate |
+| 16 | g | s | p | o | Exact quad |
+
+##### Table Design
+
+Cassandra constraint: You can only efficiently query by partition key, then
+filter on clustering columns left-to-right. For g-wildcard queries, g must be
+a clustering column. For g-specified queries, g in the partition key is more
+efficient.
+
+**Two table families needed:**
+
+**Family A: g-wildcard queries** (g in clustering columns)
+
+| Table | Partition | Clustering | Supports patterns |
+|-------|-----------|------------|-------------------|
+| SPOG | (user, collection, s) | p, o, g | 5, 7, 8 |
+| POSG | (user, collection, p) | o, s, g | 3, 4 |
+| OSPG | (user, collection, o) | s, p, g | 2, 6 |
+
+**Family B: g-specified queries** (g in partition key)
+
+| Table | Partition | Clustering | Supports patterns |
+|-------|-----------|------------|-------------------|
+| GSPO | (user, collection, g, s) | p, o | 9, 13, 15, 16 |
+| GPOS | (user, collection, g, p) | o, s | 11, 12 |
+| GOSP | (user, collection, g, o) | s, p | 10, 14 |
+
+**Collection table** (for iteration and bulk deletion)
 
-For triples, typical indexes are SPO, POS, OSP (partition by first, cluster by
-rest). For quads, the graph dimension adds: SPOG, POSG, OSPG, GSPO, etc.
+| Table | Partition | Clustering | Purpose |
+|-------|-----------|------------|---------|
+| COLL | (user, collection) | g, s, p, o | Enumerate all quads in collection |
 
-**Challenge: Quoted Triples**
+##### Write and Delete Paths
+
+**Write path**: Insert into all 7 tables.
+
+**Delete collection path**:
+1. Iterate COLL table for `(user, collection)`
+2. For each quad, delete from all 6 query tables
+3. Delete from COLL table (or range delete)
+
+**Delete single quad path**: Delete from all 7 tables directly.
+
+##### Storage Cost
+
+Each quad is stored 7 times. This is the cost of flexible querying combined
+with efficient collection deletion.
+
+##### Quoted Triples in Storage
 
 Subject or object can be a triple itself. Options:
 
@@ -425,29 +496,9 @@ Metadata table:
 - Pro: Clean separation, can index triple IDs
 - Con: Requires computing/managing triple identity, two-phase lookups
 
-**Option C: Hybrid**
-- Store quads normally with serialized quoted triple strings for simple cases
-- Maintain a separate triple ID lookup for advanced queries
-- Pro: Flexibility
-- Con: Complexity
-
-**Recommendation**: TBD after prototyping. Option A is simplest for initial
-implementation; Option B may be needed for advanced query patterns.
-
-#### Indexing Strategy
-
-Indexes must support the defined query patterns:
-
-| Query Type | Access Pattern | Index Needed |
-|------------|----------------|--------------|
-| Facts by date | P=discoveredOn, O>date | POG (predicate, object, graph) |
-| Facts by source | P=supportedBy, O=source | POG |
-| Facts by asserter | P=assertedBy, O=person | POG |
-| Metadata for a fact | S=quotedTriple | SPO/SPOG |
-| All facts in graph | G=graphIRI | GSPO |
-
-For temporal range queries (dates), Cassandra clustering column ordering
-enables efficient scans when date is a clustering column.
+**Recommendation**: Start with Option A (serialized strings) for simplicity.
+Option B may be needed if advanced query patterns over quoted triple
+components are required.
 
 2. **Phase 2+: Other Backends**
    - Neo4j and other stores implemented in subsequent stages

tests/contract/conftest.py

@@ -15,10 +15,10 @@
     TextCompletionRequest, TextCompletionResponse,
     DocumentRagQuery, DocumentRagResponse,
     AgentRequest, AgentResponse, AgentStep,
-    Chunk, Triple, Triples, Value, Error,
+    Chunk, Triple, Triples, Term, Error,
     EntityContext, EntityContexts,
     GraphEmbeddings, EntityEmbeddings,
-    Metadata
+    Metadata, IRI, LITERAL
 )
 
 
@@ -43,7 +43,7 @@ def schema_registry():
         "Chunk": Chunk,
         "Triple": Triple,
         "Triples": Triples,
-        "Value": Value,
+        "Term": Term,
         "Error": Error,
         "EntityContext": EntityContext,
         "EntityContexts": EntityContexts,
@@ -98,26 +98,22 @@ def sample_message_data():
             "collection": "test_collection",
             "metadata": []
         },
-        "Value": {
-            "value": "http://example.com/entity",
-            "is_uri": True,
-            "type": ""
+        "Term": {
+            "type": IRI,
+            "iri": "http://example.com/entity"
         },
         "Triple": {
-            "s": Value(
-                value="http://example.com/subject",
-                is_uri=True,
-                type=""
+            "s": Term(
+                type=IRI,
+                iri="http://example.com/subject"
             ),
-            "p": Value(
-                value="http://example.com/predicate", 
-                is_uri=True,
-                type=""
+            "p": Term(
+                type=IRI,
+                iri="http://example.com/predicate"
             ),
-            "o": Value(
-                value="Object value",
-                is_uri=False,
-                type=""
+            "o": Term(
+                type=LITERAL,
+                value="Object value"
             )
         }
     }
@@ -139,10 +135,10 @@ def invalid_message_data():
             {"query": "test", "user": "test", "collection": "test", "doc_limit": -1},  # Invalid doc_limit
             {"query": "test"},  # Missing required fields
         ],
-        "Value": [
-            {"value": None, "is_uri": True, "type": ""},  # Invalid value (None)
-            {"value": "test", "is_uri": "not_boolean", "type": ""},  # Invalid is_uri
-            {"value": 123, "is_uri": True, "type": ""},  # Invalid value (not string)
+        "Term": [
+            {"type": IRI, "iri": None},  # Invalid iri (None)
+            {"type": "invalid_type", "value": "test"},  # Invalid type
+            {"type": LITERAL, "value": 123},  # Invalid value (not string)
         ]
     }