Skip to content

Commit b945501

Browse files
authored
feat(hql): Intersect function (#860)
<!-- greptile_comment --> <h2>Greptile Overview</h2> <h3>Greptile Summary</h3> This PR adds a new `INTERSECT` traversal step to HelixQL end-to-end: Pest grammar support, parser `StepType::Intersect`, analyzer validation and codegen, plus a new engine adapter (`IntersectAdapter`) and multiple test fixtures (engine-level Rust tests and HQL tests). At runtime, the engine implementation collects upstream items, executes a provided sub-traversal closure per item to get candidate results, then returns only values whose IDs appear in every sub-result set (set intersection semantics). The generator emits `.intersect(|val, db, txn, arena| { G::from_iter(..) <substeps> ... })` to wire the HQL syntax to the engine adapter. Main issue to address before merge: one existing HQL test (`user_test_12`) changes the meaning of `GetIndicatorsWithTimeParams` from “has any matching time parameter” to “connected to all matching time parameters provided”, which will change expected results when multiple `time_vals` are passed and should be clarified/adjusted. <details><summary><h3>Important Files Changed</h3></summary> | Filename | Overview | |----------|----------| | helix-db/src/grammar.pest | Adds `INTERSECT(_::<traversal>)` as a new util step in the Pest grammar. | | helix-db/src/helix_engine/tests/traversal_tests/intersect_tests.rs | Adds engine-level Rust tests for IntersectAdapter across several graph scenarios. | | helix-db/src/helix_engine/tests/traversal_tests/mod.rs | Registers the new intersect test module. | | helix-db/src/helix_engine/traversal_core/ops/util/intersect.rs | Introduces IntersectAdapter for RoTraversalIterator; currently collects upstream and sub-traversal results and intersects by id. | | helix-db/src/helix_engine/traversal_core/ops/util/mod.rs | Exports the new util::intersect module. | | helix-db/src/helixc/analyzer/methods/traversal_validation.rs | Adds analyzer support for StepType::Intersect by inferring sub-traversal type and emitting generator step. | | helix-db/src/helixc/generator/traversal_steps.rs | Adds generator Step::Intersect and emits `intersect(...)` closure code (currently drops sub-traversal errors). | | helix-db/src/helixc/generator/utils.rs | Imports IntersectAdapter into generated code prelude. | | helix-db/src/helixc/parser/graph_step_parse_methods.rs | Parses `intersect_step` into StepType::Intersect using parse_expression. | | helix-db/src/helixc/parser/types.rs | Adds StepType::Intersect and updates PartialEq accordingly. | | hql-tests/tests/intersect/helix.toml | Adds new HQL test project config for intersect feature. | | hql-tests/tests/intersect/queries.hx | Adds HQL queries that use INTERSECT to find articles by all tags. | | hql-tests/tests/intersect/schema.hx | Defines Tag/Article/HasTag schema for intersect tests. | | hql-tests/tests/search_v_with_embed/queries.hx | Updates test query to take `item` param and pass it to Embed(). | | hql-tests/tests/user_test_12/queries.hx | Replaces EXISTS-based semantics with INTERSECT-based query; changes meaning from 'any matching time parameter' to 'all matching time parameters'. | | hql-tests/tests/user_test_12/queries.rs | Adds generated Rust code for user_test_12; includes new intersect usage and embedded schema/config. | </details> </details> <details><summary><h3>Sequence Diagram</h3></summary> ```mermaid sequenceDiagram participant User as HQL User participant Parser as helixc parser (pest) participant Analyzer as helixc analyzer participant Gen as helixc generator participant Engine as traversal engine User->>Parser: Parse traversal with ::INTERSECT(_::<sub-traversal>) Parser->>Analyzer: Emit StepType::Intersect(Expression) Analyzer->>Analyzer: infer_expr_type(expr, Some(cur_ty)) Analyzer->>Gen: GenerateStep::Intersect { traversal } Gen->>Engine: Emit Rust: .intersect(|val, db, txn, arena| G::from_iter(..).<substeps>...) Engine->>Engine: Collect upstream values loop for each upstream value Engine->>Engine: Run sub-traversal closure => Vec<TraversalValue> end Engine->>Engine: Intersect result sets by TraversalValue.id() Engine-->>User: Iterator of intersected TraversalValue results ``` </details> <!-- greptile_other_comments_section --> <!-- /greptile_comment -->
2 parents 41b7503 + d53553a commit b945501

File tree

16 files changed

+1347
-13
lines changed

16 files changed

+1347
-13
lines changed

helix-db/src/grammar.pest

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ get_stmt = { identifier ~ "<-" ~ evaluates_to_anything }
6565
traversal = { (start_node | start_edge | search_vector | start_vector) ~ step* ~ last_step? }
6666
id_traversal = { identifier ~ ((step+ ~ last_step?) | last_step) }
6767
anonymous_traversal = { "_" ~ ((step+ ~ last_step?) | last_step)? }
68-
step = { "::" ~ (graph_step | order_by| aggregate | group_by | where_step | closure_step | object_step | exclude_field | count | ID | range_step | AddE | rerank_rrf | rerank_mmr) }
68+
step = { "::" ~ (graph_step | order_by| aggregate | group_by | where_step | intersect_step | closure_step | object_step | exclude_field | count | ID | range_step | AddE | rerank_rrf | rerank_mmr) }
6969
last_step = { "::" ~ (bool_operations | update | upsert_v | upsert_e | upsert_n | first) }
7070
// change this for loop to be able to take traversals etc in the future.
7171
for_loop = { "FOR" ~ for_argument ~ "IN" ~ identifier ~ "{" ~ query_body ~ "}" }
@@ -192,6 +192,7 @@ shortest_path_astar ={ "ShortestPathAStar" ~ ("<" ~ type_args ~ ">")? ~ "(" ~ ma
192192
// Util steps
193193
// ---------------------------------------------------------------------
194194
where_step = { "WHERE" ~ "(" ~ (evaluates_to_bool | anonymous_traversal) ~ ")" }
195+
intersect_step = { "INTERSECT" ~ "(" ~ anonymous_traversal ~ ")" }
195196
exists = { negate? ~ "EXISTS" ~ "(" ~ (traversal | id_traversal | anonymous_traversal) ~ ")" }
196197
negate = { "!" }
197198
range_step = { "RANGE" ~ "(" ~ (evaluates_to_number) ~ "," ~ (evaluates_to_number) ~ ")" }

0 commit comments

Comments
 (0)