Skip to content

add support for complex boolean logic within addWhere and addHaving#2646

Open
albbird wants to merge 1 commit intomalloydata:mainfrom
albbird:feat-filter-conjunction-operators
Open

add support for complex boolean logic within addWhere and addHaving#2646
albbird wants to merge 1 commit intomalloydata:mainfrom
albbird:feat-filter-conjunction-operators

Conversation

@albbird
Copy link

@albbird albbird commented Feb 3, 2026

Problem

The addWhere and addHaving functions in malloy-query-builder couldn't express complex boolean logic (AND/OR) between filter conditions across different fields. All filters were joined with commas, limiting the ability to build queries with OR conditions or explicit AND groupings.

Note: The existing ClauseChain in malloy-filter only supports AND/OR operators within a single field's filter expression (e.g., carrier ~ f'WN and AA'). It cannot combine conditions across different fields like flight_count > 100 AND total_distance > 1000.

Solution

Extended the addWhere and addHaving functions to accept an optional conjunction parameter ('and' | 'or') that specifies how the current filter connects to the previous one, enabling boolean logic across multiple fields.

Changes

packages/malloy-interfaces/src/types.ts

  • Added Conjunction type ('and' | 'or')
  • Added optional conjunction property to FilterOperation type

packages/malloy-interfaces/src/to_malloy.ts

  • Created formatFilterBlock function to handle per-item conjunction operators
  • Updated serialization for where and having clauses to output and/or instead of comma when conjunction is specified

packages/malloy-query-builder/src/query-ast.ts

  • Extended addWhere function overloads to accept optional conjunction parameter
  • Extended addHaving function similarly for consistency

Usage Example

const segment = query.getOrAddDefaultSegment();
segment.addWhere('carrier', 'WN');
segment.addWhere('carrier', 'AA', 'and');
segment.addWhere('carrier', 'DL', 'or');

Generates:

run: flights -> {
  where:
    carrier ~ f`WN`
    and carrier ~ f`AA`
    or carrier ~ f`DL`
}

ClauseChain vs Conjunction Parameter

Feature ClauseChain (existing) Conjunction (this PR)
Scope Single field Multiple fields
Where applied Inside filter string Between filter operations
Example carrier ~ f'WN and AA' carrier ~ f'WN' and origin ~ f'TX'
Use case Same field, multiple conditions Different fields, boolean logic

Combined Usage

The two approaches are complementary. You can use ClauseChain's complex within-field logic inside each addWhere call, and then use the conjunction parameter to connect those filters across fields:

segment.addWhere('carrier', '(WN; AA) | DL'); // Complex within-field logic
segment.addWhere('origin', 'TX, CA', 'and'); // Connected via AND

Limitations

Cross-field grouped logic with parentheses is not supported:

(country = 'US' AND age = '10') OR (state = 'CA' OR color = 'white')

This would require a filter tree builder, which is beyond the scope of this PR.

Key Design Decisions

  • The conjunction relates to the current filter being added (how it connects to the previous filter)
  • First filter's conjunction is ignored (no previous filter to connect to)
  • Without a conjunction parameter, the default comma separator is used (backward compatible)
  • Prevents broken queries (no trailing operators like where: field ~ f'value' and)

Tests

Added comprehensive tests in packages/malloy-query-builder/src/query-ast.spec.ts:

  • Multiple wheres with AND conjunction
  • Multiple wheres with OR conjunction
  • Mixed AND and OR conjunctions
  • Where with path and conjunction
  • Where with ParsedFilter and conjunction
  • First where with conjunction (verifies conjunction is ignored)
  • Default behavior (comma separator when no conjunction)
  • Having with AND/OR conjunctions

All existing tests continue to pass.

…unctions

Signed-off-by: Vinicius A <vinicius.albuquerquebarros@bird.com>
@mtoy-googly-moogly
Copy link
Collaborator

OK, this is potentially awesome or awful.

There are reasons for the limitations on filtering.
There need to be some conversations.

DO NOT MERGE, until the conversations are complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants