feat(queryopt): set simplification optimizer by jzelinskie · Pull Request #3051 · authzed/spicedb

jzelinskie · 2026-04-15T20:58:13Z

Description

Adds a new query-plan optimizer, set-simplification, that eliminates structurally redundant nodes from outline trees using ten set-theoretic laws. The optimizer runs as a standard pass (registered in StandardOptimizations) and requires no cardinality or schema information — all laws hold unconditionally.

Laws implemented

Union

A ∪ A = A (idempotency)
A ∪ (A ∩ B) = A (absorption)
A ∪ (A − B) = A (complement-absorption)

Intersection

A ∩ A = A (idempotency)
A ∩ (A ∪ B) = A (absorption)
Y ∩ (A − C) = ∅ when Y ⊆ C (complement-annihilation)

Exclusion

X − A = ∅ when X ⊆ A (annihilation, generalized from A − A = ∅)
(A ∪ B) − Y = B − Y when A ⊆ Y (left-pruning)
A − ∅ = A (null-identity)

Structural prerequisite

(A ∪ B) ∪ C = A ∪ B ∪ C and (A ∩ B) ∩ C = A ∩ B ∩ C (associativity flattening, runs first so absorption sees all
peers)

Implementation notes

The optimizer is structured as one function per law, all composed via MutateOutline in a single bottom-up pass. Subsumption tests reuse the existing isSubsumedBy predicate; no new structural predicates were introduced.

A − ∅ = A lives in the absorption mutation chain rather than in the shared NullPropagation utility. NullPropagation is also called by reachability-pruning, which intentionally leaves A − ∅ intact (the null subtrahend is a meaningful pruning artifact in that context).

The optimizer was initially named absorption-idempotency; it is renamed to set-simplification to reflect the full scope of rules it now covers.

Testing

Added unit tests for everything including those missing for NullPropagation.
LMK if I need to add any other kind of tests when adding optimizers

References

Wikipedia for set theory absorption

codecov · 2026-04-15T21:02:11Z

Codecov Report

❌ Patch coverage is 96.96970% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.67%. Comparing base (97811cf) to head (a5e257d).

Files with missing lines	Patch %	Lines
pkg/query/queryopt/set_simplification.go	96.93%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3051      +/-   ##
==========================================
+ Coverage   75.57%   75.67%   +0.11%     
==========================================
  Files         489      490       +1     
  Lines       59837    60067     +230     
==========================================
+ Hits        45213    45447     +234     
+ Misses      11332    11325       -7     
- Partials     3292     3295       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

tstirrat15

LGTM but would like barak's eyes on it as well

tstirrat15 · 2026-04-16T17:18:40Z

 		Pushes caveat evalution to the lowest point in the tree.
 		Cannot push through intersection arrows
 		`,
+		Priority: 20,


Do we have a means of constructing a list of optimizers at startup time rather than taking the registration approach? This smells like it's going to turn into z-index eventually.

This is definitely a place that could use work. I think for now it'd probably help to just have a central list where optimizers are registered, similar how to we register gRPC middleware. If it ever got too confusing doing that, it'd make sense to explicitly add "before" and "after" properties to each optimizer and let the system compute the order based on those (kind like systemd startup ordering)

@jzelinskie I'd say we do the before/after now: its going to rapidly become untenable, so I recommend as a followup PR

miparnisari · 2026-04-20T17:36:34Z

could we add more tests? the coverage is quite poor: https://app.codecov.io/gh/authzed/spicedb/pull/3051/blob/pkg/query/queryopt/set_simplification.go

tstirrat15

See comments

tstirrat15 · 2026-04-20T21:53:53Z

+		}
+		for _, factor := range intersectionFactors(x) {
+			if !slices.ContainsFunc(y.SubOutlines, func(c query.Outline) bool {
+				return query.OutlineCompare(factor, c) == 0


This doesn't necessarily need to be a part of this PR, but it'd be nice to have an OutlineEquals alias for this.

tstirrat15 · 2026-04-20T21:58:21Z

+		}
+		for _, factor := range intersectionFactors(x) {
+			if !slices.ContainsFunc(y.SubOutlines, func(c query.Outline) bool {
+				return query.OutlineCompare(factor, c) == 0


Is this potentially expensive for an intersection at the top of a bunch of deep trees? Or are we expecting that the shapes we're working with here aren't going to be problematic in that way?

tstirrat15 · 2026-04-20T21:59:56Z

+func unionComplementAbsorption(outline query.Outline) query.Outline {
+	return eliminateRedundantChildren(outline, query.UnionIteratorType, func(y, x query.Outline) bool {
+		return y.Type == query.ExclusionIteratorType &&
+			len(y.SubOutlines) == 2 &&


Is this not satisfied by construction? Or are we being defensive here?

This was just being defensive in case of refactoring.

tstirrat15 · 2026-04-20T22:01:40Z

+	}
+
+	for i, child := range outline.SubOutlines {
+		if child.Type != query.ExclusionIteratorType || len(child.SubOutlines) != 2 {


Same question here

defensiveness

…standard set

…A−B) = A)

… ∩ (A ∪ B) = A)

… laws

… ⊆ A Extends exclusionSelfAnnihilation (A − A = ∅) to the full subset case: any exclusion node is annihilated when its minuend is subsumed by its subtrahend. This covers (A ∩ B) − A = ∅, (A − B) − A = ∅, and A − (A ∪ B) = ∅ by reusing the existing isSubsumedBy predicate, making the change a one-liner.

Covers every branch of NullPropagation: all iterator types (union, intersection, arrow, intersection arrow, exclusion, caveat, alias, recursive), the defensive len guards, ID preservation on null output, and leaf/unhandled types that pass through unchanged.

Extends the absorption-idempotency optimizer with four additional cardinality-free structural transformations: - flattenAssociativity: inlines nested same-type union/intersection children into their parent so that absorption rules see all peers at the same level. Without this, Union[A, Union[A∩B, C]] cannot be reduced to Union[A, C] because A and A∩B are never compared as siblings. - exclusionNullIdentity: simplifies A − ∅ = A by dropping the exclusion wrapper when the subtrahend is NullIteratorType. Placed in absorption.go rather than NullPropagation to avoid affecting the reachability-pruning optimizer, which calls NullPropagation directly. - exclusionLeftPruning: removes union children from the left side of an exclusion that are subsumed by the subtrahend, since those elements would be fully removed by the subtraction regardless. Generalizes to (A ∪ B) − Y = B − Y when A ⊆ Y; when all children are pruned the union is replaced with null and NullPropagation propagates ∅ − Y = ∅. - intersectionComplementAnnihilation: replaces an intersection with ∅ when it contains an exclusion child (A − C) and any sibling Y where Y ⊆ C, since elements of Y are inside C and elements of A − C are outside C, making the intersection empty.

absorptionIdempotency combined union idempotency, union absorption, and complement-absorption into one function. intersectionIdempotencyAbsorption combined intersection idempotency and absorption. Split both into one function per law: unionIdempotency, unionAbsorption, unionComplementAbsorption intersectionIdempotency, intersectionAbsorption Each function now has an inline predicate that expresses exactly its rule, making it straightforward to match the code to the law it implements. Also reorders functions to match mutation execution order (normalization, union rules, intersection rules, exclusion rules, helpers), standardizes all doc comments to the "Law: X" convention, and moves the caveat/arrow opacity note from absorptionIdempotency into isSubsumedBy and the init description where it actually applies.

The optimizer now implements ten set-theoretic laws spanning union, intersection, and exclusion operators. "absorption-idempotency" described only two of them; "set-simplification" covers the full scope.

github-actions Bot added the area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools) label Apr 15, 2026

jzelinskie force-pushed the absorption-optim branch from 3e14dd6 to 6e45bdc Compare April 15, 2026 22:09

jzelinskie marked this pull request as ready for review April 15, 2026 22:52

jzelinskie requested a review from a team as a code owner April 15, 2026 22:52

jzelinskie force-pushed the absorption-optim branch from 6e45bdc to a6cf7a3 Compare April 16, 2026 16:12

tstirrat15 previously approved these changes Apr 16, 2026

View reviewed changes

jzelinskie dismissed tstirrat15’s stale review via 1e73d17 April 16, 2026 20:25

jzelinskie requested a review from barakmich April 16, 2026 20:53

jzelinskie added the area/perf Affects performance or scalability label Apr 16, 2026

miparnisari reviewed Apr 17, 2026

View reviewed changes

Comment thread pkg/query/mutations.go Outdated

jzelinskie force-pushed the absorption-optim branch from 9ac6f21 to 6a4e842 Compare April 17, 2026 18:05

miparnisari reviewed Apr 17, 2026

View reviewed changes

Comment thread pkg/query/mutations_test.go Outdated

jzelinskie changed the title ~~feat(queryopt): absorption-idempotency optimizer~~ feat(queryopt): set simplification optimizer Apr 17, 2026

jzelinskie force-pushed the absorption-optim branch 3 times, most recently from 5ec75be to 24e2fbd Compare April 18, 2026 00:19

miparnisari reviewed Apr 20, 2026

View reviewed changes

tstirrat15 reviewed Apr 20, 2026

View reviewed changes

jzelinskie added 11 commits May 1, 2026 17:12

chore(queryopt): add explicit priorities to existing optimizers

1c499b0

feat(queryopt): implement absorption-idempotency optimizer mutation

19ebb98

feat(queryopt): register absorption-idempotency optimizer and add to …

ed2bfed

…standard set

chore(CHANGELOG): add absorption optimizer entry

ee1317a

feat(queryopt): add complement-absorption to union subsumption (A ∪ (…

446bda2

…A−B) = A)

feat(queryopt): add exclusion self-annihilation (A − A = ∅)

d42a3f3

feat(queryopt): add intersection idempotency/absorption (A ∩ A = A, A…

681a82f

… ∩ (A ∪ B) = A)

docs(queryopt): update absorption optimizer description with all five…

592d0b3

… laws

refactor(queryopt): dedup child iteration

2d654c5

fix(NullPropagation): null if either arrow child is null

bbe9540

jzelinskie added 4 commits May 1, 2026 17:13

rename(queryopt): absorption-idempotency → set-simplification

a5e257d

The optimizer now implements ten set-theoretic laws spanning union, intersection, and exclusion operators. "absorption-idempotency" described only two of them; "set-simplification" covers the full scope.

miparnisari force-pushed the absorption-optim branch from 24e2fbd to a5e257d Compare May 2, 2026 00:14

Conversation

jzelinskie commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Implementation notes

Testing

References

Uh oh!

codecov Bot commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tstirrat15 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tstirrat15 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jzelinskie commented Apr 15, 2026 •

edited

Loading

codecov Bot commented Apr 15, 2026 •

edited

Loading