optimizer: Make EQProp call MSE::reduce only after a change in reduce_expr#31979
Merged
ggevay merged 2 commits intoMaterializeInc:mainfrom Mar 21, 2025
Merged
optimizer: Make EQProp call MSE::reduce only after a change in reduce_expr#31979ggevay merged 2 commits intoMaterializeInc:mainfrom
EQProp call MSE::reduce only after a change in reduce_expr#31979ggevay merged 2 commits intoMaterializeInc:mainfrom
Conversation
f8baa21 to
763d310
Compare
763d310 to
3b3fdea
Compare
frankmcsherry
approved these changes
Mar 21, 2025
Contributor
frankmcsherry
left a comment
There was a problem hiding this comment.
I can't really tell what this does to the overall behavior, but I think MSE::reduce doesn't have specific behavior in any case (e.g. not necessarily idempotent; may or may not perform various simplifications) so it should be equally correct w/o calling it, and the behavior of the whole thing can't rely on it being called or not. E.g. no clue whether we produce the same plans but have to run EQProp more often to tickle reductions we miss for some reason.
Contributor
Author
|
My thinking is that we do call |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We have a CPU profile where a user running some very complex ad-hoc queries made envd quite busy with optimization. (Slack discussion.) The profile is showing that most of the time is spent in
MirScalarExpr::reducecalls fromEquivalencePropagation.This PR modifies
EquivalencePropagationto callreduceonly if there was a recent change in the expression. Specifically, all of EQProp's calls toreduceare just after areduce_expr, which conveniently returns a bool that indicates whether it made any changes.There are no EXPLAIN changes across all of our slts. I've also done a run on all slts where I added a temporary assert that the
reduces that we are skipping wouldn't do anything. This failed only one time across all of our slts: injoins.slt:1275. But looking at the EXPLAINs of this query onmainvs. the PR, the final plan does not change even for this query.Added a feature flag, enabled by default.
The first commit is the actual functionality, the second just adds the feature flag and wires it up.
Motivation
Tips for reviewer
Checklist
$T ⇔ Proto$Tmapping (possibly in a backwards-incompatible way), then it is tagged with aT-protolabel.