-
Notifications
You must be signed in to change notification settings - Fork 4.1k
release-24.3: opt: fix optional filter removal for lookup join #159774
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
release-24.3: opt: fix optional filter removal for lookup join #159774
Conversation
Fixes #159660 Previously, we don't reset the `optionalMultiValFilterSuffixLen` when we find a necessary multi-val filter for a column. This would cause an incorrect removal of an unoptional filter. For example, - Index: (x, y ,z) - ON filters: x = 1, z IN (1, 2) - Optional filters: y IN (3, 4) Since the filters are processed with the order of columns in the index, the filters will be processed in the order of `x = 1`, `y IN (3, 4)`, `z IN (1, 2)`. When the processing reaches `y IN (3, 4)`, since it is an optional filter, `optionalMultiValFilterSuffixLen = 1`. But `z IN (1, 2)` is actually a necessary filter, with `optionalMultiValFilterSuffixLen` not being reset, it will be eventually wrongly removed. Release note (bug fix): Fixed a bug causing a query predicate to be ignored when the predicate is on a column following one or more ENUM columns in an index, the predicate constrains the column to multiple values, and a lookup join to the index is chosen for the query plan. This bug was introduced in 24.3.0 and present in all versions since.
|
Thanks for opening a backport. Before merging, please confirm that it falls into one of the following categories (select one):
Add a brief release justification to the PR description explaining your selection. Also, confirm that the change does not break backward compatibility and complies with all aspects of the backport policy. All backports must be reviewed by the TL and EM for the owning area. |
|
✅ PR #159774 is compliant with backport policy Confidence: high 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
michae2
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Backport 1/1 commits from #159722 on behalf of @ZhouXing19.
Fixes #159660
Previously, we don't reset the
optionalMultiValFilterSuffixLenwhen wefind a necessary multi-val filter for a column. This would cause an incorrect
removal of an unoptional filter.
For example,
Since the filters are processed with the order of columns in the index,
the filters will be processed in the order of
x = 1,y IN (3, 4),z IN (1, 2).When the processing reaches
y IN (3, 4), since it is an optional filter,optionalMultiValFilterSuffixLen = 1. Butz IN (1, 2)is actually anecessary filter, with
optionalMultiValFilterSuffixLennot being reset,it will be eventually wrongly removed.
Release note (bug fix): Fixed a bug causing a query predicate to be ignored
when the predicate is on a column following one or more ENUM columns in an
index, the predicate constrains the column to multiple values, and a lookup
join to the index is chosen for the query plan. This bug was introduced in
24.3.0 and present in all versions since.
Release justification: critical fix for the customer, fix for query plan generation that caused incorrect result.