Commit 3aa08ad
fix: InList Dictionary filter pushdown type mismatch (apache#20962)
Closes apache#20937
`ArrayStaticFilter::contains()` unconditionally unwraps
dictionary-encoded needles to evaluate against distinct values. This
only works when `in_array` has the same type as the dictionary's value
type. When `in_array` is also Dictionary, the unwrap creates a type
mismatch in `make_comparator`. This became reachable after:
- apache#20505
which removed dictionary flattening in the InList builder, allowing
Dictionary arrays to reach `ArrayStaticFilter` via HashJoin dynamic
filter pushdown with `pushdown_filters` enabled.
- Guard the dictionary unwrap in `ArrayStaticFilter::contains()` to only
fire when the dictionary value type matches `in_array`'s type. When both
sides are Dictionary, fall through to `make_comparator(Dict, Dict)`
which arrow-ord handles natively.
- Update the sqllogictest from apache#20960 to expect success.
- Add unit tests covering all `try_new_from_array` type combinations
(primitive specializations, Utf8, Dictionary, Struct).
Yes — unit tests and sqllogictest.1 parent 5881ede commit 3aa08ad
File tree
2 files changed
+641
-3
lines changed- datafusion
- physical-expr/src/expressions
- sqllogictest/test_files
2 files changed
+641
-3
lines changed
0 commit comments