Is there a join type that can pushdown join filter to one side's scan? #7158
alexwilcoxson-rel
started this conversation in
Ideas
Replies: 1 comment 1 reply
-
To the best of my knowledge this is not supported today by DataFusion -- I think this type of filter pushdown is called "sideways information passing" in database literature and is a typical technique to improve the performance for column stores. I think it has come up before in the context of improving join performance (for example, on the TPCH benchmarks -- cc @Dandandan and @mingmwang ) @alexwilcoxson-rel would you be willing to help file a feature request for this item? I can provide some additional commentary / pointers as well |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I was looking through the join options' code and it didn't look like this was possible. e.g.
LHS table produces small number of rows. Collect these and provide them as a filter pushed down to RHS scan to reduce files read.
Beta Was this translation helpful? Give feedback.
All reactions