fix: Restore pyarrow predicate conversion for is_in#26811
fix: Restore pyarrow predicate conversion for is_in#26811erandagan wants to merge 2 commits intopola-rs:mainfrom
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #26811 +/- ##
==========================================
+ Coverage 81.30% 81.66% +0.35%
==========================================
Files 1802 1805 +3
Lines 246972 248038 +1066
Branches 3086 3132 +46
==========================================
+ Hits 200810 202549 +1739
+ Misses 45371 44683 -688
- Partials 791 806 +15 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
This doesn't seem right.. we cannot apply the current conversion on AnyValue::List as we would lose a nesting level on list conversion.
Could you re-work this to ensure the conversion happens as an exception only for the RHS of the is_in() operation?
@nameexhaustion Apologies for missing this, I'm not too familiar with the intricacies of the codebase. Pushed a fix. |
| let mut args = args; | ||
| args.allow_literal_series = true; | ||
| let values = predicate_to_pa(input.get(1)?.node(), expr_arena, args)?; | ||
| let values = predicate_to_pa(rhs_node, expr_arena, args) |
There was a problem hiding this comment.
Can we make this only do the conversion if we have Literal(List(s)) (i.e. remove the leading predicate_to_pa().or_else. It's otherwise not correct to convert the RHS since the semantics are different
Closes #26802
AI DIsclosure: