-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge?
After BatchCoalescer has been integrated into AsyncFuncExec by @feniljain in
There is nothing remaining in the CoalesceBatches rule
datafusion/datafusion/physical-optimizer/src/coalesce_batches.rs
Lines 58 to 75 in fd26321
| if let Some(async_exec) = plan_any.downcast_ref::<AsyncFuncExec>() { | |
| // Coalesce inputs to async functions to reduce number of async function invocations | |
| let children = async_exec.children(); | |
| assert_eq_or_internal_err!( | |
| children.len(), | |
| 1, | |
| "Expected AsyncFuncExec to have exactly one child" | |
| ); | |
| let coalesce_exec = Arc::new(CoalesceBatchesExec::new( | |
| Arc::clone(children[0]), | |
| target_batch_size, | |
| )); | |
| let new_plan = plan.with_new_children(vec![coalesce_exec])?; | |
| Ok(Transformed::yes(new_plan)) | |
| } else { | |
| Ok(Transformed::no(plan)) | |
| } |
Describe the solution you'd like
We should now remove the CoalesceBatches optimizer rule and mark the CoalesceBatches execution operator as deprecated (per https://datafusion.apache.org/contributor-guide/api-health.html#deprecation-guidelines) -- and refer people to the BatchCoalescer
Describe alternatives you've considered
No response
Additional context
No response
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request