Skip to content

Remove CoalesceBatches optimizer rule #19591

@alamb

Description

@alamb

Is your feature request related to a problem or challenge?

After BatchCoalescer has been integrated into AsyncFuncExec by @feniljain in

There is nothing remaining in the CoalesceBatches rule

if let Some(async_exec) = plan_any.downcast_ref::<AsyncFuncExec>() {
// Coalesce inputs to async functions to reduce number of async function invocations
let children = async_exec.children();
assert_eq_or_internal_err!(
children.len(),
1,
"Expected AsyncFuncExec to have exactly one child"
);
let coalesce_exec = Arc::new(CoalesceBatchesExec::new(
Arc::clone(children[0]),
target_batch_size,
));
let new_plan = plan.with_new_children(vec![coalesce_exec])?;
Ok(Transformed::yes(new_plan))
} else {
Ok(Transformed::no(plan))
}

Describe the solution you'd like

We should now remove the CoalesceBatches optimizer rule and mark the CoalesceBatches execution operator as deprecated (per https://datafusion.apache.org/contributor-guide/api-health.html#deprecation-guidelines) -- and refer people to the BatchCoalescer

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions