feat: Improve sort memory resilience (#19494)

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

Closes #19493 .

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

Greatly reduces the memory requested by ExternalSorter to perform sorts,
adds much more granularity to the reservations, and Tries to do this
with minimal overhead by merging the splitting and sorting processes.

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

The sort stream will calculate the indices once, but the take will be
done in batches, so we create batch_size sized RecordBatches, whose
get_record_batch_size results return info that is very close to their
sliced sizes(if not completely the same), this means there is no need
for the precaution of reserving a huge amount of memory in order to do
the merge sort, meaning we can merge more streams at the same time, and
so on and so forth.

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

Yes

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

There is a new sort_batch_chunked function, which returns a Vec of
RecordBatch, based on the provided batch_size.
Some docs are updated.

---------

Co-authored-by: Raz Luvaton <[email protected]>

Author: EmilyMatt

datafusion/physical-plan/src/aggregates/row_hash.rs

@@ -1198,7 +1198,7 @@ impl GroupedHashAggregateStream {
             // instead.
             // Spilling to disk and reading back also ensures batch size is consistent
             // rather than potentially having one significantly larger last batch.
-            self.spill()?;
+            self.spill()?; // TODO: use sort_batch_chunked instead?
 
             // Mark that we're switching to stream merging mode.
             self.spill_state.is_stream_merging = true;

datafusion/physical-plan/src/sorts/multi_level_merge.rs

@@ -30,7 +30,7 @@ use arrow::datatypes::SchemaRef;
 use datafusion_common::Result;
 use datafusion_execution::memory_pool::MemoryReservation;
 
-use crate::sorts::sort::get_reserved_byte_for_record_batch_size;
+use crate::sorts::sort::get_reserved_bytes_for_record_batch_size;
 use crate::sorts::streaming_merge::{SortedSpillFile, StreamingMergeBuilder};
 use crate::stream::RecordBatchStreamAdapter;
 use datafusion_execution::{RecordBatchStream, SendableRecordBatchStream};
@@ -360,9 +360,13 @@ impl MultiLevelMergeBuilder {
         for spill in &self.sorted_spill_files {
             // For memory pools that are not shared this is good, for other this is not
             // and there should be some upper limit to memory reservation so we won't starve the system
-            match reservation.try_grow(get_reserved_byte_for_record_batch_size(
-                spill.max_record_batch_memory * buffer_len,
-            )) {
+            match reservation.try_grow(
+                get_reserved_bytes_for_record_batch_size(
+                    spill.max_record_batch_memory,
+                    // Size will be the same as the sliced size, bc it is a spilled batch.
+                    spill.max_record_batch_memory,
+                ) * buffer_len,
+            ) {
                 Ok(_) => {
                     number_of_spills_to_read_for_current_phase += 1;
                 }