Skip to content

Commit 84232d8

Browse files
authored
Add additional info about memory reservation to the doc of MemoryPool (apache#14789)
1 parent 568a2b1 commit 84232d8

File tree

1 file changed

+4
-1
lines changed
  • datafusion/execution/src/memory_pool

1 file changed

+4
-1
lines changed

datafusion/execution/src/memory_pool/mod.rs

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,10 @@ pub use pool::*;
5555
/// "large" amounts of memory (proportional to number of input rows), such as
5656
/// `GroupByHashExec`. It does NOT track and limit memory used internally by
5757
/// other operators such as `DataSourceExec` or the `RecordBatch`es that flow
58-
/// between operators.
58+
/// between operators. Furthermore, operators should not reserve memory for the
59+
/// batches they produce. Instead, if a parent operator needs to hold batches
60+
/// from its children in memory for an extended period, it is the parent
61+
/// operator's responsibility to reserve the necessary memory for those batches.
5962
///
6063
/// In order to avoid allocating memory until the OS or the container system
6164
/// kills the process, DataFusion `ExecutionPlan`s (operators) that consume

0 commit comments

Comments
 (0)