Replies: 1 comment
-
The comments on this issue may possibly be relevant (not 100% sure though) |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Through examining the current implementation of multi-column sort's spill-to-disk strategies, I find we are asking for more memory during spill, which I think is worth discussing:
During the spill,
Rows
are created for comparison efficiency for each in-memory RecordBatch. Considering why we spill in the first place, does thisRows
optimization increase the possibility of execution failure due to memory shortage?Please share your insights or corrections if I misunderstand any part of the code.
Beta Was this translation helpful? Give feedback.
All reactions