Skip to content

Commit d32337b

Browse files
juliuszsompolskihvanhovell
authored andcommitted
[SPARK-22721] BytesToBytesMap peak memory usage not accurate after reset()
## What changes were proposed in this pull request? BytesToBytesMap doesn't update peak memory usage before shrinking back to initial capacity in reset(), so after a disk spill one never knows what was the size of hash table was before spilling. ## How was this patch tested? Checked manually. Author: Juliusz Sompolski <[email protected]> Closes #19915 from juliuszsompolski/SPARK-22721.
1 parent 8ae004b commit d32337b

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -879,6 +879,7 @@ public LongArray getArray() {
879879
* Reset this map to initialized state.
880880
*/
881881
public void reset() {
882+
updatePeakMemoryUsed();
882883
numKeys = 0;
883884
numValues = 0;
884885
freeArray(longArray);

0 commit comments

Comments
 (0)