Skip to content

Commit 3864480

Browse files
sarutakmengxr
authored andcommitted
[SPARK-25266][CORE] Fix memory leak in Barrier Execution Mode
## What changes were proposed in this pull request? BarrierCoordinator uses Timer and TimerTask. `TimerTask#cancel()` is invoked in ContextBarrierState#cancelTimerTask but `Timer#purge()` is never invoked. Once a TimerTask is scheduled, the reference to it is not released until `Timer#purge()` is invoked even though `TimerTask#cancel()` is invoked. ## How was this patch tested? I checked the number of instances related to the TimerTask using jmap. Closes apache#22258 from sarutak/fix-barrierexec-oom. Authored-by: sarutak <[email protected]> Signed-off-by: Xiangrui Meng <[email protected]>
1 parent 1fd59c1 commit 3864480

File tree

2 files changed

+2
-0
lines changed

2 files changed

+2
-0
lines changed

core/src/main/scala/org/apache/spark/BarrierCoordinator.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,7 @@ private[spark] class BarrierCoordinator(
123123
private def cancelTimerTask(): Unit = {
124124
if (timerTask != null) {
125125
timerTask.cancel()
126+
timer.purge()
126127
timerTask = null
127128
}
128129
}

core/src/main/scala/org/apache/spark/BarrierTaskContext.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,7 @@ class BarrierTaskContext(
140140
throw e
141141
} finally {
142142
timerTask.cancel()
143+
timer.purge()
143144
}
144145
}
145146

0 commit comments

Comments
 (0)