Skip to content

Commit ca426bf

Browse files
committed
[SPARK-26042][SS][TESTS] Fix a potential hang in KafkaContinuousSourceTopicDeletionSuite
## What changes were proposed in this pull request? As initializing lazy vals shares the same lock, a thread is trying to initialize `executedPlan` when `isRDD` is running, this thread will hang forever. This PR just materializes `executedPlan` so that accessing it when `toRdd` is running doesn't need to wait for a lock ## How was this patch tested? Jenkins Closes apache#23023 from zsxwing/SPARK-26042. Authored-by: Shixiong Zhu <[email protected]> Signed-off-by: Shixiong Zhu <[email protected]> (cherry picked from commit 4035c98) Signed-off-by: Shixiong Zhu <[email protected]>
1 parent e2e1f0a commit ca426bf

File tree

1 file changed

+6
-1
lines changed

1 file changed

+6
-1
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -259,7 +259,12 @@ class ContinuousExecution(
259259

260260
reportTimeTaken("runContinuous") {
261261
SQLExecution.withNewExecutionId(
262-
sparkSessionForQuery, lastExecution)(lastExecution.toRdd)
262+
sparkSessionForQuery, lastExecution) {
263+
// Materialize `executedPlan` so that accessing it when `toRdd` is running doesn't need to
264+
// wait for a lock
265+
lastExecution.executedPlan
266+
lastExecution.toRdd
267+
}
263268
}
264269
} catch {
265270
case t: Throwable

0 commit comments

Comments
 (0)