Skip to content

Commit 2c5b4b0

Browse files
committed
[SPARK-54861][CORE] Reset task thread name to IDLE_TASK_THREAD_NAME when task completes
### What changes were proposed in this pull request? This PR proposes to reset the task thread name to `Executor task idle worker#threadId` when a task completes. ### Why are the changes needed? This change provides better observablity for users and developers. Currently, when a task completes, the [task thread name](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/executor/Executor.scala#L642) remains unchanged until a new task launched overrides the original name. Therefore, we can still see the specific task name from the task thread stacktrace even if the task has been completed. And this is misleading as people may mistakenly think that the task thread is actively running for some task but is idle indeed. ### Does this PR introduce _any_ user-facing change? Yes. Users see different thread name from the executor thread dump page. Before this change: <img width="1554" height="492" alt="image (6)" src="https://github.com/user-attachments/assets/eba7bb8c-652d-4aa4-9b58-a8748f89ee7e" /> After this change: <img width="1560" height="516" alt="image (7)" src="https://github.com/user-attachments/assets/964cfbda-11af-4e96-a492-16eb1d0ec6ab" /> ### How was this patch tested? Manually tested (verified by Spark UI). ### Was this patch authored or co-authored using generative AI tooling? No. Closes #53634 from Ngone51/unset-task-thread-name. Authored-by: Yi Wu <[email protected]> Signed-off-by: Yi Wu <[email protected]>
1 parent ea8e941 commit 2c5b4b0

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

core/src/main/scala/org/apache/spark/executor/Executor.scala

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ import org.slf4j.{MDC => SLF4JMDC}
4040

4141
import org.apache.spark._
4242
import org.apache.spark.deploy.SparkHadoopUtil
43-
import org.apache.spark.executor.Executor.TASK_THREAD_NAME_PREFIX
43+
import org.apache.spark.executor.Executor.{IDLE_TASK_THREAD_NAME, TASK_THREAD_NAME_PREFIX}
4444
import org.apache.spark.internal.{Logging, LogKeys}
4545
import org.apache.spark.internal.LogKeys._
4646
import org.apache.spark.internal.config._
@@ -944,6 +944,7 @@ private[spark] class Executor(
944944
}
945945
// Release the session reference. If evicted and this was the last task, cleanup happens.
946946
isolatedSession.release()
947+
Thread.currentThread().setName(s"$IDLE_TASK_THREAD_NAME#$threadId" )
947948
}
948949
}
949950

@@ -1388,6 +1389,7 @@ private[spark] class Executor(
13881389

13891390
private[spark] object Executor extends Logging {
13901391
val TASK_THREAD_NAME_PREFIX = "Executor task launch worker"
1392+
val IDLE_TASK_THREAD_NAME = "Executor task idle worker"
13911393

13921394
// This is reserved for internal use by components that need to read task properties before a
13931395
// task is fully deserialized. When possible, the TaskContext.getLocalProperty call should be

0 commit comments

Comments
 (0)