Skip to content

Commit 4da541a

Browse files
ScrapCodesdongjoon-hyun
authored andcommitted
[SPARK-25543][K8S] Print debug message iff execIdsRemovedInThisRound is not empty.
## What changes were proposed in this pull request? Spurious logs like /sec. 2018-09-26 09:33:57 DEBUG ExecutorPodsLifecycleManager:58 - Removed executors with ids from Spark that were either found to be deleted or non-existent in the cluster. 2018-09-26 09:33:58 DEBUG ExecutorPodsLifecycleManager:58 - Removed executors with ids from Spark that were either found to be deleted or non-existent in the cluster. 2018-09-26 09:33:59 DEBUG ExecutorPodsLifecycleManager:58 - Removed executors with ids from Spark that were either found to be deleted or non-existent in the cluster. 2018-09-26 09:34:00 DEBUG ExecutorPodsLifecycleManager:58 - Removed executors with ids from Spark that were either found to be deleted or non-existent in the cluster. The fix is easy, first check if there are any removed executors, before producing the log message. ## How was this patch tested? Tested by manually deploying to a minikube cluster. Closes apache#22565 from ScrapCodes/spark-25543/k8s/debug-log-spurious-warning. Authored-by: Prashant Sharma <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 40e6ed8 commit 4da541a

File tree

1 file changed

+5
-2
lines changed

1 file changed

+5
-2
lines changed

resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -100,8 +100,11 @@ private[spark] class ExecutorPodsLifecycleManager(
100100
}
101101
}
102102
}
103-
logDebug(s"Removed executors with ids ${execIdsRemovedInThisRound.mkString(",")}" +
104-
s" from Spark that were either found to be deleted or non-existent in the cluster.")
103+
104+
if (execIdsRemovedInThisRound.nonEmpty) {
105+
logDebug(s"Removed executors with ids ${execIdsRemovedInThisRound.mkString(",")}" +
106+
s" from Spark that were either found to be deleted or non-existent in the cluster.")
107+
}
105108
}
106109

107110
private def onFinalNonDeletedState(

0 commit comments

Comments
 (0)