[SPARK-52661] Suppress pending pod log messages

dongjoon-hyun · dongjoon-hyun · commit b0255cfa9bc0 · 2025-07-02T18:43:14.000-07:00
### What changes were proposed in this pull request? This PR aims to suppress pending pod log messages from WARN to DEBUG. ### Why are the changes needed? Since the pending status is a legit status, we don't need to WARN. Frequently, it can happen during the K8s cluster autoscaling-up. ``` 25/07/02 22:58:44 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:44 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:44 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:44 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:44 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:44 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:46 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:46 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:46 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:46 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:46 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:46 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:50 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:55 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:55 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:55 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:55 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:55 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:55 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:58 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:58 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:58 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:58 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:58 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:58:58 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:03 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:05 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:05 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:05 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:05 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:05 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:05 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:09 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:09 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:09 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:09 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:09 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:09 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:13 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:13 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:13 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:13 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. 25/07/02 22:59:13 WARN BaseAppDriverObserver: Cannot determine driver pod status, the pod may in pending state. ``` ### Does this PR introduce _any_ user-facing change? No behavior change because this changes only log levels. ### How was this patch tested? Manual review. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#272 from dongjoon-hyun/SPARK-52661. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
diff --git a/spark-operator/src/main/java/org/apache/spark/k8s/operator/reconciler/observers/BaseAppDriverObserver.java b/spark-operator/src/main/java/org/apache/spark/k8s/operator/reconciler/observers/BaseAppDriverObserver.java
@@ -82,7 +82,7 @@ protected Optional<ApplicationState> observeDriverTermination(
     if (status == null
         || status.getContainerStatuses() == null
         || status.getContainerStatuses().isEmpty()) {
-      log.warn("Cannot determine driver pod status, the pod may in pending state.");
+      log.debug("Cannot determine driver pod status, the pod may in pending state.");
       return Optional.empty();
     }
 

Original file line number	Diff line number	Diff line change
`@@ -82,7 +82,7 @@ protected Optional<ApplicationState> observeDriverTermination(`
`82`	`82`	`if (status == null`
`83`	`83`	`\|\| status.getContainerStatuses() == null`
`84`	`84`	`\|\| status.getContainerStatuses().isEmpty()) {`
`85`		`- log.warn("Cannot determine driver pod status, the pod may in pending state.");`
	`85`	`+ log.debug("Cannot determine driver pod status, the pod may in pending state.");`
`86`	`86`	`return Optional.empty();`
`87`	`87`	`}`
`88`	`88`