Skip to content

Commit 76b2766

Browse files
committed
[SPARK-53983] Increase s.k.o.reconciler.foregroundRequestTimeoutSeconds to 60s
### What changes were proposed in this pull request? This PR aims to increase `s.k.o.reconciler.foregroundRequestTimeoutSeconds` to `60s`. ### Why are the changes needed? While testing on EKS clusters, `30s` timeout happens too frequently. ### Does this PR introduce _any_ user-facing change? This only increases the timeout. `Spark Operator` will wait more instead of timing out. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #395 from dongjoon-hyun/SPARK-53983. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 67da720 commit 76b2766

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

docs/config_properties.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
| spark.kubernetes.operator.terminateOnInformerFailureEnabled | Boolean | false | false | Enable to indicate informer errors should stop operator startup. If disabled, operator startup will ignore recoverable errors, caused for example by RBAC issues and will retry periodically. |
99
| spark.kubernetes.operator.reconciler.terminationTimeoutSeconds | Integer | 30 | false | Grace period for operator shutdown before reconciliation threads are killed. |
1010
| spark.kubernetes.operator.reconciler.parallelism | Integer | 50 | false | Thread pool size for Spark Operator reconcilers. Unbounded pool would be used if set to non-positive number. |
11-
| spark.kubernetes.operator.reconciler.foregroundRequestTimeoutSeconds | Long | 30 | true | Timeout (in seconds) for requests made to API server. This applies only to foreground requests. |
11+
| spark.kubernetes.operator.reconciler.foregroundRequestTimeoutSeconds | Long | 60 | true | Timeout (in seconds) for requests made to API server. This applies only to foreground requests. |
1212
| spark.kubernetes.operator.reconciler.intervalSeconds | Long | 120 | true | Interval (in seconds, non-negative) to reconcile Spark applications. Note that reconciliation is always expected to be triggered when app spec / status is updated. This interval controls the reconcile behavior of operator reconciliation even when there's no update on SparkApplication, e.g. to determine whether a hanging app needs to be proactively terminated. Thus this is recommended to set to above 2 minutes to avoid unnecessary no-op reconciliation. |
1313
| spark.kubernetes.operator.reconciler.trimStateTransitionHistoryEnabled | Boolean | true | true | When enabled, operator would trim state transition history when a new attempt starts, keeping previous attempt summary only. |
1414
| spark.kubernetes.operator.reconciler.appStatusListenerClassNames | String | | false | Comma-separated names of SparkAppStatusListener class implementations |

spark-operator/src/main/java/org/apache/spark/k8s/operator/config/SparkOperatorConf.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,7 @@ public final class SparkOperatorConf {
123123
"Timeout (in seconds) for requests made to API server. This "
124124
+ "applies only to foreground requests.")
125125
.typeParameterClass(Long.class)
126-
.defaultValue(30L)
126+
.defaultValue(60L)
127127
.build();
128128

129129
/**

0 commit comments

Comments
 (0)