Skip to content

Conversation

@dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented May 26, 2025

What changes were proposed in this pull request?

This PR aims to add pi-with-driver-timeout example.

Why are the changes needed?

spark.driver.timeout with DriverTimeoutPlugin is a new feature of Apache Spark 4.0.0 in order to guarantee the maximum run time of the jobs in the best effort approach.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manual test the example (timeout: 1minutes)

$ kubectl apply -f examples/pi-with-driver-timeout.yaml

$ kubectl get sparkapp
NAME                     CURRENT STATE                       AGE
pi-with-driver-timeout   TerminatedWithoutReleaseResources   2m59s

Was this patch authored or co-authored using generative AI tooling?

No.

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-52304] Add pi-with-driver-timeout.yaml example [SPARK-52304] Add pi-with-driver-timeout example May 26, 2025
@dongjoon-hyun
Copy link
Member Author

Since this is an example, I'll merge this.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-52304 branch May 26, 2025 05:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant