Commit 88d106d
committed
[SPARK-52471] Add
### What changes were proposed in this pull request?
This PR aims to add [Swift-based SparkPi](https://github.com/apache/spark-connect-swift/tree/main/Examples/pi) K8s [CronJob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/) example.
### Why are the changes needed?
To provide a working example with
- [Apache Spark 4.0.0 Connect Server](https://spark.apache.org/docs/4.0.0/) launched by [Apache Spark Kubernetes Operator 0.3.0](https://artifacthub.io/packages/helm/spark-kubernetes-operator/spark-kubernetes-operator) Helm chart.
- [SparkPi Swift App](https://github.com/apache/spark-connect-swift/tree/main/Examples/pi) developed with [Apache Spark Connect for Swift 0.3.0](https://swiftpackageindex.com/apache/spark-connect-swift) library.
- [Swift 6.1](https://www.swift.org) language
### Does this PR introduce _any_ user-facing change?
No behavior change because this is a new example.
### How was this patch tested?
Manual tests.
```bash
# Install Spark K8s Operator
$ helm install spark spark/spark-kubernetes-operator
# Launch Spark 4.0 Connect Server
$ kubectl apply -f examples/spark-connect-server.yaml
# Launch `Swift-based SparkPi` Application
$ kubectl apply -f examples/job/cron-pi-swift.yaml
$ kubectl get cronjob
NAME SCHEDULE TIMEZONE SUSPEND ACTIVE LAST SCHEDULE AGE
spark-connect-swift-pi * * * * * <none> False 1 2s 7m51s
$ kubectl get pod -l job-name
NAME READY STATUS RESTARTS AGE
spark-connect-swift-pi-29163859-mkh5p 0/1 Completed 0 2m31s
spark-connect-swift-pi-29163860-btc66 0/1 Completed 0 91s
spark-connect-swift-pi-29163861-68mmt 0/1 Completed 0 31s
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes apache#243 from dongjoon-hyun/SPARK-52471.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>Swift-based SparkPi K8s CronJob example1 parent 21db3b2 commit 88d106d
1 file changed
+31
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
0 commit comments