Commit 38c2deb
committed
[SPARK-52297] Add
### What changes were proposed in this pull request?
This PR aims to add [Swift-based SparkPi](https://github.com/apache/spark-connect-swift/tree/main/Examples/pi) K8s [Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) example.
### Why are the changes needed?
To provide a working example with
- [Apache Spark 4.0.0 Connect Server](https://spark.apache.org/docs/4.0.0/) launched by [Apache Spark Kubernetes Operator 0.2.0](https://artifacthub.io/packages/helm/spark-kubernetes-operator/spark-kubernetes-operator) Helm chart.
- [SparkPi Swift App](https://github.com/apache/spark-connect-swift/tree/main/Examples/pi) developed with [Apache Spark Connect for Swift 0.2.0](https://swiftpackageindex.com/apache/spark-connect-swift) library.
- [Swift 6.1](https://www.swift.org) language
### Does this PR introduce _any_ user-facing change?
No behavior change because this is a new example.
### How was this patch tested?
Manual tests.
```bash
# Install Spark K8s Operator
$ helm install spark spark/spark-kubernetes-operator
# Launch Spark 4.0 Connect Server
$ kubectl apply -f examples/spark-connect-server.yaml
# Launch `Swift-based SparkPi` Application
$ kubectl apply -f examples/job/pi-swift.yaml
# Check the log
$ kubectl logs -f job/spark-connect-swift-pi
Pi is roughly 3.142923142923143
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes apache#227 from dongjoon-hyun/SPARK-52297.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>Swift-based SparkPi K8s Job example1 parent 4523b6d commit 38c2deb
1 file changed
+28
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
0 commit comments