Skip to content

Commit 88d106d

Browse files
committed
[SPARK-52471] Add Swift-based SparkPi K8s CronJob example
### What changes were proposed in this pull request? This PR aims to add [Swift-based SparkPi](https://github.com/apache/spark-connect-swift/tree/main/Examples/pi) K8s [CronJob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/) example. ### Why are the changes needed? To provide a working example with - [Apache Spark 4.0.0 Connect Server](https://spark.apache.org/docs/4.0.0/) launched by [Apache Spark Kubernetes Operator 0.3.0](https://artifacthub.io/packages/helm/spark-kubernetes-operator/spark-kubernetes-operator) Helm chart. - [SparkPi Swift App](https://github.com/apache/spark-connect-swift/tree/main/Examples/pi) developed with [Apache Spark Connect for Swift 0.3.0](https://swiftpackageindex.com/apache/spark-connect-swift) library. - [Swift 6.1](https://www.swift.org) language ### Does this PR introduce _any_ user-facing change? No behavior change because this is a new example. ### How was this patch tested? Manual tests. ```bash # Install Spark K8s Operator $ helm install spark spark/spark-kubernetes-operator # Launch Spark 4.0 Connect Server $ kubectl apply -f examples/spark-connect-server.yaml # Launch `Swift-based SparkPi` Application $ kubectl apply -f examples/job/cron-pi-swift.yaml $ kubectl get cronjob NAME SCHEDULE TIMEZONE SUSPEND ACTIVE LAST SCHEDULE AGE spark-connect-swift-pi * * * * * <none> False 1 2s 7m51s $ kubectl get pod -l job-name NAME READY STATUS RESTARTS AGE spark-connect-swift-pi-29163859-mkh5p 0/1 Completed 0 2m31s spark-connect-swift-pi-29163860-btc66 0/1 Completed 0 91s spark-connect-swift-pi-29163861-68mmt 0/1 Completed 0 31s ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#243 from dongjoon-hyun/SPARK-52471. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 21db3b2 commit 88d106d

File tree

1 file changed

+31
-0
lines changed

1 file changed

+31
-0
lines changed

examples/job/cron-pi-swift.yaml

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one or more
2+
# contributor license agreements. See the NOTICE file distributed with
3+
# this work for additional information regarding copyright ownership.
4+
# The ASF licenses this file to You under the Apache License, Version 2.0
5+
# (the "License"); you may not use this file except in compliance with
6+
# the License. You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
apiVersion: batch/v1
16+
kind: CronJob
17+
metadata:
18+
name: spark-connect-swift-pi
19+
spec:
20+
schedule: "* * * * *"
21+
jobTemplate:
22+
spec:
23+
template:
24+
spec:
25+
containers:
26+
- name: job
27+
image: apache/spark-connect-swift:pi
28+
env:
29+
- name: SPARK_REMOTE
30+
value: 'sc://spark-connect-server-0-driver-svc:15002'
31+
restartPolicy: Never

0 commit comments

Comments
 (0)