@@ -37,16 +37,17 @@ found [here](https://github.com/apache/datafusion-comet/tree/main/benchmarks).
3737
3838Install helm Spark operator for Kubernetes
3939``` bash
40+ # Add the Helm repository
4041helm repo add spark-operator https://kubeflow.github.io/spark-operator
41-
4242helm repo update
4343
44- helm install my-release spark-operator/spark-operator --namespace spark-operator --create-namespace --set webhook.enable=true
45- ````
44+ # Install the operator into the spark-operator namespace and wait for deployments to be ready
45+ helm install spark-operator spark-operator/spark-operator --namespace spark-operator --create-namespace --wait
46+ ```
4647
4748Check the operator is deployed
4849``` bash
49- helm status --namespace spark-operator my-release
50+ helm status --namespace spark-operator spark-operator
5051
5152NAME: my-release
5253NAMESPACE: spark-operator
9192 labels:
9293 version: 3.5.4
9394 instances: 1
94- cores: 2
95+ cores: 1
9596 coreLimit: 1200m
9697 memory: 512m
9798```
@@ -100,10 +101,35 @@ Refer to [Comet builds](#comet-docker-images)
100101Run Apache Spark application with Comet enabled
101102``` bash
102103kubectl apply -f spark-pi.yaml
104+ sparkapplication.sparkoperator.k8s.io/spark-pi created
103105```
104106
105107Check application status
106108``` bash
107- kubectl describe sparkapplication --namespace=spark-operator
109+ kubectl get sparkapp spark-pi
110+
111+ NAME STATUS ATTEMPTS START FINISH AGE
112+ spark-pi RUNNING 1 2025-03-18T21:19:48Z < no value> 65s
113+ ```
114+ To check more runtime details
115+ ``` bash
116+ kubectl describe sparkapplication spark-pi
117+
118+ ....
119+ Events:
120+ Type Reason Age From Message
121+ ---- ------ ---- ---- -------
122+ Normal SparkApplicationSubmitted 8m15s spark-application-controller SparkApplication spark-pi was submitted successfully
123+ Normal SparkDriverRunning 7m18s spark-application-controller Driver spark-pi-driver is running
124+ Normal SparkExecutorPending 7m11s spark-application-controller Executor [spark-pi-68732195ab217303-exec-1] is pending
125+ Normal SparkExecutorRunning 7m10s spark-application-controller Executor [spark-pi-68732195ab217303-exec-1] is running
126+ Normal SparkExecutorCompleted 7m5s spark-application-controller Executor [spark-pi-68732195ab217303-exec-1] completed
127+ Normal SparkDriverCompleted 7m4s spark-application-controller Driver spark-pi-driver completed
128+
129+ ```
130+
131+ Get Driver Logs
132+ ``` bash
133+ kubectl logs spark-pi-driver
108134```
109135More info on [ Kube Spark operator] ( https://www.kubeflow.org/docs/components/spark-operator/getting-started/ )
0 commit comments