Skip to content

Commit 46b162c

Browse files
authored
doc: Renew kubernetes.md (#1549)
1 parent 2226cfa commit 46b162c

File tree

1 file changed

+32
-6
lines changed

1 file changed

+32
-6
lines changed

docs/source/user-guide/kubernetes.md

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -37,16 +37,17 @@ found [here](https://github.com/apache/datafusion-comet/tree/main/benchmarks).
3737

3838
Install helm Spark operator for Kubernetes
3939
```bash
40+
# Add the Helm repository
4041
helm repo add spark-operator https://kubeflow.github.io/spark-operator
41-
4242
helm repo update
4343

44-
helm install my-release spark-operator/spark-operator --namespace spark-operator --create-namespace --set webhook.enable=true
45-
````
44+
# Install the operator into the spark-operator namespace and wait for deployments to be ready
45+
helm install spark-operator spark-operator/spark-operator --namespace spark-operator --create-namespace --wait
46+
```
4647

4748
Check the operator is deployed
4849
```bash
49-
helm status --namespace spark-operator my-release
50+
helm status --namespace spark-operator spark-operator
5051

5152
NAME: my-release
5253
NAMESPACE: spark-operator
@@ -91,7 +92,7 @@ spec:
9192
labels:
9293
version: 3.5.4
9394
instances: 1
94-
cores: 2
95+
cores: 1
9596
coreLimit: 1200m
9697
memory: 512m
9798
```
@@ -100,10 +101,35 @@ Refer to [Comet builds](#comet-docker-images)
100101
Run Apache Spark application with Comet enabled
101102
```bash
102103
kubectl apply -f spark-pi.yaml
104+
sparkapplication.sparkoperator.k8s.io/spark-pi created
103105
```
104106

105107
Check application status
106108
```bash
107-
kubectl describe sparkapplication --namespace=spark-operator
109+
kubectl get sparkapp spark-pi
110+
111+
NAME STATUS ATTEMPTS START FINISH AGE
112+
spark-pi RUNNING 1 2025-03-18T21:19:48Z <no value> 65s
113+
```
114+
To check more runtime details
115+
```bash
116+
kubectl describe sparkapplication spark-pi
117+
118+
....
119+
Events:
120+
Type Reason Age From Message
121+
---- ------ ---- ---- -------
122+
Normal SparkApplicationSubmitted 8m15s spark-application-controller SparkApplication spark-pi was submitted successfully
123+
Normal SparkDriverRunning 7m18s spark-application-controller Driver spark-pi-driver is running
124+
Normal SparkExecutorPending 7m11s spark-application-controller Executor [spark-pi-68732195ab217303-exec-1] is pending
125+
Normal SparkExecutorRunning 7m10s spark-application-controller Executor [spark-pi-68732195ab217303-exec-1] is running
126+
Normal SparkExecutorCompleted 7m5s spark-application-controller Executor [spark-pi-68732195ab217303-exec-1] completed
127+
Normal SparkDriverCompleted 7m4s spark-application-controller Driver spark-pi-driver completed
128+
129+
```
130+
131+
Get Driver Logs
132+
```bash
133+
kubectl logs spark-pi-driver
108134
```
109135
More info on [Kube Spark operator](https://www.kubeflow.org/docs/components/spark-operator/getting-started/)

0 commit comments

Comments
 (0)