Skip to content

Commit 798259e

Browse files
committed
update docs
1 parent 5038692 commit 798259e

File tree

3 files changed

+14
-13
lines changed

3 files changed

+14
-13
lines changed

docs/modules/spark-k8s/pages/usage-guide/history-server.adoc

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -157,10 +157,9 @@ By setting up port forwarding on 18080 the UI can be opened by pointing your bro
157157

158158
image::history-server-ui.png[History Server Console]
159159

160-
== Metrics
160+
== Monitoring
161161

162-
[NOTE]
163-
====
164-
Starting with version 25.7, the built-in Prometheus servlet is enabled in addition to the existing JMX exporter.
165-
The JMX exporter is still available but it is deprecated and will be removed in a future release.
166-
====
162+
The operator creates a Kubernetes service dedicated specifically to collect metrics for Spark History instances with Prometheus.
163+
These metrics are exported via the JMX exporter as the history server doesn't support the built in Spark prometheus servlet.
164+
The service name follows the convention `<stacklet name>-history-metrics`.
165+
Metrics can be scraped at the endpoint `<service name>:18081/metrics`.

docs/modules/spark-k8s/pages/usage-guide/operations/applications.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ As the operator creates the necessary resources, the status of the application t
99
NOTE: The operator never reconciles an application once it has been created.
1010
To resubmit an application, a new SparkApplication resource must be created.
1111

12-
== Metrics
12+
== Monitoring
1313

1414
[NOTE]
1515
====

docs/modules/spark-k8s/pages/usage-guide/spark-connect.adoc

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -26,12 +26,14 @@ include::example$example-spark-connect.yaml[]
2626
<7> Customize the driver properties in the `server` role. The number of cores here is not related to Kubernetes cores!
2727
<8> Customize `spark.executor.\*` and `spark.kubernetes.executor.*` in the `executor` role.
2828

29-
== Metrics
29+
== Monitoring
3030

31-
The server pod exposes Prometheus metrics at the following endpoints:
31+
The operator creates a Kubernetes service dedicated specifically to collect metrics for Spark Connect instances with Prometheus.
32+
The service name follows the convention `<stacklet name>-server-metrics`.
33+
This service exposes Prometheus metrics at the following endpoints:
3234

33-
* `/metrics/prometheus` for driver instances.
34-
* `/metrics/executors/prometheus` for executor instances.
35+
* `<service name>:4040/metrics/prometheus` for driver instances.
36+
* `<service name>:4040/metrics/executors/prometheus` for executor instances.
3537

3638
To customize the metrics configuration use the `spec.server.configOverrides' like this:
3739

@@ -47,8 +49,8 @@ The example above adds a new endpoint for application metrics.
4749

4850
== Spark History Server
4951

50-
Unforunately integration with the Spark History Server is not supported yet.
51-
The connect server seems to ignore the `spark.eventLog` properties while also prohibiting clients to set them programatically.
52+
Unfortunately integration with the Spark History Server is not supported yet.
53+
The connect server seems to ignore the `spark.eventLog` properties while also prohibiting clients to set them programmatically.
5254

5355
== Notable Omissions
5456

0 commit comments

Comments
 (0)