From c44db7d71d16e1d6385c063f3c726413dec63c64 Mon Sep 17 00:00:00 2001 From: Zhou JIANG Date: Fri, 8 Aug 2025 13:13:33 +0200 Subject: [PATCH] [SPARK-53187] Support SparkCluster event related metrics set ### What changes were proposed in this pull request? This PR adds SparkCluster as a supported type in metrics and therefore supports publishing execution metrics for it. ### Why are the changes needed? Operator by default publish metrics by resource type (SparkApplication) - by adding this support, we support the same set of counter and histogram for received event and timed execution ### Does this PR introduce _any_ user-facing change? More Metrics becomes available for SparkClusters ### How was this patch tested? From dev sandbox we can see metrics like ``` metrics_operator.sdk_sparkapplication_added_resource_event_Count{type="counters"} 9010 metrics_operator.sdk_sparkapplication_reconciliation_failed_Count{type="counters"} 9 metrics_operator.sdk_sparkapplication_reconciliation_finished_Count{type="counters"} 182841 metrics_operator.sdk_sparkapplication_reconciliation_retries_Count{type="counters"} 9 metrics_operator.sdk_sparkcluster_added_resource_event_Count{type="counters"} 9009 metrics_operator.sdk_sparkcluster_reconciliation_failed_Count{type="counters"} 0 metrics_operator.sdk_sparkcluster_reconciliation_finished_Count{type="counters"} 182821 metrics_operator.sdk_sparkcluster_reconciliation_retries_Count{type="counters"} 0 ``` ### Was this patch authored or co-authored using generative AI tooling? No Closes #295 from jiangzho/cluster_metrics. Authored-by: Zhou JIANG Signed-off-by: Peter Toth --- .../helm/spark-kubernetes-operator/templates/_helpers.tpl | 1 + 1 file changed, 1 insertion(+) diff --git a/build-tools/helm/spark-kubernetes-operator/templates/_helpers.tpl b/build-tools/helm/spark-kubernetes-operator/templates/_helpers.tpl index bd8cc86b..f5764393 100644 --- a/build-tools/helm/spark-kubernetes-operator/templates/_helpers.tpl +++ b/build-tools/helm/spark-kubernetes-operator/templates/_helpers.tpl @@ -115,6 +115,7 @@ spark.kubernetes.operator.namespace={{ .Release.Namespace }} spark.kubernetes.operator.name={{- include "spark-operator.name" . }} spark.kubernetes.operator.dynamicConfig.enabled={{ .Values.operatorConfiguration.dynamicConfig.enable }} spark.kubernetes.operator.metrics.port={{ include "spark-operator.metricsPort" . }} +spark.kubernetes.operator.health.probePort={{ include "spark-operator.probePort" . }} {{- if .Values.workloadResources.namespaces.overrideWatchedNamespaces }} spark.kubernetes.operator.watchedNamespaces={{ include "spark-operator.workloadNamespacesStr" . | trim }} {{- end }}