Merge pull request #23408 from rh-max/monitoring-timeseries-metadata

Monitoring timeseries metadata

Author: Bob Furu

modules/monitoring-attaching-additional-labels-to-your-time-series-and-alerts.adoc

@@ -0,0 +1,62 @@
+// Module included in the following assemblies:
+//
+// * monitoring/cluster_monitoring/configuring-the-monitoring-stack.adoc
+
+[id="attaching-additional-labels-to-your-time-series-and-alerts_{context}"]
+= Attaching additional labels to your time series and alerts
+
+Using the external labels feature of Prometheus, you can attach additional custom labels to all time series and alerts leaving the Prometheus cluster.
+
+.Prerequisites
+
+* Make sure you have the `cluster-monitoring-config` ConfigMap object with the `data/config.yaml` section.
+
+.Procedure
+
+. Start editing the `cluster-monitoring-config` ConfigMap:
++
+----
+$ oc -n openshift-monitoring edit configmap cluster-monitoring-config
+----
+
+. Define a map of labels you want to add for every metric under `data/config.yaml`:
++
+[source, yaml]
+----
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: cluster-monitoring-config
+  namespace: openshift-monitoring
+data:
+  config.yaml: |
+    prometheusK8s:
+      externalLabels:
+        <key>: <value> <1>
+----
++
+<1> Substitute `<key>: <value>` with a map of key-value pairs where `<key>` is a unique name of the new label and `<value>` is its value.
++
+[WARNING]
+====
+Do not use `prometheus` or `prometheus_replica` as key names, because they are reserved and would be overwritten.
+====
++
+For example, to add metadata about the region and environment to all time series and alerts, use:
++
+[source, yaml]
+----
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: cluster-monitoring-config
+  namespace: openshift-monitoring
+data:
+  config.yaml: |
+    prometheusK8s:
+      externalLabels:
+        region: eu
+        environment: prod
+----
+
+. Save the file to apply the changes. The new configuration is applied automatically.

monitoring/cluster_monitoring/configuring-the-monitoring-stack.adoc

@@ -76,6 +76,8 @@ include::modules/monitoring-listing-acting-alerting-rules.adoc[leveloffset=+2]
 
 * See also link:https://prometheus.io/docs/alerting/alertmanager/[the Alertmanager documentation].
 
+include::modules/monitoring-attaching-additional-labels-to-your-time-series-and-alerts.adoc[leveloffset=+1]
+
 .Next steps
 
 * xref:../../monitoring/cluster_monitoring/managing-cluster-alerts.adoc#managing-cluster-alerts[Manage cluster alerts.]