Merge pull request #48050 from bburt-rh/RHDEVDOCS-3956-document-per-label-scrape-limit-settings-for-UWM

RHDEVDOCS-3956 - document per label scrape limit settings for UWM

Author: bburt-rh

modules/monitoring-limiting-scrape-samples-in-user-defined-projects.adoc

@@ -2,19 +2,22 @@
 //
 // * monitoring/configuring-the-monitoring-stack.adoc
 
+:_content-type: CONCEPT
 [id="controlling-the-impact-of-unbound-attributes-in-user-defined-projects_{context}"]
 = Controlling the impact of unbound metrics attributes in user-defined projects
 
 Developers can create labels to define attributes for metrics in the form of key-value pairs. The number of potential key-value pairs corresponds to the number of possible values for an attribute. An attribute that has an unlimited number of potential values is called an unbound attribute. For example, a `customer_id` attribute is unbound because it has an infinite number of possible values.
 
-Every assigned key-value pair has a unique time series. The use of many unbound attributes in labels can result in an exponential increase in the number of time series created. This can impact Prometheus performance and can consume a lot of disk space.
+Every assigned key-value pair has a unique time series. Using many unbound attributes in labels can create exponentially more time series, which can impact Prometheus performance and available disk space.
 
 Cluster administrators can use the following measures to control the impact of unbound metrics attributes in user-defined projects:
 
-* *Limit the number of samples that can be accepted* per target scrape in user-defined projects
-* *Create alerts* that fire when a scrape sample threshold is reached or when the target cannot be scraped
+* Limit the number of samples that can be accepted per target scrape in user-defined projects
+* Limit the number of scraped labels, the length of label names, and the length of label values.
+* Create alerts that fire when a scrape sample threshold is reached or when the target cannot be scraped
 
 [NOTE]
 ====
-Limiting scrape samples can help prevent the issues caused by adding many unbound attributes to labels. Developers can also prevent the underlying cause by limiting the number of unbound attributes that they define for metrics. Using attributes that are bound to a limited set of possible values reduces the number of potential key-value pair combinations.
+To prevent issues caused by adding many unbound attributes, limit the number of scrape samples, label names, and unbound attributes you define for metrics. 
+Also reduce the number of potential key-value pair combinations by using attributes that are bound to a limited set of possible values.
 ====

modules/monitoring-setting-a-scrape-sample-limit-for-user-defined-projects.adoc renamed to modules/monitoring-setting-scrape-sample-and-label-limits-for-user-defined-projects.adoc

@@ -3,20 +3,21 @@
 // * monitoring/configuring-the-monitoring-stack.adoc
 
 :_content-type: PROCEDURE
-[id="setting-a-scrape-sample-limit-for-user-defined-projects_{context}"]
-= Setting a scrape sample limit for user-defined projects
+[id="setting-scrape-sample-and-label-limits-for-user-defined-projects_{context}"]
+= Setting scrape sample and label limits for user-defined projects
 
 You can limit the number of samples that can be accepted per target scrape in user-defined projects.
+You can also limit the number of scraped labels, the length of label names, and the length of label values.
 
 [WARNING]
 ====
-If you set a sample limit, no further sample data is ingested for that target scrape after the limit is reached.
+If you set sample or label limits, no further sample data is ingested for that target scrape after the limit is reached.
 ====
 
 .Prerequisites
 
 * You have access to the cluster as a user with the `cluster-admin` role, or as a user with the `user-workload-monitoring-config-edit` role in the `openshift-user-workload-monitoring` project.
-* You have created the `user-workload-monitoring-config` `ConfigMap` object.
+* You have enabled monitoring for user-defined projects.
 * You have installed the OpenShift CLI (`oc`).
 
 .Procedure
@@ -44,7 +45,30 @@ data:
 ----
 <1> A value is required if this parameter is specified. This `enforcedSampleLimit` example limits the number of samples that can be accepted per target scrape in user-defined projects to 50,000.
 
-. Save the file to apply the changes. The limit is applied automatically.
+. Add the `enforcedLabelLimit`, `enforcedLabelNameLengthLimit`, and `enforcedLabelValueLengthLimit` configurations to `data/config.yaml` to limit the number of scraped labels, the length of label names, and the length of label values in user-defined projects:
++
+[source,yaml]
+----
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: user-workload-monitoring-config
+  namespace: openshift-user-workload-monitoring
+data:
+  config.yaml: |
+    prometheus:
+      enforcedLabelLimit: 500 <1>
+      enforcedLabelNameLengthLimit: 50 <2>
+      enforcedLabelValueLengthLimit: 600 <3>
+----
+<1> Specifies the maximum number of labels per scrape. 
+The default value is `0`, which specifies no limit.
+<2> Specifies the maximum length in characters of a label name.
+The default value is `0`, which specifies no limit.
+<3> Specifies the maximum length in characters of a label value.
+The default value is `0`, which specifies no limit.
+
+. Save the file to apply the changes. The limits are applied automatically.
 +
 [NOTE]
 ====

monitoring/configuring-the-monitoring-stack.adoc

@@ -108,7 +108,7 @@ include::modules/monitoring-creating-cluster-id-labels-for-metrics.adoc[leveloff
 
 // Managing scrape sample limits for user-defined projects
 include::modules/monitoring-limiting-scrape-samples-in-user-defined-projects.adoc[leveloffset=+1]
-include::modules/monitoring-setting-a-scrape-sample-limit-for-user-defined-projects.adoc[leveloffset=+2]
+include::modules/monitoring-setting-scrape-sample-and-label-limits-for-user-defined-projects.adoc[leveloffset=+2]
 include::modules/monitoring-creating-scrape-sample-alerts.adoc[leveloffset=+2]
 
 [role="_additional-resources"]

monitoring/troubleshooting-monitoring-issues.adoc

@@ -23,5 +23,5 @@ include::modules/monitoring-determining-why-prometheus-is-consuming-disk-space.a
 [role="_additional-resources"]
 .Additional resources
 
-* See xref:../monitoring/configuring-the-monitoring-stack.adoc#setting-a-scrape-sample-limit-for-user-defined-projects_configuring-the-monitoring-stack[Setting a scrape sample limit for user-defined projects] for details on how to set a scrape sample limit and create related alerting rules
+* See xref:../monitoring/configuring-the-monitoring-stack.adoc#setting-scrape-sample-and-label-limits-for-user-defined-projects_configuring-the-monitoring-stack[Setting a scrape sample limit for user-defined projects] for details on how to set a scrape sample limit and create related alerting rules
 * xref:../support/getting-support.html#support-submitting-a-case_getting-support[Submitting a support case]

support/troubleshooting/investigating-monitoring-issues.adoc

@@ -27,4 +27,4 @@ include::modules/monitoring-determining-why-prometheus-is-consuming-disk-space.a
 [role="_additional-resources"]
 .Additional resources
 
-* See xref:../../monitoring/configuring-the-monitoring-stack.adoc#setting-a-scrape-sample-limit-for-user-defined-projects_configuring-the-monitoring-stack[Setting a scrape sample limit for user-defined projects] for details on how to set a scrape sample limit and create related alerting rules
+* See xref:../../monitoring/configuring-the-monitoring-stack.adoc#setting-scrape-sample-and-label-limits-for-user-defined-projects_configuring-the-monitoring-stack[Setting a scrape sample limit for user-defined projects] for details on how to set a scrape sample limit and create related alerting rules