Skip to content

Commit 5053714

Browse files
authored
Added info on creating alert rule in Prometheus (#547)
* Added info on creating alert rule in Prometheus. * Updated info on creating alert rule in Prometheus * Rewrote update as procedure and implemented rest of reviewer's feedback. * Corrected output error.
1 parent 4c81fe3 commit 5053714

File tree

1 file changed

+35
-0
lines changed

1 file changed

+35
-0
lines changed

docs/kafka/metrics-monitoring-kafka/README.adoc

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -250,8 +250,43 @@ spec:
250250
name: additional-scrape-configs
251251
key: kafka-federate.yaml
252252
----
253+
253254
====
254255

256+
[id="proc-configuring-prometheus-alerts_{context}"]
257+
== Configuring Prometheus alerts for Kafka instance limits
258+
259+
.Prerequisites
260+
* You have successfully configured metrics monitoring for a Kafka instance in Prometheus.
261+
* You use the Prometheus Operator in your monitoring environment.
262+
* You can define https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/[alerting rules] in Prometheus and can deploy an https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/user-guides/alerting.md/[Alertmanager cluster] in Prometheus Operator.
263+
264+
265+
.Procedure
266+
. Create a `PrometheusRule` custom resource with alerts defined for the https://access.redhat.com/articles/5979061[capacity of your Kafka instance].
267+
. Apply the `PrometheusRule` to the cluster that you are federating the metrics to.
268+
269+
.Example `PrometheusRule` custom resource for a Kafka broker storage limit alert
270+
[source,subs="+quotes"]
271+
----
272+
apiVersion: monitoring.coreos.com/v1
273+
kind: PrometheusRule
274+
spec:
275+
groups:
276+
- name: limits
277+
rules:
278+
- alert: KafkaBrokerStorageFillingUp
279+
expr: predict_linear(kubelet_volume_stats_available_bytes{persistentvolumeclaim=~"data-(.+)-kafka-[0-9]+"}[1h], 4 * 3600)
280+
labels:
281+
severity: <SOME_SEVERITY>
282+
annotations:
283+
summary: 'Broker PersistentVolume is filling up.'
284+
description: 'Based on recent sampling, the Broker PersistentVolume claimed by {{ $labels.persistentvolumeclaim }} is expected to fill up within four days.
285+
----
286+
287+
288+
289+
255290
[role="_additional-resources"]
256291
.Additional resources
257292
* {base-url}{getting-started-url-kafka}[_Getting started with {product-long-kafka}_^]

0 commit comments

Comments
 (0)