Skip to content

Commit 1ca8fe0

Browse files
committed
manifests: add pvc related alerts to prometheus rule
alerts when detecting metrics for read-only filesystem and device out of space errors. the alerts in this commit follow the conventions specified by https://github.com/openshift/enhancements/blob/master/enhancements/monitoring/alerting-consistency.md.
1 parent acc522a commit 1ca8fe0

File tree

1 file changed

+24
-1
lines changed

1 file changed

+24
-1
lines changed

manifests/09-prometheus-rules.yaml

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,31 @@ metadata:
55
namespace: openshift-image-registry
66
annotations:
77
capability.openshift.io/name: ImageRegistry
8-
release.openshift.io/delete: "true"
98
include.release.openshift.io/hypershift: "true"
109
include.release.openshift.io/ibm-cloud-managed: "true"
1110
include.release.openshift.io/self-managed-high-availability: "true"
1211
include.release.openshift.io/single-node-developer: "true"
12+
spec:
13+
groups:
14+
- name: pvc-problem-detector.rules
15+
rules:
16+
- alert: ImageRegistryStorageReadOnly
17+
for: 10m
18+
expr: sum without(instance, pod, operation) (rate(imageregistry_storage_errors_total{code="READ_ONLY_FILESYSTEM"}[5m])) > 0
19+
labels:
20+
kubernetes_operator_part_of: image-registry
21+
severity: warning
22+
annotations:
23+
summary: The image registry storage is read-only and no images will be committed to storage.
24+
description: The image registry storage is read-only. Read-only storage affects direct pushes to the image registry, and pull-through proxy caching. In the case of pull-through proxy caching, read-only storage is particularly important because without it the image registry won't be actually caching anything. Please verify your backing storage solution and make sure the volume mounted on the image-registry pods is writable to avoid potential outages.
25+
message: The image registry storage is read-only and no images will be committed to storage.
26+
- alert: ImageRegistryStorageFull
27+
for: 10m
28+
expr: sum without(instance, pod, operation) (rate(imageregistry_storage_errors_total{code="DEVICE_OUT_OF_SPACE"}[5m])) > 0
29+
labels:
30+
kubernetes_operator_part_of: image-registry
31+
severity: warning
32+
annotations:
33+
summary: The image registry storage disk is full and no images will be committed to storage.
34+
description: The image registry storage disk is full. A full disk affects direct pushes to the image registry, and pull-through proxy caching. In the case of pull-through proxy caching, disk space is particularly important because without it the image registry won't be actually caching anything. Please verify your backing storage solution and make sure the volume mounted on the image-registry pods have enough free disk space to avoid potential outages.
35+
message: The image registry storage disk is full and no images will be committed to storage.

0 commit comments

Comments
 (0)