feat: Add KubeDeploymentRolloutStuck

This add an alert when a deployment rollout hits its
`spec.progressDeadlineSeconds`.
Those could have a number of causes, which can originates from the
cluster or the deployment:
- pods taking too much time to start
- cluster at full capacity and deployment surging during upgrade
  (maxSurge > 0).

Author: VannTen

alerts/apps_alerts.libsonnet

@@ -87,6 +87,21 @@
             'for': '15m',
             alert: 'KubeDeploymentReplicasMismatch',
           },
+          {
+            expr: |||
+              kube_deployment_status_condition{condition="Progressing", status="false",%(prefixedNamespaceSelector)s%(kubeStateMetricsSelector)s}
+              != 0
+            ||| % $._config,
+            labels: {
+              severity: 'warning',
+            },
+            annotations: {
+              description: 'Rollout of deployment {{ $labels.namespace }}/{{ $labels.deployment }} is not progressing for longer than 15 minutes.',
+              summary: 'Deployment rollout is not progressing.',
+            },
+            'for': '15m',
+            alert: 'KubeDeploymentRolloutStuck',
+          },
           {
             expr: |||
               (

runbook.md

@@ -37,6 +37,9 @@ This page collects this repositories alerts and begins the process of describing
 ##### Alert Name: "KubeDeploymentReplicasMismatch"
 + *Message*: `Deployment {{ $labels.namespace }}/{{ $labels.deployment }} replica mismatch`
 + *Severity*: warning
+##### Alert Name: "KubeDeploymentRolloutStuck"
++ *Message*: `Rollout of deployment {{ $labels.namespace }}/{{ $labels.deployment }} is not progressing`
++ *Severity*: warning
 ##### Alert Name: "KubeStatefulSetReplicasMismatch"
 + *Message*: `StatefulSet {{ $labels.namespace }}/{{ $labels.statefulset }} replica mismatch`
 + *Severity*: warning