Skip to content

Commit 387cbea

Browse files
richardtiefibakshay
richardtief
and
ibakshay
authored
feat(alerts): CA cert creation via cert-manager (#637)
* feat(alerts): CA cert creation via cert-manager fixes #546 Helm `genCA` function led to reconciliation loops in Greenhouse. Signed-off-by: Richard Tief <richard.tief@sap.com> * fix(alerts): correct formatting in values.yaml for caCert comment Signed-off-by: IB Akshay <akshay.iyyadurai.balasundaram@sap.com> * chore(alerts): improve description in values.yaml Signed-off-by: Richard Tief <richard.tief@sap.com> * chore(alerts): disable certManager for helm-lint and helm-test Signed-off-by: Richard Tief <richard.tief@sap.com> * chore(alerts): move ci folder into charts/ Signed-off-by: Richard Tief <richard.tief@sap.com> * chore(alerts): increase versions Signed-off-by: Richard Tief <richard.tief@sap.com> * chore(alerts): name cert-manager resources like the organisation Signed-off-by: Richard Tief <richard.tief@sap.com> * chore(alerts): add shared prometheus cert For consecutive Prometheus installations. Signed-off-by: Richard Tief <richard.tief@sap.com> * docs(alerts): adds cert-manager option values and explaination Signed-off-by: Richard Tief <richard.tief@sap.com> * chore(alerts): add cert-manager options to plugindefinition Signed-off-by: Richard Tief <richard.tief@sap.com> * docs(alerts): improve wording Co-authored-by: Akshay Iyyadurai Balasundaram <akshay.iyyadurai.balasundaram@sap.com> --------- Signed-off-by: Richard Tief <richard.tief@sap.com> Signed-off-by: IB Akshay <akshay.iyyadurai.balasundaram@sap.com> Co-authored-by: IB Akshay <akshay.iyyadurai.balasundaram@sap.com>
1 parent 39f7000 commit 387cbea

13 files changed

+160
-191
lines changed

‎alerts/README.md‎

Lines changed: 12 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,10 @@ Greenhouse regularly performs integration tests that are bundled with **alerts**
6060

6161
| Name | Description | Value |
6262
| ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------- | ------- |
63+
| `global.caCert` | Additional caCert to add to the CA bundle | `""` |
6364
| `alerts.commonLabels` | Labels to apply to all resources | `{}` |
65+
| `alerts.defaultRules.create` | Creates community Alertmanager alert rules. | `true` |
66+
| `alerts.defaultRules.labels` | kube-monitoring `plugin: <plugin.name>` to evaluate Alertmanager rules. | `{}` |
6467
| `alerts.alertmanager.enabled` | Deploy Prometheus Alertmanager | `true` |
6568
| `alerts.alertmanager.annotations` | Annotations for Alertmanager | `{}` |
6669
| `alerts.alertmanager.config` | Alertmanager configuration directives. | `{}` |
@@ -76,18 +79,15 @@ Greenhouse regularly performs integration tests that are bundled with **alerts**
7679
| `alerts.alertmanager.alertmanagerConfig.webhook.routes[].name` | Name of the webhook route. | `""` |
7780
| `alerts.alertmanager.alertmanagerConfig.webhook.routes[].url` | Webhook url to post alerts to. | `""` |
7881
| `alerts.alertmanager.alertmanagerConfig.webhook.routes[].matchers` | List of matchers that the alert's label should match. matchType <string>, name <string>, regex <boolean>, value <string> | `[]` |
82+
| `alerts.alertmanager.alertmanagerSpec.alertmanagerConfiguration` | AlermanagerConfig to be used as top level configuration | `false` |
83+
| `alerts.alertmanager.alertmanagerConfig.webhook.routes[].matchers` | List of matchers that the alert's label should match. matchType <string>, name <string>, regex <boolean>, value <string> | `[]` |
7984

80-
| `alerts.auth.secretName` | Use custom secret for Alertmanager authentication | `""` |
81-
| `alerts.auth.autoGenerateCert.enabled` | TLS Certificate Option 1: Use Helm to automatically generate self-signed certificate. | `true` |
82-
| `alerts.auth.autoGenerateCert.recreate` | If set to true, new key/certificate is generated on Helm upgrade. | `false` |
83-
| `alerts.auth.autoGenerateCert.certPeriodDays` | Cert period time in days. The default is 365 days. | `365` |
84-
| `alerts.auth.certFile` | Path to your own PEM-encoded certificate. | `""` |
85-
| `alerts.auth.keyFile` | Path to your own PEM-encoded private key. | `""` |
86-
| `alerts.auth.caFile` | Path to CA cert. | `""` |
85+
### cert-manager options
8786

88-
| `alerts.defaultRules.create` | Creates community Alertmanager alert rules. | `true` |
89-
| `alerts.defaultRules.labels` | kube-monitoring `plugin: <plugin.name>` to evaluate Alertmanager rules. | `{}` |
90-
| `alerts.alertmanager.alertmanagerSpec.alertmanagerConfiguration` | AlermanagerConfig to be used as top level configuration | `false` |
87+
| `alerts.certManager.enabled` | Creates `jetstack/cert-manager` resources to generate Issuer and Certificates for Prometheus authentication. | `true` |
88+
| `alerts.certManager.rootCert.duration` | Duration, how long the root certificate is valid. | `"5y"` |
89+
| `alerts.certManager.admissionCert.duration` | Duration, how long the admission certificate is valid. | `"1y"` |
90+
| `alerts.certManager.issuerRef.name` | Name of the existing Issuer to use. | `""` |
9191

9292
### Supernova options
9393

@@ -216,11 +216,9 @@ alertmanagerConfiguration:
216216

217217
## TLS Certificate Requirement
218218

219-
Greenhouse onboarded Prometheus installations need to communicate with the Alertmanager component to enable advanced processing of alerts. The Alertmanager Ingress requires a TLS certificate to be configured and trusted by Prometheus to ensure the communication. There are various ways in which you can generate/configure the required TLS certificate.
219+
Greenhouse onboarded Prometheus installations need to communicate with the Alertmanager component to enable processing of alerts. If an Alertmanager Ingress is enabled, this requires a TLS certificate to be configured and trusted by Alertmanger to ensure the communication. To enable automatic self-signed TLS certificate provisioning via cert-manager, set the `alerts.certManager.enabled` value to `true`.
220220

221-
- You can use an automatically generated self-signed certificate by setting `alerts.auth.autoGenerateCert.enabled` to `true`. Helm will create a self-signed cert and a secret for you.
222-
- You can use your own generated self-signed certificate by setting `alerts.auth.autoGenerateCert.enabled` to `false`. You should provide the necessary values to `alerts.auth.certFile`, `alerts.auth.keyFile`, and `alerts.auth.caFile`.
223-
- You can also sideload custom certificate by disabling `alerts.auth.autoGenerateCert.enabled` to `false` while setting your custom cert secret name in `alerts.auth.secretName`
221+
Note: Prerequisite of this feature is a installed [jetstack/cert-manager](https://github.com/jetstack/cert-manager) which can be implemented via the Greenhouse [cert-manager](https://github.com/cloudoperators/greenhouse-extensions/tree/main/cert-manager) Plugin.
224222

225223
## Examples
226224

‎alerts/charts/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ maintainers:
88
name: alerts
99
sources:
1010
- https://github.com/cloudoperators/greenhouse-extensions
11-
version: 0.18.0
11+
version: 0.19.0
1212
keywords:
1313
- prometheus-alertmanager
1414
dependencies:

alerts/ci/test-values.yaml renamed to alerts/charts/ci/test-values.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,5 @@ alerts:
66
enabled: true
77
hosts:
88
- dummy-host
9+
certManager:
10+
enabled: false

‎alerts/charts/templates/_helper.tpl‎

Lines changed: 0 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -47,36 +47,3 @@ plugin: {{ $root.Release.Name }}
4747
{{- end }}
4848
{{- end }}
4949
{{- end }}
50-
51-
{{/*
52-
Return certificate and CA for Prometheus to push alerts.
53-
It handles variants when a cert has to be generated by Helm,
54-
a cert is loaded from an existing secret or is provided via `.Values`
55-
*/}}
56-
{{- define "alerts.generateCert" -}}
57-
{{- $caCertEnc := "" }}
58-
{{- $certCrtEnc := "" }}
59-
{{- $certKeyEnc := "" }}
60-
{{- if .Values.alerts.auth.autoGenerateCert.enabled }}
61-
{{- $prevSecret := (lookup "v1" "Secret" .Release.Namespace (default (printf "%s-monitoring-ca" .Release.Namespace) .Values.alerts.auth.secretName )) }}
62-
{{- if and (not .Values.alerts.auth.autoGenerateCert.recreate) $prevSecret }}
63-
{{- $certCrtEnc = index $prevSecret "data" "tls.crt" }}
64-
{{- $certKeyEnc = index $prevSecret "data" "tls.key" }}
65-
{{- $caCertEnc = index $prevSecret "data" "ca.crt" }}
66-
{{- else }}
67-
{{- $altNames := list ( printf "%s-alertmanager.%s" .Release.Name .Release.Namespace ) ( printf "%s-alertmanager.%s.svc" .Release.Name .Release.Namespace ) -}}
68-
{{- $tmpperioddays := int .Values.alerts.auth.autoGenerateCert.certPeriodDays | default 365 }}
69-
{{- $ca := genCA "greenhouse-monitoring-ca" $tmpperioddays }}
70-
{{- $cert := genSignedCert .Release.Name nil $altNames $tmpperioddays $ca }}
71-
{{- $certCrtEnc = b64enc $cert.Cert }}
72-
{{- $certKeyEnc = b64enc $cert.Key }}
73-
{{- $caCertEnc = b64enc $ca.Cert }}
74-
{{- end }}
75-
{{- else }}
76-
{{- $certCrtEnc = .Files.Get .Values.alerts.auth.certFile | b64enc }}
77-
{{- $certKeyEnc = .Files.Get .Values.alerts.auth.keyFile | b64enc }}
78-
{{- $caCertEnc = .Files.Get .Values.alerts.auth.caFile | b64enc }}
79-
{{- end }}
80-
{{- $result := dict "crt" $certCrtEnc "key" $certKeyEnc "ca" $caCertEnc }}
81-
{{- $result | toYaml }}
82-
{{- end }}

‎alerts/charts/templates/am-ca-bundle-secret.yaml‎

Lines changed: 0 additions & 18 deletions
This file was deleted.

‎alerts/charts/templates/am-webhook-secrets.yaml‎

Lines changed: 0 additions & 23 deletions
This file was deleted.

‎alerts/charts/templates/ca-secret-issuer-cert.yaml‎

Lines changed: 0 additions & 55 deletions
This file was deleted.

‎alerts/charts/templates/certmanager.yaml‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
{{- if .Values.alerts.certManager.enabled -}}
2+
{{- if not .Values.alerts.certManager.issuerRef -}}
3+
# Create a selfsigned Issuer in order to create a root CA certificate for the org
4+
# signing alertmanager serving certificates
5+
apiVersion: cert-manager.io/v1
6+
kind: Issuer
7+
metadata:
8+
name: {{ .Release.Namespace }}-prometheus-issuer
9+
labels:
10+
{{- include "kube-prometheus-stack.labels" . | indent 4 }}
11+
spec:
12+
selfSigned: {}
13+
---
14+
# Generate a CA Certificate used to sign certificates for the alertmanager
15+
apiVersion: cert-manager.io/v1
16+
kind: Certificate
17+
metadata:
18+
name: {{ .Release.Namespace }}-prometheus-root-cert
19+
labels:
20+
{{- include "kube-prometheus-stack.labels" . | indent 4 }}
21+
spec:
22+
secretName: {{ .Release.Namespace }}-prometheus-root-cert
23+
duration: {{ .Values.alerts.certManager.rootCert.duration | default "43800h0m0s" | quote }}
24+
issuerRef:
25+
name: {{ .Release.Namespace }}-prometheus-issuer
26+
commonName: "ca.prometheus.greenhouse"
27+
isCA: true
28+
---
29+
# Create an Issuer that uses the above generated CA certificate to issue certs
30+
apiVersion: cert-manager.io/v1
31+
kind: Issuer
32+
metadata:
33+
name: {{ .Release.Namespace }}-prometheus-root-issuer
34+
labels:
35+
{{- include "kube-prometheus-stack.labels" . | indent 4 }}
36+
spec:
37+
ca:
38+
secretName: {{ .Release.Namespace }}-root-cert
39+
{{- end }}
40+
---
41+
# generate a server certificate for the alertmanager to use
42+
apiVersion: cert-manager.io/v1
43+
kind: Certificate
44+
metadata:
45+
name: {{ include "kube-prometheus-stack.fullname" . }}-cert
46+
labels:
47+
{{- include "kube-prometheus-stack.labels" . | indent 4 }}
48+
spec:
49+
secretName: tls-{{ include "kube-prometheus-stack.fullname" . }}-cert
50+
duration: {{ .Values.alerts.certManager.admissionCert.duration | default "8760h0m0s" | quote }}
51+
issuerRef:
52+
{{- if .Values.alerts.certManager.issuerRef }}
53+
{{- toYaml .Values.alerts.certManager.issuerRef | nindent 4 }}
54+
{{- else }}
55+
name: {{ .Release.Namespace }}-root-issuer
56+
{{- end }}
57+
dnsNames:
58+
- {{ include "kube-prometheus-stack.fullname" . }}
59+
- {{ include "kube-prometheus-stack.fullname" . }}.{{ .Release.Namespace }}
60+
- {{ include "kube-prometheus-stack.fullname" . }}.{{ .Release.Namespace }}.svc
61+
{{- if and .Values.alerts.alertmanager.ingress.enabled .Values.alerts.alertmanager.ingress.hosts }}
62+
{{ toYaml .Values.alerts.alertmanager.ingress.hosts | indent 2 }}
63+
{{- end }}
64+
---
65+
# generate a shared client certificate for prometheus to use
66+
apiVersion: cert-manager.io/v1
67+
kind: Certificate
68+
metadata:
69+
name: prometheus-{{ .Release.Namespace }}-cert
70+
labels:
71+
{{- include "kube-prometheus-stack.labels" . | indent 4 }}
72+
spec:
73+
secretName: tls-prometheus-{{ .Release.Namespace }}
74+
duration: {{ .Values.alerts.certManager.admissionCert.duration | default "8760h0m0s" | quote }}
75+
issuerRef:
76+
{{- if .Values.alerts.certManager.issuerRef }}
77+
{{- toYaml .Values.alerts.certManager.issuerRef | nindent 4 }}
78+
{{- else }}
79+
name: {{ .Release.Namespace }}-root-issuer
80+
{{- end }}
81+
dnsNames:
82+
{{- if .Values.global.greenhouse.baseDomain }}
83+
- {{ printf "*.s%" .Values.global.greenhouse.baseDomain }}
84+
{{- end }}
85+
{{- end -}}

‎alerts/charts/templates/hooks/caBundle-secret.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{{- if and .Values.alerts.alertmanager.enabled .Values.alerts.alertmanager.ingress.enabled .Values.global.caCert }}
2+
{{- $extCABundle := .Values.global.caCert }}
3+
{{- $intCASecret := (lookup "v1" "Secret" $.Release.Namespace (printf "%s-%s" (include "kube-prometheus-stack.fullname") "-root-cert")).data }}
4+
{{- $intCABundle := get $intCASecret "ca.crt" | b64dec }}
5+
apiVersion: v1
6+
kind: Secret
7+
type: Opaque
8+
metadata:
9+
name: {{ $.Release.Namespace }}-ca-bundle
10+
labels:
11+
{{- include "kube-prometheus-stack.labels" . | indent 4 }}
12+
annotations:
13+
"helm.sh/hook": post-install,post-upgrade
14+
"helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
15+
## Ensure this is run before release installation and upgrade
16+
"helm.sh/hook-weight": "-5"
17+
data:
18+
ca.crt: {{ printf "%s%s" $extCABundle $intCABundle | b64enc | quote }}
19+
{{- end }}

‎alerts/charts/templates/tests/test-alerts-config.yaml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,11 @@ data:
4949
}
5050
{{- end }}
5151
52-
{{- if and .Values.alerts.auth.autoGenerateCert.enabled .Values.alerts.alertmanager.ingress.enabled .Values.alerts.alertmanager.ingress.hosts (.Capabilities.APIVersions.Has "cert-manager.io/v1") }}
53-
@test "Verify certificate creation and validation against the API endpoint of the alert manager" {
52+
{{- if and .Values.alerts.alertmanager.ingress.enabled .Values.alerts.alertmanager.ingress.hosts .Values.alerts.certManager.enabled }}
53+
@test "Generated server certificate for the Prometheus Alertmanager to use" {
5454
verify "there is 1 issuer named '{{ .Release.Namespace }}-monitoring-issuer'"
55-
verify "there is 1 certificate named '{{ .Release.Namespace }}-prometheus-auth'"
56-
verify "there is 1 secret named 'tls-{{ .Release.Namespace }}-prometheus-auth'"
55+
verify "there is 1 certificate named '{{ include "kube-prometheus-stack.fullname" . }}-cert'"
56+
verify "there is 1 secret named '{{ include "kube-prometheus-stack.fullname" . }}-cert'"
5757
5858
url="https://{{ first .Values.alerts.alertmanager.ingress.hosts }}/-/healthy"
5959

0 commit comments

Comments
 (0)