Skip to content

Commit 935870d

Browse files
Update mtls doc , rearrange some parts
1 parent 08a221f commit 935870d

File tree

1 file changed

+86
-47
lines changed

1 file changed

+86
-47
lines changed

articles/azure-monitor/containers/prometheus-metrics-scrape-configuration.md

Lines changed: 86 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -458,57 +458,20 @@ metric_relabel_configs:
458458
459459
---
460460
461-
### TLS based scraping
462-
463-
If you have a Prometheus instance served with TLS and you want to scrape metrics from it, you need to set scheme to `https` and set the TLS settings in your configmap or respective CRD. You can use the `tls_config` configuration property inside a custom scrape job to configure the TLS settings either using a CRD or a configmap. You need to provide a CA certificate to validate API server certificate with. The CA certificate is used to verify the authenticity of the server's certificate when Prometheus connects to the target over TLS. It helps ensure that the server's certificate is signed by a trusted authority.
464-
465-
The secret should be created in kube-system namespace and then the configmap/CRD should be created in kube-system namespace. The order of secret creation matters. When there's no secret but a valid CRD/config map, you will find errors in collector log -> `no file found for cert....`
466-
467-
Below are the details about how to provide the TLS config settings through a configmap or CRD.
468-
469-
- To provide the TLS config setting in a configmap, please create the self-signed certificate and key inside your mtls enabled app.
470-
An example tlsConfig inside the config map should look like this:
471461
472-
```yaml
473-
tls_config:
474-
ca_file: /etc/prometheus/certs/client-cert.pem
475-
cert_file: /etc/prometheus/certs/client-cert.pem
476-
key_file: /etc/prometheus/certs/client-key.pem
477-
insecure_skip_verify: false
478-
```
462+
Prometheus supports TLS and basic authentication over its HTTP endpoints. Scraping target using HTTPS instead of HTTP is supported. You could fetch metrics using HTTPS, client-certificate authentication, and basic authentication.
479463
480-
- To provide the TLS config setting in a CRD, please create the self-signed certificate and key inside your mtls enabled app.
481-
An example tlsConfig inside a Podmonitor should look like this:
464+
Below are the details on 2 mechanisms of authentication.
465+
1. Basic authentication
466+
2. TLS based authentication
482467
483-
```yaml
484-
tlsConfig:
485-
ca:
486-
secret:
487-
key: "client-cert.pem" # since it is self-signed
488-
name: "ama-metrics-mtls-secret"
489-
cert:
490-
secret:
491-
key: "client-cert.pem"
492-
name: "ama-metrics-mtls-secret"
493-
keySecret:
494-
key: "client-key.pem"
495-
name: "ama-metrics-mtls-secret"
496-
insecureSkipVerify: false
497-
```
498-
> [!NOTE]
499-
> Make sure that the certificate file name and key name inside the mtls app is in the following format in case of a CRD based scraping.
500-
For example: secret_kube-system_ama-metrics-mtls-secret_cert-name.pem and secret_kube-system_ama-metrics-mtls-secret_key-name.pem.
501-
> The CRD needs to be created in kube-system namespace.
502-
> The secret name should exactly be ama-metrics-mtls-secret in kube-system namespace. An example command for creating secret: kubectl create secret generic ama-metrics-mtls-secret --from-file=secret_kube-system_ama-metrics-mtls-secret_client-cert.pem=secret_kube-system_ama-metrics-mtls-secret_client-cert.pem --from-file=secret_kube-system_ama-metrics-mtls-secret_client-key.pem=secret_kube-system_ama-metrics-mtls-secret_client-key.pem -n kube-system
468+
### Basic Authentication
503469
504-
To read more on TLS authentication, the following documents might be helpful.
470+
If you are using `basic_auth` setting in your prometheus configuration, please follow the steps -
505471

506-
- Generating TLS certificates -> https://o11y.eu/blog/prometheus-server-tls/
507-
- Configurations -> https://prometheus.io/docs/alerting/latest/configuration/#tls_config
472+
Below is an example of creating a secret.
508473

509-
### Basic Authentication
510-
If you are using `basic_auth` setting in your prometheus configuration, please follow the steps -
511-
1. Create a secret in the **kube-system** namespace named **ama-metrics-mtls-secret**
474+
1. Create a secret object in the **kube-system** namespace named **ama-metrics-mtls-secret**.
512475

513476

514477
The value for password1 is **base64encoded**
@@ -524,6 +487,12 @@ type: Opaque
524487
data:
525488
password1: <base64-encoded-string>
526489
```
490+
> [!NOTE]
491+
>
492+
> Make sure the name is **ama-metrics-mtls-secret** and it is in **kube-system** namespace.
493+
>
494+
> Inside the secret object , you can specify as many number of secret values under data section and name them how ever you want. Each secret name-value pair specified in the data section of the secret object will be mounted as a seperate file in this /etc/prometheus/certs location with filename(s) same as key(s) specified in the data section. The secret values should be base64 encoded before putting them under the data section.
495+
527496

528497
2. In the configmap for the custom scrape configuration use the following setting -
529498
```yaml
@@ -535,15 +504,85 @@ basic_auth:
535504

536505
> [!NOTE]
537506
>
538-
> Make sure the name is **ama-metrics-mtls-secret** and it is in **kube-system** namespace.
539-
>
540507
> The **/etc/prometheus/certs/** path is mandatory, but *password1* can be any string and needs to match the key for the data in the secret created above.
541508
This is because the secret **ama-metrics-mtls-secret** is mounted in the path **/etc/prometheus/certs/** within the container.
542509
>
543510
> The base64 encoded value is automatically decoded by the agent pods when the secret is mounted as file.
544511
>
545512
> Any other configuration setting for authorization that is considered as a secret in the [prometheus configuration](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) needs to use the file setting alternative instead as described above.
546513

514+
515+
### TLS based scraping
516+
517+
If you are using `tls_config` setting in your prometheus configuration, please follow the steps -
518+
519+
Below is an example of creating a secret.
520+
521+
1. Create a secret object in the **kube-system** namespace named **ama-metrics-mtls-secret**.
522+
523+
524+
The value for password1 is **base64encoded**
525+
The key *password1* can be anything, but just needs to match your scrapeconfig *password_file* filepath.
526+
527+
```yaml
528+
apiVersion: v1
529+
kind: Secret
530+
metadata:
531+
name: ama-metrics-mtls-secret
532+
namespace: kube-system
533+
type: Opaque
534+
data:
535+
password1: <base64-encoded-string>
536+
```
537+
> [!NOTE]
538+
>
539+
> Make sure the name is **ama-metrics-mtls-secret** and it is in **kube-system** namespace.
540+
>
541+
> Inside the secret object , you can specify as many number of secret values under data section and name them how ever you want. Each secret name-value pair specified in the data section of the secret object will be mounted as a seperate file in this /etc/prometheus/certs location with filename(s) same as key(s) specified in the data section. The secret values should be base64 encoded before putting them under the data section.
542+
543+
2. Below are the details about how to provide the TLS config settings through a configmap or CRD.
544+
545+
- To provide the TLS config setting in a configmap, please create the self-signed certificate and key inside your mtls enabled app.
546+
An example tlsConfig inside the config map should look like this:
547+
548+
```yaml
549+
tls_config:
550+
ca_file: /etc/prometheus/certs/client-cert.pem
551+
cert_file: /etc/prometheus/certs/client-cert.pem
552+
key_file: /etc/prometheus/certs/client-key.pem
553+
insecure_skip_verify: false
554+
```
555+
556+
- To provide the TLS config setting in a CRD, please create the self-signed certificate and key inside your mtls enabled app.
557+
An example tlsConfig inside a Podmonitor should look like this:
558+
559+
```yaml
560+
tlsConfig:
561+
ca:
562+
secret:
563+
key: "client-cert.pem" # since it is self-signed
564+
name: "ama-metrics-mtls-secret"
565+
cert:
566+
secret:
567+
key: "client-cert.pem"
568+
name: "ama-metrics-mtls-secret"
569+
keySecret:
570+
key: "client-key.pem"
571+
name: "ama-metrics-mtls-secret"
572+
insecureSkipVerify: false
573+
```
574+
> [!NOTE]
575+
>
576+
> Make sure that the certificate file name and key name inside the mtls app is in the following format in case of a CRD based scraping. For example: secret_kube-system_ama-metrics-mtls-secret_cert-name.pem and secret_kube-system_ama-metrics-mtls-secret_key-name.pem.
577+
> The CRD needs to be created in kube-system namespace.
578+
> The secret name should exactly be ama-metrics-mtls-secret in kube-system namespace. An example command for creating secret: kubectl create secret generic ama-metrics-mtls-secret --from-file=secret_kube-system_ama-metrics-mtls-secret_client-cert.pem=secret_kube-system_ama-metrics-mtls-secret_client-cert.pem --from-file=secret_kube-system_ama-metrics-mtls-secret_client-key.pem=secret_kube-system_ama-metrics-mtls-secret_client-key.pem -n kube-system
579+
> If you have a Prometheus instance served with TLS and you want to scrape metrics from it, you need to set scheme to `https` and set the TLS settings in your configmap or respective CRD. You can use the `tls_config` configuration property inside a custom scrape job to configure the TLS settings either using a CRD or a configmap. You need to provide a CA certificate to validate API server certificate with. The CA certificate is used to verify the authenticity of the server's certificate when Prometheus connects to the target over TLS. It helps ensure that the server's certificate is signed by a trusted authority.
580+
> The secret should be created in kube-system namespace and then the configmap/CRD should be created in kube-system namespace. The order of secret creation matters. When there's no secret but a valid CRD/config map, you will find errors in collector log -> `no file found for cert....`
581+
> To read more on TLS authentication, the following documents might be helpful.
582+
>
583+
> - Generating TLS certificates -> https://o11y.eu/blog/prometheus-server-tls/
584+
> - Configurations -> https://prometheus.io/docs/alerting/latest/configuration/#tls_config
585+
547586
## Next steps
548587

549588
[Setup Alerts on Prometheus metrics](./container-insights-metric-alerts.md)<br>

0 commit comments

Comments
 (0)