You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/azure-monitor/containers/prometheus-metrics-scrape-configuration.md
+86-47Lines changed: 86 additions & 47 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -458,57 +458,20 @@ metric_relabel_configs:
458
458
459
459
---
460
460
461
-
### TLS based scraping
462
-
463
-
If you have a Prometheus instance served with TLS and you want to scrape metrics from it, you need to set scheme to `https` and set the TLS settings in your configmap or respective CRD. You can use the `tls_config` configuration property inside a custom scrape job to configure the TLS settings either using a CRD or a configmap. You need to provide a CA certificate to validate API server certificate with. The CA certificate is used to verify the authenticity of the server's certificate when Prometheus connects to the target over TLS. It helps ensure that the server's certificate is signed by a trusted authority.
464
-
465
-
The secret should be created in kube-system namespace and then the configmap/CRD should be created in kube-system namespace. The order of secret creation matters. When there's no secret but a valid CRD/config map, you will find errors in collector log -> `no file found for cert....`
466
-
467
-
Below are the details about how to provide the TLS config settings through a configmap or CRD.
468
-
469
-
- To provide the TLS config setting in a configmap, please create the self-signed certificate and key inside your mtls enabled app.
470
-
An example tlsConfig inside the config map should look like this:
471
461
472
-
```yaml
473
-
tls_config:
474
-
ca_file: /etc/prometheus/certs/client-cert.pem
475
-
cert_file: /etc/prometheus/certs/client-cert.pem
476
-
key_file: /etc/prometheus/certs/client-key.pem
477
-
insecure_skip_verify: false
478
-
```
462
+
Prometheus supports TLS and basic authentication over its HTTP endpoints. Scraping target using HTTPS instead of HTTP is supported. You could fetch metrics using HTTPS, client-certificate authentication, and basic authentication.
479
463
480
-
- To provide the TLS config setting in a CRD, please create the self-signed certificate and key inside your mtls enabled app.
481
-
An example tlsConfig inside a Podmonitor should look like this:
464
+
Below are the details on 2 mechanisms of authentication.
465
+
1. Basic authentication
466
+
2. TLS based authentication
482
467
483
-
```yaml
484
-
tlsConfig:
485
-
ca:
486
-
secret:
487
-
key: "client-cert.pem" # since it is self-signed
488
-
name: "ama-metrics-mtls-secret"
489
-
cert:
490
-
secret:
491
-
key: "client-cert.pem"
492
-
name: "ama-metrics-mtls-secret"
493
-
keySecret:
494
-
key: "client-key.pem"
495
-
name: "ama-metrics-mtls-secret"
496
-
insecureSkipVerify: false
497
-
```
498
-
> [!NOTE]
499
-
> Make sure that the certificate file name and key name inside the mtls app is in the following format in case of a CRD based scraping.
500
-
For example: secret_kube-system_ama-metrics-mtls-secret_cert-name.pem and secret_kube-system_ama-metrics-mtls-secret_key-name.pem.
501
-
> The CRD needs to be created in kube-system namespace.
502
-
> The secret name should exactly be ama-metrics-mtls-secret in kube-system namespace. An example command for creating secret: kubectl create secret generic ama-metrics-mtls-secret --from-file=secret_kube-system_ama-metrics-mtls-secret_client-cert.pem=secret_kube-system_ama-metrics-mtls-secret_client-cert.pem --from-file=secret_kube-system_ama-metrics-mtls-secret_client-key.pem=secret_kube-system_ama-metrics-mtls-secret_client-key.pem -n kube-system
468
+
### Basic Authentication
503
469
504
-
To read more on TLS authentication, the following documents might be helpful.
470
+
If you are using `basic_auth` setting in your prometheus configuration, please follow the steps -
If you are using `basic_auth` setting in your prometheus configuration, please follow the steps -
511
-
1. Create a secret in the **kube-system** namespace named **ama-metrics-mtls-secret**
474
+
1. Create a secret object in the **kube-system** namespace named **ama-metrics-mtls-secret**.
512
475
513
476
514
477
The value for password1 is **base64encoded**
@@ -524,6 +487,12 @@ type: Opaque
524
487
data:
525
488
password1: <base64-encoded-string>
526
489
```
490
+
> [!NOTE]
491
+
>
492
+
> Make sure the name is **ama-metrics-mtls-secret** and it is in **kube-system** namespace.
493
+
>
494
+
> Inside the secret object , you can specify as many number of secret values under data section and name them how ever you want. Each secret name-value pair specified in the data section of the secret object will be mounted as a seperate file in this /etc/prometheus/certs location with filename(s) same as key(s) specified in the data section. The secret values should be base64 encoded before putting them under the data section.
495
+
527
496
528
497
2. In the configmap for the custom scrape configuration use the following setting -
529
498
```yaml
@@ -535,15 +504,85 @@ basic_auth:
535
504
536
505
> [!NOTE]
537
506
>
538
-
> Make sure the name is **ama-metrics-mtls-secret** and it is in **kube-system** namespace.
539
-
>
540
507
> The **/etc/prometheus/certs/** path is mandatory, but *password1* can be any string and needs to match the key for the data in the secret created above.
541
508
This is because the secret **ama-metrics-mtls-secret** is mounted in the path **/etc/prometheus/certs/** within the container.
542
509
>
543
510
> The base64 encoded value is automatically decoded by the agent pods when the secret is mounted as file.
544
511
>
545
512
> Any other configuration setting for authorization that is considered as a secret in the [prometheus configuration](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) needs to use the file setting alternative instead as described above.
546
513
514
+
515
+
### TLS based scraping
516
+
517
+
If you are using `tls_config` setting in your prometheus configuration, please follow the steps -
518
+
519
+
Below is an example of creating a secret.
520
+
521
+
1. Create a secret object in the **kube-system** namespace named **ama-metrics-mtls-secret**.
522
+
523
+
524
+
The value for password1 is **base64encoded**
525
+
The key *password1* can be anything, but just needs to match your scrapeconfig *password_file* filepath.
526
+
527
+
```yaml
528
+
apiVersion: v1
529
+
kind: Secret
530
+
metadata:
531
+
name: ama-metrics-mtls-secret
532
+
namespace: kube-system
533
+
type: Opaque
534
+
data:
535
+
password1: <base64-encoded-string>
536
+
```
537
+
> [!NOTE]
538
+
>
539
+
> Make sure the name is **ama-metrics-mtls-secret** and it is in **kube-system** namespace.
540
+
>
541
+
> Inside the secret object , you can specify as many number of secret values under data section and name them how ever you want. Each secret name-value pair specified in the data section of the secret object will be mounted as a seperate file in this /etc/prometheus/certs location with filename(s) same as key(s) specified in the data section. The secret values should be base64 encoded before putting them under the data section.
542
+
543
+
2. Below are the details about how to provide the TLS config settings through a configmap or CRD.
544
+
545
+
- To provide the TLS config setting in a configmap, please create the self-signed certificate and key inside your mtls enabled app.
546
+
An example tlsConfig inside the config map should look like this:
547
+
548
+
```yaml
549
+
tls_config:
550
+
ca_file: /etc/prometheus/certs/client-cert.pem
551
+
cert_file: /etc/prometheus/certs/client-cert.pem
552
+
key_file: /etc/prometheus/certs/client-key.pem
553
+
insecure_skip_verify: false
554
+
```
555
+
556
+
- To provide the TLS config setting in a CRD, please create the self-signed certificate and key inside your mtls enabled app.
557
+
An example tlsConfig inside a Podmonitor should look like this:
558
+
559
+
```yaml
560
+
tlsConfig:
561
+
ca:
562
+
secret:
563
+
key: "client-cert.pem" # since it is self-signed
564
+
name: "ama-metrics-mtls-secret"
565
+
cert:
566
+
secret:
567
+
key: "client-cert.pem"
568
+
name: "ama-metrics-mtls-secret"
569
+
keySecret:
570
+
key: "client-key.pem"
571
+
name: "ama-metrics-mtls-secret"
572
+
insecureSkipVerify: false
573
+
```
574
+
> [!NOTE]
575
+
>
576
+
> Make sure that the certificate file name and key name inside the mtls app is in the following format in case of a CRD based scraping. For example: secret_kube-system_ama-metrics-mtls-secret_cert-name.pem and secret_kube-system_ama-metrics-mtls-secret_key-name.pem.
577
+
> The CRD needs to be created in kube-system namespace.
578
+
> The secret name should exactly be ama-metrics-mtls-secret in kube-system namespace. An example command for creating secret: kubectl create secret generic ama-metrics-mtls-secret --from-file=secret_kube-system_ama-metrics-mtls-secret_client-cert.pem=secret_kube-system_ama-metrics-mtls-secret_client-cert.pem --from-file=secret_kube-system_ama-metrics-mtls-secret_client-key.pem=secret_kube-system_ama-metrics-mtls-secret_client-key.pem -n kube-system
579
+
> If you have a Prometheus instance served with TLS and you want to scrape metrics from it, you need to set scheme to `https` and set the TLS settings in your configmap or respective CRD. You can use the `tls_config` configuration property inside a custom scrape job to configure the TLS settings either using a CRD or a configmap. You need to provide a CA certificate to validate API server certificate with. The CA certificate is used to verify the authenticity of the server's certificate when Prometheus connects to the target over TLS. It helps ensure that the server's certificate is signed by a trusted authority.
580
+
> The secret should be created in kube-system namespace and then the configmap/CRD should be created in kube-system namespace. The order of secret creation matters. When there's no secret but a valid CRD/config map, you will find errors in collector log -> `no file found for cert....`
581
+
> To read more on TLS authentication, the following documents might be helpful.
0 commit comments