Skip to content

Commit 380caef

Browse files
v-dirichardsv-dirichards
authored
Merge pull request #252451 from seenu433/main
Added a new article on how to configure prometheus remote write to Azure Managed Prometheus from ARO
2 parents 665edff + 9c8d42c commit 380caef

File tree

2 files changed

+115
-0
lines changed

2 files changed

+115
-0
lines changed

‎articles/openshift/howto-remotewrite-prometheus.md

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
---
2+
title: Remote write to Azure Monitor Managed Service
3+
description: Describes how to configure remote write to send data from the default Prometheus server running in your ARO cluster
4+
author: srpadala
5+
ms.topic: conceptual
6+
ms.date: 09/21/2023
7+
---
8+
# Configure remote write to send data to Azure Monitor Workspace from the default Prometheus server in your Azure Red Hat OpenShift (ARO) cluster
9+
10+
Azure Red Hat OpenShift comes preinstalled with a default Prometheus server that shouldn't be removed as per the [support policy](support-policies-v4.md). There are scenarios where there's a need to centralize data from self-managed Prometheus clusters for long-term data retention and to create a centralized view across your clusters. Azure Monitor managed service for Prometheus allows you to collect and analyze metrics at scale using a Prometheus-compatible monitoring solution, based on the [Prometheus](https://aka.ms/azureprometheus-promio) project from the Cloud Native Computing Foundation. you can use [remote_write](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage) to send data from the in-cluster Prometheus servers into the Azure managed service.
11+
12+
## Prerequisites
13+
- Data for Azure Monitor managed service for Prometheus is stored in an [Azure Monitor workspace](../azure-monitor/essentials/azure-monitor-workspace-overview.md). You must [create a new workspace](../azure-monitor/essentials/azure-monitor-workspace-manage.md#create-an-azure-monitor-workspace) if you don't already have one.
14+
15+
## Create Microsoft Entra ID application
16+
Follow the procedure at [Register an application with Microsoft Entra ID and create a service principal](../active-directory/develop/howto-create-service-principal-portal.md#register-an-application-with-azure-ad-and-create-a-service-principal) to register an application for Prometheus remote-write and create a service principal.
17+
18+
Copy the tenant ID and client ID of the created service principal
19+
1. Browse to **Identity > Applications > App registrations**, then select your application.
20+
2. On the app's overview page, copy the Directory (tenant) ID value and store it in your application code.
21+
3. Copy the Application (client) ID value and store it in your application code.
22+
23+
Create a new client secret as described in [Create new client secret](../active-directory/develop/howto-create-service-principal-portal.md#option-3-create-a-new-client-secret) and copy the value of the created secret.
24+
25+
set the values of the collected tenant ID, client ID and client secret
26+
```
27+
export TENANT_ID=<tenant-id>
28+
export CLIENT_ID=<client-id>
29+
export CLIENT_SECRET=<client-secret>
30+
```
31+
32+
## Assign Monitoring Metrics Publisher role on the data collection rule to the application
33+
The application requires the *Monitoring Metrics Publisher* role on the data collection rule associated with your Azure Monitor workspace.
34+
35+
1. From the menu of your Azure Monitor Workspace account, select the **Data collection rule** to open the **Overview** page for the data collection rule.
36+
37+
2. Select **Access control (IAM)** in the **Overview** page for the data collection rule.
38+
39+
3. Select **Add** and then **Add role assignment**.
40+
41+
4. Select **Monitoring Metrics Publisher** role and select **Next**.
42+
43+
5. Select **User, group, or service principal** and then select **Select members**. Select the application that you created and select **Select**.
44+
45+
6. Select **Review + assign** to complete the role assignment.
46+
47+
## Create secret in the ARO cluster
48+
49+
We're using the OAuth 2.0 authentication method from the [supported remote write authentication settings](https://docs.openshift.com/container-platform/4.11/monitoring/configuring-the-monitoring-stack.html#supported_remote_write_authentication_settings_configuring-the-monitoring-stack)
50+
To facilitate this approach, create a secret with the client ID and client secret
51+
52+
```
53+
cat << EOF | oc apply -f -
54+
apiVersion: v1
55+
kind: Secret
56+
metadata:
57+
name: oauth2-credentials
58+
namespace: openshift-monitoring
59+
stringData:
60+
id: "${CLIENT_ID}"
61+
secret: "${CLIENT_SECRET}"
62+
EOF
63+
```
64+
65+
## Configure remote write
66+
67+
To [configure](https://docs.openshift.com/container-platform/4.11/monitoring/configuring-the-monitoring-stack.html#configuring_remote_write_storage_configuring-the-monitoring-stack) remote write for default platform monitoring, we need to update the cluster-monitoring-config config map in the openshift-monitoring namespace
68+
1. Replace the INGESTION-URL in the configuration with the **Metrics ingestion endpoint** from the **Overview** page for the Azure Monitor workspace
69+
2. Replace the TENANT_ID in the configuration with the tenant ID of the service principal
70+
71+
Edit the configmap
72+
73+
```
74+
oc edit -n openshift-monitoring cm cluster-monitoring-config
75+
```
76+
77+
and update the configuration
78+
79+
```
80+
data:
81+
config.yaml: |
82+
prometheusK8s:
83+
remoteWrite:
84+
- url: "<INGESTION-URL>"
85+
oauth2:
86+
clientId:
87+
secret:
88+
name: oauth2-credentials
89+
key: id
90+
clientSecret:
91+
name: oauth2-credentials
92+
key: secret
93+
tokenUrl: "https://login.microsoftonline.com/<TENANT_ID>/oauth2/v2.0/token"
94+
scopes:
95+
- "https://monitor.azure.com/.default"
96+
```
97+
98+
## Visualize metrics using Azure Managed Grafana Workspace
99+
The captured metrics can be visualized using community Grafana dashboards or create contextual dashboards as required.
100+
101+
1. Create an [Azure Managed Grafana workspace](../managed-grafana/quickstart-managed-grafana-portal.md)
102+
2. [Link](../azure-monitor/essentials/azure-monitor-workspace-manage.md?tabs=azure-portal#link-a-grafana-workspace) the created Grafana workspace to the Azure Monitor workspace
103+
3. [Import](../managed-grafana/how-to-create-dashboard.md?tabs=azure-portal#import-a-grafana-dashboard) the community Grafana Dashboard with ID 3870 [OpenShift/K8 Cluster Overview](https://grafana.com/grafana/dashboards/3870-openshift-k8-cluster-overview/) into the Grafana workspace
104+
4. Specify the Azure Monitor Workspace as the datasource
105+
5. Save the dashboard
106+
6. Access the dashboard from **Home -> Dashboards**
107+
108+
## Troubleshooting
109+
See [Azure Monitor managed service for Prometheus remote write](../azure-monitor/containers/prometheus-remote-write.md#hitting-your-ingestion-quota-limit).
110+
111+
## Next steps
112+
113+
- [Learn more about Azure Monitor managed service for Prometheus](../azure-monitor/essentials/prometheus-metrics-overview.md).

‎articles/openshift/toc.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,8 @@
101101
href : ../azure-monitor/insights/container-insights-enable-arc-enabled-clusters.md
102102
- name: Disable Azure Monitor container insights in Azure Red Hat OpenShift 4
103103
href: ../azure-monitor/insights/container-insights-enable-arc-enabled-clusters.md#delete-extension-instance
104+
- name: Configure Azure Monitor managed service for prometheus remote write
105+
href: howto-remotewrite-prometheus.md
104106
- name: Develop and run applications
105107
items:
106108
- name: Deploy an application from source code

0 commit comments

Comments
 (0)