|
1 | 1 | ---
|
2 | 2 | title: Remote write to Azure Monitor Managed Service
|
3 |
| -description: Describes how to configure remote write to send data from the default Prometheus server running in your ARO cluster |
4 |
| -author: srpadala |
| 3 | +description: Describes how to configure remote write to send data from the default Prometheus server running in your ARO cluster. |
| 4 | +author: johnmarco |
| 5 | +ms.author: johnmarc |
| 6 | +ms.service: azure-redhat-openshift |
5 | 7 | ms.topic: conceptual
|
6 |
| -ms.date: 09/21/2023 |
| 8 | +ms.date: 01/02/2023 |
7 | 9 | ---
|
8 |
| -# Configure remote write to send data to Azure Monitor Workspace from the default Prometheus server in your Azure Red Hat OpenShift (ARO) cluster |
| 10 | +# Send data to Azure Monitor workspace from the Prometheus server in your Azure Red Hat OpenShift (ARO) cluster |
9 | 11 |
|
10 |
| -Azure Red Hat OpenShift comes preinstalled with a default Prometheus server that shouldn't be removed as per the [support policy](support-policies-v4.md). There are scenarios where there's a need to centralize data from self-managed Prometheus clusters for long-term data retention and to create a centralized view across your clusters. Azure Monitor managed service for Prometheus allows you to collect and analyze metrics at scale using a Prometheus-compatible monitoring solution, based on the [Prometheus](https://aka.ms/azureprometheus-promio) project from the Cloud Native Computing Foundation. you can use [remote_write](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage) to send data from the in-cluster Prometheus servers into the Azure managed service. |
| 12 | +Azure Red Hat OpenShift comes preinstalled with a default Prometheus server. As per the [support policy](support-policies-v4.md), this default Prometheus server shouldn't be removed. Some scenarios need to centralize data from self-managed Prometheus clusters for long-term data retention to create a centralized view across your clusters. Azure Monitor managed service for Prometheus allows you to collect and analyze metrics at scale using a Prometheus-compatible monitoring solution based on the [Prometheus](https://aka.ms/azureprometheus-promio) project from the Cloud Native Computing Foundation. You can use [remote_write](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage) to send data from the in-cluster Prometheus servers to the Azure managed service. |
11 | 13 |
|
12 | 14 | ## Prerequisites
|
13 | 15 | - Data for Azure Monitor managed service for Prometheus is stored in an [Azure Monitor workspace](../azure-monitor/essentials/azure-monitor-workspace-overview.md). You must [create a new workspace](../azure-monitor/essentials/azure-monitor-workspace-manage.md#create-an-azure-monitor-workspace) if you don't already have one.
|
14 | 16 |
|
15 | 17 | ## Create Microsoft Entra ID application
|
16 | 18 | Follow the procedure at [Register an application with Microsoft Entra ID and create a service principal](../active-directory/develop/howto-create-service-principal-portal.md#register-an-application-with-azure-ad-and-create-a-service-principal) to register an application for Prometheus remote-write and create a service principal.
|
17 | 19 |
|
18 |
| -Copy the tenant ID and client ID of the created service principal |
19 |
| -1. Browse to **Identity > Applications > App registrations**, then select your application. |
20 |
| -2. On the app's overview page, copy the Directory (tenant) ID value and store it in your application code. |
21 |
| -3. Copy the Application (client) ID value and store it in your application code. |
| 20 | +1. Copy the tenant ID and client ID of the created service principal. |
| 21 | + 1. Browse to **Identity > Applications > App registrations**, then select your application. |
| 22 | + 1. On the overview page for the app, copy the Directory (tenant) ID value and store it in your application code. |
| 23 | + 1. Copy the Application (client) ID value and store it in your application code. |
| 24 | + |
| 25 | +1. Create a new client secret as directed in [Create new client secret](../active-directory/develop/howto-create-service-principal-portal.md#option-3-create-a-new-client-secret).Copy the value of the created secret. |
22 | 26 |
|
23 |
| -Create a new client secret as described in [Create new client secret](../active-directory/develop/howto-create-service-principal-portal.md#option-3-create-a-new-client-secret) and copy the value of the created secret. |
| 27 | +1. Set the values of the collected tenant ID, client ID and client secret: |
24 | 28 |
|
25 |
| -set the values of the collected tenant ID, client ID and client secret |
26 |
| -``` |
27 |
| -export TENANT_ID=<tenant-id> |
28 |
| -export CLIENT_ID=<client-id> |
29 |
| -export CLIENT_SECRET=<client-secret> |
30 |
| -``` |
| 29 | + ``` |
| 30 | + export TENANT_ID=<tenant-id> |
| 31 | + export CLIENT_ID=<client-id> |
| 32 | + export CLIENT_SECRET=<client-secret> |
| 33 | + ``` |
| 34 | + |
| 35 | +## Assign Monitoring Metrics Publisher role to the application |
31 | 36 |
|
32 |
| -## Assign Monitoring Metrics Publisher role on the data collection rule to the application |
33 | 37 | The application requires the *Monitoring Metrics Publisher* role on the data collection rule associated with your Azure Monitor workspace.
|
34 | 38 |
|
35 | 39 | 1. From the menu of your Azure Monitor Workspace account, select the **Data collection rule** to open the **Overview** page for the data collection rule.
|
36 | 40 |
|
37 |
| -2. Select **Access control (IAM)** in the **Overview** page for the data collection rule. |
| 41 | +2. On the **Overview** page, select **Access control (IAM)**. |
38 | 42 |
|
39 |
| -3. Select **Add** and then **Add role assignment**. |
| 43 | +3. Select **Add**, and then select **Add role assignment**. |
40 | 44 |
|
41 |
| -4. Select **Monitoring Metrics Publisher** role and select **Next**. |
| 45 | +4. Select **Monitoring Metrics Publisher** role, and then select **Next**. |
42 | 46 |
|
43 |
| -5. Select **User, group, or service principal** and then select **Select members**. Select the application that you created and select **Select**. |
| 47 | +5. Select **User, group, or service principal**, and then select **Select members**. Select the application that you created and select **Select**. |
44 | 48 |
|
45 | 49 | 6. Select **Review + assign** to complete the role assignment.
|
46 | 50 |
|
47 | 51 | ## Create secret in the ARO cluster
|
48 | 52 |
|
49 |
| -We're using the OAuth 2.0 authentication method from the [supported remote write authentication settings](https://docs.openshift.com/container-platform/4.11/monitoring/configuring-the-monitoring-stack.html#supported_remote_write_authentication_settings_configuring-the-monitoring-stack) |
50 |
| -To facilitate this approach, create a secret with the client ID and client secret |
| 53 | +To authenticate with a remote write endpoint, the OAuth 2.0 authentication method from the [supported remote write authentication settings](https://docs.openshift.com/container-platform/4.11/monitoring/configuring-the-monitoring-stack.html#supported_remote_write_authentication_settings_configuring-the-monitoring-stack) is used. To facilitate this approach, create a secret with the client ID and client secret: |
51 | 54 |
|
52 | 55 | ```
|
53 | 56 | cat << EOF | oc apply -f -
|
|
64 | 67 |
|
65 | 68 | ## Configure remote write
|
66 | 69 |
|
67 |
| -To [configure](https://docs.openshift.com/container-platform/4.11/monitoring/configuring-the-monitoring-stack.html#configuring_remote_write_storage_configuring-the-monitoring-stack) remote write for default platform monitoring, we need to update the cluster-monitoring-config config map in the openshift-monitoring namespace |
68 |
| -1. Replace the INGESTION-URL in the configuration with the **Metrics ingestion endpoint** from the **Overview** page for the Azure Monitor workspace |
69 |
| -2. Replace the TENANT_ID in the configuration with the tenant ID of the service principal |
| 70 | +To [configure](https://docs.openshift.com/container-platform/4.11/monitoring/configuring-the-monitoring-stack.html#configuring_remote_write_storage_configuring-the-monitoring-stack) remote write for default platform monitoring, update the *cluster-monitoring-config* config map in the openshift-monitoring namespace. |
| 71 | +
|
| 72 | +1. Open the config map for editing: |
| 73 | +
|
| 74 | + ``` |
| 75 | + oc edit -n openshift-monitoring cm cluster-monitoring-config |
| 76 | + ``` |
| 77 | + |
| 78 | + ``` |
| 79 | + data: |
| 80 | + config.yaml: | |
| 81 | + prometheusK8s: |
| 82 | + remoteWrite: |
| 83 | + - url: "<INGESTION-URL>" |
| 84 | + oauth2: |
| 85 | + clientId: |
| 86 | + secret: |
| 87 | + name: oauth2-credentials |
| 88 | + key: id |
| 89 | + clientSecret: |
| 90 | + name: oauth2-credentials |
| 91 | + key: secret |
| 92 | + tokenUrl: "https://login.microsoftonline.com/<TENANT_ID>/oauth2/v2.0/token" |
| 93 | + scopes: |
| 94 | + - "https://monitor.azure.com/.default" |
| 95 | + ``` |
| 96 | + |
| 97 | +1. Update the configuration. |
| 98 | +
|
| 99 | + 1. Replace `INGESTION-URL` in the configuration with the **Metrics ingestion endpoint** from the **Overview** page for the Azure Monitor workspace. |
| 100 | + |
| 101 | + 1. Replace `TENANT_ID` in the configuration with the tenant ID of the service principal. |
70 | 102 |
|
71 |
| -Edit the configmap |
72 | 103 |
|
73 |
| -``` |
74 |
| -oc edit -n openshift-monitoring cm cluster-monitoring-config |
75 |
| -``` |
| 104 | +## Visualize metrics using Azure Managed Grafana Workspace |
76 | 105 |
|
77 |
| -and update the configuration |
| 106 | +The captured metrics can be visualized using community Grafana dashboards, or you can create contextual dashboards as required. |
78 | 107 |
|
79 |
| -``` |
80 |
| -data: |
81 |
| - config.yaml: | |
82 |
| - prometheusK8s: |
83 |
| - remoteWrite: |
84 |
| - - url: "<INGESTION-URL>" |
85 |
| - oauth2: |
86 |
| - clientId: |
87 |
| - secret: |
88 |
| - name: oauth2-credentials |
89 |
| - key: id |
90 |
| - clientSecret: |
91 |
| - name: oauth2-credentials |
92 |
| - key: secret |
93 |
| - tokenUrl: "https://login.microsoftonline.com/<TENANT_ID>/oauth2/v2.0/token" |
94 |
| - scopes: |
95 |
| - - "https://monitor.azure.com/.default" |
96 |
| -``` |
| 108 | +1. Create an [Azure Managed Grafana workspace](../managed-grafana/quickstart-managed-grafana-portal.md). |
97 | 109 |
|
98 |
| -## Visualize metrics using Azure Managed Grafana Workspace |
99 |
| -The captured metrics can be visualized using community Grafana dashboards or create contextual dashboards as required. |
| 110 | +1. [Link](../azure-monitor/essentials/azure-monitor-workspace-manage.md?tabs=azure-portal#link-a-grafana-workspace) the created Grafana workspace to the Azure Monitor workspace. |
| 111 | +
|
| 112 | +1. [Import](../managed-grafana/how-to-create-dashboard.md?tabs=azure-portal#import-a-grafana-dashboard) the community Grafana Dashboard with ID 3870 [OpenShift/K8 Cluster Overview](https://grafana.com/grafana/dashboards/3870-openshift-k8-cluster-overview/) into the Grafana workspace. |
100 | 113 |
|
101 |
| -1. Create an [Azure Managed Grafana workspace](../managed-grafana/quickstart-managed-grafana-portal.md) |
102 |
| -2. [Link](../azure-monitor/essentials/azure-monitor-workspace-manage.md?tabs=azure-portal#link-a-grafana-workspace) the created Grafana workspace to the Azure Monitor workspace |
103 |
| -3. [Import](../managed-grafana/how-to-create-dashboard.md?tabs=azure-portal#import-a-grafana-dashboard) the community Grafana Dashboard with ID 3870 [OpenShift/K8 Cluster Overview](https://grafana.com/grafana/dashboards/3870-openshift-k8-cluster-overview/) into the Grafana workspace |
104 |
| -4. Specify the Azure Monitor Workspace as the datasource |
105 |
| -5. Save the dashboard |
106 |
| -6. Access the dashboard from **Home -> Dashboards** |
| 114 | +1. Specify the Azure Monitor workspace as the data source. |
| 115 | +
|
| 116 | +1. Save the dashboard. |
| 117 | +
|
| 118 | +1. Access the dashboard from **Home -> Dashboards**. |
107 | 119 |
|
108 | 120 | ## Troubleshooting
|
| 121 | +
|
109 | 122 | See [Azure Monitor managed service for Prometheus remote write](../azure-monitor/containers/prometheus-remote-write.md#hitting-your-ingestion-quota-limit).
|
110 | 123 |
|
111 | 124 | ## Next steps
|
|
0 commit comments