Skip to content

Commit 4c2fc3f

Browse files
rashmichandrashekarrashmichandrashekar
committed
fixing docs
1 parent c7543e8 commit 4c2fc3f

File tree

1 file changed

+168
-4
lines changed

1 file changed

+168
-4
lines changed

‎articles/azure-monitor/containers/prometheus-metrics-multiple-workspaces.md

Lines changed: 168 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,182 @@ ms.reviewer: aul
88

99
# Send Prometheus metrics to multiple Azure Monitor workspaces
1010

11-
Routing metrics to more Azure Monitor workspaces can be done through the creation of additional data collection rules.
11+
Routing different metrics to more Azure Monitor workspaces can be done through the creation of additional data collection rules.
1212

13-
## Send different metrics to different Azure Monitor workspaces
13+
You can create Data Collection Rules with corresponding Data Collection Endpoints for different metrics to be sent to additional Azure Monitor workspaces from the same Kubernetes cluster.
14+
Currently, this is only available through onboarding through Resource Manager templates. You can follow the [regular onboarding process](kubernetes-monitoring-enable.md#enable-prometheus-and-grafana) and then edit the same Resource Manager templates to add additional DCRs and DCEs for your additional Azure Monitor workspaces. You'll need to edit the template to add an additional parameters for every additional Azure Monitor workspace, add another DCR for every additional Azure Monitor workspace, add another DCE , add the Monitor Reader Role for the new Azure Monitor workspace and add an additional Azure Monitor workspace integration for Grafana.
1415

15-
If you want to send some metrics to one Azure Monitor workspace and other metrics to a different one, follow the above steps to add additional DCRs. The value of `microsoft_metrics_include_label` under the `labelIncludeFilter` in the DCR is the identifier for the workspace. To then configure which metrics are routed to which workspace, you can add an extra pre-defined label, `microsoft_metrics_account` to the metrics. The value should be the same as the corresponding `microsoft_metrics_include_label` in the DCR for that workspace. To add the label to the metrics, you can utilize `relabel_configs` in your scrape config. To send all metrics from one job to a certain workspace, add the following relabel config:
16+
- Add the following parameters:
17+
```json
18+
"parameters": {
19+
"azureMonitorWorkspaceResourceId2": {
20+
"type": "string"
21+
},
22+
"azureMonitorWorkspaceLocation2": {
23+
"type": "string",
24+
"defaultValue": "",
25+
"allowedValues": [
26+
"eastus2euap",
27+
"centraluseuap",
28+
"centralus",
29+
"eastus",
30+
"eastus2",
31+
"northeurope",
32+
"southcentralus",
33+
"southeastasia",
34+
"uksouth",
35+
"westeurope",
36+
"westus",
37+
"westus2"
38+
]
39+
},
40+
...
41+
}
42+
```
43+
44+
- Add an additional Data Collection Endpoint. You *must* replace `<dceName>`:
45+
```json
46+
{
47+
"type": "Microsoft.Insights/dataCollectionEndpoints",
48+
"apiVersion": "2021-09-01-preview",
49+
"name": "[variables('dceName')]",
50+
"location": "[parameters('azureMonitorWorkspaceLocation2')]",
51+
"kind": "Linux",
52+
"properties": {}
53+
}
54+
```
55+
- Add an additional DCR with the new Data Collection Endpoint. You *must* replace `<dcrName>`:
56+
```json
57+
{
58+
"type": "Microsoft.Insights/dataCollectionRules",
59+
"apiVersion": "2021-09-01-preview",
60+
"name": "<dcrName>",
61+
"location": "[parameters('azureMonitorWorkspaceLocation2')]",
62+
"kind": "Linux",
63+
"properties": {
64+
"dataCollectionEndpointId": "[resourceId('Microsoft.Insights/dataCollectionEndpoints/', variables('dceName'))]",
65+
"dataFlows": [
66+
{
67+
"destinations": ["MonitoringAccount2"],
68+
"streams": ["Microsoft-PrometheusMetrics"]
69+
}
70+
],
71+
"dataSources": {
72+
"prometheusForwarder": [
73+
{
74+
"name": "PrometheusDataSource",
75+
"streams": ["Microsoft-PrometheusMetrics"],
76+
"labelIncludeFilter":
77+
"microsoft_metrics_include_label": "MonitoringAccountLabel2"
78+
}
79+
]
80+
},
81+
"description": "DCR for Azure Monitor Metrics Profile (Managed Prometheus)",
82+
"destinations": {
83+
"monitoringAccounts": [
84+
{
85+
"accountResourceId": "[parameters('azureMonitorWorkspaceResourceId2')]",
86+
"name": "MonitoringAccount2"
87+
}
88+
]
89+
}
90+
},
91+
"dependsOn": [
92+
"[resourceId('Microsoft.Insights/dataCollectionEndpoints/', variables('dceName'))]"
93+
]
94+
}
95+
```
96+
97+
- Add an additional Data Collection Rule Association (DCRA) with the relevant Data Collection Rule (DCR). This associates the DCR with the cluster. You must replace `<dcraName>`:
98+
```json
99+
{
100+
"type": "Microsoft.Resources/deployments",
101+
"name": "<dcraName>",
102+
"apiVersion": "2017-05-10",
103+
"subscriptionId": "[variables('clusterSubscriptionId')]",
104+
"resourceGroup": "[variables('clusterResourceGroup')]",
105+
"dependsOn": [
106+
"[resourceId('Microsoft.Insights/dataCollectionEndpoints/', variables('dceName'))]",
107+
"[resourceId('Microsoft.Insights/dataCollectionRules', variables('dcrName'))]"
108+
],
109+
"properties": {
110+
"mode": "Incremental",
111+
"template": {
112+
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
113+
"contentVersion": "1.0.0.0",
114+
"parameters": {},
115+
"variables": {},
116+
"resources": [
117+
{
118+
"type": "Microsoft.ContainerService/managedClusters/providers/dataCollectionRuleAssociations",
119+
"name": "[concat(variables('clusterName'),'/microsoft.insights/', variables('dcraName'))]",
120+
"apiVersion": "2021-09-01-preview",
121+
"location": "[parameters('clusterLocation')]",
122+
"properties": {
123+
"description": "Association of data collection rule. Deleting this association will break the data collection for this AKS Cluster.",
124+
"dataCollectionRuleId": "[resourceId('Microsoft.Insights/dataCollectionRules', variables('dcrName'))]"
125+
}
126+
}
127+
]
128+
},
129+
"parameters": {}
130+
}
131+
}
132+
```
133+
- Add an additional Grafana integration:
134+
```json
135+
{
136+
"type": "Microsoft.Dashboard/grafana",
137+
"apiVersion": "2022-08-01",
138+
"name": "[split(parameters('grafanaResourceId'),'/')[8]]",
139+
"sku": {
140+
"name": "[parameters('grafanaSku')]"
141+
},
142+
"location": "[parameters('grafanaLocation')]",
143+
"properties": {
144+
"grafanaIntegrations": {
145+
"azureMonitorWorkspaceIntegrations": [
146+
// Existing azureMonitorWorkspaceIntegrations values (if any)
147+
// {
148+
// "azureMonitorWorkspaceResourceId": "<value>"
149+
// },
150+
// {
151+
// "azureMonitorWorkspaceResourceId": "<value>"
152+
// },
153+
{
154+
"azureMonitorWorkspaceResourceId": "[parameters('azureMonitorWorkspaceResourceId')]"
155+
},
156+
{
157+
"azureMonitorWorkspaceResourceId": "[parameters('azureMonitorWorkspaceResourceId2')]"
158+
}
159+
]
160+
}
161+
}
162+
}
163+
```
164+
- Assign `Monitoring Data Reader` role to read data from the new Azure Monitor Workspace:
165+
166+
```json
167+
{
168+
"type": "Microsoft.Authorization/roleAssignments",
169+
"apiVersion": "2022-04-01",
170+
"name": "[parameters('roleNameGuid')]",
171+
"scope": "[parameters('azureMonitorWorkspaceResourceId2')]",
172+
"properties": {
173+
"roleDefinitionId": "[concat('/subscriptions/', variables('clusterSubscriptionId'), '/providers/Microsoft.Authorization/roleDefinitions/', 'b0d8363b-8ddd-447d-831f-62ca05bff136')]",
174+
"principalId": "[reference(resourceId('Microsoft.Dashboard/grafana', split(parameters('grafanaResourceId'),'/')[8]), '2022-08-01', 'Full').identity.principalId]"
175+
}
176+
}
177+
178+
179+
Then configure which metrics are routed to which workspace, by adding an extra pre-defined label, `microsoft_metrics_account` to the metrics. The value should be the same as the corresponding `microsoft_metrics_include_label` in the DCR for that workspace. To add the label to the metrics, you can utilize `relabel_configs` in your scrape config. To send all metrics from one job to a certain workspace, add the following relabel config:
16180

17181
```yaml
18182
relabel_configs:
19183
- source_labels: [__address__]
20184
target_label: microsoft_metrics_account
21185
action: replace
22-
replacement: "MonitoringAccountLabel1"
186+
replacement: "MonitoringAccountLabel2"
23187
```
24188

25189
The source label is `__address__` because this label will always exist so this relabel config will always be applied. The target label will always be `microsoft_metrics_account` and its value should be replaced with the corresponding label value for the workspace.

0 commit comments

Comments
 (0)