Skip to content

Commit e156c6e

Browse files
author
Sreekanth Iyer (Ushta Te Consultancy Services)
committed
Added Approach 2 and 3
1 parent b6ce770 commit e156c6e

File tree

1 file changed

+145
-11
lines changed

1 file changed

+145
-11
lines changed

articles/hdinsight/azure-monitor-agent.md

Lines changed: 145 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -92,11 +92,146 @@ Activate the new integration by going to your cluster's portal page and scrollin
9292

9393
1. Select Save once precondition steps are complete.
9494

95-
### Approach 2: Enable Azure monitor agent using Azure CLI
95+
### Approach 2: Enable Azure monitor agent using Azure Powershell
9696

9797
1. Enable system-assigned MSI
9898

99-
1. First get cluster information to check the MSI of cluster
99+
1. First get cluster information to check the MSI of cluster.
100+
101+
102+
`Get-AzHDInsightCluster -ResourceGroupName $resourceGroup –ClusterName $cluster`
103+
104+
105+
106+
1. If this cluster has no MSI, directly enable system assigned MSI
107+
108+
`Update-AzHDInsightCluster -ResourceGroupName $resourceGroup -ClusterName $cluster -IdentityType "SystemAssigned"`
109+
110+
111+
1. If this cluster only has user assigned MSI, add system assigned MSI to identity.
112+
113+
114+
`Update-AzHDInsightCluster -ResourceGroupName $resourceGroup -ClusterName $cluster -IdentityType "SystemAssigned,UserAssigned" -IdentityId "$userAssignedIdentityResourceId"`
115+
116+
117+
118+
1. If this cluster already system assigned MSI, no need to anything.
119+
120+
121+
1. Creation of DCR
122+
123+
For more information, see [Create and edit data collection rules (DCRs)](/azure/azure-monitor/essentials/data-collection-rule-create-edit?tabs=powershell#create-a-dcr).
124+
125+
```
126+
# The URL of the DCR template file, change {HDIClusterType} to your cluster type.
127+
128+
# The valid types are: hadoop, hbase, interactivehive, kafka, llap, spark
129+
130+
$dcrTemplatejsonUrl = "https://hdiconfigactions.blob.core.windows.net/azuremonitoriningagent/DCR/{HDIClusterType}_dcr_template.json"
131+
132+
$dcrJsonContent = Invoke-RestMethod -Uri $dcrTemplatejsonUrl
133+
134+
135+
136+
# Get details of your Log Analytics workspace, if your workspace is in another subscription, you need to change context to the subscription
137+
138+
$workspaceResourceGroupName = "{yourWorkspaceResourceGroup}"
139+
140+
$workspaceName = {yourWorkspaceName}
141+
142+
$workspace = Get-AzOperationalInsightsWorkspace -ResourceGroupName $workspaceResourceGroupName -Name $workspaceName
143+
144+
145+
146+
# Customize the DCR content
147+
148+
$dcrJsonContent.properties.destinations.logAnalytics[0].workspaceResourceId = $workspace.ResourceId
149+
150+
$dcrJsonContent.properties.destinations.logAnalytics[0].workspaceId = $workspace.CustomerId
151+
152+
$dcrJsonContent.location = $workspace.Location
153+
154+
155+
156+
# Create the DCR using the customized JSON (DCR needs to be in the same location as Log Analytics workspace).
157+
158+
# If your HDInsight cluster is in another subscription, you need to change context to your cluster’s subscription
159+
160+
$dcrName = " {yourDcrName} "
161+
162+
$resourceGroupName = " {YourDcrResourceGroup} "
163+
164+
$dcrStr = $dcrJsonContent | ConvertTo-Json -Depth 10
165+
166+
$dcr = New-AzDataCollectionRule -Name $dcrName -ResourceGroupName $resourceGroupName -JsonString $dcrStr
167+
```
168+
169+
170+
1. Association of DCR.
171+
172+
For more information, see [Set up the Azure Monitor agent on Windows client devices](/azure/azure-monitor/agents/azure-monitor-agent-windows-client#create-and-associate-a-monitored-object).
173+
174+
175+
```
176+
# Associate DCR to HDInsight cluster
177+
178+
$hdinsightClusterResourceId = "/subscriptions/{subscription}/resourceGroups/{resourceGroup}/providers/Microsoft.HDInsight/clusters/{clusterName}"
179+
180+
$dcrAssociationName = "dcrAssociationName {yourDcrAssociation} "
181+
182+
New-AzDataCollectionRuleAssociation -AssociationName $dcrAssociationName -ResourceUri $hdinsightClusterResourceId -DataCollectionRuleId $dcr.Id
183+
```
184+
185+
1. Enabling Azure Monitor Agent.
186+
187+
```
188+
# Enter user information
189+
190+
$resourceGroup = "<your-resource-group>"
191+
192+
$cluster = "<your-cluster>"
193+
194+
$LAW = "<your-Log-Analytics-workspace>"
195+
196+
# End of user input
197+
198+
199+
# obtain workspace id for defined Log Analytics workspace
200+
201+
$WorkspaceId = (Get-AzOperationalInsightsWorkspace -ResourceGroupName $resourceGroup -Name $LAW).CustomerId
202+
203+
204+
205+
# obtain primary key for defined Log Analytics workspace
206+
207+
$PrimaryKey = (Get-AzOperationalInsightsWorkspace -ResourceGroupName $resourceGroup -Name $LAW | Get-AzOperationalInsightsWorkspaceSharedKeys).PrimarySharedKey
208+
209+
210+
211+
# Enables monitoring and relevant logs will be sent to the specified workspace.
212+
213+
Enable-AzHDInsightAzureMonitorAgent -ResourceGroupName $resourceGroup -ClusterName $cluster -WorkspaceId $WorkspaceId -PrimaryKey $PrimaryKey
214+
215+
216+
217+
# Gets the status of monitoring installation on the cluster.
218+
219+
Get-AzHDInsightAzureMonitorAgent -ResourceGroupName $resourceGroup -ClusterName $cluster
220+
```
221+
222+
223+
1. (Optional) disabling Azure Monitor Agent.
224+
225+
```
226+
Disable-AzHDInsightAzureMonitorAgent -ResourceGroupName $resourceGroup -ClusterName $cluster
227+
```
228+
229+
230+
### Approach 3: Enable Azure monitor agent using Azure CLI
231+
232+
1. Enable system-assigned MSI.
233+
234+
1. First get cluster information to check the MSI of cluster.
100235
101236
102237
```
@@ -109,7 +244,7 @@ Activate the new integration by going to your cluster's portal page and scrollin
109244
url="https://management.azure.com/subscriptions/${subscriptionId}/resourcegroups/${resourceGroupName}/providers/Microsoft.HDInsight/clusters/${clusterName}?api-version=2024-08-01-preview"
110245
```
111246
112-
1. If this cluster has no MSI, directly enable system assigned MSI via rest API
247+
1. If this cluster has no MSI, directly enable system assigned MSI via rest API.
113248
114249
```
115250
body="{\"identity\": {\"type\": \"SystemAssigned\"}}"
@@ -124,12 +259,12 @@ Activate the new integration by going to your cluster's portal page and scrollin
124259
```
125260
126261
127-
1. If this cluster already system assigned MSI, no need to anything .
262+
1. If this cluster already system assigned MSI, no need to anything.
128263
129264
130-
1. Creation of DCR
265+
1. Creation of DCR.
131266
132-
For more insormation, see [Create and edit data collection rules (DCRs)](/azure/azure-monitor/essentials/data-collection-rule-create-edit?tabs=CLI#create-a-dcr)
267+
For more information, see [Create and edit data collection rules (DCRs)](/azure/azure-monitor/essentials/data-collection-rule-create-edit?tabs=CLI#create-a-dcr)
133268
134269
```
135270
# The URL of the DCR template file, change {HDIClusterType} to your cluster type.
@@ -254,12 +389,11 @@ Activate the new integration by going to your cluster's portal page and scrollin
254389
```
255390
256391
257-
1. (Optional) disabling Azure Monitor Agent
392+
1. (Optional) disabling Azure Monitor Agent.
258393
259394
```
260395
az hdinsight azure-monitor-agent disable --name $cluster --resource-group $resourceGroup
261396
```
262-
263397
264398
### Enable Azure Monitor Agent logging for Spark cluster
265399
@@ -281,7 +415,7 @@ The following steps describe how customers can enable the new Azure Monitor Agen
281415
282416
There are two ways you can access the new tables.
283417
284-
#### Approach 1:
418+
#### Approach 1
285419
286420
1. The first way to access the new tables is through the Log Analytics workspace.
287421
@@ -296,7 +430,7 @@ There are two ways you can access the new tables.
296430
> [!NOTE]
297431
>This process describes how the logs were accessed in the old integration. This requires the user to have access to the workspace.
298432
299-
#### Approach 2:
433+
#### Approach 2
300434
301435
The second way to access the new tables is through Cluster portal access.
302436
@@ -356,7 +490,7 @@ We provide a [mapping table](./log-analytics-migration.md#appendix-table-mappi
356490
357491
### Update dashboards for HDInsight clusters
358492
359-
If you build multiple dashboards to monitor your HDInsight clusters, you need to adjust the query behind the table once you enable the new Azure Monitor integration. The table name or the field name might change in the new integration, but all the information you have in old integration is included.
493+
If you build multiple dashboards to monitor your HDInsight clusters, you need to adjust the query behind the table once you enable the new Azure Monitor integration. The table name or the field name might change in the new integration, but all the information you have in old integration is included.
360494
361495
Refer to the [mapping table](log-analytics-migration.md#appendix-table-mapping) between the old table/schema to the new table/schema to update the query behind the dashboards
362496

0 commit comments

Comments
 (0)