You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-oms-log-analytics-use-queries.md
+53-48Lines changed: 53 additions & 48 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,9 +5,9 @@ author: hrasheed-msft
5
5
ms.author: hrasheed
6
6
ms.reviewer: jasonh
7
7
ms.service: hdinsight
8
-
ms.custom: hdinsightactive
9
8
ms.topic: conceptual
10
-
ms.date: 11/05/2018
9
+
ms.custom: hdinsightactive
10
+
ms.date: 12/02/2019
11
11
---
12
12
13
13
# Query Azure Monitor logs to monitor HDInsight clusters
@@ -29,96 +29,101 @@ You must have configured an HDInsight cluster to use Azure Monitor logs, and add
29
29
Learn how to look for specific metrics for your HDInsight cluster.
30
30
31
31
1. Open the Log Analytics workspace that is associated to your HDInsight cluster from the Azure portal.
32
-
1.Select the **Log Search** tile.
33
-
1. Type the following query in the search box to search for all metrics for all available metrics for all HDInsight clusters configured to use Azure Monitor logs, and then select **RUN**.
32
+
1.Under **General**, select **Logs**.
33
+
1. Type the following query in the search box to search for all metrics for all available metrics for all HDInsight clusters configured to use Azure Monitor logs, and then select **Run**. Review the results.
34
34
35
-
search *
35
+
```kusto
36
+
search *
37
+
```
36
38
37
39
38
40
39
-
The output shall look like:
40
-
41
-
![log analytics search all metrics](./media/hdinsight-hadoop-oms-log-analytics-use-queries/hdinsight-log-analytics-search-all-metrics-output.png"Search all metrics output")
42
-
43
-
1. From the left pane, under **Type**, select a metric that you want to dig deep into, and then select **Apply**. The following screenshot shows the `metrics_resourcemanager_queue_root_default_CL` type is selected.
41
+
1. From the left menu, select the **Filter** tab.
44
42
45
-
> [!NOTE]
46
-
> You may need to select the **[+]More** button to find the metric you are looking for. Also, the **Apply** button is at the bottom of the list so you must scroll down to see it.
47
-
48
-
Notice that the query in the text box changes to one shown in the highlighted box in the following screenshot:
43
+
1. Under **Type**, select **Heartbeat**. Then select **Apply & Run**.
49
44
50
45
51
46
52
-
1. To dig deeper into this specific metric. For example, you can refine the existing output based on the average of resources used in a 10-minute interval, categorized by cluster name using the following query:
53
-
54
-
search in (metrics_resourcemanager_queue_root_default_CL) * | summarize AggregatedValue = avg(UsedAMResourceMB_d) by ClusterName_s, bin(TimeGenerated, 10m)
55
-
56
-
1. Instead of refining based on the average of resources used, you can use the following query to refine the results based on when the maximum resources were used (as well as 90th and 95th percentile) in a 10-minute window:
47
+
1. Notice that the query in the text box changes to:
1. You can dig deeper by using the options available in the left menu. For example:
61
55
62
-
Learn how to look error messages during a specific time window. The steps here are just one example on how you can arrive at the error message you are interested in. You can use any property that is available to look for the errors you are trying to find.
56
+
- To see logs from a specific node:
63
57
64
-
1. Open the Log Analytics workspace that is associated to your HDInsight cluster from the Azure portal.
65
-
2. Select the **Log Search** tile.
66
-
3. Type the following query to search for all error messages for all HDInsight clusters configured to use Azure Monitor logs, and then select **RUN**.
67
-
68
-
search "Error"
58
+
69
59
70
-
You shall see an output like the following output:
60
+
- To see logs at certain times:
71
61
72
-
![Azure portal log search errors](./media/hdinsight-hadoop-oms-log-analytics-use-queries/hdinsight-log-analytics-search-all-errors-output.png"Search all errors output")
62
+
73
63
74
-
4. From the left pane, under **Type** category, select an error type that you want to dig deep into, and then select **Apply**. Notice the results are refined to only show the error of the type you selected.
64
+
1. Select **Apply & Run** and review the results. Also note that the query was updated to:
75
65
76
-
5. You can dig deeper into this specific error list by using the options available in the left pane. For example:
66
+
```kusto
67
+
search *
68
+
| where Type == "Heartbeat"
69
+
| where (Computer == "zk2-myhado") and (TimeGenerated == "2019-12-02T23:15:02.69Z" or TimeGenerated == "2019-12-02T23:15:08.07Z" or TimeGenerated == "2019-12-02T21:09:34.787Z")
70
+
```
77
71
78
-
- To see error messages from a specific worker node:
72
+
### Additional sample queries
79
73
80
-
![Search for specific errors output1](./media/hdinsight-hadoop-oms-log-analytics-use-queries/hdinsight-log-analytics-search-specific-error-refined.png"Search for specific errors output1")
74
+
A sample query based on the average of resources used in a 10-minute interval, categorized by cluster name:
81
75
82
-
- To see an error occurred at a certain time:
76
+
```kusto
77
+
search in (metrics_resourcemanager_queue_root_default_CL) *
78
+
| summarize AggregatedValue = avg(UsedAMResourceMB_d) by ClusterName_s, bin(TimeGenerated, 10m)
79
+
```
83
80
84
-
![Search for specific errors output2](./media/hdinsight-hadoop-oms-log-analytics-use-queries/hdinsight-log-analytics-search-specific-error-time.png"Search for specific errors output2")
81
+
Instead of refining based on the average of resources used, you can use the following query to refine the results based on when the maximum resources were used (as well as 90th and 95th percentile) in a 10-minute window:
85
82
86
-
6. To see the specific error. You can select **[+]show more** to look at the actual error message.
87
-
88
-
![Search for specific errors output3](./media/hdinsight-hadoop-oms-log-analytics-use-queries/hdinsight-log-analytics-search-specific-error-arrived.png"Search for specific errors output3")
83
+
```kusto
84
+
search in (metrics_resourcemanager_queue_root_default_CL) *
The first step to create an alert is to arrive at a query based on which the alert is triggered. You can use any query that you want to create an alert.
93
91
94
92
1. Open the Log Analytics workspace that is associated to your HDInsight cluster from the Azure portal.
95
-
2. Select the **Log Search** tile.
96
-
3. Run the following query on which you want to create an alert, and then select **RUN**.
93
+
1. Under **General**, select **Logs**.
94
+
1. Run the following query on which you want to create an alert, and then select **Run**.
97
95
98
-
metrics_resourcemanager_queue_root_default_CL | where AppsFailed_d > 0
96
+
```kusto
97
+
metrics_resourcemanager_queue_root_default_CL | where AppsFailed_d > 0
98
+
```
99
99
100
100
The query provides list of failed applications running on HDInsight clusters.
101
101
102
-
4. Select **New Alert Rule** on the top of the page.
102
+
1. Select **New alert rule** on the top of the page.
103
103
104
104
105
105
106
-
5. In the **Create rule** window, enter the query and other details to create an alert, and then select **Create alert rule**.
106
+
1. In the **Create rule** window, enter the query and other details to create an alert, and then select **Create alert rule**.
107
107
108
108
109
109
110
-
To edit or delete an existing alert:
110
+
### Edit or delete an existing alert
111
111
112
112
1. Open the Log Analytics workspace from the Azure portal.
113
-
2. From the left menu, select **Alert**.
114
-
3. Select the alert you want to edit or delete.
115
-
4. You have the following options: **Save**, **Discard**, **Disable**, and **Delete**.
113
+
114
+
1. From the left menu, under **Monitoring**, select **Alerts**.
115
+
116
+
1. Towards the top, select **Manage alert rules**.
117
+
118
+
1. Select the alert you want to edit or delete.
119
+
120
+
1. You have the following options: **Save**, **Discard**, **Disable**, and **Delete**.
0 commit comments