Skip to content

Commit 653f00a

Browse files
authored
Merge pull request #207640 from paulth1/alerts-log-query
edit pass: alerts-log-query
2 parents c57a6dc + 1d92921 commit 653f00a

File tree

1 file changed

+106
-99
lines changed

1 file changed

+106
-99
lines changed
Lines changed: 106 additions & 99 deletions
Original file line numberDiff line numberDiff line change
@@ -1,56 +1,58 @@
11
---
2-
title: Optimizing log alert queries | Microsoft Docs
3-
description: Recommendations for writing efficient alert queries
2+
title: Optimize log alert queries | Microsoft Docs
3+
description: This article gives recommendations for writing efficient alert queries.
44
author: yanivlavi
55
ms.author: yalavi
66
ms.topic: conceptual
77
ms.date: 2/23/2022
88
---
9-
# Optimizing log alert queries
10-
This article describes how to write and convert [Log Alert](./alerts-unified-log.md) queries to achieve optimal performance. Optimized queries reduce latency and load of alerts, which run frequently.
9+
# Optimize log alert queries
1110

12-
## How to start writing an alert log query
11+
This article describes how to write and convert [log alert](./alerts-unified-log.md) queries to achieve optimal performance. Optimized queries reduce latency and load of alerts, which run frequently.
1312

14-
Alert queries start from [querying the log data in Log Analytics](alerts-log.md#create-a-new-log-alert-rule-in-the-azure-portal) that indicates the issue. You can use the [alert query examples topic](../logs/queries.md) to understand what you can discover. You may also [get started on writing your own query](../logs/log-analytics-tutorial.md).
13+
## Start writing an alert log query
14+
15+
Alert queries start from [querying the log data in Log Analytics](alerts-log.md#create-a-new-log-alert-rule-in-the-azure-portal) that indicates the issue. To understand what you can discover, see [Using queries in Azure Monitor Log Analytics](../logs/queries.md). You can also [get started on writing your own query](../logs/log-analytics-tutorial.md).
1516

1617
### Queries that indicate the issue and not the alert
1718

18-
The alert flow was built to transform the results that indicate the issue to an alert. For example, in a case of a query like:
19+
The alert flow was built to transform the results that indicate the issue to an alert. For example, in the case of a query like:
1920

2021
``` Kusto
2122
SecurityEvent
2223
| where EventID == 4624
2324
```
2425

25-
If the intent of the user is to alert, when this event type happens, the alerting logic appends `count` to the query. The query that will run will be:
26+
If the intent of the user is to alert, when this event type happens, the alerting logic appends `count` to the query. The query that runs will be:
2627

2728
``` Kusto
2829
SecurityEvent
2930
| where EventID == 4624
3031
| count
3132
```
3233

33-
There's no need to add alerting logic to the query and doing that may even cause issues. In the above example, if you include `count` in your query, it will always result in the value 1, since the alert service will do `count` of `count`.
34+
There's no need to add alerting logic to the query, and doing that might even cause issues. In the preceding example, if you include `count` in your query, it will always result in the value 1, because the alert service will do `count` of `count`.
3435

35-
### Avoid `limit` and `take` operators
36+
### Avoid limit and take operators
3637

37-
Using `limit` and `take` in queries can increase latency and load of alerts as the results aren't consistent over time. It's preferred you use it only if needed.
38+
Using `limit` and `take` in queries can increase latency and load of alerts because the results aren't consistent over time. Use them only if needed.
3839

3940
## Log query constraints
41+
4042
[Log queries in Azure Monitor](../logs/log-query-overview.md) start with either a table, [`search`](/azure/kusto/query/searchoperator), or [`union`](/azure/kusto/query/unionoperator) operator.
4143

42-
Queries for log alert rules should always start with a table to define a clear scope, which improves both query performance and the relevance of the results. Queries in alert rules run frequently, so using `search` and `union` can result in excessive overhead adding latency to the alert, as it requires scanning across multiple tables. These operators also reduce the ability of the alerting service to optimize the query.
44+
Queries for log alert rules should always start with a table to define a clear scope, which improves query performance and the relevance of the results. Queries in alert rules run frequently. Using `search` and `union` can result in excessive overhead that adds latency to the alert because it requires scanning across multiple tables. These operators also reduce the ability of the alerting service to optimize the query.
4345

4446
We don't support creating or modifying log alert rules that use `search` or `union` operators, except for cross-resource queries.
4547

46-
For example, the following alerting query is scoped to the _SecurityEvent_ table and searches for specific event ID. It's the only table that the query must process.
48+
For example, the following alerting query is scoped to the _SecurityEvent_ table and searches for a specific event ID. It's the only table that the query must process.
4749

4850
``` Kusto
4951
SecurityEvent
5052
| where EventID == 4624
5153
```
5254

53-
Log alert rules using [cross-resource queries](../logs/cross-workspace-query.md) are not affected by this change since cross-resource queries use a type of `union`, which limits the query scope to specific resources. The following example would be valid log alert query:
55+
Log alert rules using [cross-resource queries](../logs/cross-workspace-query.md) aren't affected by this change because cross-resource queries use a type of `union`, which limits the query scope to specific resources. The following example would be a valid log alert query:
5456

5557
```Kusto
5658
union
@@ -60,68 +62,71 @@ workspace('Contoso-workspace1').Perf
6062
```
6163

6264
>[!NOTE]
63-
> [Cross-resource queries](../logs/cross-workspace-query.md) are supported in the new [scheduledQueryRules API](/rest/api/monitor/scheduledqueryrule-2021-08-01/scheduled-query-rules). If you still use the [legacy Log Analytics Alert API](./api-alerts.md) for creating log alerts, you can learn about switching [here](../alerts/alerts-log-api-switch.md).
65+
> [Cross-resource queries](../logs/cross-workspace-query.md) are supported in the new [scheduledQueryRules API](/rest/api/monitor/scheduledqueryrule-2021-08-01/scheduled-query-rules). If you still use the [legacy Log Analytics Alert API](./api-alerts.md) for creating log alerts, see [Upgrade legacy rules management to the current Azure Monitor Log Alerts API](../alerts/alerts-log-api-switch.md) to learn about switching.
6466
6567
## Examples
66-
The following examples include log queries that use `search` and `union` and provide steps you can use to modify these queries for use in alert rules.
68+
69+
The following examples include log queries that use `search` and `union`. They provide steps you can use to modify these queries for use in alert rules.
6770

6871
### Example 1
69-
You want to create a log alert rule using the following query that retrieves performance information using `search`:
72+
73+
You want to create a log alert rule by using the following query that retrieves performance information using `search`:
7074

7175
``` Kusto
7276
search *
7377
| where Type == 'Perf' and CounterName == '% Free Space'
7478
| where CounterValue < 30
7579
```
7680

77-
To modify this query, start by using the following query to identify the table that the properties belong to:
78-
79-
``` Kusto
80-
search *
81-
| where CounterName == '% Free Space'
82-
| summarize by $table
83-
```
81+
1. To modify this query, start by using the following query to identify the table that the properties belong to:
82+
83+
``` Kusto
84+
search *
85+
| where CounterName == '% Free Space'
86+
| summarize by $table
87+
```
8488
85-
The result of this query would show that the _CounterName_ property came from the _Perf_ table.
89+
The result of this query would show that the _CounterName_ property came from the _Perf_ table.
8690
87-
You can use this result to create the following query that you would use for the alert rule:
91+
1. Use this result to create the following query that you would use for the alert rule:
8892
89-
``` Kusto
90-
Perf
91-
| where CounterName == '% Free Space'
92-
| where CounterValue < 30
93-
```
93+
``` Kusto
94+
Perf
95+
| where CounterName == '% Free Space'
96+
| where CounterValue < 30
97+
```
9498
9599
### Example 2
96-
You want to create a log alert rule using the following query that retrieves performance information using `search`:
100+
101+
You want to create a log alert rule by using the following query that retrieves performance information using `search`:
97102
98103
``` Kusto
99104
search ObjectName =="Memory" and CounterName=="% Committed Bytes In Use"
100105
| summarize Avg_Memory_Usage =avg(CounterValue) by Computer
101106
| where Avg_Memory_Usage between(90 .. 95)
102107
```
103108

104-
To modify this query, start by using the following query to identify the table that the properties belong to:
109+
1. To modify this query, start by using the following query to identify the table that the properties belong to:
110+
111+
``` Kusto
112+
search ObjectName=="Memory" and CounterName=="% Committed Bytes In Use"
113+
| summarize by $table
114+
```
115+
116+
The result of this query would show that the _ObjectName_ and _CounterName_ properties came from the _Perf_ table.
105117
106-
``` Kusto
107-
search ObjectName=="Memory" and CounterName=="% Committed Bytes In Use"
108-
| summarize by $table
109-
```
110-
111-
The result of this query would show that the _ObjectName_ and _CounterName_ property came from the _Perf_ table.
118+
1. Use this result to create the following query that you would use for the alert rule:
112119
113-
You can use this result to create the following query that you would use for the alert rule:
114-
115-
``` Kusto
116-
Perf
117-
| where ObjectName =="Memory" and CounterName=="% Committed Bytes In Use"
118-
| summarize Avg_Memory_Usage=avg(CounterValue) by Computer
119-
| where Avg_Memory_Usage between(90 .. 95)
120-
```
120+
``` Kusto
121+
Perf
122+
| where ObjectName =="Memory" and CounterName=="% Committed Bytes In Use"
123+
| summarize Avg_Memory_Usage=avg(CounterValue) by Computer
124+
| where Avg_Memory_Usage between(90 .. 95)
125+
```
121126
122127
### Example 3
123128
124-
You want to create a log alert rule using the following query that uses both `search` and `union` to retrieve performance information:
129+
You want to create a log alert rule by using the following query that uses both `search` and `union` to retrieve performance information:
125130
126131
``` Kusto
127132
search (ObjectName == "Processor" and CounterName == "% Idle Time" and InstanceName == "_Total")
@@ -132,39 +137,40 @@ search (ObjectName == "Processor" and CounterName == "% Idle Time" and InstanceN
132137
| summarize Avg_Idle_Time = avg(CounterValue) by Computer
133138
```
134139

135-
To modify this query, start by using the following query to identify the table that the properties in the first part of the query belong to:
140+
1. To modify this query, start by using the following query to identify the table that the properties in the first part of the query belong to:
136141

137-
``` Kusto
138-
search (ObjectName == "Processor" and CounterName == "% Idle Time" and InstanceName == "_Total")
139-
| summarize by $table
140-
```
142+
``` Kusto
143+
search (ObjectName == "Processor" and CounterName == "% Idle Time" and InstanceName == "_Total")
144+
| summarize by $table
145+
```
141146
142-
The result of this query would show that all these properties came from the _Perf_ table.
147+
The result of this query would show that all these properties came from the _Perf_ table.
143148
144-
Now use `union` with `withsource` command to identify which source table has contributed each row.
149+
1. Use `union` with the `withsource` command to identify which source table has contributed each row:
145150
146-
``` Kusto
147-
union withsource=table *
148-
| where CounterName == "% Processor Utility"
149-
| summarize by table
150-
```
151+
``` Kusto
152+
union withsource=table *
153+
| where CounterName == "% Processor Utility"
154+
| summarize by table
155+
```
151156
152-
The result of this query would show that these properties also came from the _Perf_ table.
157+
The result of this query would show that these properties also came from the _Perf_ table.
153158
154-
You can use these results to create the following query that you would use for the alert rule:
159+
1. Use these results to create the following query that you would use for the alert rule:
155160
156-
``` Kusto
157-
Perf
158-
| where ObjectName == "Processor" and CounterName == "% Idle Time" and InstanceName == "_Total"
159-
| where Computer !in (
160-
(Perf
161-
| where CounterName == "% Processor Utility"
162-
| summarize by Computer))
163-
| summarize Avg_Idle_Time = avg(CounterValue) by Computer
164-
```
161+
``` Kusto
162+
Perf
163+
| where ObjectName == "Processor" and CounterName == "% Idle Time" and InstanceName == "_Total"
164+
| where Computer !in (
165+
(Perf
166+
| where CounterName == "% Processor Utility"
167+
| summarize by Computer))
168+
| summarize Avg_Idle_Time = avg(CounterValue) by Computer
169+
```
165170
166171
### Example 4
167-
You want to create a log alert rule using the following query that joins the results of two `search` queries:
172+
173+
You want to create a log alert rule by using the following query that joins the results of two `search` queries:
168174
169175
```Kusto
170176
search Type == 'SecurityEvent' and EventID == '4625'
@@ -176,38 +182,39 @@ search Type == 'SecurityEvent' and EventID == '4625'
176182
) on Hour
177183
```
178184

179-
To modify the query, start by using the following query to identify the table that contains the properties in the left side of the join:
180-
181-
``` Kusto
182-
search Type == 'SecurityEvent' and EventID == '4625'
183-
| summarize by $table
184-
```
185+
1. To modify the query, start by using the following query to identify the table that contains the properties in the left side of the join:
186+
187+
``` Kusto
188+
search Type == 'SecurityEvent' and EventID == '4625'
189+
| summarize by $table
190+
```
185191
186-
The result indicates that the properties in the left side of the join belong to _SecurityEvent_ table.
192+
The result indicates that the properties in the left side of the join belong to the _SecurityEvent_ table.
187193
188-
Now use the following query to identify the table that contains the properties in the right side of the join:
194+
1. Use the following query to identify the table that contains the properties in the right side of the join:
189195
190-
``` Kusto
191-
search in (Heartbeat) OSType == 'Windows'
192-
| summarize by $table
193-
```
196+
``` Kusto
197+
search in (Heartbeat) OSType == 'Windows'
198+
| summarize by $table
199+
```
194200
195-
The result indicates that the properties in the right side of the join belong to _Heartbeat_ table.
196-
197-
You can use these results to create the following query that you would use for the alert rule:
198-
199-
``` Kusto
200-
SecurityEvent
201-
| where EventID == '4625'
202-
| summarize by Computer, Hour = bin(TimeGenerated, 1h)
203-
| join kind = leftouter (
204-
Heartbeat
205-
| where OSType == 'Windows'
206-
| summarize arg_max(TimeGenerated, Computer) by Computer , Hour = bin(TimeGenerated, 1h)
207-
| project Hour , Computer
208-
) on Hour
209-
```
201+
The result indicates that the properties in the right side of the join belong to the _Heartbeat_ table.
202+
203+
1. Use these results to create the following query that you would use for the alert rule:
204+
205+
``` Kusto
206+
SecurityEvent
207+
| where EventID == '4625'
208+
| summarize by Computer, Hour = bin(TimeGenerated, 1h)
209+
| join kind = leftouter (
210+
Heartbeat
211+
| where OSType == 'Windows'
212+
| summarize arg_max(TimeGenerated, Computer) by Computer , Hour = bin(TimeGenerated, 1h)
213+
| project Hour , Computer
214+
) on Hour
215+
```
210216
211217
## Next steps
218+
212219
- Learn about [log alerts](alerts-log.md) in Azure Monitor.
213220
- Learn about [log queries](../logs/log-query-overview.md).

0 commit comments

Comments
 (0)