Skip to content

Commit 913a961

Browse files
committed
Push Efrat's changes to troubleshooting articles
1 parent 6356130 commit 913a961

File tree

3 files changed

+89
-235
lines changed

3 files changed

+89
-235
lines changed

articles/azure-monitor/alerts/alerts-troubleshoot-log.md

Lines changed: 11 additions & 112 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,17 @@
11
---
2-
title: Troubleshoot log search alerts in Azure Monitor | Microsoft Docs
3-
description: Common issues, errors, and resolutions for log search alert rules in Azure.
2+
title: Troubleshoot log alerts in Azure Monitor | Microsoft Docs
3+
description: Common issues, errors, and resolutions for log alert rules in Azure.
44
ms.author: abbyweisberg
55
ms.topic: conceptual
6-
ms.custom: build-2023
7-
ms.date: 02/13/2024
8-
ms.reviewer: yalavi
6+
ms.date: 02/28/2024
7+
ms.reviewer: nolavime
98
---
109

1110
# Troubleshoot log search alerts in Azure Monitor
1211

13-
This article describes how to resolve common issues with log search alerts in Azure Monitor. It also provides solutions to common problems with the functionality and configuration of log search alerts.
12+
This article describes how to resolve common issues with log search alerts in Azure Monitor. It also provides solutions to common problems with the functionality and configuration of log alerts.
1413

15-
You can use log search alerts to evaluate resources logs every set frequency by using a [Log Analytics](../logs/log-analytics-tutorial.md) query, and fire an alert that's based on the results. Rules can trigger one or more actions using [Action Groups](./action-groups.md). To learn more about functionality and terminology of log search alerts, see [Log search alerts in Azure Monitor](alerts-types.md#log-alerts).
14+
You can use log alerts to evaluate resources logs every set frequency by using a [Log Analytics](../logs/log-analytics-tutorial.md) query, and fire an alert that's based on the results. Rules can trigger one or more actions using [Action Groups](./action-groups.md). To learn more about functionality and terminology of log search alerts, see [Log alerts in Azure Monitor](alerts-types.md#log-alerts).
1615

1716
> [!NOTE]
1817
> This article doesn't consider cases where the Azure portal shows that an alert rule was triggered but a notification isn't received. For such cases, see [Action or notification on my alert did not work as expected](./alerts-troubleshoot.md#action-or-notification-on-my-alert-did-not-work-as-expected).
@@ -31,7 +30,6 @@ View the health status of your log search alert rule:
3130
:::image type="content" source="media/log-search-alert-health/log-search-alert-resource-health.png" alt-text="Screenshot of the Resource health section in a log search alert rule.":::
3231

3332
See [Monitor the health of log search alert rules](log-alert-rule-health.md#monitor-the-health-of-log-search-alert-rules) to learn more.
34-
## Log search alert didn't fire
3533

3634
### Data ingestion time for logs
3735

@@ -43,119 +41,32 @@ To mitigate latency, the system retries the alert evaluation multiple times. Aft
4341

4442
### Actions are muted or alert rule is defined to resolve automatically
4543

46-
Log search alerts provide an option to mute fired alert actions for a set amount of time using **Mute actions** and to only fire once per condition being met using **Automatically resolve alerts**.
44+
Log alerts provide an option to mute fired alert actions for a set amount of time using **Mute actions** and to only fire once per condition being met using **Automatically resolve alerts**.
4745

4846
A common issue is that you think that the alert didn't fire, but it was actually the rule configuration.
4947

5048
:::image type="content" source="media/alerts-troubleshoot-log/LogAlertSuppress.png" lightbox="media/alerts-troubleshoot-log/LogAlertSuppress.png" alt-text="Suppress alerts":::
5149

5250
## Log search alert fired when it shouldn't have
53-
### Alert scope resource has been moved, renamed, or deleted
5451

55-
When you author an alert rule, Log Analytics creates a permission snapshot for your user ID. This snapshot is saved in the rule and contains the rule scope resource, Azure Resource Manager ID. If the rule scope resource moves, gets renamed, or is deleted, all log search alert rules that refer to that resource will break. To work correctly, alert rules need to be recreated using the new Azure Resource Manager ID.
56-
57-
### The alert rule uses a system-assigned managed identity
58-
59-
When you create a log search alert rule with system-assigned managed identity, the identity is created without any permissions. After you create the rule, you need to assign the appropriate roles to the rule’s identity so that it can access the data you want to query. For example, you might need to give it a Reader role for the relevant Log Analytics workspaces, or a Reader role and a Database Viewer role for the relevant ADX cluster. See [managed identities](/azure/azure-monitor/alerts/alerts-create-log-alert-rule#configure-the-alert-rule-details) for more information about using managed identities in log search alerts.
60-
61-
### Metric measurement alert rule with splitting using the legacy Log Analytics API
62-
63-
[Metric measurement](alerts-types.md#log-alerts) is a type of log search alert that's based on summarized time series results. You can use these rules to group by columns to [split alerts](alerts-types.md#monitor-the-same-condition-on-multiple-resources-using-splitting-by-dimensions-1). If you're using the legacy Log Analytics API, splitting doesn't work as expected because it doesn't support grouping.
64-
65-
You can use the current ScheduledQueryRules API to set **Aggregate On** in [Metric measurement](alerts-types.md#log-alerts) rules, which work as expected. To learn more about switching to the current ScheduledQueryRules API, see [Upgrade to the current Log Alerts API from legacy Log Analytics Alert API](./alerts-log-api-switch.md).
66-
67-
### Override query time range
68-
69-
As a part of the configuration of the alert, in the section of the "Advance Options", there is an option to configure "Override query time range" parameter.
70-
If you want the alert evaluation period to be different than the query time range, enter a time range here.
71-
The alert time range is limited to a maximum of two days. Even if the query contains an ago command with a time range of longer than two days, the two-day maximum time range is applied. For example, even if the query text contains ago(7d), the query only scans up to two days of data.
72-
If the query requires more data than the alert evaluation, you can change the time range manually.
73-
If there's ago command in the query, it will be changed automatically to be 2 days (48 hours).
74-
75-
:::image type="content" source="media/alerts-troubleshoot-log/alerts-rule-preview-advanced-options.png" lightbox="media/alerts-troubleshoot-log/alerts-rule-preview-advanced-options.png" alt-text="Screenshot of advanced settings for log search alerts.":::
76-
77-
## Log search alert fired unnecessarily
78-
79-
A configured [log search alert rule in Azure Monitor](./alerts-log.md) might be triggered unexpectedly. The following sections describe some common reasons.
52+
A configured [log alert rule in Azure Monitor](./alerts-log.md) might be triggered unexpectedly. The following sections describe some common reasons.
8053

8154
### Alert triggered by partial data
8255

8356
Azure Monitor processes terabytes of customers' logs from across the world, which can cause [logs ingestion latency](../logs/data-ingestion-time.md).
8457

8558
Logs are semi-structured data and are inherently more latent than metrics. If you're experiencing many misfires in fired alerts, you should consider using [metric alerts](alerts-metric-overview.md). You can send data to the metric store from logs using [metric alerts for logs](alerts-metric-logs.md).
8659

87-
Log search alerts work best when you try to detect data in the logs. It works less well when you try to detect lack of data in the logs, like alerting on virtual machine heartbeat.
60+
Log alerts work best when you try to detect data in the logs. It works less well when you try to detect lack of data in the logs, like alerting on virtual machine heartbeat.
8861

8962
There are built-in capabilities to prevent false alerts, but they can still occur on very latent data (over ~30 minutes) and data with latency spikes.
9063

9164
## Log search alert rule was disabled
9265

9366
If a log search alert rule query fails to evaluate continuously for a week, Azure Monitor disables it automatically.
9467
The following sections list some reasons why Azure Monitor might disable a log search alert rule. Additionally, there's an example of the [Activity log](../../azure-monitor/essentials/activity-log.md) event that is submitted when a rule is disabled.
95-
## Log search alert was disabled
96-
97-
The following sections list some reasons why Azure Monitor might disable a log search alert rule. After those sections, there's an [example of the activity log that is sent when a rule is disabled](#activity-log-example-when-rule-is-disabled).
98-
99-
### Alert scope no longer exists or was moved
100-
101-
When the scope resources of an alert rule are no longer valid, rule execution fails, and billing stops.
102-
103-
If a log search alert fails continuously for a week, Azure Monitor disables it.
104-
105-
### <a name="query-used-in-a-log-alert-isnt-valid"></a>Query used in a log search alert isn't valid
106-
107-
When a log search alert rule is created, the query is validated for correct syntax. But sometimes, the query provided in the log search alert rule can start to fail. Some common reasons are:
108-
109-
- Rules were created via the API, and validation was skipped by the user.
110-
- The query [runs on multiple resources](../logs/cross-workspace-query.md), and one or more of the resources was deleted or moved.
111-
- The [query fails](../logs/api/errors.md) because:
112-
- The logging solution wasn't [deployed to the workspace](../insights/solutions.md#install-a-monitoring-solution), so tables aren't created.
113-
- Data stopped flowing to a table in the query for more than 30 days.
114-
- [Custom logs tables](../agents/data-sources-custom-logs.md) aren't yet created, because the data flow hasn't started.
115-
- Changes in [query language](/azure/kusto/query/) include a revised format for commands and functions, so the query provided earlier is no longer valid.
116-
117-
[Azure Advisor](../../advisor/advisor-overview.md) warns you about this behavior. It adds a recommendation about the affected log search alert rule. The category used is 'High Availability' with medium impact and a description of 'Repair your log search alert rule to ensure monitoring'.
118-
119-
## Alert rule quota was reached
120-
121-
For details about the number of log search alert rules per subscription and maximum limits of resources, see [Azure Monitor service limits](../service-limits.md).
122-
123-
### Recommended Steps
124-
125-
If you've reached the quota limit, the following steps might help resolve the issue.
126-
127-
1. Delete or disable log search alert rules that aren’t used anymore.
128-
1. Use [splitting of alerts by dimensions](alerts-types.md#monitor-the-same-condition-on-multiple-resources-using-splitting-by-dimensions-1) to reduce rules count. These rules can monitor many resources and detection cases.
129-
1. If you need the quota limit to be increased, continue to open a support request, and provide the following information:
130-
131-
- The Subscription IDs and Resource IDs for which the quota limit needs to be increased
132-
- The reason for quota increase
133-
- The resource type for the quota increase, such as **Log Analytics** or **Application Insights**
134-
- The requested quota limit
135-
136-
### To check the current usage of new log search alert rules
137-
138-
#### From the Azure portal
139-
140-
1. On the Alerts screen in Azure Monitor, select **Alert rules**.
141-
1. In the **Subscription** dropdown control, filter to the subscription you want. (Make sure you don't filter to a specific resource group, resource type, or resource.)
142-
1. In the **Signal type** dropdown control, select **Log Search**.
143-
1. Verify that the **Status** dropdown control is set to **Enabled**.
144-
145-
The total number of log search alert rules is displayed above the rules list.
14668

14769
#### Activity log example when rule is disabled
148-
#### From API
149-
150-
- PowerShell - [Get-AzScheduledQueryRule](/powershell/module/az.monitor/get-azscheduledqueryrule)
151-
- CLI: [az monitor scheduled-query list](/cli/azure/monitor/scheduled-query#az-monitor-scheduled-query-list)
152-
- REST API - [List by subscription](/rest/api/monitor/scheduledqueryrule-2021-08-01/scheduled-query-rules/list-by-subscription)
153-
154-
## Activity log example when rule is disabled
155-
156-
If query fails for seven days continuously, Azure Monitor disables the log search alert and stops the billing of the rule. You can see the exact time when Azure Monitor disabled the log search alert in the [Azure activity log](../../azure-monitor/essentials/activity-log.md).
157-
158-
See this example:
15970

16071
```json
16172
{
@@ -309,20 +220,8 @@ The total number of log search alert rules is displayed above the rules list.
309220
- CLI: [az monitor scheduled-query list](/cli/azure/monitor/scheduled-query#az-monitor-scheduled-query-list)
310221
- REST API - [List by subscription](/rest/api/monitor/scheduledqueryrule-2021-08-01/scheduled-query-rules/list-by-subscription)
311222

312-
## Query syntax validation error
313-
314-
If you get an error message that says "Couldn't validate the query syntax because the service can't be reached", it could be either:
315-
- A query syntax error.
316-
- A problem connecting to the service that validates the query.
317-
318-
Try the following steps to resolve the problem:
319-
1. Try running the query in Azure Monitor Logs, and fix any syntax issues.
320-
2. If your query syntax is valid, check the connection to the service.
321-
- Flush the DNS cache on your local machine, by opening a command prompt and running the following command: `ipconfig /flushdns`, and then check again. If you still get the same error message, try the next step.
322-
- Copy and paste this URL into the browser: [https://api.loganalytics.io/v1/version](https://api.loganalytics.io/v1/version). If you get an error, contact your IT administrator to allow the IP addresses associated with **api.loganalytics.io** listed [here](../ip-addresses.md#application-insights-and-log-analytics-apis).
323-
324223
## Next steps
325224

326-
- Learn about [log search alerts in Azure](./alerts-types.md#log-alerts).
327-
- Learn more about [configuring log search alerts](../logs/log-query-overview.md).
225+
- Learn about [log alerts in Azure](./alerts-unified-log.md).
226+
- Learn more about [configuring log alerts](../logs/log-query-overview.md).
328227
- Learn more about [log queries](../logs/log-query-overview.md).

0 commit comments

Comments
 (0)