Skip to content

Commit 4bc8bf8

Browse files
committed
Update summary-rules.md
1 parent 23c563d commit 4bc8bf8

File tree

1 file changed

+30
-27
lines changed

1 file changed

+30
-27
lines changed

articles/azure-monitor/logs/summary-rules.md

Lines changed: 30 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -43,9 +43,14 @@ Summary rules perform batch processing directly in your Log Analytics workspace.
4343

4444
:::image type="content" source="media/summary-rules/ingestion-flow.png" alt-text="A diagram that shows how data is ingested from various data sources to a Log Analytics workspace and is aggregated and reingested into the workspace by using a summary rule." lightbox="media/summary-rules/ingestion-flow.png":::
4545

46-
You can aggregate data you ingest into any table, including both [Analytics and Basic](basic-logs-query.md) tables.
46+
You can aggregate data from any table, regardless of whether the table has an [Analytics or Basic data plan](basic-logs-query.md). Azure Monitor creates the destination table schema based on the query you define. If the destination table already exists, Azure Monitor adds any columns required to support the query results.
4747

48-
You can configure up to 30 rules to aggregate data from multiple tables and send the aggregated data to the same destination table or to separate tables.
48+
You can configure up to 30 active rules to aggregate data from multiple tables and send the aggregated data to separate destination tables or the same table. All destination tables include a set of standard fields with summary rule information, including:
49+
50+
- `_RuleName`: The summary rule that generated the aggregated log entry.
51+
- `_RuleLastModifiedTime`: When the rule was last modified.
52+
- `_BinSize`: The aggregation interval.
53+
- `_BinStartTime` The aggregation start time.
4954

5055
You can export summarized data from a custom log table to a storage account or Event Hubs for further integrations by defining a [data export rule](logs-data-export.md).
5156

@@ -69,29 +74,25 @@ Here's the aggregated data that the summary rule sends to the destination table:
6974

7075
Instead of logging hundreds of similar entries within an hour, the destination shows the count of each unique entry, as defined in the KQL query. Set the [Basic data plan](basic-logs-configure.md) on the `ContainerLogsV2` table for cheap retention of the raw data, and use the summarized data in the destination table for your analysis needs.
7176

72-
All entries in the destination table include a set of standard fields:
73-
74-
- `_RuleName`: The summary rule that generated the aggregated log entry.
75-
- `_RuleLastModifiedTime`: When the rule was last modified.
76-
- `_BinSize`: The aggregation interval.
77-
- `_BinStartTime` The aggregation start time.
78-
7977
## Restrictions and limitations
8078

81-
| Category | Limit | Comments |
82-
|:---|:---|:---|
83-
| Maximum number of active rules in a workspace | 30 | |
84-
| Maximum number of results per bin | 500,000 | |
85-
| Maximum results set volume | 100 MB | |
86-
| Query time-out for bin processing | 10 minutes | |
79+
| Category | Limit |
80+
|:---|:---|
81+
| Maximum number of active rules in a workspace | 30 |
82+
| Maximum number of results per bin | 500,000 |
83+
| Maximum results set volume | 100 MB |
84+
| Query time-out for bin processing | 10 minutes |
8785

8886
- The summary rule processes incoming data and can't be configured on a historical time range.
8987
- When bin execution retries are exhausted, the bin is skipped and can't be re-executed.
9088
- Querying a Log Analytics workspace in another tenant by using Lighthouse isn't supported.
9189
- KQL limits depend on the table plan of the source table.
9290

93-
- Analytics: Supports all KQL commands, except for data reshaping plugins, including [bag unpack](/azure/data-explorer/kusto/query/bag-unpack-plugin), [narrow](/azure/data-explorer/kusto/query/narrow-plugin), and [pivot](/azure/data-explorer/kusto/query/pivot-plugin).
94-
- Basic: Supports all KQL commands on a single Basic or Auxiliary table. Because `summarize` and `join` aren't supported, use lookup for up to five Analytics tables.
91+
- Analytics: Supports all KQL commands, except for:
92+
93+
- [Cross-resource queries](cross-workspace-query.md), using the `workspaces()`, `app()`, and `resource()` expressions, and [cross-service queries](azure-monitor-data-explorer-proxy.md), using the `ADX()` and `ARG()` expressions.
94+
- Plugins that reshape the data schema, including [bag unpack](/azure/data-explorer/kusto/query/bag-unpack-plugin), [narrow](/azure/data-explorer/kusto/query/narrow-plugin), and [pivot](/azure/data-explorer/kusto/query/pivot-plugin).
95+
- Basic: Supports all KQL commands on a single table. You can join up to five Analytics tables using the [lookup](/azure/data-explorer/kusto/query/lookup-operator) operator.
9596
- Functions: User-defined functions aren't supported. System functions provided by Microsoft are supported.
9697

9798
## Pricing model
@@ -108,7 +109,7 @@ For example, this is the cost calculation for hourly rule that returns 100 recor
108109
| Rule configuration | Monthly price calculation
109110
| --- | --- |
110111
| Query Analytics table | Ingestion price x record size x number of records x 24 hours x 30 days |
111-
| Query Basic table scanning 1 GB each bin | Scanned GB price x scanned size + Ingestion price x record size x number of records x 24 hours x 30 days |
112+
| Query Basic table | Scanned GB price x scanned size + Ingestion price x record size x number of records x 24 hours x 30 days |
112113

113114
For more information, see [Azure Monitor pricing](https://azure.microsoft.com/pricing/details/monitor/).
114115

@@ -308,7 +309,7 @@ This table describes the summary rule parameters:
308309
| --- | --- |
309310
| `ruleType` | `User` or `System` | Specifies the type of rule. <br> - `User`: Rules you define. <br> - `System`: Predefined rules managed by Azure services. |
310311
| `description` | String | Describes the rule and its function. This parameter is helpful when you have several rules and can help with rule management. |
311-
| `binSize` |`20`, `30`, `60`, `120`, `180`, `360`, `720`, or `1,440` (minutes) | Defines the aggregation interval lookback time range. For values over an hour, the aggregation starts at the beginning of the whole hour - if you set `"binSize": 120`, you might get entries for `02:00 to 04:00` and `04:00 to 06:00`. When the bin size is smaller than an hour, the rule begins aggregating immediately. |
312+
| `binSize` |`20`, `30`, `60`, `120`, `180`, `360`, `720`, or `1,440` (minutes) | Defines the aggregation interval and lookback time range. For values over an hour, the aggregation starts at the beginning of the whole hour - if you set `"binSize": 120`, you might get entries for `02:00 to 04:00` and `04:00 to 06:00`. When the bin size is smaller than an hour, the rule begins aggregating immediately. |
312313
| `query` | [Kusto Query Language (KQL) query](get-started-queries.md) | Defines the query to execute in the rule. You don't need to specify a time range because the `binSize` parameter determines the aggregation - for example, `02:00 to 03:00` if `"binSize": 60`. If you add a time filter in the query, the time rage used in the query is the intersection between the filter and the bin size. |
313314
| `destinationTable` | `tablename_CL` | Specifies the name of the destination custom log table. The name value must have the suffix `_CL`. Azure Monitor creates the table in the workspace, if it doesn't already exist, based on the query you set in the rule. If the table already exists in the workspace, Azure Monitor adds any new columns introduced in the query. <br><br> If the summary results include a reserved column name - such as `TimeGenerated`, `_IsBillable`, `_ResourceId`, `TenantId`, or `Type` - Azure Monitor appends the `_Original` prefix to the original fields to preserve their original values.|
314315
| `binDelay` (optional) | Integer (minutes) | Sets a time to delay before bin execution for late arriving data, also known as [ingestion latency](data-ingestion-time.md). The delay allows for most data to arrive and for service load distribution. The default delay is from three and a half minutes to 10% of the `binSize` value. <br><br> If you know that the data you query is typically ingested with delay, set the `binDelay` parameter with the known delay value or greater. For more information, see [Configure the aggregation timing](#configure-the-aggregation-timing).|
@@ -320,7 +321,7 @@ This table describes the summary rule parameters:
320321

321322
By default, the summary rule creates the first aggregation shortly after the next whole hour.
322323

323-
The short delay Azure Monitor adds accounts for ingestion latency - or the time between when the data is created in the monitored system and the time that it becomes available for analysis in Azure Monitor. By default, this delay is between three and a half minutes to 10% of the `binSize` value before aggregating each chunk of data. In most cases, this delay ensures that Azure Monitor aggregates all data logged within each bin period and doesn't miss late arriving data.
324+
The short delay Azure Monitor adds accounts for ingestion latency - or the time between when the data is created in the monitored system and the time that it becomes available for analysis in Azure Monitor. By default, this delay is between three and a half minutes to 10% of the `binSize` value before aggregating each chunk of data. In most cases, this delay ensures that Azure Monitor aggregates all data logged within each bin period.
324325

325326
For example:
326327

@@ -403,7 +404,7 @@ Authorization: {credential}
403404

404405
## Delete a summary rule
405406

406-
You can have up to 30 active summary rules in your Log Analytics workspace. If you want to create a new rule, but you already have 10 active rules, you must stop or delete an active summary rule.
407+
You can have up to 30 active summary rules in your Log Analytics workspace. If you want to create a new rule, but you already have 30 active rules, you must stop or delete an active summary rule.
407408

408409
To delete a rule, use this `DELETE` API call:
409410

@@ -460,24 +461,26 @@ A KQL query can contain sensitive information in comments or in the query syntax
460461

461462
Considerations when you work with encrypted queries:
462463

463-
- If you already have summary rule queries before you link a storage account to your Log Analytics workspace, update your existing query rules to encrypt the queries.
464464
- Linking a storage account to encrypt your queries doesn’t interrupt existing rules.
465-
- Queries that you save in a storage account are considered service artifacts and their format might change.
466-
- You can use the same storage account for summary rules queries, [saved queries in Log Analytics](save-query.md), and [log alerts](../alerts/alerts-types.md#log-alerts).
465+
- By default, Azure Monitor stores summary rule queries in Log Analytics storage. If you have existing summary rules before you link a storage account to your Log Analytics workspace, update your summary rules so the queries to save the existing queries in the storage account.
466+
- Queries that you save in a storage account are located in the `CustomerConfigurationStoreTable` table. These queries are considered service artifacts and their format might change.
467+
- You can use the same storage account for summary rule queries, [saved queries in Log Analytics](save-query.md), and [log alerts](../alerts/alerts-types.md#log-alerts).
467468

468469
## Troubleshoot summary rules
469470

470471
This section provides tips for troubleshooting summary rules.
471472

472473
### Summary rule destination table accidentally deleted
473474

474-
If you delete the destination table while the summary rule is active, the rule remains active even though it doesn't return data. The logs in the LASummaryLogs table show the run as successful.
475+
If you delete the destination table while the summary rule is active, the rule gets suspended and Azure Monitor sends an event to the `LASummaryLogs` table with a message indicating that the rule was suspended.
476+
477+
If you don't need the summary results in the destination table, delete the rule and table. If you need to recover summary results, follow the steps in Create or update summary rules section to recreate the table. The destination table is restored, including the data ingested before the delete, depending on the retention policy in the table.
475478

476-
If you don't need the summary results in the destination table, delete the rule. If you need the summary results, follow the steps in [Create or update summary rules](#create-or-update-a-summary-rule) section to recreate the table. The results table restores all data, including the data ingested before the delete, depending on the retention policy in the table.
479+
If you don't need the summary results in the destination table, delete the rule and table. If you need the summary results, follow the steps in the [Create or update summary rules](#create-or-update-a-summary-rule) section to recreate the destination table and restore all data, including the data ingested before the delete, depending on the retention policy in the table.
477480

478481
### Query uses operators that create new columns in the destination table
479482

480-
If the query in the summary rule includes operators that allow output schema expansion based on incoming data - for example, if the query the `arg_max(expression, *)` function - the summary rule needs to create new columns in the destination table. However, Azure Monitor doesn't add new columns to the destination table after you create or update the summary rule, and the output data that requires these columns will be dropped. To add the new fields to the destination table, [update the summary rule](#create-or-update-a-summary-rule) or [add a column to your table manually](create-custom-table.md#add-or-delete-a-custom-column).
483+
The destination table schema is defined when you create or update a summary rule. If the query in the summary rule includes operators that allow output schema expansion based on incoming data - for example, if the query uses the `arg_max(expression, *)` function - Azure Monitor doesn't add new columns to the destination table after you create or update the summary rule, and the output data that requires these columns will be dropped. To add the new fields to the destination table, [update the summary rule](#create-or-update-a-summary-rule) or [add a column to your table manually](create-custom-table.md#add-or-delete-a-custom-column).
481484

482485
### Deleted data remains in workspace, subject to retention period
483486

0 commit comments

Comments
 (0)