Skip to content

Commit c3ccb21

Browse files
learn-build-service-prod[bot]learn-build-service-prod[bot]
authored
Merge pull request #2792 from MicrosoftDocs/main639002269642170087sync_temp
For protected branch, push strategy should use PR and merge to target branch method to work around git push error
2 parents 6b7a4cd + f75c6af commit c3ccb21

File tree

3 files changed

+255
-0
lines changed

3 files changed

+255
-0
lines changed

‎data-explorer/kusto/management/query-acceleration-policy.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@ Query acceleration is supported in Eventhouse over OneLake, Azure Data Lake Stor
2121
To enable query acceleration in the Fabric UI, see [Query acceleration over OneLake shortcuts](https://go.microsoft.com/fwlink/?linkid=2296674).
2222
::: moniker-end
2323

24+
> [!IMPORTANT]
25+
> To troubleshoot problems with query acceleration, see [Troubleshoot query acceleration over external delta tables](query-acceleration-tsg.md).
26+
2427
## Limitations
2528

2629
* The number of columns in the external table can't exceed 900.

‎data-explorer/kusto/management/query-acceleration-tsg.md‎

Lines changed: 249 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,249 @@
1+
---
2+
title: query acceleration troubleshooting guide
3+
description: Learn how to troubleshoot query acceleration issues
4+
ms.reviewer: urishapira
5+
ms.topic: reference
6+
ms.date: 12/01/2025
7+
---
8+
9+
# Troubleshoot query acceleration over external delta tables
10+
11+
> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)]
12+
13+
The [query acceleration policy](query-acceleration-policy.md) enables accelerating queries over external delta tables by caching delta table metadata and data files. The policy defines which date ranges (number of days back and hot windows) are accelerated so that queries over those ranges can run faster.
14+
15+
The query acceleration feature consists of the following components:
16+
17+
- A background job that maintains a local snapshot (**catalog**) of the delta table metadata.
18+
- A background job that caches delta table data files.
19+
- Query-time enhancements that utilize the catalog and the cached data.
20+
21+
To understand why things aren't working as expected, it's important to identify which of these components isn't functioning properly.
22+
23+
This article helps you troubleshoot scenarios where:
24+
25+
- Query over an accelerated external delta table returns **stale data**, or
26+
- Query over an accelerated external delta table is **slower than expected**
27+
28+
## Prerequisites
29+
30+
1. **Ensure query acceleration is enabled on the external table** by running the following command:
31+
32+
```kusto
33+
.show external table <ETName> policy query_acceleration
34+
| project isnotnull(Policy) and todynamic(Policy).IsEnabled
35+
```
36+
37+
If this command returns `false`, enable the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md).
38+
39+
2. **Ensure the delta table complies with the Delta protocol.**
40+
41+
The query acceleration feature assumes a delta table that complies with the Delta protocol. Manual operations executed directly on the delta table (for example, editing transaction logs or parquet files) aren't supported and may result in unexpected behavior.
42+
43+
If such operations have been executed on the delta table, recreate the external table and re-enable the query acceleration policy.
44+
45+
46+
## Query is returning stale data
47+
48+
This is a data freshness issue: query results don't reflect the latest data from the underlying delta table.
49+
50+
Query acceleration refreshes the accelerated data periodically, so that results are no older than the configured `MaxAge` value in the policy.
51+
By design, queries over accelerated external tables may return data that lags behind the latest delta table version by up to `MaxAge`. Set `MaxAge` to the maximum data staleness that is acceptable at query time.
52+
53+
You can control the effective `MaxAge` in two ways:
54+
55+
- Configure the `MaxAge` property in the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md).
56+
- Override `MaxAge` per query by using the [`external_table()` operator's](../query/external-table-function.md) `MaxAgeOverride` parameter.
57+
58+
## Query isn't running fast enough
59+
60+
This is a performance issue: query is slower than expected, and acceleration doesn't appear to improve performance.
61+
62+
There are a few reasons why this could happen:
63+
64+
- The query acceleration catalog is unusable (out-of-date or never built) - see section [Check if catalog is unusable](#check-if-catalog-is-unusable)
65+
- The query scans nonaccelerated data - see section [Check if query is over nonaccelerated data](#check-if-query-is-over-nonaccelerated-data)
66+
- The query doesn't comply with KQL best practices - see [KQL best practices](../query/best-practices.md)
67+
68+
## Check if catalog is unusable
69+
70+
Query acceleration uses a local catalog for the external table containing a snapshot of the delta table metadata. If this catalog hasn't been updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **unusable** and isn't used at query time. In that case, queries fall back to reading the remote delta table directly, which can be significantly slower.
71+
72+
Fetch the current state of the catalog using the following command:
73+
74+
```kusto
75+
.show external table [ETName] details
76+
| extend MinimumUpdateTime = now() - totimespan(todynamic(QueryAccelerationPolicy).MaxAge)
77+
| project IsCatalogUnusable = MinimumUpdateTime > todatetime(todynamic(QueryAccelerationState).LastUpdatedDateTime)
78+
```
79+
80+
`IsCatalogUnusable == true` indicates the catalog is stale and query acceleration won't be used.
81+
82+
### Troubleshoot unusable catalogs
83+
84+
To understand why a catalog is unusable, first check if the query acceleration state is healthy and resolve unhealthy reasons as needed.
85+
86+
Run:
87+
88+
```kusto
89+
.show external table [ETName] details
90+
| project state = todynamic(QueryAccelerationState)
91+
| project IsHealthy = state.IsHealthy, UnhealthyReason = state.NotHealthyReason
92+
```
93+
94+
- If the state is **healthy** but the catalog is still stale, it could be that the query acceleration policy was enabled recently. See [Query acceleration policy was enabled recently](#query-acceleration-policy-was-enabled-recently)
95+
- If the state is **unhealthy**, refer to [Query acceleration unhealthy state – understanding and mitigating](#query-acceleration-unhealthy-state--understanding-and-mitigating).
96+
97+
98+
#### Query acceleration policy was enabled recently
99+
100+
When the query acceleration policy is enabled for the first time, building the initial catalog needs to complete before it can be used in queries. During this period, the `LastUpdatedDateTime` value is empty:
101+
102+
```kusto
103+
.show external table [ETName] details
104+
| project todynamic(QueryAccelerationState).LastUpdatedDateTime
105+
```
106+
107+
If `LastUpdatedDateTime` is empty, allow some time for the first update to complete. This usually takes up to several minutes. Subsequent updates are expected to be significantly faster.
108+
109+
#### Query acceleration unhealthy state – understanding and mitigating
110+
111+
When an external table's query acceleration is unhealthy, you can retrieve the unhealthy reason using the following command:
112+
113+
```kusto
114+
.show external table [ETName] details
115+
| project todynamic(QueryAccelerationState).NotHealthyReason
116+
```
117+
118+
Use the following table to understand and mitigate common unhealthy states.
119+
120+
> [!NOTE]
121+
>
122+
> To re-enable an external table's query acceleration policy, run the following commands:
123+
>```kusto
124+
> .execute database script <|
125+
> .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":false}'
126+
> .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":true}'
127+
>```
128+
129+
130+
::: moniker range="azure-data-explorer"
131+
132+
| Unhealthy reason | Example `NotHealthyReason` | Action |
133+
| --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
134+
| External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Verify the connection string of the external table, including the authentication method, and that the permissions on the underlying storage are correct. |
135+
| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table’s connection string so it targets a valid Delta table location. Make sure the path points to the table’s root folder - not the _delta_log directory. |
136+
| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table and re-enable the query acceleration policy. |
137+
| Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. |
138+
| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. |
139+
| Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn` (a column of type `datetime` in the delta table), or leave the property empty if not required. |
140+
| Delta table has one of the following unsupported features for query acceleration <br/>• Column mapping mode 'Name' <br/>• AddFile transactions referencing files with absolute path <br/>• deletion vectors with absolute path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. |
141+
| Managed identity error | `Managed identity must be specified for external tables with impersonation authentication.` | Ensure that the query acceleration policy contains a valid managed identity that has:<br/>• Appropriate permissions on the Delta table<br/>• The `AutomatedFlows` usage type in the cluster or database managed identity policy. |
142+
143+
::: moniker-end
144+
145+
::: moniker range="microsoft-fabric"
146+
147+
| Unhealthy reason | Example `NotHealthyReason` | Action |
148+
| --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
149+
| External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Verify the connection string of the external table, including the authentication method, and that the permissions on the underlying storage are correct. |
150+
| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table’s connection string so it targets a valid Delta table location. Make sure the path points to the table’s root folder - not the _delta_log directory. |
151+
| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table and re-enable the query acceleration policy. |
152+
| Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. |
153+
| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. |
154+
| Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn` (a column of type `datetime` in the delta table), or leave the property empty if not required. |
155+
| Delta table has one of the following unsupported features for query acceleration <br/>• Column mapping mode 'Name' <br/>• AddFile transactions referencing files with absolute path <br/>• deletion vectors with absolute path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. |
156+
157+
::: moniker-end
158+
159+
160+
## Check if query is over nonaccelerated data
161+
162+
To fully benefit from query acceleration, queries must be executed over **accelerated data**. Non-accelerated data is read directly from the remote delta table, which may result in significant latency.
163+
164+
Use the following command and filter on a time frame that includes the relevant query:
165+
166+
```kusto
167+
.show queries
168+
| where StartedOn > ago(1h)
169+
| extend ExternalDataStats = OverallQueryStats.input_dataset_statistics.external_data
170+
```
171+
172+
If `ExternalDataStats.iterated_artifacts` or `ExternalDataStats.downloaded_items` are greater than `0`, it means data was read from the remote delta table (non-accelerated path).
173+
The following section helps you understand why.
174+
175+
### Troubleshoot queries over nonaccelerated data
176+
177+
There are two main reasons why a query might read non-accelerated data:
178+
179+
- The query-time filter isn't fully within the query acceleration hot period or hot windows.
180+
- The data within the policy hot period isn't fully cached.
181+
182+
#### Query filter isn't fully within the hot period or hot windows
183+
184+
Run the following command to view the hot caching properties and make sure the query filters match them:
185+
186+
```kusto
187+
.show external table [ETName] policy query_acceleration
188+
| project Policy = todynamic(Policy)
189+
| project Policy.Hot, Policy.HotWindows
190+
```
191+
192+
Ensure your query's time filter is fully contained within the configured `Hot` period or the defined `HotWindows`.
193+
194+
If it's a one-time query, policy change isn't recommended. However, if you anticipate running multiple queries over the same time range that lies outside the configured `Hot` period or defined `HotWindows` and require improved performance, alter the policy by:
195+
196+
- Increasing the hot period, and/or
197+
- Adding additional hot windows that match your query patterns.
198+
199+
#### Data within the hot period isn't fully cached
200+
201+
Use the following command to check the acceleration progress:
202+
203+
```kusto
204+
.show external table [ETName] details
205+
| project state = todynamic(QueryAccelerationState)
206+
| project state.CompletionPercentage, state.PendingDataFilesCount
207+
```
208+
209+
- If `CompletionPercentage < 100`, allow more time for data to be accelerated.
210+
- If `CompletionPercentage` doesn't increase over time, follow the guidance in [Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues).
211+
212+
### Understanding and mitigating data acceleration issues
213+
214+
Unaccelerated data (`CompletionPercentage < 100`) can stem from several issues.
215+
216+
#### Data is currently being accelerated
217+
218+
Data acceleration might take time, especially when:
219+
220+
- A query acceleration policy has recently been enabled, or
221+
- A significant amount of data was recently added to the delta table, or
222+
- The delta table has undergone an optimization operation such as `OPTIMIZE` that results in many deleted and recreated files.
223+
224+
Frequently running `OPTIMIZE` or `MERGE` operations on the source delta table that cause large-scale rewrites of data files can negatively affect acceleration performance because data files are repeatedly rewritten, and need to be accelerated.
225+
226+
#### Data files aren't eligible for acceleration
227+
228+
Parquet data files larger than **1 GB** won't be cached.
229+
230+
If your delta table includes many large files, consider adjusting your data generation or optimization strategy to produce smaller parquet files.
231+
If this requires recreating the Delta table, make sure you recreate the external table and re-enable query acceleration policy.
232+
233+
#### Insufficient cluster capacity or resources
234+
235+
Query acceleration operations are restricted by the cluster's available query acceleration capacity.
236+
237+
Run the following command to view the remaining capacity:
238+
239+
```kusto
240+
.show capacity
241+
| where Resource == 'QueryAcceleration'
242+
| project Remaining
243+
```
244+
245+
- If `Remaining == 0` consistently and `CompletionPercentage` isn't increasing, consider:
246+
247+
- Scaling the cluster out or up to provide more resources.
248+
- Increasing the `QueryAcceleration` capacity by [altering the capacity policy](alter-capacity-policy-command.md). NOTE: altering the capacity policy may have adverse effects on other operations. Alter the policy as a last resort at your own discretion.
249+

‎data-explorer/kusto/management/toc.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -586,6 +586,9 @@ items:
586586
- name: Query acceleration policy
587587
href: query-acceleration-policy.md
588588
displayName: external table, delta table
589+
- name: Query acceleration troubleshooting guide
590+
href: query-acceleration-tsg.md
591+
displayName: external table, delta table
589592
- name: .alter query acceleration policy command
590593
href: alter-query-acceleration-policy-command.md
591594
displayName: external table, delta table

0 commit comments

Comments
 (0)