You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The `.update table` command performs data updates in a specified table by deleting and appending records atomically.
13
+
There are two options for appending the records:
14
+
15
+
- Ingest the records based on a provided query. The query is noted using the *AppendIdentifier*.
16
+
- Move extents containing the records to append from another table to the target table. For more information, see [Update using move extents](#update-using-move-extents).
13
17
14
18
> [!WARNING]
15
19
> This command is unrecoverable.
@@ -27,9 +31,20 @@ You must have at least [Table Admin](../access-control/role-based-access-control
@@ -39,21 +54,25 @@ You must have at least [Table Admin](../access-control/role-based-access-control
39
54
| *TableName* | `string` | :heavy_check_mark: | The name of the table to update.
40
55
|*DeleteIdentifier*|`string`|:heavy_check_mark:| The identifier name used to specify the delete predicate applied to the updated table. |
41
56
|*DeletePredicate*|`string`|:heavy_check_mark:| The text of a query whose results are used as data to delete. The predicate has the same [limitations as the soft delete predicate](../concepts/data-soft-delete.md#limitations-and-considerations). |
42
-
|*AppendIdentifier*|`string`|:heavy_check_mark:| The identifier name used to specify the append predicate applied to the updated table. |
43
-
|*AppendPredicate*|`string`|:heavy_check_mark:| The text of a query whose results are used as data to append. |
57
+
|*AppendIdentifier*|`string`|| The identifier name used to specify the append predicate applied to the updated table. Required if using update based on ingest from query. |
58
+
|*AppendPredicate*|`string`|| The text of a query whose results are used as data to append. Required if using update based on ingest from query. |
59
+
|*SourceTableName*|`string`|| The name of the table to move extents from. Must be a local table in current database. Required if using [Update using move extents](#update-using-move-extents).|
44
60
45
61
> [!IMPORTANT]
62
+
>
46
63
> * Both delete and append predicates can't use remote entities, cross-db, and cross-cluster entities. Predicates can't reference an external table or use the `externaldata` operator.
47
-
> * Append and delete queries are expected to produce deterministic results. Nondeterministic queries can lead to unexpected results. A query is deterministic if and only if it would return the same data if executed multiple times.
64
+
> * Append and delete queries are expected to produce deterministic results. Nondeterministic queries can lead to unexpected results. A query is deterministic if and only if it would return the same data if executed multiple times.
48
65
> * For example, use of [`take` operator](../query/take-operator.md), [`sample` operator](../query/sample-operator.md), [`rand` function](../query/rand-function.md), and other such operators isn't recommended because these operators aren't deterministic.
49
66
> * Queries might be executed more than once within the `update` execution. If the intermediate query results are inconsistent, the update command can produce unexpected results.
67
+
> * The delete and append predicates are based on the same snapshot of the table, and therefore they can't depend on each other. In other words, the append predicate executes on a snapshot of the source table *before* the deletion and vice versa - the delete predicate executes on a snapshot of the source table *before* the append.
|*whatif*| bool | If `true`, returns the number of records that will be appended / deleted in every shard, without appending / deleting any records. The default is `false`. |
56
-
|*distributed*| bool | If `true`, the command ingests from all nodes executing the query in parallel. Default is `false`. See [performance tips](#performance-tips). |
73
+
|`whatif`| bool | If `true`, returns the number of records that are appended / deleted in every shard, without appending / deleting any records. The default is `false`. |
74
+
|`distributed`| bool | If `true`, the command ingests from all nodes executing the query in parallel. This option is relevant only for update based on ingest from query. Default is `false`. See [performance tips](#performance-tips). |
75
+
|`extentTagsToMove`| string | Optional [Extent tags](extent-tags.md) to filter only specific extents, when using update using move extents. The tags are provided as an array, in the same format as in [Ingestion properties](../ingestion-properties.md). See examples in [Update using move extents](#update-using-move-extents).|
57
76
58
77
> [!IMPORTANT]
59
78
> We recommend running in `whatif` mode first before executing the update to validate the predicates before deleting or appending data.
@@ -69,9 +88,15 @@ The result of the command is a table where each record represents an [extent](ex
69
88
| ExtentId |`guid`| The unique identifier for the extent created or deleted by the command. |
70
89
| RowCount |`long`| The number of rows created or deleted in the specified extent by the command. |
71
90
91
+
## Update using move extents
92
+
93
+
When you use this option, the records to append to target table are moved from the provided *SourceTableName* using [move extents](move-extents.md). The update moves *all* extents from the table, or only those that match the provided *extentTagsToMove*, if *extentTagsToMove* are specified in the command properties. The source table must be a local table in the current database and must the same schema as the target table.
94
+
95
+
The move extents option is useful when the records to append are already ingested to a staging table in the database, and should replace existing records in the target table. In this case, using the move extents option is more efficient than ingest from query, as this option doesn't require reingesting the records. See examples in [Update rows from a staging table using move extents](#update-rows-from-a-staging-table-using-move-extents).
96
+
72
97
## Choose between `.update table` and materialized views
73
98
74
-
There are scenarios where you could use either the `.update table` command or a [materialized view](materialized-views/materialized-view-overview.md) to achieve the same goal in a table. For example, a materialized view could be used to keep the latest version of each record or an update could be used to update records when a new version is available.
99
+
There are scenarios where you could use either the `.update table` command or a [materialized view](materialized-views/materialized-view-overview.md) to achieve the same goal in a table. For example, a materialized view could be used to keep the latest version of each record or an update could be used to update records when a new version is available.
75
100
76
101
Use the following guidelines to decide which method to use:
77
102
@@ -81,9 +106,9 @@ Use the following guidelines to decide which method to use:
81
106
82
107
## Performance tips
83
108
84
-
* Data ingestion is a resource-intensive operation that might affect concurrent activities on the cluster, including running queries. We recommend that you avoid the following resource-intensive actions: running many `.update` commands at once, and intensive use of the *distributed* property.
109
+
* Data ingestion is a resource-intensive operation that might affect concurrent activities on the cluster, including running queries. We recommend that you avoid the following resource-intensive actions: running many `.update` commands at once, and intensive use of the *distributed* property.
85
110
* Limit the append data to less than 1 GB per operation. If necessary, use multiple update commands.
86
-
* Set the `distributed` flag to `true` only if the amount of data being produced by the query is large, exceeds 1 GB and doesn't require serialization: multiple nodes can then produce output in parallel. Don't use this flag when query results are small, since it might needlessly generate many small data shards.
111
+
* Set the `distributed` flag to `true` only if the amount of data being produced by the query is large, exceeds 1 GB, and doesn't require serialization: multiple nodes can then produce output in parallel. Don't use this flag when query results are small, since it might needlessly generate many small data shards.
87
112
88
113
## Examples
89
114
@@ -163,7 +188,6 @@ The following example updates multiple columns on all rows with color gray.
163
188
| extend Color="";
164
189
```
165
190
166
-
167
191
### Update rows using another table (reference values)
168
192
169
193
In this example, the first step is to create the following mapping table:
@@ -206,7 +230,7 @@ Sometimes values to update are known without being stored in a table, and the [d
206
230
| where true;
207
231
```
208
232
209
-
### Update rows with a staging table
233
+
### Update rows from a staging table using move extents
210
234
211
235
A popular pattern is to first land data in a staging table before updating the main table.
212
236
@@ -220,17 +244,47 @@ The first command creates a staging table:
220
244
| extend Color = tostring(dynamic(["Red", "Blue", "Gray"])[Id %3])
221
245
```
222
246
223
-
The next command updates the main table with the data in the staging table:
247
+
The next command updates the main table with the data in the staging table, by moving all extents from staging table to main table:
224
248
225
249
```kusto
226
-
.update table Employees delete D append A <|
227
-
let A = MyStagingTable;
250
+
.update table Employees delete D move MyStagingTable <|
228
251
let D = Employees
229
252
| join kind=leftsemi MyStagingTable on Id
230
253
| where true;
231
254
```
232
255
233
-
Some records in the staging table didn't exist in the main table (that is, had `Id>100`) but were still inserted in the main table (upsert behavior).
256
+
Some records in the staging table didn't exist in the main table (that is, had `Id>100`) but were still inserted in the main table (upsert behavior). Since the command uses move extents, MyStagingTable will be empty after the update.
257
+
258
+
#### Update from a staging table using move extents with extent tags
259
+
260
+
In this example, the extents in the staging table include extent tags, and we're only interested in updating based on a subset of the tags:
This advanced schema discovery query identifies all property names that exist across nodes of each label type. Unlike the previous query that shows the schema structure, this query aggregates property names across all nodes of the same type, helping you understand which properties are consistently available and which might be optional or rare. This example uses the [`LDBC_SNB_Interactive` graph](graph-sample-data.md#ldbc-snb-interactive) to explore the complete property landscape of different entity types in the social network dataset.
348
+
349
+
:::moniker range="azure-data-explorer"
350
+
> [!div class="nextstepaction"]
351
+
> <ahref="https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAA22PP2%2BDQAzF93wKbzmkIKXqnCXpUinKkjGqkAMOXOH%2ByOempeLDlwOUUAkvd9b7Pfu5ZPSVWh%2Ff9ofsfNpn71aIMRd9p3Wy6qCMempQ8gqUdQUlK%2BjLs%2FukXOLriUVTgB1EeQMNXqmJ7fiZTB2Ye4reN%2B3c4yyoYV6sHqAfj7aAWttih8w4hx9cB98VMYEO1gkZL616UpeXj2RGhi9jkPUv%2FU9qsKYskMyN28k4ZZ2iDEc8jgFxIK0nd1NBWNsywss7FlYkcG3HQb1Leh1e%2FwDFz22ZfwEAAA%3D%3D"target="_blank">Run the query</a>
352
+
::: moniker-end
353
+
354
+
```kusto
355
+
graph('LDBC_SNB_Interactive')
356
+
| graph-match (node)
357
+
project properties = node, labels = labels(node)
358
+
| mv-apply properties on (
359
+
mv-expand kind=array properties
360
+
| where isnotempty(properties[1])
361
+
| summarize properties = make_set(properties[0])
362
+
)
363
+
| mv-expand label = labels to typeof(string)
364
+
| summarize properties =make_set(properties) by label
This query performs schema discovery for edge (relationship) properties, showing what information is stored with each type of relationship in your graph. Understanding edge properties is crucial for analyzing relationship metadata such as timestamps, weights, confidence scores, or other attributes that provide context about connections. This example uses the [`BloodHound_AD` graph](graph-sample-data.md#bloodhound-active-directory-dataset) to explore the properties available on different types of Active Directory privilege relationships.
377
+
378
+
:::moniker range="azure-data-explorer"
379
+
> [!div class="nextstepaction"]
380
+
> <ahref="https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAA22PywrCQAxF9%2F2K7JwBC4rrLhQX%2FoOIpG1sx3YezMRHpR9vR4tWMJsEcu7NTeXR1WK2aa0td%2FZiyuN6O5NJD1VcpBq5qEHIdE%2BHVMgEhnLenqng2B15VhQgA5pDizm1cX4PgqKNvqboXNtNaWtAvJxiDQDdHZoSGmXKDL3HKfzherjV5AlUMJZJO%2B7El9ovD3JChovW6NWDfjNqbOgYiKfCxSgcs45RXh98PgG2wJ0jexKBvTJVhP%2Ff%2BHNCQt69jQYVD3tYPQGByglydQEAAA%3D%3D"target="_blank">Run the query</a>
381
+
::: moniker-end
382
+
383
+
```kusto
384
+
graph('BloodHound_AD')
385
+
| graph-match ()-[e]-()
386
+
project properties = e, labels = labels(e)
387
+
| mv-apply properties on (
388
+
mv-expand kind=array properties
389
+
| where isnotempty(properties[1])
390
+
| summarize properties = make_set(properties[0])
391
+
)
392
+
| mv-expand label = labels to typeof(string)
393
+
| summarize properties =make_set(properties) by label
Use this pattern to locate entities with particular characteristics or to validate data quality by checking for expected property values. This example uses the [`BloodHound_Entra` graph](graph-sample-data.md#bloodhound-entra-dataset) to find nodes with specific name properties in Microsoft Entra environments.
Understanding the overall topology of your graph reveals the types of connections that exist between different entity types. This analysis helps you understand the data model, identify the most common relationship patterns, and discover potential paths for traversal queries. The topology query shows which node labels connect to which other node labels through specific edge types, providing a comprehensive view of your graph's structure.
429
+
430
+
:::moniker range="azure-data-explorer"
431
+
> [!div class="nextstepaction"]
432
+
> <ahref="https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAA32QwW7CMAyG730K35pKi3gCNgnYTtxA4oAmFFLTZmrjKHHZinj4me6wQqf5Esv29%2FuPZ7MtBWqo6oFOwDVCFU2oQcOuNpwnsOQ9WsYSmOBTai%2FZMKHy9WqxPLw5b7x1psmL7PrD6tawrUGlaAu9x3f9rMrERQYSIdKHqMGGumhxbY7YJJhDMyQD8QSvZTXpoNRXmFi2sSP%2F2B70r9CeNX4F48tfDZkZ6ckXuA9IJ5U4Ol%2FdQyNTgt1Z%2FB98dCb01OxfEqlrWxPdBWFJnWfh7O1VBRz7sYHRUaZ3%2BAY9FMKWwwEAAA%3D%3D"target="_blank">Run the query</a>
433
+
::: moniker-end
434
+
435
+
```kusto
436
+
//Topology of the graph - What's connected to what?
![learn-build-service-prod[bot]](https://avatars.githubusercontent.com/in/237442?v=4&size=40){kind=avatar}
0 commit comments