Repo sync for protected branch

data-explorer/kusto/management/update-table-command.md

@@ -1,15 +1,19 @@
 ---
-title:  .update table command
-description: Learn how to use the .update table command to perform transactional data updates.
+title:  The update table command
+description: Learn how to use update table command to perform transactional data updates.
 ms.reviewer: vplauzon
 ms.topic: reference
 ms.date: 11/18/2024
 ---
-# .update table command
+# `.update table` command
 
 > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)]
 
 The `.update table` command performs data updates in a specified table by deleting and appending records atomically.
+There are two options for appending the records:
+
+- Ingest the records based on a provided query. The query is noted using the *AppendIdentifier*.
+- Move extents containing the records to append from another table to the target table. For more information, see [Update using move extents](#update-using-move-extents).
 
 > [!WARNING]
 > This command is unrecoverable.
@@ -27,9 +31,20 @@ You must have at least [Table Admin](../access-control/role-based-access-control
 
 [!INCLUDE [syntax-conventions-note](../includes/syntax-conventions-note.md)]
 
-`.update` `[async]` `table` *TableName* `delete` *DeleteIdentifier* `append` *AppendIdentifier* [`with` `(` *propertyName* `=` *propertyValue* `)`] `<|` <br>
-`let` *DeleteIdentifier*`=` *DeletePredicate*`;` <br>
-`let` *AppendIdentifier*`=` *AppendPredicate*`;`
+**Update using `append` and `delete`:**
+
+```kusto
+.update [async] table TableName delete DeleteIdentifier append AppendIdentifier [with ( propertyName = propertyValue )] <|
+let DeleteIdentifier = DeletePredicate;
+let AppendIdentifier = AppendPredicate;
+```
+
+**Update using `move extents` and `delete`:**
+
+```kusto
+.update [async] table TableName delete DeleteIdentifier move SourceTableName [with ( *propertyName = propertyValue )] <|
+let DeleteIdentifier = DeletePredicate;
+```
 
 ### Parameters
 
@@ -39,21 +54,25 @@ You must have at least [Table Admin](../access-control/role-based-access-control
 | *TableName*        | `string` | :heavy_check_mark: | The name of the table to update.
 | *DeleteIdentifier* | `string` | :heavy_check_mark: | The identifier name used to specify the delete predicate applied to the updated table.                                                                                                                                                              |
 | *DeletePredicate*  | `string` | :heavy_check_mark: | The text of a query whose results are used as data to delete. The predicate has the same [limitations as the soft delete predicate](../concepts/data-soft-delete.md#limitations-and-considerations). |
-| *AppendIdentifier* | `string` | :heavy_check_mark: | The identifier name used to specify the append predicate applied to the updated table.                                                                                                                                                              |
-| *AppendPredicate*  | `string` | :heavy_check_mark: | The text of a query whose results are used as data to append.                                                                                                                                                               |
+| *AppendIdentifier* | `string` || The identifier name used to specify the append predicate applied to the updated table. Required if using update based on ingest from query.  |
+| *AppendPredicate*  | `string` || The text of a query whose results are used as data to append. Required if using update based on ingest from query. |
+| *SourceTableName*  | `string` || The name of the table to move extents from. Must be a local table in current database. Required if using [Update using move extents](#update-using-move-extents).|
 
 > [!IMPORTANT]
+>
 > * Both delete and append predicates can't use remote entities, cross-db, and cross-cluster entities. Predicates can't reference an external table or use the `externaldata` operator.
-> * Append and delete queries are expected to produce deterministic results.  Nondeterministic queries can lead to unexpected results. A query is deterministic if and only if it would return the same data if executed multiple times.
+> * Append and delete queries are expected to produce deterministic results. Nondeterministic queries can lead to unexpected results. A query is deterministic if and only if it would return the same data if executed multiple times.
 >    * For example, use of [`take` operator](../query/take-operator.md), [`sample` operator](../query/sample-operator.md), [`rand` function](../query/rand-function.md), and other such operators isn't recommended because these operators aren't deterministic.
 > * Queries might be executed more than once within the `update` execution. If the intermediate query results are inconsistent, the update command can produce unexpected results.
+> * The delete and append predicates are based on the same snapshot of the table, and therefore they can't depend on each other. In other words, the append predicate executes on a snapshot of the source table *before* the deletion and vice versa - the delete predicate executes on a snapshot of the source table *before* the append.
 
 ## Supported properties
 
 | Name     | Type | Description                                                                                                                                                |
 | -------- | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| *whatif* | bool | If `true`, returns the number of records that will be appended / deleted in every shard, without appending / deleting any records. The default is `false`. |
-| *distributed* | bool | If `true`, the command ingests from all nodes executing the query in parallel. Default is `false`. See [performance tips](#performance-tips). |
+| `whatif` | bool | If `true`, returns the number of records that are appended / deleted in every shard, without appending / deleting any records. The default is `false`. |
+| `distributed` | bool | If `true`, the command ingests from all nodes executing the query in parallel. This option is relevant only for update based on ingest from query. Default is `false`. See [performance tips](#performance-tips). |
+| `extentTagsToMove` | string | Optional [Extent tags](extent-tags.md) to filter only specific extents, when using update using move extents. The tags are provided as an array, in the same format as in [Ingestion properties](../ingestion-properties.md). See examples in [Update using move extents](#update-using-move-extents).|
 
 > [!IMPORTANT]
 > We recommend running in `whatif` mode first before executing the update to validate the predicates before deleting or appending data.
@@ -69,9 +88,15 @@ The result of the command is a table where each record represents an [extent](ex
 | ExtentId | `guid`   | The unique identifier for the extent created or deleted by the command. |
 | RowCount | `long`   | The number of rows created or deleted in the specified extent by the command.    |
 
+## Update using move extents
+
+When you use this option, the records to append to target table are moved from the provided *SourceTableName* using [move extents](move-extents.md). The update moves *all* extents from the table, or only those that match the provided *extentTagsToMove*, if *extentTagsToMove* are specified in the command properties. The source table must be a local table in the current database and must the same schema as the target table.
+
+The move extents option is useful when the records to append are already ingested to a staging table in the database, and should replace existing records in the target table. In this case, using the move extents option is more efficient than ingest from query, as this option doesn't require reingesting the records. See examples in [Update rows from a staging table using move extents](#update-rows-from-a-staging-table-using-move-extents).
+
 ## Choose between `.update table` and materialized views
 
-There are scenarios where you could use either the `.update table` command or a [materialized view](materialized-views/materialized-view-overview.md) to achieve the same goal in a table.  For example, a materialized view could be used to keep the latest version of each record or an update could be used to update records when a new version is available.
+There are scenarios where you could use either the `.update table` command or a [materialized view](materialized-views/materialized-view-overview.md) to achieve the same goal in a table. For example, a materialized view could be used to keep the latest version of each record or an update could be used to update records when a new version is available.
 
 Use the following guidelines to decide which method to use:
 
@@ -81,9 +106,9 @@ Use the following guidelines to decide which method to use:
 
 ## Performance tips
 
-* Data ingestion is a resource-intensive operation that might affect concurrent activities on the cluster, including running queries.  We recommend that you avoid the following resource-intensive actions: running many `.update` commands at once, and intensive use of the *distributed* property.
+* Data ingestion is a resource-intensive operation that might affect concurrent activities on the cluster, including running queries. We recommend that you avoid the following resource-intensive actions: running many `.update` commands at once, and intensive use of the *distributed* property.
 * Limit the append data to less than 1 GB per operation. If necessary, use multiple update commands.
-* Set the `distributed` flag to `true` only if the amount of data being produced by the query is large, exceeds 1 GB and doesn't require serialization:  multiple nodes can then produce output in parallel.  Don't use this flag when query results are small, since it might needlessly generate many small data shards.
+* Set the `distributed` flag to `true` only if the amount of data being produced by the query is large, exceeds 1 GB, and doesn't require serialization:  multiple nodes can then produce output in parallel. Don't use this flag when query results are small, since it might needlessly generate many small data shards.
 
 ## Examples
 
@@ -163,7 +188,6 @@ The following example updates multiple columns on all rows with color gray.
       | extend Color="";
 ```
 
-
 ### Update rows using another table (reference values)
 
 In this example, the first step is to create the following mapping table:
@@ -206,7 +230,7 @@ Sometimes values to update are known without being stored in a table, and the [d
       | where true;
 ```
 
-### Update rows with a staging table
+### Update rows from a staging table using move extents
 
 A popular pattern is to first land data in a staging table before updating the main table.
 
@@ -220,17 +244,47 @@ The first command creates a staging table:
     | extend Color = tostring(dynamic(["Red", "Blue", "Gray"])[Id %3])
 ```
 
-The next command updates the main table with the data in the staging table:
+The next command updates the main table with the data in the staging table, by moving all  extents from staging table to main table:
 
 ```kusto
-.update table Employees delete D append A <|
-    let A = MyStagingTable;
+.update table Employees delete D move MyStagingTable <|
     let D = Employees
         | join kind=leftsemi MyStagingTable on Id
         | where true;
 ```
 
-Some records in the staging table didn't exist in the main table (that is, had `Id>100`) but were still inserted in the main table (upsert behavior).
+Some records in the staging table didn't exist in the main table (that is, had `Id>100`) but were still inserted in the main table (upsert behavior). Since the command uses move extents, MyStagingTable will be empty after the update.
+
+#### Update from a staging table using move extents with extent tags
+
+In this example, the extents in the staging table include extent tags, and we're only interested in updating based on a subset of the tags:
+
+```kusto
+.set-or-replace MyStagingTable with(tags='["drop-by:tag1"]')<|
+    range i from 70 to 100 step 5
+    | project Id=i
+    | extend Code = tostring(dynamic(["Customer", "Employee"])[Id %2])
+    | extend Color = tostring(dynamic(["Red", "Blue", "Gray"])[Id %3])
+
+.set-or-replace MyStagingTable with(tags='["drop-by:tag2"]')<|
+    range i from 100 to 150 step 5
+    | project Id=i
+    | extend Code = tostring(dynamic(["Customer", "Employee"])[Id %2])
+    | extend Color = tostring(dynamic(["Red", "Blue", "Gray"])[Id %3])
+```
+
+The following command updates the main table with the data from the first ingestion ("drop-by:tag1") in the staging table.
+The delete part must filter on the extent tags as well, if you would like to only delete records based on this tag.
+The entire MyStagingTable is considered for the delete query, not only the records in the extents with "drop-by:tag1" so the filter
+must be explicitly added to the delete query.
+
+```kusto
+.update table Employees delete D move MyStagingTable with (extentTagsToMove='["drop-by:tag1"]') <|
+    let D = Employees | where Id in (MyStagingTable | where extent_tags() has "drop-by:tag1" | project Id);
+```
+
+At the end of the command, MyStagingTable includes only the records from the second ingestion ("drop-by:tag2"), as the command
+moves the extents with `drop-by:tag1`.
 
 ### Compound key
 

data-explorer/kusto/query/graph-exploration-basics.md

@@ -342,6 +342,64 @@ graph('BloodHound_Entra')
 |---|
 |{<br>  "onpremsyncenabled": "bool",<br>  "system_tags": "string",<br>  "lastcollected": "string",<br>  "pwdlastset": "string",<br>  "usertype": "string",<br>  "userprincipalname": "string",<br>  "email": "string",<br>  "tenantid": "guid",<br>  "name": "string",<br>  "lastseen": "string",<br>  "displayname": "string",<br>  "enabled": "bool",<br>  "title": "string",<br>  "onpremid": "string",<br>  "objectid": "guid",<br>  "whencreated": "string"<br>}|
 
+**Find all properties of all nodes by label**:
+
+This advanced schema discovery query identifies all property names that exist across nodes of each label type. Unlike the previous query that shows the schema structure, this query aggregates property names across all nodes of the same type, helping you understand which properties are consistently available and which might be optional or rare. This example uses the [`LDBC_SNB_Interactive` graph](graph-sample-data.md#ldbc-snb-interactive) to explore the complete property landscape of different entity types in the social network dataset.
+
+:::moniker range="azure-data-explorer"
+> [!div class="nextstepaction"]
+> <a href="https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAA22PP2%2BDQAzF93wKbzmkIKXqnCXpUinKkjGqkAMOXOH%2ByOempeLDlwOUUAkvd9b7Pfu5ZPSVWh%2Ff9ofsfNpn71aIMRd9p3Wy6qCMempQ8gqUdQUlK%2BjLs%2FukXOLriUVTgB1EeQMNXqmJ7fiZTB2Ye4reN%2B3c4yyoYV6sHqAfj7aAWttih8w4hx9cB98VMYEO1gkZL616UpeXj2RGhi9jkPUv%2FU9qsKYskMyN28k4ZZ2iDEc8jgFxIK0nd1NBWNsywss7FlYkcG3HQb1Leh1e%2FwDFz22ZfwEAAA%3D%3D" target="_blank">Run the query</a>
+::: moniker-end
+
+```kusto
+graph('LDBC_SNB_Interactive')
+| graph-match (node)
+    project properties = node, labels = labels(node)
+| mv-apply properties on (
+        mv-expand kind=array properties
+        | where isnotempty(properties[1])
+        | summarize properties = make_set(properties[0])
+    )
+| mv-expand label = labels to typeof(string)
+| summarize properties =make_set(properties) by label
+| take 3
+```
+
+|label|properties|
+|---|---|
+|TAGCLASS|[<br>  "id",<br>  "node_id",<br>  "lbl",<br>  "name",<br>  "url"<br>]|
+|TAG|[<br>  "id",<br>  "node_id",<br>  "lbl",<br>  "name",<br>  "url"<br>]|
+|FORUM|[<br>  "id",<br>  "creationDate",<br>  "node_id",<br>  "lbl",<br>  "title"<br>]|
+
+**Find all properties of all edges by label**:
+
+This query performs schema discovery for edge (relationship) properties, showing what information is stored with each type of relationship in your graph. Understanding edge properties is crucial for analyzing relationship metadata such as timestamps, weights, confidence scores, or other attributes that provide context about connections. This example uses the [`BloodHound_AD` graph](graph-sample-data.md#bloodhound-active-directory-dataset) to explore the properties available on different types of Active Directory privilege relationships.
+
+:::moniker range="azure-data-explorer"
+> [!div class="nextstepaction"]
+> <a href="https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAA22PywrCQAxF9%2F2K7JwBC4rrLhQX%2FoOIpG1sx3YezMRHpR9vR4tWMJsEcu7NTeXR1WK2aa0td%2FZiyuN6O5NJD1VcpBq5qEHIdE%2BHVMgEhnLenqng2B15VhQgA5pDizm1cX4PgqKNvqboXNtNaWtAvJxiDQDdHZoSGmXKDL3HKfzherjV5AlUMJZJO%2B7El9ovD3JChovW6NWDfjNqbOgYiKfCxSgcs45RXh98PgG2wJ0jexKBvTJVhP%2Ff%2BHNCQt69jQYVD3tYPQGByglydQEAAA%3D%3D" target="_blank">Run the query</a>
+::: moniker-end
+
+```kusto
+graph('BloodHound_AD')
+| graph-match ()-[e]-()
+    project properties = e, labels = labels(e)
+| mv-apply properties on (
+        mv-expand kind=array properties
+        | where isnotempty(properties[1])
+        | summarize properties = make_set(properties[0])
+    )
+| mv-expand label = labels to typeof(string)
+| summarize properties =make_set(properties) by label
+| take 3
+```
+
+|label|properties|
+|---|---|
+|GetChangesAll|[<br>  "id",<br>  "lbl",<br>  "src",<br>  "dst",<br>  "properties",<br>  "lastseen"<br>]|
+|OwnsRaw|[<br>  "id",<br>  "lbl",<br>  "src",<br>  "dst",<br>  "properties",<br>  "lastseen"<br>]|
+|AddKeyCredentialLink|[<br>  "id",<br>  "lbl",<br>  "src",<br>  "dst",<br>  "properties",<br>  "lastseen"<br>]|
+
 **Find nodes with specific property values**:
 
 Use this pattern to locate entities with particular characteristics or to validate data quality by checking for expected property values. This example uses the [`BloodHound_Entra` graph](graph-sample-data.md#bloodhound-entra-dataset) to find nodes with specific name properties in Microsoft Entra environments.
@@ -365,6 +423,42 @@ graph('BloodHound_Entra')
 |10|[email protected]|
 |12|[email protected]|
 
+### Topology of the graph
+
+Understanding the overall topology of your graph reveals the types of connections that exist between different entity types. This analysis helps you understand the data model, identify the most common relationship patterns, and discover potential paths for traversal queries. The topology query shows which node labels connect to which other node labels through specific edge types, providing a comprehensive view of your graph's structure.
+
+:::moniker range="azure-data-explorer"
+> [!div class="nextstepaction"]
+> <a href="https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAA32QwW7CMAyG730K35pKi3gCNgnYTtxA4oAmFFLTZmrjKHHZinj4me6wQqf5Esv29%2FuPZ7MtBWqo6oFOwDVCFU2oQcOuNpwnsOQ9WsYSmOBTai%2FZMKHy9WqxPLw5b7x1psmL7PrD6tawrUGlaAu9x3f9rMrERQYSIdKHqMGGumhxbY7YJJhDMyQD8QSvZTXpoNRXmFi2sSP%2F2B70r9CeNX4F48tfDZkZ6ckXuA9IJ5U4Ol%2FdQyNTgt1Z%2FB98dCb01OxfEqlrWxPdBWFJnWfh7O1VBRz7sYHRUaZ3%2BAY9FMKWwwEAAA%3D%3D" target="_blank">Run the query</a>
+::: moniker-end
+
+```kusto
+//Topology of the graph - What's connected to what?
+graph('LDBC_Financial')
+| graph-match (src)-[e]->(dst)
+    project SourceLabels = labels(src), EdgeLabels = labels(e), DestinationLabels = labels(dst)
+| mv-expand EdgeLabel = EdgeLabels to typeof(string)
+| mv-expand SourceLabel = SourceLabels to typeof(string)
+| mv-expand DestinationLabel = DestinationLabels to typeof(string)
+| summarize Count = count() by SourceLabel, EdgeLabel, DestinationLabel
+```
+
+|SourceLabel|EdgeLabel|DestinationLabel|Count|
+|---|---|---|---|
+|COMPANY|GUARANTEE|COMPANY|202|
+|COMPANY|APPLY|LOAN|449|
+|PERSON|APPLY|LOAN|927|
+|ACCOUNT|REPAY|LOAN|2747|
+|LOAN|DEPOSIT|ACCOUNT|2758|
+|ACCOUNT|TRANSFER|ACCOUNT|8132|
+|ACCOUNT|WITHDRAW|ACCOUNT|9182|
+|PERSON|GUARANTEE|PERSON|377|
+|COMPANY|OWN|ACCOUNT|671|
+|COMPANY|INVEST|COMPANY|679|
+|PERSON|OWN|ACCOUNT|1384|
+|MEDIUM|SIGN_IN|ACCOUNT|2489|
+|PERSON|INVEST|COMPANY|1304|
+
 ## Related content
 
 - [Graph semantics overview](graph-semantics-overview.md)