diff --git a/data-explorer/kusto/access-control/role-based-access-control.md b/data-explorer/kusto/access-control/role-based-access-control.md index f944c9dfa0..cd112d49b5 100644 --- a/data-explorer/kusto/access-control/role-based-access-control.md +++ b/data-explorer/kusto/access-control/role-based-access-control.md @@ -73,11 +73,13 @@ The **Manage** column offers ways to add or remove role principals. | External Table | Admin | Full permission in the scope of a particular external table. | Database User or Database Viewer | [management commands](../management/manage-external-table-security-roles.md) | | Materialized view | Admin | Full permission to alter the view, delete the view, and grant admin permissions to another principal. | Database User or Table Admin | [management commands](../management/manage-materialized-view-security-roles.md) | | Function | Admin | Full permission to alter the function, delete the function, and grant admin permissions to another principal. | Database User or Table Admin | [management commands](../management/manage-function-security-roles.md) | +| Graph | GraphAdmin | Full permission in the scope of a particular graph model. | Database User | | ::: moniker-end ::: moniker range="microsoft-fabric" -| Scope | Role | Permissions | How the role is obtained | -| ----------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + +| Scope | Role | Permissions | How the role is obtained | +| ---- | ---- | ----------- | ------------------------- | | Eventhouse | AllDatabasesAdmin | Full permission to all databases in the Eventhouse. May show and alter certain Eventhouse-level policies. Includes all permissions. | - Inherited as workspace **admin**, workspace **member**, or workspace **contributor**.

Can't be assigned with management commands. | | Database | Admin | Full permission in the scope of a particular database. Includes all lower level permissions. | - Inherited as workspace **admin**, workspace **member**, or workspace **contributor**
- [Item shared](/fabric/get-started/share-items#item-permission-model) with editing permissions.
- Assigned with [management commands](../management/manage-database-security-roles.md) | | Database | User | Read all data and metadata of the database. Create tables and functions, and become the admin for those tables and functions. | - Assigned with [management commands](../management/manage-database-security-roles.md) | @@ -90,6 +92,7 @@ The **Manage** column offers ways to add or remove role principals. | External Table | Admin | Full permission in the scope of a particular external table. | - Assigned with [management commands](../management/manage-external-table-security-roles.md). Dependent on having **Database User** or **Database Viewer** on the parent database. | | Materialized view | Admin | Full permission to alter the view, delete the view, and grant admin permissions to another principal. | - Inherited as workspace **admin**, workspace **member**, or workspace **contributor**
- Parent item (KQL Database) [shared](/fabric/get-started/share-items#item-permission-model) with editing permissions.
- Assigned with [management commands](../management/manage-database-security-roles.md). Dependent on having **Database User** or **Table Admin** on the parent items. | | Function | Admin | Full permission to alter the function, delete the function, and grant admin permissions to another principal. | - Inherited as workspace **admin**, workspace **member**, or workspace **contributor**
- Parent item (KQL Database) [shared](/fabric/get-started/share-items#item-permission-model) with editing permissions.
- Assigned with [management commands](../management/manage-database-security-roles.md). Dependent on having **Database User** or **Table Admin** on the parent items. | +| Graph | GraphAdmin | Full permission in the scope of a particular graph model. | - Inherited as workspace **admin**, workspace **member**, or workspace **contributor**
- Parent item (KQL Database) [shared](/fabric/get-started/share-items#item-permission-model) with editing permissions.
- Assigned with [management commands](../management/manage-database-security-roles.md). Dependent on having **Database User** or **Table Admin** on the parent items. | | ::: moniker-end ## Related content diff --git a/data-explorer/kusto/management/capacity-policy.md b/data-explorer/kusto/management/capacity-policy.md index dc272bbc78..ad152f237e 100644 --- a/data-explorer/kusto/management/capacity-policy.md +++ b/data-explorer/kusto/management/capacity-policy.md @@ -27,6 +27,7 @@ The capacity policy is made of the following components: * [PurgeStorageArtifactsCleanupCapacity](#purge-storage-artifacts-cleanup-capacity) * [PeriodicStorageArtifactsCleanupCapacity](#periodic-storage-artifacts-cleanup-capacity) * [QueryAccelerationCapacity](#query-acceleration-capacity) +* [GraphSnapshotsCapacity](#graph-snapshots-capacity) To view the capacity of your cluster, use the [.show capacity](show-capacity-command.md) command. @@ -193,6 +194,18 @@ The [.show capacity](show-capacity-command.md) command returns the cluster's que `Minimum(ClusterMaximumConcurrentOperations` `,` *Number of nodes in cluster* `*` `Maximum(1,` *Core count per node* `*` `CoreUtilizationCoefficient))` +### Graph Snapshots capacity + +| Property | Type | Description | +|--|--|--| +| `ClusterMaximumConcurrentOperations` | `long` | The maximum number of concurrent snapshot creation operations on cluster. | + +**Formula** + +The [.show capacity](show-capacity-command.md) command returns the cluster's periodic storage artifacts cleanup capacity based on the following formula: + +`ClusterMaximumConcurrentOperations` + ## Defaults The default capacity policy has the following JSON representation: @@ -241,6 +254,9 @@ The default capacity policy has the following JSON representation: "QueryAccelerationCapacity": { "ClusterMaximumConcurrentOperations": 100, "CoreUtilizationCoefficient": 0.5 + }, + "GraphSnapshotsCapacity": { + "ClusterMaximumConcurrentOperations": 5 } } ``` diff --git a/data-explorer/kusto/management/graph/graph-model-create-or-alter.md b/data-explorer/kusto/management/graph/graph-model-create-or-alter.md index 490879dd9b..8f78df968a 100644 --- a/data-explorer/kusto/management/graph/graph-model-create-or-alter.md +++ b/data-explorer/kusto/management/graph/graph-model-create-or-alter.md @@ -1,6 +1,6 @@ --- title: .create-or-alter graph_model command -description: Learn how to create or alter a graph model using the .create-or-alter graph_model command with syntax, parameters, and examples. +description: Learn how to create or alter a graph model using the create-or-alter graph_model command with syntax, parameters, and examples. ms.reviewer: herauch ms.topic: reference ms.date: 05/24/2025 @@ -17,7 +17,7 @@ Creates a new graph model or alters an existing one using the provided model def ## Permissions -To run this command, the user needs [Database Admin permissions](../../access-control/role-based-access-control.md). +To run this command, the user needs [Database User permissions](../../access-control/role-based-access-control.md). ## Syntax @@ -106,13 +106,13 @@ This command returns a table with the following columns: |Name|CreationTime|ID|SnapshotsCount|Model|AuthorizedPrincipals|RetentionPolicy| |---|---|---|---|---|---|---| -|SocialNetwork|2025-05-23 14:42:37.5128901|aaaaaaaa-0b0b-1c1c-2d2d-333333333333|0|model from above|[
{
"Type": "AAD User",
"DisplayName": "Alex Johnson (upn: alex.johnson@contoso.com)",
"ObjectId": "aaaaaaaa-bbbb-cccc-1111-22222222222",
"FQN": "aaduser=aaaaaaaa-bbbb-cccc-1111-22222222222;aaaabbbb-0000-cccc-1111-dddd2222eeee",
"Notes": "",
"RoleAssignmentIdentifier": "a0a0a0a0-bbbb-cccc-dddd-e1e1e1e1e1e1"
}
]|{
"SoftDeletePeriod": "3650.00:00:00"
}| +|SocialNetwork|2025-05-23 14:42:37.5128901|aaaaaaaa-0b0b-1c1c-2d2d-333333333333|0|model from above|[
{
"Type": "Microsoft Entra ID User",
"DisplayName": "Alex Johnson (upn: alex.johnson@contoso.com)",
"ObjectId": "aaaaaaaa-bbbb-cccc-1111-22222222222",
"FQN": "aaduser=aaaaaaaa-bbbb-cccc-1111-22222222222;aaaabbbb-0000-cccc-1111-dddd2222eeee",
"Notes": "",
"RoleAssignmentIdentifier": "a0a0a0a0-bbbb-cccc-dddd-e1e1e1e1e1e1"
}
]|{
"SoftDeletePeriod": "3650.00:00:00"
}| ## Notes * If a graph model with the specified name doesn't exist, a new one is created when using `.create-or-alter graph_model`. If one already exists, it's updated with the new definition. * Each time a graph model is altered, a new version is created, allowing you to track changes over time and revert to previous versions if needed. -* To generate a graph snapshot from the model, use the [.make graph_snapshot](graph-snapshot-make.md) command. +* To generate a graph snapshot from the model, use the [`.make graph_snapshot`](graph-snapshot-make.md) command. ## Related content diff --git a/data-explorer/kusto/management/graph/graph-model-drop.md b/data-explorer/kusto/management/graph/graph-model-drop.md index 684829875b..7f72982456 100644 --- a/data-explorer/kusto/management/graph/graph-model-drop.md +++ b/data-explorer/kusto/management/graph/graph-model-drop.md @@ -17,7 +17,7 @@ Deletes an existing graph model and all its versions from the database, includin ## Permissions -To run this command, you need [Database admin permissions](../../access-control/role-based-access-control.md). +To run this command, you need [Database admin permissions](../../access-control/role-based-access-control.md) or [Graph admin permissions](../../access-control/role-based-access-control.md). ## Syntax diff --git a/data-explorer/kusto/management/graph/graph-model-overview.md b/data-explorer/kusto/management/graph/graph-model-overview.md index 4be6664f19..aea5f21d0e 100644 --- a/data-explorer/kusto/management/graph/graph-model-overview.md +++ b/data-explorer/kusto/management/graph/graph-model-overview.md @@ -47,18 +47,56 @@ A graph model consists of two main components: ### Schema (optional) -The schema defines the structure of the nodes and edges in the graph: +The schema defines the structure and properties of nodes and edges in the graph model. While optional, the schema serves several important purposes: -- **Nodes**: Defines the types of nodes in the graph and their properties -- **Edges**: Defines the types of relationships between nodes and their properties +- **Type safety**: Schema properties define the expected data types for node and edge properties, ensuring type consistency during graph queries +- **Property validation**: All properties defined in the schema become valid properties for nodes/edges with the corresponding labels, regardless of whether these properties appear in the step query columns +- **Query compatibility**: Schema properties can be safely referenced in graph-match queries without type collisions with step query columns + +#### Schema structure + +- **Nodes**: Defines node label types and their typed properties (e.g., `"Person": {"Name": "string", "Age": "long"}`) +- **Edges**: Defines edge label types and their typed properties (e.g., `"WORKS_AT": {"StartDate": "datetime", "Position": "string"}`) ### Definition -The Definition specifies how to build the graph from tabular data: +The Definition specifies how to build the graph from tabular data through a series of sequential operations. This section is the core of the graph model, as it transforms your relational data into a graph structure. + +#### Key characteristics of the Definition: + +* **Sequential execution**: Steps are executed in the exact order they appear in the Definition array. This order is critical because: + - Nodes must typically be created before edges that reference them + - Later steps can build upon or modify the results of earlier steps + - The sequence affects performance and memory usage during graph construction + +* **Incremental construction**: Each step adds to the graph being built, allowing you to: + - Combine data from multiple tables or sources + - Apply different logic for different types of nodes or edges + - Build complex graph structures incrementally + +#### Step types: + +* **AddNodes**: Steps that define how to create nodes from tabular data + - Can be used multiple times to add different types of nodes + - Each step can pull from different data sources or apply different filters + - Node properties are derived from the columns in the query result -* **Steps**: A sequence of operations to add nodes and edges to the graph - * **AddNodes**: Steps that define how to create nodes from tabular data - * **AddEdges**: Steps that define how to create edges from tabular data +* **AddEdges**: Steps that define how to create edges from tabular data + - Can reference nodes that don't yet exist (the system will create placeholder nodes and update them when AddNodes steps are processed later) + - Can create relationships between nodes from the same or different AddNodes steps + - Edge properties are derived from the columns in the query result + - While it's possible to add edges before nodes, it's recommended to add nodes first for better readability and understanding + +#### Execution flow example: + +``` +Step 1 (AddNodes): Create Person nodes from Employees table +Step 2 (AddNodes): Create Company nodes from Organizations table +Step 3 (AddEdges): Create WORKS_AT edges between Person and Company nodes +Step 4 (AddEdges): Create KNOWS edges between Person nodes +``` + +This sequential approach ensures that when Step 3 creates WORKS_AT edges, both the Person nodes (from Step 1) and Company nodes (from Step 2) already exist in the graph. ## Labels in Graph models @@ -278,8 +316,16 @@ To refresh a graph: ### What if different steps create duplicate edges or nodes? -- **Edges**: Duplicates remain as duplicates by default (edges don't have unique identifiers) -- **Nodes**: "Duplicates" are merged - the system assumes they represent the same entity. If there are conflicting property values, the last value processed takes precedence +The Definition steps execute sequentially, and duplicate handling differs between nodes and edges: + +- **Edges**: Duplicates remain as duplicates by default since edges don't have unique identifiers. If multiple steps create identical source-target relationships, each one becomes a separate edge in the graph. This behavior is intentional as multiple relationships between the same nodes can represent different interactions or events over time. + +- **Nodes**: "Duplicates" are automatically merged based on the NodeIdColumn value - the system assumes they represent the same entity. When multiple steps create nodes with the same identifier: + - All properties from different steps are combined into a single node + - If there are conflicting property values for the same property name, the value from the step that executed last takes precedence + - Properties that exist in one step but not another are preserved + +This merge behavior allows you to build nodes incrementally across steps, such as adding basic information in one step and enriching with additional properties in subsequent steps. ### How do graph models handle schema changes? diff --git a/data-explorer/kusto/management/graph/graph-snapshot-drop.md b/data-explorer/kusto/management/graph/graph-snapshot-drop.md index dcfea21f22..812b6ab20a 100644 --- a/data-explorer/kusto/management/graph/graph-snapshot-drop.md +++ b/data-explorer/kusto/management/graph/graph-snapshot-drop.md @@ -17,7 +17,7 @@ Deletes a specific graph snapshot from a graph model. ## Permissions -To run this command, the user needs [Database Admin permissions](../../access-control/role-based-access-control.md). +To run this command, the user needs [Database Admin permissions](../../access-control/role-based-access-control.md) or [Graph admin permissions](../../access-control/role-based-access-control.md). ## Syntax diff --git a/data-explorer/kusto/management/graph/graph-snapshot-make.md b/data-explorer/kusto/management/graph/graph-snapshot-make.md index 19238ec7a7..00d37bb2cd 100644 --- a/data-explorer/kusto/management/graph/graph-snapshot-make.md +++ b/data-explorer/kusto/management/graph/graph-snapshot-make.md @@ -17,7 +17,7 @@ Creates a new graph snapshot from a specified graph model. A graph snapshot is a ## Permissions -To run this command, the user needs [Database admin permissions](../../access-control/role-based-access-control.md). +To run this command, the user needs [Database admin permissions](../../access-control/role-based-access-control.md) or [Graph admin permissions](../../access-control/role-based-access-control.md). ## Syntax diff --git a/data-explorer/kusto/management/graph/graph-snapshot-show.md b/data-explorer/kusto/management/graph/graph-snapshot-show.md index 57f76b9edf..ca328ab935 100644 --- a/data-explorer/kusto/management/graph/graph-snapshot-show.md +++ b/data-explorer/kusto/management/graph/graph-snapshot-show.md @@ -17,7 +17,7 @@ Shows detailed information about a specific graph snapshot. ## Permissions -To run this command, the user needs [Database admin permissions](../../access-control/role-based-access-control.md). +To run this command, the user needs [Database viewer permissions](../../access-control/role-based-access-control.md). ## Syntax diff --git a/data-explorer/kusto/management/graph/graph-snapshots-drop.md b/data-explorer/kusto/management/graph/graph-snapshots-drop.md index e5330ab619..52615a238a 100644 --- a/data-explorer/kusto/management/graph/graph-snapshots-drop.md +++ b/data-explorer/kusto/management/graph/graph-snapshots-drop.md @@ -1,6 +1,6 @@ --- title: .drop graph_snapshots command -description: Learn how to delete all graph snapshots for a specific graph model using the .drop graph_snapshots command. +description: Learn how to delete all graph snapshots for a specific graph model using the drop graph_snapshots command. ms.reviewer: herauch ms.topic: reference ms.date: 05/24/2025 @@ -17,7 +17,7 @@ Deletes all graph snapshots associated with a specific graph model. ## Permissions -To run this command, you need [Database admin permissions](../../access-control/role-based-access-control.md). +To run this command, you need [Database admin permissions](../../access-control/role-based-access-control.md) or [Graph admin permissions](../../access-control/role-based-access-control.md). ## Syntax @@ -45,9 +45,9 @@ The command completes successfully without returning any output. ## Important notes -- The `.drop graph_snapshots` command permanently deletes all snapshots associated with a graph model. This operation cannot be undone. +- The `.drop graph_snapshots` command permanently deletes all snapshots associated with a graph model. This operation can't be undone. - Dropping snapshots doesn't affect the graph model itself. -- To drop a specific snapshot instead of all snapshots, use the [.drop graph_snapshot](graph-snapshot-drop.md) command. +- To drop a specific snapshot instead of all snapshots, use the [`.drop graph_snapshot`](graph-snapshot-drop.md) command. ## Next steps diff --git a/data-explorer/kusto/management/graph/graph-snapshots-show.md b/data-explorer/kusto/management/graph/graph-snapshots-show.md index be61b18a17..cfdb9901e0 100644 --- a/data-explorer/kusto/management/graph/graph-snapshots-show.md +++ b/data-explorer/kusto/management/graph/graph-snapshots-show.md @@ -17,7 +17,7 @@ Lists all graph snapshots for a specific graph model or for all graph models. ## Permissions -To run this command, the user needs [Database admin permissions](../../access-control/role-based-access-control.md). +To run this command, the user needs [Database viewer permissions](../../access-control/role-based-access-control.md). ## Syntax diff --git a/data-explorer/kusto/query/array-iff-function.md b/data-explorer/kusto/query/array-iff-function.md index 7e120e8e4c..ffb89dcade 100644 --- a/data-explorer/kusto/query/array-iff-function.md +++ b/data-explorer/kusto/query/array-iff-function.md @@ -3,13 +3,13 @@ title: array_iff() description: Learn how to use the array_iff() function to scan and evaluate elements in an array. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 06/04/2025 --- # array_iff() > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)] [!INCLUDE [sentinel](../includes/applies-to-version/sentinel.md)] -Element-wise iif function on dynamic arrays. +Element-wise `iif` function on dynamic arrays. > The `array_iff()` and `array_iif()` functions are equivalent @@ -24,15 +24,15 @@ Element-wise iif function on dynamic arrays. | Name | Type | Required | Description | |--|--|--|--| | *condition_array*| `dynamic` | :heavy_check_mark:| An array of *boolean* or numeric values.| -| *when_true* | dynamic or scalar | :heavy_check_mark: | An array of values or primitive value. This will be the result when *condition_array* is *true*.| -| *when_false* | dynamic or scalar | :heavy_check_mark: | An array of values or primitive value. This will be the result when *condition_array* is *false*.| +| *when_true* | dynamic or scalar | :heavy_check_mark: | An array of values or primitive value. It's the result when *condition_array* is *true*.| +| *when_false* | dynamic or scalar | :heavy_check_mark: | An array of values or primitive value. It's the result when *condition_array* is *false*.| > [!NOTE] > -> * The length of the return value will be the same as the input *condition_array*. +> * The length of the return value is same as the input *condition_array*. > * Numeric condition values are considered `true` if not equal to 0. -> * Non-numeric and non-boolean condition values will be null in the corresponding index of the return value. -> * If *when_true* or *when_false* is shorter than *condition_array*, missing values will be treated as null. +> * Non-numeric and non-boolean condition values is null in the corresponding index of the return value. +> * If *when_true* or *when_false* is shorter than *condition_array*, missing values are treated as null. ## Returns @@ -111,3 +111,8 @@ print condition=dynamic([true,true,true]), if_true=dynamic([1,2]), if_false=dyna |condition|if_true|if_false|res| |---|---|---|---| |[true, true, true]|[1, 2]|[3, 4]|[1, 2, null]| + +## Related content + +* [mv-apply](./mv-apply-operator.md) operator +* [mv-expand](./mv-expand-operator.md) operator diff --git a/data-explorer/kusto/query/graph-function.md b/data-explorer/kusto/query/graph-function.md index 5968590f75..72644800eb 100644 --- a/data-explorer/kusto/query/graph-function.md +++ b/data-explorer/kusto/query/graph-function.md @@ -12,7 +12,11 @@ ms.date: 05/28/2025 > [!NOTE] > This feature is currently in public preview. Functionality and syntax are subject to change before General Availability. -The `graph` function is an intrinsic function that enables querying of a persisted graph entity, similar to the `cluster()`, `database()`, `external_table()`, and `table()` functions. It supports retrieving either the most recent snapshot of the graph or a specific snapshot. +The `graph` function is an intrinsic function that enables querying of a persisted graph entity, similar to the `cluster()`, `database()`, `external_table()`, and `table()` functions. It supports retrieving either the most recent snapshot of the graph, a specific snapshot, or creating a transient graph from the model. + +## Permissions + +To run this function, the user needs [Database viewer permissions](../access-control/role-based-access-control.md). ## Syntax @@ -22,16 +26,23 @@ The `graph` function is an intrinsic function that enables querying of a persist `graph(` *GraphName* `,` `snapshot=` *SnapshotName* `)` +`graph(` *GraphName* `,` *Transient* `)` + ## Parameters | Name | Type | Required | Description | |----------------|----------|--------------------|-----------------------------------------------------------------------------| | *GraphName* | `string` | :heavy_check_mark: | The name of the [graph model](../management/graph/graph-model-overview.md) to query. | | *SnapshotName* | `string` | | The name of a specific snapshot to retrieve. If not specified, the most recent snapshot is used. | +| *Transient* | `bool` | | If `true`, creates a transient graph from the model (no snapshot is used). If `false`, uses the latest snapshot (same as omitting this parameter). | ## Returns -The `graph` function returns a graph and must be followed by a [graph operator](graph-operators.md#supported-graph-operators). The function retrieves the specified graph model name, either as the latest snapshot or a specific named snapshot. +The `graph` function returns a graph and must be followed by a [graph operator](graph-operators.md#supported-graph-operators). The function retrieves the specified graph model name, either as: + +- The latest snapshot (default or when `false` is specified) +- A specific named snapshot +- A transient graph from the model (when `true` is specified) ## Examples @@ -68,6 +79,28 @@ graph("SecurityGraph", snapshot="Snapshot_2025_05_01") project PathLength = array_length(e), Path = e ``` +### Create a transient graph from the model + +The following example creates a transient graph from the model, similar to the `make-graph` operator: + +```kusto +graph("SecurityGraph", true) +| graph-match (user)-[permission]->(resource) + where user.type == "User" and resource.type == "Database" + project UserName = user.name, ResourceName = resource.name, Permission = permission.type +``` + +### Use false to specify latest snapshot + +The following example explicitly specifies `false` to use the latest snapshot, which is equivalent to omitting the second parameter: + +```kusto +graph("SecurityGraph", false) +| graph-match (user)-[permission]->(resource) + where user.type == "User" and resource.type == "Database" + project UserName = user.name, ResourceName = resource.name, Permission = permission.type +``` + ## Related content * [Graph semantics overview](graph-semantics-overview.md) diff --git a/data-explorer/kusto/query/make-series-operator.md b/data-explorer/kusto/query/make-series-operator.md index 0ce49e0ac8..9af5ef945c 100644 --- a/data-explorer/kusto/query/make-series-operator.md +++ b/data-explorer/kusto/query/make-series-operator.md @@ -3,7 +3,7 @@ title: make-series operator description: Learn how to use the make-series operator to create a series of specified aggregated values along a specified axis. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 06/04/2025 ms.localizationpriority: high --- # make-series operator @@ -27,17 +27,17 @@ Create series of specified aggregated values along a specified axis. | Name | Type | Required | Description | |--|--|--|--| |*Column*| `string` | | The name for the result column. Defaults to a name derived from the expression.| -|*DefaultValue* | scalar | | A default value to use instead of absent values. If there's no row with specific values of *AxisColumn* and *GroupExpression*, then the corresponding element of the array will be assigned a *DefaultValue*. Default is 0.| +|*DefaultValue* | scalar | | A default value to use instead of absent values. If there's no row with specific values of *AxisColumn* and *GroupExpression*, then the corresponding element of the array is assigned a *DefaultValue*. Default is 0.| |*Aggregation*| `string` | :heavy_check_mark: | A call to an [aggregation function](make-series-operator.md#list-of-aggregation-functions), such as `count()` or `avg()`, with column names as arguments. See the [list of aggregation functions](make-series-operator.md#list-of-aggregation-functions). Only aggregation functions that return numeric results can be used with the `make-series` operator.| -|*AxisColumn*| `string` | :heavy_check_mark: | The column by which the series will be ordered. Usually the column values will be of type `datetime` or `timespan` but all numeric types are accepted.| -|*start* | scalar | :heavy_check_mark: | The low bound value of the *AxisColumn* for each of the series to be built. If *start* is not specified, it will be the first bin, or step, that has data in each series.| -|*end*| scalar| :heavy_check_mark: | The high bound non-inclusive value of the *AxisColumn*. The last index of the time series is smaller than this value and will be *start* plus integer multiple of *step* that is smaller than *end*. If *end* is not specified, it will be the upper bound of the last bin, or step, that has data per each series.| +|*AxisColumn*| `string` | :heavy_check_mark: | The column by which the series is ordered. Usually the column values are of type `datetime` or `timespan` but all numeric types are accepted.| +|*start* | scalar | :heavy_check_mark: | The low bound value of the *AxisColumn* for each of the series to be built. If *start* isn't specified, it's the first bin, or step, that has data in each series.| +|*end*| scalar| :heavy_check_mark: | The high bound non-inclusive value of the *AxisColumn*. The last index of the time series is smaller than this value and is *start* plus integer multiple of *step* that is smaller than *end*. If *end* isn't specified, it's the upper bound of the last bin, or step, that has data per each series.| |*step*| scalar | :heavy_check_mark: | The difference, or bin size, between two consecutive elements of the *AxisColumn* array. For a list of possible time intervals, see [timespan](scalar-data-types/timespan.md).| |*GroupExpression* | | |An expression over the columns that provides a set of distinct values. Typically it's a column name that already provides a restricted set of values. | |*MakeSeriesParameters*| | | Zero or more space-separated parameters in the form of *Name* `=` *Value* that control the behavior. See [supported make series parameters](#supported-make-series-parameters).| > [!NOTE] -> The *start*, *end*, and *step* parameters are used to build an array of *AxisColumn* values. The array consists of values between *start* and *end*, with the *step* value representing the difference between one array element to the next. All *Aggregation* values are ordered respectively to this array. +> The `start`, `end`, and `step` parameters are used to build an array of *AxisColumn* values. The array consists of values between `start` and `end`, with the `step` value representing the difference between one array element to the next. All *Aggregation* values are ordered respectively to this array. ### Supported make series parameters @@ -48,7 +48,7 @@ Create series of specified aggregated values along a specified axis. > [!NOTE] > -> The arrays generated by make-series are limited to 1048576 values (2^20). Trying to generate a larger array with make-series would result in either an error or a truncated array. +> The arrays generated by make-series are limited to 1,048,576 values (2^20). Trying to generate a larger array with make-series would result in either an error or a truncated array. **Alternate Syntax** @@ -61,9 +61,9 @@ Create series of specified aggregated values along a specified axis. The generated series from the alternate syntax differs from the main syntax in two aspects: * The *stop* value is inclusive. -* Binning the index axis is generated with bin() and not bin_at(), which means that *start* may not be included in the generated series. +* Binning the index axis is generated with bin() and not bin_at(), which means that *start* might not be included in the generated series. -It's recommended to use the main syntax of make-series and not the alternate syntax. +We recommend that you use the main syntax of make-series and not the alternate syntax. ## Returns @@ -219,7 +219,7 @@ data |---| |0| -Using `kind=nonempty` in `make-series` will produce a non-empty result of the default values: +Using `kind=nonempty` in `make-series` produces a non-empty result of the default values: :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -257,3 +257,36 @@ data |avg_metric|timestamp| |---|---| |[
1.0,
1.0,
1.0,
1.0,
1.0,
1.0,
1.0,
1.0,
1.0
]|[
"2017-01-01T00:00:00.0000000Z",
"2017-01-02T00:00:00.0000000Z",
"2017-01-03T00:00:00.0000000Z",
"2017-01-04T00:00:00.0000000Z",
"2017-01-05T00:00:00.0000000Z",
"2017-01-06T00:00:00.0000000Z",
"2017-01-07T00:00:00.0000000Z",
"2017-01-08T00:00:00.0000000Z",
"2017-01-09T00:00:00.0000000Z"
]| + +Using `make-series` and `mv-expand` to fill values for missing records: + +:::moniker range="azure-data-explorer" +> [!div class="nextstepaction"] +> Run the query +::: moniker-end + +```kusto +let startDate = datetime(2025-01-06); +let endDate = datetime(2025-02-09); +let data = datatable(Time: datetime, Value: int, other:int) +[ + datetime(2025-01-07), 10, 11, + datetime(2025-01-16), 20, 21, + datetime(2025-02-01), 30, 5 +]; +data +| make-series Value=sum(Value), other=-1 default=-2 on Time from startDate to endDate step 7d +| mv-expand Value, Time, other +| extend Time=todatetime(Time), Value=toint(Value), other=toint(other) +| project-reorder Time, Value, other +``` + +**Output** + +|Time|Value|other| +|---|---|---| +|2025-01-06T00:00:00Z|10|-1| +|2025-01-13T00:00:00Z|20|-1| +|2025-01-20T00:00:00Z|0|-2| +|2025-01-27T00:00:00Z|30|-1| +|2025-02-03T00:00:00Z|0|-2| \ No newline at end of file diff --git a/data-explorer/kusto/query/mv-apply-operator.md b/data-explorer/kusto/query/mv-apply-operator.md index 22d6fd3f86..e1dbcccd7b 100644 --- a/data-explorer/kusto/query/mv-apply-operator.md +++ b/data-explorer/kusto/query/mv-apply-operator.md @@ -3,7 +3,7 @@ title: mv-apply operator description: Learn how to use the mv-apply operator to apply a subquery to each record and union the results of each subquery. ms.reviewer: alexans ms.topic: reference -ms.date: 01/28/2025 +ms.date: 06/04/2025 --- # mv-apply operator @@ -14,7 +14,7 @@ all subqueries. For example, assume a table `T` has a column `Metric` of type `dynamic` whose values are arrays of `real` numbers. The following query locates the -two biggest values in each `Metric` value, and return the records corresponding +two biggest values in each `Metric` value, and returns the records corresponding to these values. ```kusto @@ -43,10 +43,10 @@ The `mv-apply` operator gets the following inputs: If not specified, the original name of the column is used when the expression is a column reference. A random name is used otherwise. > [!NOTE] - > It is recommended to use the default column names. + > We recommend that you use the default column names. 1. The data types of the elements of those dynamic arrays, after expansion. - These become the column types of the columns in the subtables. + These data types become the column types of the columns in the subtables. If not specified, `dynamic` is used. 1. Optionally, the name of a column to add to the subtables that specifies the @@ -74,7 +74,7 @@ Where *ItemIndex* has the syntax: `limit` *RowLimit* -and *SubQuery* has the same syntax of any query statement. +*SubQuery* has the same syntax of any query statement. [!INCLUDE [syntax-conventions-note](../includes/syntax-conventions-note.md)] @@ -82,15 +82,15 @@ and *SubQuery* has the same syntax of any query statement. |Name|Type|Required|Description| |--|--|--|--| -|*ItemIndex*| `string` ||Indicates the name of a column of type `long` that's appended to the input as part of the array-expansion phase and indicates the 0-based array index of the expanded value.| -|*Name*| `string` ||The name to assign the array-expanded values of each array-expanded expression. If not specified, the name of the column is used if available. A random name is generated if *ArrayExpression* isn't a simple column name.| -|*ArrayExpression*| `dynamic` | :heavy_check_mark:|The array whose values are array-expanded. If the expression is the name of a column in the input, the input column is removed from the input and a new column of the same name, or *ColumnName* if specified, appears in the output.| -|*Typename*| `string` ||The name of the type that the individual elements of the `dynamic` array *ArrayExpression* take. Elements that don't conform to this type are replaced by a null value. If unspecified, `dynamic` is used by default.| -|*RowLimit*| `int` ||A limit on the number of records to generate from each record of the input. If unspecified, 2147483647 is used.| -|*SubQuery*| `string` ||A tabular query expression with an implicit tabular source that gets applied to each array-expanded subtable.| +|`ItemIndex`| `string` ||Indicates the name of a column of type `long` that's appended to the input as part of the array-expansion phase and indicates the 0-based array index of the expanded value.| +|`Name`| `string` ||The name to assign the array-expanded values of each array-expanded expression. If not specified, the name of the column is used if available. A random name is generated if *ArrayExpression* isn't a simple column name.| +|`ArrayExpression`| `dynamic` | :heavy_check_mark:|The array whose values are array-expanded. If the expression is the name of a column in the input, the input column is removed from the input and a new column of the same name, or *ColumnName* if specified, appears in the output.| +|`Typename`| `string` ||The name of the type that the individual elements of the `dynamic` array *ArrayExpression* take. Elements that don't conform to this type are replaced by a null value. If unspecified, `dynamic` is used by default.| +|`RowLimit`| `int` ||A limit on the number of records to generate from each record of the input. If unspecified, 2147483647 is used.| +|`SubQuery`| `string` ||A tabular query expression with an implicit tabular source that gets applied to each array-expanded subtable.| >[!NOTE] -> Unlike the [`mv-expand`](mv-expand-operator.md) operator, the `mv-apply` operator doesn't support `bagexpand=array` expansion. If the expression to be expanded is a property bag and not an array, you can use an inner `mv-expand` operator (see example below). +> Unlike the [`mv-expand`](mv-expand-operator.md) operator, the `mv-apply` operator doesn't support `bagexpand=array` expansion. If the expression to be expanded is a property bag and not an array, you can use an inner `mv-expand` operator (see the following example). ## Examples @@ -126,7 +126,7 @@ _data ### Calculating the sum of the largest two elements in an array -The query outputs the sum of the top 2 even numbers (6 + 8 = 14) and the sum of the top 2 odd numbers (5 + 7 = 12). +The query outputs the sum of the top two even numbers (6 + 8 = 14) and the sum of the top two odd numbers (5 + 7 = 12). :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -154,7 +154,7 @@ _data ### Select elements in arrays -The query identifies the top 2 elements from each dynamic array based on the Arr2 values and summarizes them into new lists. +The query identifies the top two elements from each dynamic array based on the Arr2 values and summarizes them into new lists. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -177,9 +177,9 @@ datatable (Val:int, Arr1:dynamic, Arr2:dynamic) | Val1 | Arr1 | Arr2 | `NewArr1` | `NewArr2` | |--|--|--|--|--| -| 1 | ["A1","A2","A3"] | [10,30,7] | ["A2',"A1"] | [30,10] | -| 7 | ["B1","B2","B5"] | [15,11,50] | ["B5","B1"] | [50,15] | -| 3 | ["C1","C2","C3","C4"] | [6,40,20,8] | ["C2","C3"] | [40,20] | +| 1 | `["A1","A2","A3"]` | `[10,30,7]` | `["A2',"A1"]` | `[30,10]` | +| 7 | `["B1","B2","B5"]` | `[15,11,50]` | `["B5","B1"]` | `[50,15]` | +| 3 | `["C1","C2","C3","C4"]` | `[6,40,20,8]` | `["C2","C3"]` | `[40,20]` | ### Using `with_itemindex` for working with a subset of the array @@ -212,7 +212,7 @@ _data | 3 | 8 | | 4 | 10 | -### Using mutiple columns to join element of 2 arrays +### Using multiple columns to join element of two arrays The query combines elements from two dynamic arrays into a new concatenated format and then summarizes them into lists. @@ -242,7 +242,7 @@ datatable (Val: int, Arr1: dynamic, Arr2: dynamic) ### Applying mv-apply to a property bag -This query dynamically removes properties from the packed values object based on the criteria that their values do not start with "555". The final result contains the original columns with unwanted properties removed. +This query dynamically removes properties from the packed values object based on the criteria that their values don't start with 555. The final result contains the original columns with unwanted properties removed. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -276,3 +276,4 @@ datatable(SourceNumber: string, TargetNumber: string, CharsCount: long) ## Related content * [mv-expand](mv-expand-operator.md) operator +* [array_iff](./array-iff-function.md) function diff --git a/data-explorer/kusto/query/mv-expand-operator.md b/data-explorer/kusto/query/mv-expand-operator.md index 407c4e43d0..f39fa9040b 100644 --- a/data-explorer/kusto/query/mv-expand-operator.md +++ b/data-explorer/kusto/query/mv-expand-operator.md @@ -3,7 +3,7 @@ title: mv-expand operator description: Learn how to use the mv-expand operator to expand multi-value dynamic arrays or property bags into multiple records. ms.reviewer: alexans ms.topic: reference -ms.date: 01/20/2025 +ms.date: 06/04/2025 monikerRange: "microsoft-fabric || azure-data-explorer || azure-monitor || microsoft-sentinel " --- # mv-expand operator @@ -144,14 +144,14 @@ datatable (a: int, b: dynamic) | a | b | key | val | |--|--|--|--| -| 1 | ["prop1","a1"] | prop1 | a1 | -| 1 | ["prop2","b1"] | prop2 | b1 | -| 2 | ["prop1","a2"] | prop1 | a2 | -| 2 | ["prop2","b2"] | prop2 | b2 | +| 1 | `["prop1","a1"]` | prop1 | a1 | +| 1 | `["prop2","b1"]` | prop2 | b1 | +| 2 | `["prop1","a2"]` | prop1 | a2 | +| 2 | `["prop2","b2"]` | prop2 | b2 | ### Zipped two columns -Expanding two columns will first 'zip' the applicable columns and then expand them: +Expanding two columns first 'zip' the applicable columns and then expand them: :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -221,9 +221,9 @@ datatable (a: string, b: dynamic, c: dynamic)[ | ColumnName | ColumnOrdinal | DateType | ColumnType | |--|--|--|--| -| a | 0 | System.String | `string` | -| b | 1 | System.Object | `dynamic` | -| c | 2 | System.Int32 | `int` | +| a | 0 | `System.String` | `string` | +| b | 1 | `System.Object` | `dynamic` | +| c | 2 | `System.Int32` | `int` | Notice column `b` is returned as `dynamic` while `c` is returned as `int`. @@ -257,6 +257,7 @@ range x from 1 to 4 step 1 * [mv-apply](mv-apply-operator.md) operator. * For the opposite of the mv-expand operator, see [summarize make_list()](make-list-aggregation-function.md). * For expanding dynamic JSON objects into columns using property bag keys, see [bag_unpack()](bag-unpack-plugin.md) plugin. -* [parse_json function](./parse-json-function.md) +* [parse_json](./parse-json-function.md) function +* [array_iff](./array-iff-function.md) function ::: moniker-end diff --git a/data-explorer/kusto/query/parse-operator.md b/data-explorer/kusto/query/parse-operator.md index 2a89a34523..0cd6499c14 100644 --- a/data-explorer/kusto/query/parse-operator.md +++ b/data-explorer/kusto/query/parse-operator.md @@ -314,7 +314,7 @@ Traces ## Related content -* [parse-json function](parse-json-function.md) -* [extract function](extract-function.md) -* [extract-all function](extract-all-function.md) -* [extract-json function](extract-json-function.md) +* [parse-json](parse-json-function.md) function +* [extract](extract-function.md) function +* [extract-all](extract-all-function.md) function +* [extract-json](extract-json-function.md) function diff --git a/data-explorer/kusto/query/print-operator.md b/data-explorer/kusto/query/print-operator.md index 6c1b280ecc..82b6060e5a 100644 --- a/data-explorer/kusto/query/print-operator.md +++ b/data-explorer/kusto/query/print-operator.md @@ -3,7 +3,7 @@ title: print operator description: Learn how to use the print operator to output a single row with one or more scalar expression results as columns. ms.reviewer: alexans ms.topic: reference -ms.date: 01/20/2025 +ms.date: 06/04/2025 --- # print operator @@ -51,7 +51,7 @@ print 0 + 1 + 2 + 3 + 4 + 5, x = "Wow!" |print_0|x| |--|--| -|15| Wow!| +|15| `Wow!`| ### Print concatenated string @@ -70,4 +70,4 @@ print banner=strcat("Hello", ", ", "World!") |banner| |--| -|Hello, World!| +|`Hello, World!`| diff --git a/data-explorer/kusto/query/todatetime-function.md b/data-explorer/kusto/query/todatetime-function.md index 20db38d1ea..babf9022c1 100644 --- a/data-explorer/kusto/query/todatetime-function.md +++ b/data-explorer/kusto/query/todatetime-function.md @@ -3,7 +3,7 @@ title: todatetime() description: Learn how to use the todatetime() function to convert the input expression to a datetime value. ms.reviewer: alexans ms.topic: reference -ms.date: 03/09/2025 +ms.date: 06/04/2025 --- # todatetime() @@ -65,9 +65,9 @@ print todatetime('12-02-2022') == datetime('12-02-2022') ## Related content -* [make-datetime function](./make-datetime-function.md) +* [make-datetime](./make-datetime-function.md) function * [Scalar function types at a glance](scalar-functions.md) * [The datetime data type](scalar-data-types/datetime.md) -* [datetime_add()](datetime-add-function.md) +* [datetime_add](datetime-add-function.md) function * [Datetime / timespan arithmetic](datetime-timespan-arithmetic.md) -* [totimespan()](totimespan-function.md) +* [totimespan](totimespan-function.md) function diff --git a/data-explorer/kusto/query/totimespan-function.md b/data-explorer/kusto/query/totimespan-function.md index 06184051c2..bc4ce7a66c 100644 --- a/data-explorer/kusto/query/totimespan-function.md +++ b/data-explorer/kusto/query/totimespan-function.md @@ -35,16 +35,17 @@ Else, result is null. ## Example -The following example checks whether the input matches the defined timespan. +The following example shows different usage of totimespan: :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] -> Run the query +> Run the query ::: moniker-end ```kusto let value=5; -print minsStr=totimespan("0.00:03:00"), days=totimespan(4d), hour=totimespan(value * 1h), mins=totimespan(value * 2m), seconds=totimespan(5 * 4s), timespanMin=timespan(25min) +print minsStr=totimespan("0.00:03:00"), days=totimespan(4d), hour=totimespan(value * 1h), + mins=totimespan(value * 2m), seconds=totimespan(5 * 4s), timespanMin=timespan(25min) ``` **Output**