diff --git a/data-explorer/kusto/includes/help-cluster-samples-stormevents.md b/data-explorer/kusto/includes/help-cluster-samples-stormevents.md new file mode 100644 index 0000000000..3b2b161c57 --- /dev/null +++ b/data-explorer/kusto/includes/help-cluster-samples-stormevents.md @@ -0,0 +1,6 @@ +--- +ms.topic: include +ms.date: 01/13/2025 +--- + +In the [help cluster](https://dataexplorer.azure.com/clusters/help/), there's a `Samples` database with a `StormEvents` table. \ No newline at end of file diff --git a/data-explorer/kusto/query/alias-statement.md b/data-explorer/kusto/query/alias-statement.md index 785a23e0d3..0ce2adbd05 100644 --- a/data-explorer/kusto/query/alias-statement.md +++ b/data-explorer/kusto/query/alias-statement.md @@ -3,18 +3,19 @@ title: Alias statement description: Learn how to use an alias statement to define an alias for a database that is used for a query. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/12/2025 --- # Alias statement > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)] [!INCLUDE [sentinel](../includes/applies-to-version/sentinel.md)] -Alias statements allow you to define an alias for databases, which can be used later in the same query. +Alias statements allow you to define an alias for a database, which can be used in the same query. :::moniker range="azure-data-explorer" This is useful when you're working with several clusters but want to appear as if you're working on fewer clusters. The alias must be defined according to the following syntax, where *clustername* and *databasename* are existing and valid entities. + ## Syntax `alias` database *DatabaseAliasName* `=` cluster("https://*clustername*.kusto.windows.net").database("*DatabaseName*") @@ -50,7 +51,7 @@ The alias must be defined according to the following syntax, where *Eventhouse* ## Examples -In the [help cluster](https://dataexplorer.azure.com/clusters/help/), there's a `Samples` database with a `StormEvents` table. +[!INCLUDE [help-cluster](../includes/help-cluster-samples-stormevents.md)] First, count the number of records in that table. diff --git a/data-explorer/kusto/query/batches.md b/data-explorer/kusto/query/batches.md index 3998315bcb..eec2c34b76 100644 --- a/data-explorer/kusto/query/batches.md +++ b/data-explorer/kusto/query/batches.md @@ -3,7 +3,7 @@ title: Batches description: This article describes Batches. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/13/2025 --- # Batches @@ -18,6 +18,8 @@ A query can include multiple tabular expression statements, as long as they're d ## Examples +[!INCLUDE [help-cluster](../includes/help-cluster-samples-stormevents.md)] + ### Name tabular results The following query produces two tabular results. User agent tools can then display those results with the appropriate name associated with each (`Count of events in Florida` and `Count of events in Guam`, respectively). @@ -32,9 +34,25 @@ StormEvents | where State == "FLORIDA" | count | as ['Count of events in Florida StormEvents | where State == "GUAM" | count | as ['Count of events in Guam'] ``` +**Output** + +### [Count of events in Florida](#tab/florida) + +| Count | +| -- | +| 1042 | + +### [Count of events in Guam](#tab/guam) + +| Count | +| -- | +| 4 | + +--- + ### Share a calculation -Batching is useful for scenarios where a common calculation is shared by multiple subqueries, such as for dashboards. If the common calculation is complex, use the [materialize() function](materialize-function.md) and construct the query so that it will be executed only once: +Batching is useful for scenarios where a common calculation is shared by multiple subqueries, such as for dashboards. If the common calculation is complex, use the [materialize() function](materialize-function.md) and construct the query so that it will be executed only once. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -46,3 +64,26 @@ let m = materialize(StormEvents | summarize n=count() by State); m | where n > 2000; m | where n < 10 ``` + +**Output** + +### [Table 1](#tab/table-1) + +| State | n | +|--|--| +| ILLINOIS | 2022 | +| IOWA | 2337 | +| KANSAS | 3166 | +| MISSOURI | 2016 | +| TEXAS | 4701 | + +### [Table 2](#tab/table-2) + +| State | n | +|--|--| +| GUAM | 2022 | +| GULF OF ALASKA | 2337 | +| HAWAII WATERS | 3166 | +| LAKE ONTARIO | 2016 | + +--- diff --git a/data-explorer/kusto/query/let-statement.md b/data-explorer/kusto/query/let-statement.md index e04078be5b..df7d3b2e60 100644 --- a/data-explorer/kusto/query/let-statement.md +++ b/data-explorer/kusto/query/let-statement.md @@ -65,6 +65,8 @@ To optimize multiple uses of the `let` statement within a single query, see [Opt ## Examples +[!INCLUDE [help-cluster](../includes/help-cluster-samples-stormevents.md)] + ### Define scalar values The following example uses a scalar expression statement. @@ -91,6 +93,16 @@ let ['some number'] = 20; range y from 0 to ['some number'] step 5 ``` +**Output** + +|y| +|---| +|0| +|5| +|10| +|15| +|20| + ### Create a user defined function with scalar calculation This example uses the let statement with arguments for scalar calculation. The query defines function `MultiplyByN` for multiplying two numbers. diff --git a/data-explorer/kusto/query/pattern-statement.md b/data-explorer/kusto/query/pattern-statement.md index d9e9840310..d21b6850f6 100644 --- a/data-explorer/kusto/query/pattern-statement.md +++ b/data-explorer/kusto/query/pattern-statement.md @@ -3,7 +3,7 @@ title: Pattern statement description: Learn how to use pattern statements to map string tuples to tabular expressions. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/15/2025 monikerRange: "microsoft-fabric || azure-data-explorer" --- @@ -11,9 +11,13 @@ monikerRange: "microsoft-fabric || azure-data-explorer" > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)] [!INCLUDE [sentinel](../includes/applies-to-version/sentinel.md)] -A **pattern** is a construct that maps string tuples to tabular expressions. Each pattern must *declare* a pattern name and optionally *define* a pattern mapping. Patterns that define a mapping return a tabular expression when invoked. Any two statements must be separated by a semicolon. +A **pattern** is a construct that maps string tuples to tabular expressions. -*Empty patterns* are patterns that are declared but don't define a mapping. When invoked, they return error *SEM0036* along with the details of the missing pattern definitions in the HTTP header. Middle-tier applications that provide a Kusto Query Language (KQL) experience can use the returned details as part of their process to enrich KQL query results. +Each pattern must *declare* a pattern name and optionally *define* a pattern mapping. Patterns that define a mapping return a tabular expression when invoked. Separate any two statements by a semicolon. + +Empty patterns are patterns that are declared but don't define a mapping. When invoked, they return error *SEM0036* along with the details of the missing pattern definitions in the HTTP header. + +Middle-tier applications that provide a Kusto Query Language (KQL) experience can use the returned details as part of their process to enrich KQL query results. For more information, see [Working with middle-tier applications](#work-with-middle-tier-applications). ## Syntax @@ -56,11 +60,13 @@ For more information, see [Working with middle-tier applications](#work-with-mid ## Examples -In each of the following examples, a pattern is declared, defined, and then invoked. +[!INCLUDE [help-cluster](../includes/help-cluster-samples-stormevents.md)] -### Define simple patterns +In these examples, a pattern is defined. -The following example defines a pattern that maps states to an expression that returns its capital/major city. +### Define a simple pattern + +This example defines a pattern that maps states to an expression that returns its capital/major city. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -77,21 +83,23 @@ declare pattern country = (name:string)[state:string] country("Canada").Alberta ``` -**Output** +**Output** |Capital| |-------| |Edmonton| +### Define a scoped pattern + +This example defines a pattern to scope data and metrics of application data. The pattern is invoked to return a union of the data. + :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] > Run the query ::: moniker-end -The following example defines a pattern that defines some scoped application data. - ```kusto -declare pattern App = (applicationId:string)[scope:string] +declare pattern App = (applicationId:string)[scope:string] { ('a1').['Data'] = { range x from 1 to 5 step 1 | project App = "App #1", Data = x }; ('a1').['Metrics'] = { range x from 1 to 5 step 1 | project App = "App #1", Metrics = rand() }; @@ -147,23 +155,34 @@ union app("ApplicationX").["*"] | count ``` -**Returns semantic error** +**Output semantic error** + > One or more pattern references were not declared. Detected pattern references: ["app('ApplicationX').['*']"] ## Work with middle-tier applications A middle-tier application provides its users with the ability to use KQL and wants to enhance the experience by enriching the query results with augmented data from its internal service. -To this end, the application provides users with a pattern statement that returns tabular data that their users can use in their queries. The pattern's arguments are the keys the application will use to retrieve the enrichment data. When the user runs the query, the application does not parse the query itself but instead plans to leverage the error returned by an empty pattern to retrieve the keys it requires. So it prepends the query with the empty pattern declaration, sends it to the cluster for processing, and then parses the returned HTTP header to retrieve the values of missing pattern arguments. The application uses these values to look up the enrichment data and builds a new declaration that defines the appropriate enrichment data mapping. Finally, the application prepends the new definition to the user's query, resends it for processing, and returns the result it receives to the user. +To this end, the application provides users with a pattern statement that returns tabular data that their users can use in their queries. The pattern's arguments are the keys the application will use to retrieve the enrichment data. + +When the user runs the query, the application doesn't parse the query itself but instead uses the error returned by an empty pattern to retrieve the keys it requires. So it prepends the query with the empty pattern declaration, sends it to the cluster for processing, and then parses the returned HTTP header to retrieve the values of missing pattern arguments. The application uses these values to look up the enrichment data and builds a new declaration that defines the appropriate enrichment data mapping. + +Finally, the application prepends the new definition to the query, resends it for processing, and returns the result it receives to the user. ### Example -In the following example, a middle-tier application provides the ability to enrich queries with longitude/latitude locations. The application uses an internal service to map IP addresses to longitude/latitude locations, and provides a pattern called `map_ip_to_longlat` for this purpose. Let's suppose the application gets the following query from the user: +In the examples, a pattern is declared, defined, and then invoked. + +#### Declare an empty pattern + +In this example, a middle-tier application enriches queries with longitude/latitude locations. The application uses an internal service to map IP addresses to longitude/latitude locations, and provides a pattern called `map_ip_to_longlat`. When the query is run, it returns an error with missing pattern definitions: ```kusto map_ip_to_longlat("10.10.10.10") ``` +#### Declare and define a pattern + The application does not parse this query and hence does not know which IP address (*10.10.10.10*) was passed to the pattern. So it prepends the user query with an empty `map_ip_to_longlat` pattern declaration and sends it for processing: ```kusto @@ -174,7 +193,11 @@ map_ip_to_longlat("10.10.10.10") The application receives the following error in response. > One or more pattern references were not declared. Detected pattern references: ["map_ip_to_longlat('10.10.10.10')"] -The application inspects the error, determines that the error indicates a missing pattern reference, and retrieves the missing IP address (*10.10.10.10*). It uses the IP address to look up the enrichment data in its internal service and builds a new pattern defining the mapping of the IP address to the corresponding longitude and latitude data. The new pattern is prepended to the user's query and run again. This time the query succeeds because the enrichment data is now declared in the query, and the result is sent to the user. +#### Invoke a pattern + +The application inspects the error, determines that the error indicates a missing pattern reference, and retrieves the missing IP address (*10.10.10.10*). It uses the IP address to look up the enrichment data in its internal service and builds a new pattern defining the mapping of the IP address to the corresponding longitude and latitude data. The new pattern is prepended to the user's query and run again. + +This time the query succeeds because the enrichment data is now declared in the query, and the result is sent to the user. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] diff --git a/data-explorer/kusto/query/query-parameters-statement.md b/data-explorer/kusto/query/query-parameters-statement.md index 64c09c16f0..3eb7e68154 100644 --- a/data-explorer/kusto/query/query-parameters-statement.md +++ b/data-explorer/kusto/query/query-parameters-statement.md @@ -3,7 +3,7 @@ title: Query parameters declaration statement description: Learn how to use the query parameters declaration statement to parameterize queries and protect against injection attacks. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/13/2025 monikerRange: "microsoft-fabric || azure-data-explorer" --- # Query parameters declaration statement @@ -44,6 +44,8 @@ To reference query parameters, the query text, or functions it uses, must first ## Example +[!INCLUDE [help-cluster](../includes/help-cluster-samples-stormevents.md)] + :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] > Run the query diff --git a/data-explorer/kusto/query/restrict-statement.md b/data-explorer/kusto/query/restrict-statement.md index bee7e2d9fc..e7db0058ed 100644 --- a/data-explorer/kusto/query/restrict-statement.md +++ b/data-explorer/kusto/query/restrict-statement.md @@ -3,7 +3,7 @@ title: Restrict statement description: Learn how to use the restrict statement to limit tabular views that are visible to subsequent query statements. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/13/2025 monikerRange: "microsoft-fabric || azure-data-explorer" --- # Restrict statement @@ -38,9 +38,11 @@ The middle-tier application can prefix the user's query with a **logical model** ## Examples +[!INCLUDE [help-cluster](../includes/help-cluster-samples-stormevents.md)] + ### Let statement -The following example uses a [let statement](let-statement.md) appearing before `restrict` statement. +The example uses a [let statement](let-statement.md) appearing before `restrict` statement. ```kusto // Limit access to 'Test' let statement only @@ -50,7 +52,7 @@ restrict access to (Test); ### Tables or functions -The following example uses references to [tables](../management/tables.md) or [functions](../management/functions.md) that are defined in the database metadata. +The example uses references to [tables](../management/tables.md) or [functions](../management/functions.md) that are defined in the database metadata. ```kusto // Assuming the database that the query uses has table Table1 and Func1 defined in the metadata, @@ -61,7 +63,7 @@ restrict access to (database().Table1, database().Func1, database('DB2').Table2) ### Patterns -The following example uses wildcard patterns that can match multiples of [let statements](let-statement.md) or tables/functions. +The example uses wildcard patterns that can match multiples of [let statements](let-statement.md) or tables/functions. ```kusto let Test1 = () { print x=1 }; @@ -82,7 +84,7 @@ restrict access to (database('DB2').*); ### Prevent user from querying other user data -The following example shows how a middle-tier application can prepend a user's query +The example shows how a middle-tier application can prepend a user's query with a logical model that prevents the user from querying any other user's data. ```kusto diff --git a/data-explorer/kusto/query/set-statement.md b/data-explorer/kusto/query/set-statement.md index 2fc5ad3fa3..4aceeb710a 100644 --- a/data-explorer/kusto/query/set-statement.md +++ b/data-explorer/kusto/query/set-statement.md @@ -3,7 +3,7 @@ title: Set statement description: Learn how to use the set statement to set a request property for the duration of the query. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/13/2025 monikerRange: "microsoft-fabric || azure-data-explorer" --- # Set statement @@ -35,6 +35,8 @@ Request properties aren't formally a part of the Kusto Query Language and may be ## Example +[!INCLUDE [help-cluster](../includes/help-cluster-samples-stormevents.md)] + ```kusto set querytrace; Events | take 100 diff --git a/data-explorer/kusto/query/tabular-expression-statements.md b/data-explorer/kusto/query/tabular-expression-statements.md index 4c5b20004f..3b5d738cb5 100644 --- a/data-explorer/kusto/query/tabular-expression-statements.md +++ b/data-explorer/kusto/query/tabular-expression-statements.md @@ -3,7 +3,7 @@ title: Tabular expression statements description: Learn how to use tabular expression statements to produce tabular datasets. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/13/2025 --- # Tabular expression statements @@ -41,9 +41,11 @@ A tabular data source produces sets of records, to be further processed by tabul ## Examples +[!INCLUDE [help-cluster](../includes/help-cluster-samples-stormevents.md)] + ### Filter rows by condition -The following query counts the number of records in the `StormEvents` table that have a value of "FLORIDA" in the `State` column. +This query counts the number of records in the `StormEvents` table that have a value of "FLORIDA" in the `State` column. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -64,7 +66,7 @@ StormEvents ### Combine data from two tables -In the following example, the [join](join-operator.md) operator is used to combine records from two tabular data sources: the `StormEvents` table and the `PopulationData` table. +In this example, the [join](join-operator.md) operator is used to combine records from two tabular data sources: the `StormEvents` table and the `PopulationData` table. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] diff --git a/data-explorer/monitor-queued-ingestion.md b/data-explorer/monitor-queued-ingestion.md index 3ce4af9eb5..74a4959358 100644 --- a/data-explorer/monitor-queued-ingestion.md +++ b/data-explorer/monitor-queued-ingestion.md @@ -269,5 +269,5 @@ Click away from the settings to get the full chart that compares the number of e ## Related content * [Monitor Azure Data Explorer performance, health, and usage with metrics](using-metrics.md) -* [Monitor Azure Data Explorer ingestion, commands, queries, and tables using diagnostic logs](using-diagnostic-logs.md) +* [Monitor Azure Data Explorer ingestion, commands, queries, and tables using diagnostic logs](monitor-data-explorer.md#monitor-azure-data-explorer-ingestion-commands-queries-and-tables-using-diagnostic-logs) * [Use Azure Monitor Insights](/azure/azure-monitor/insights/data-explorer)