Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
d4727af
replace branch PR 6555 (1)
ktalmor Jan 26, 2025
eca7a9a
replace branch PR 6555 (2)
ktalmor Jan 26, 2025
f4a69be
Replaced help cluster include + minor fixes
ktalmor Jan 28, 2025
26c9a3f
Merge branch 'main' of https://github.com/MicrosoftDocs/dataexplorer-…
ktalmor Jan 28, 2025
569e91b
Merge branch 'main' of https://github.com/ktalmor/dataexplorer-docs-p…
ktalmor Jan 28, 2025
41d3a8c
Replace help cluster include
ktalmor Jan 28, 2025
7680ed6
Merge branch 'main' of https://github.com/ktalmor/dataexplorer-docs-p…
ktalmor Jan 28, 2025
7cee000
reset file
ktalmor Jan 28, 2025
dd088df
reset file
ktalmor Jan 28, 2025
9a3f3ce
reset files
ktalmor Jan 28, 2025
bb03489
smaill fixes
ktalmor Jan 28, 2025
d555244
small change
ktalmor Jan 28, 2025
1f07f51
Merge branch 'main' of https://github.com/MicrosoftDocs/dataexplorer-…
ktalmor Jan 29, 2025
43cad46
added example description
ktalmor Jan 29, 2025
b109644
removed sentence
ktalmor Jan 29, 2025
be5d273
Merge branch 'main' of https://github.com/MicrosoftDocs/dataexplorer-…
ktalmor Jan 29, 2025
89b13a6
removed sentence
ktalmor Jan 29, 2025
af6fe33
Update callout-policy.md
danield137 Jan 31, 2025
747c03d
Add introductory examples to multiple Kusto query operator docs.
ktalmor Feb 3, 2025
ad72002
Add introductory example sentence
ktalmor Feb 3, 2025
9f12402
Update shuffle-query.md
ktalmor Feb 3, 2025
1c4a4c9
formatting
ktalmor Feb 4, 2025
f182a80
separated code from text
ktalmor Feb 4, 2025
7b2e5ca
Fix code block
ktalmor Feb 4, 2025
06f5bdf
fixed code box
ktalmor Feb 4, 2025
7a59439
Update search examples
ktalmor Feb 4, 2025
0154963
typo
ktalmor Feb 4, 2025
8d913a7
Fix case-sensitive search term description
ktalmor Feb 4, 2025
752d30a
remove empty line
ktalmor Feb 4, 2025
de14616
Terminology fix
ktalmor Feb 4, 2025
8fc7723
Type
ktalmor Feb 4, 2025
6e18d1d
Update data-explorer/kusto/query/parse-operator.md
shsagir Feb 4, 2025
0c115d2
Fix typos in 'delimiter' in parse-kv-operator.md
ktalmor Feb 5, 2025
84ab58d
Fix typo in query parameters statement documentation
ktalmor Feb 5, 2025
c397973
Specify *ContosoSales* database as scope in search examples
ktalmor Feb 5, 2025
28aed47
Acrolinx checks
ktalmor Feb 5, 2025
9feaea2
Update data-explorer/kusto/management/callout-policy.md
shsagir Feb 6, 2025
b78b9e8
Merge pull request #6602 from danield137/patch-1
prmerger-automator[bot] Feb 6, 2025
9b85900
Update data-explorer/kusto/query/parse-kv-operator.md
shsagir Feb 6, 2025
87b143d
Add note on default search behavior
ktalmor Feb 6, 2025
b6c599d
Update parameter descriptions for search operator
ktalmor Feb 6, 2025
2f71060
Add example query to serialize-operator.md
ktalmor Feb 6, 2025
a57d291
Merge pull request #6581 from ktalmor/KQL-review-batch8
PMEds28 Feb 6, 2025
6a7c28e
Apply suggestions from code review
v-ccolin Feb 6, 2025
d7580bc
Merge pull request #6582 from ktalmor/KQL-review-batch-9
v-ccolin Feb 6, 2025
b9b8f01
Merge pull request #6615 from MicrosoftDocs/main
v-ccolin Feb 6, 2025
f81ac6c
Merging changes synced from https://github.com/MicrosoftDocs/dataexpl…
Feb 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion data-explorer/kusto/management/callout-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Callout policies are managed at cluster-level and are classified into the follow
| sandbox_artifacts | Controls sandboxed plugins ([python](../query/python-plugin.md) and [R](../query/r-plugin.md)). |
| external_data | Controls access to external data through [external tables](../query/schema-entities/external-tables.md) or [externaldata](../query/externaldata-operator.md) operator. |
| webapi | Controls access to http endpoints. |
| ai_embed_text | Controls the [ai_embed_text plugin)](../query/ai-embed-text-plugin.md). |
| azure_openai | Controls calls to Azure OpenAI plugins such as the embedding plugin [ai_embed_text plugin](../query/ai-embed-text-plugin.md). |

## Predefined callout policies

Expand Down
19 changes: 10 additions & 9 deletions data-explorer/kusto/query/parse-kv-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: parse-kv operator
description: Learn how to use the parse-kv operator to represent structured information extracted from a string expression in a key/value form.
ms.reviewer: alexans
ms.topic: reference
ms.date: 08/11/2024
ms.date: 02/06/2025
---

# parse-kv operator
Expand All @@ -14,13 +14,13 @@ Extracts structured information from a string expression and represents the info

The following extraction modes are supported:

* [**Specified delimeter**](#specified-delimeter): Extraction based on specified delimiters that dictate how keys/values and pairs are separated from each other.
* [**Non-specified delimeter**](#nonspecified-delimiter): Extraction with no need to specify delimiters. Any nonalphanumeric character is considered a delimiter.
* [**Specified delimiter**](#specified-delimiter): Extraction based on specified delimiters that dictate how keys/values and pairs are separated from each other.
* [**Non-specified delimiter**](#nonspecified-delimiter): Extraction with no need to specify delimiters. Any nonalphanumeric character is considered a delimiter.
* [**Regex**](#regex): Extraction based on [regular expressions](regex.md).

## Syntax

### Specified delimeter
### Specified delimiter

*T* `|` `parse-kv` *Expression* `as` `(` *KeysList* `)` `with` `(` `pair_delimiter` `=` *PairDelimiter* `,` `kv_delimiter` `=` *KvDelimiter* [`,` `quote` `=` *QuoteChars* ... [`,` `escape` `=` *EscapeChar* ...]] [`,` `greedy` `=` `true`] `)`

Expand Down Expand Up @@ -52,16 +52,17 @@ The original input tabular expression *T*, extended with columns per specified k

> [!NOTE]
>
> * If a key doesn't appear in a record, the corresponding column value will either be `null` or an empty string, depending on the column type.
> * If a key doesn't appear in a record, the corresponding column value is either `null` or an empty string, depending on the column type.
> * Only keys that are listed in the operator are extracted.
> * The first appearance of a key is extracted, and subsequent values are ignored.
> * When extracting keys and values, leading and trailing white spaces are ignored.
> * When you extract keys and values, leading and trailing white spaces are ignored.

## Examples

The examples in this section show how to use the syntax to help you get started.
### Extraction with well-defined delimiters

In the following example, keys and values are separated by well defined delimiters. These delimeters are comma and colon characters.
In this query, keys and values are separated by well defined delimiters. These delimeters are comma and colon characters.

:::moniker range="azure-data-explorer"
> [!div class="nextstepaction"]
Expand Down Expand Up @@ -101,7 +102,7 @@ print str='src=10.1.1.123 dst=10.1.1.124 bytes=125 failure="connection aborted"
|--|--|--|--|--|
|2021-01-01 10:00:54.0000000| 10.1.1.123| 10.1.1.124| 125| connection aborted|

The following example uses different opening and closing quotes:
This query uses different opening and closing quotes:

:::moniker range="azure-data-explorer"
> [!div class="nextstepaction"]
Expand Down Expand Up @@ -221,7 +222,7 @@ print str="2021-01-01T10:00:34 [INFO] ThreadId:458745723, Machine:Node001, Text:

### Extraction using regex

When no delimiters define text structure well enough, regular expression-based extraction can be useful.
When no delimiters define text structure enough, regular expression-based extraction can be useful.

:::moniker range="azure-data-explorer"
> [!div class="nextstepaction"]
Expand Down
9 changes: 7 additions & 2 deletions data-explorer/kusto/query/parse-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: parse operator
description: Learn how to use the parse operator to parse the value of a string expression into one or more calculated columns.
ms.reviewer: alexans
ms.topic: reference
ms.date: 08/11/2024
ms.date: 01/22/2025
monikerRange: "microsoft-fabric || azure-data-explorer || azure-monitor || microsoft-sentinel "
---
# parse operator
Expand Down Expand Up @@ -37,7 +37,7 @@ Evaluates a string expression and parses its value into one or more calculated c
> * If the parsed *expression* isn't of type `string`, it will be converted to type `string`.
> * Use [`project`](project-operator.md) if you also want to drop or rename some columns.

### Supported kind values
### Supported `kind` values

|Text|Description|
|--|--|
Expand Down Expand Up @@ -67,9 +67,14 @@ The input table extended according to the list of columns that are provided to t

## Examples

The examples in this section show how to use the syntax to help you get started.

[!INCLUDE [help-cluster](../includes/help-cluster-note.md)]

The `parse` operator provides a streamlined way to `extend` a table by using multiple `extract` applications on the same `string` expression. This result is useful, when the table has a `string` column that contains several values that you want to break into individual columns. For example, a column that's produced by a developer trace ("`printf`"/"`Console.WriteLine`") statement.

### Parse and extend results

In the following example, the column `EventText` of table `Traces` contains
strings of the form `Event: NotifySliceRelease (resourceName={0}, totalSlices={1}, sliceNumber={2}, lockTime={3}, releaseTime={4}, previousLockTime={5})`.
The operation extends the table with six columns: `resourceName`, `totalSlices`, `sliceNumber`, `lockTime`, `releaseTime`, and `previousLockTime`.
Expand Down
6 changes: 5 additions & 1 deletion data-explorer/kusto/query/parse-where-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: parse-where operator
description: Learn how to use the parse-where operator to parse the value of a string expression into one or more calculated columns.
ms.reviewer: alexans
ms.topic: reference
ms.date: 08/11/2024
ms.date: 01/20/2025
---
# parse-where operator

Expand Down Expand Up @@ -70,6 +70,10 @@ The input table, which is extended according to the list of columns that are pro

## Examples

The examples in this section show how to use the syntax to help you get started.

[!INCLUDE [help-cluster](../includes/help-cluster-note.md)]

The `parse-where` operator provides a streamlined way to `extend` a table by using multiple `extract` applications on the same `string` expression. This is most useful when the table has a `string` column that contains several values that you want to break into individual columns. For example, you can break up a column that was produced by a developer trace ("`printf`"/"`Console.WriteLine`") statement.

### Using `parse`
Expand Down
24 changes: 14 additions & 10 deletions data-explorer/kusto/query/partition-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@ title: partition operator
description: Learn how to use the partition operator to partition the records of the input table into multiple subtables.
ms.reviewer: alexans
ms.topic: reference
ms.date: 08/11/2024
ms.date: 01/22/2025
---
# partition operator

> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)] [!INCLUDE [sentinel](../includes/applies-to-version/sentinel.md)]

The partition operator partitions the records of its input table into multiple subtables according to values in a key column. The operator runs a subquery on each subtable, and produces a single output table that is the union of the results of all subqueries.

This operator is useful when you need to perform a subquery only on a subset of rows that belongs to the same partition key, and not query the whole dataset. These subqueries could include aggregate functions, window functions, top *N* and others.
The partition operator is useful when you need to perform a subquery only on a subset of rows that belong to the same partition key, and not a query of the whole dataset. These subqueries could include aggregate functions, window functions, top *N* and others.

The partition operator supports several strategies of subquery operation:

Expand All @@ -36,16 +36,16 @@ The partition operator supports several strategies of subquery operation:
| *Column*| `string` | :heavy_check_mark: | The name of a column in *T* whose values determine how to partition the input tabular source.|
| *TransformationSubQuery*| `string` | :heavy_check_mark: | A tabular transformation expression. The source is implicitly the subtables produced by partitioning the records of *T*. Each subtable is homogenous on the value of *Column*.</br></br> The expression must provide only one tabular result and shouldn't have other types of statements, such as `let` statements.|
| *SubQueryWithSource*| `string` | :heavy_check_mark: | A tabular expression that includes its own tabular source, such as a table reference. This syntax is only supported with the [legacy strategy](#legacy-strategy). The subquery can only reference the key column, *Column*, from *T*. To reference the column, use the syntax `toscalar(`*Column*`)`.</br></br> The expression must provide only one tabular result and shouldn't have other types of statements, such as `let` statements.|
| *Hints*| `string` | | Zero or more space-separated parameters in the form of: *HintName* `=` *Value* that control the behavior of the operator. See the [supported hints](#supported-hints) per strategy type.
| *Hints*| `string` | | Zero or more space-separated parameters in the form of: *HintName* `=` *Value* that control the behavior of the operator. See the [supported hints](#supported-hints) per strategy type.|

### Supported hints

|Hint name|Type|Strategy|Description|
|--|--|--|--|
|`hint.shufflekey`| `string` | [shuffle](#shuffle-strategy) | The partition key used to run the partition operator with the `shuffle` strategy. |
|`hint.materialized`| `bool` | [legacy](#legacy-strategy) | If set to `true`, will materialize the source of the `partition` operator. The default value is `false`. |
|`hint.materialized`| `bool` | [legacy](#legacy-strategy) | If set to `true`, materializes the source of the `partition` operator. The default value is `false`. |
|`hint.concurrency`| `int` | [legacy](#legacy-strategy) | Determines how many partitions to run in parallel. The default value is `16`.|
|`hint.spread`| `int` | [legacy](#legacy-strategy) | Determines how to distribute the partitions among cluster nodes. The default value is `1`.</br></br> For example, if there are *N* partitions and the spread hint is set to *P*, then the *N* partitions will be processed by *P* different cluster nodes equally in parallel/sequentially depending on the concurrency hint.|
|`hint.spread`| `int` | [legacy](#legacy-strategy) | Determines how to distribute the partitions among cluster nodes. The default value is `1`.</br></br> For example, if there are *N* partitions and the spread hint is set to *P*, then the *N* partitions are processed by *P* different cluster nodes equally, in parallel/sequentially depending on the concurrency hint.|

## Returns

Expand Down Expand Up @@ -120,10 +120,14 @@ If the subquery is a tabular transformation without a tabular source, the source
To use this strategy, specify `hint.strategy=legacy` or omit any other strategy indication.

> [!NOTE]
> An error will occur if the partition column, *Column*, contains more than 64 distinct values.
> An error occurs if the partition column, *Column*, contains more than 64 distinct values.

## Examples

The examples in this section show how to use the syntax to help you get started.

[!INCLUDE [help-cluster](../includes/help-cluster-note.md)]


### Find top values

Expand All @@ -144,7 +148,7 @@ StormEvents
)
```

**Output**
**Output**

|EventType|State|Events|Injuries|
|---|---|---|---|
Expand Down Expand Up @@ -180,7 +184,7 @@ StormEvents
)
```

**Output**
**Output**

|EventType|TotalInjueries|
|---|---|
Expand Down Expand Up @@ -212,7 +216,7 @@ StormEvents
| count
```

**Output**
**Output**

|Count|
|---|
Expand All @@ -238,7 +242,7 @@ range x from 1 to 2 step 1
| count
```

**Output**
**Output**

|Count|
|---|
Expand Down
5 changes: 3 additions & 2 deletions data-explorer/kusto/query/pattern-statement.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,14 @@ For more information, see [Working with middle-tier applications](#work-with-mid
| *PathArgType* | `string` | | The type of the *PathArgType* argument. Possible values: `string` |
| *ArgValue* | `string` | :heavy_check_mark: | The *ArgName* and optional *PathName* tuple values to be mapped to an *expression*. |
| *PathValue* | `string` | | The value to map for *PathName*. |
| *expression* | `string` | :heavy_check_mark: | A tabular or lambda expression that references a function returning tabular data. For example: `Logs | where Timestamp > ago(1h)` |
| *expression* | `string` | :heavy_check_mark: | A tabular or lambda expression that references a function returning tabular data. For example: `Logs | where Timestamp > ago(1h)`|

## Examples

The examples in this section show how to use the syntax to help you get started.

[!INCLUDE [help-cluster](../includes/help-cluster-note.md)]

In these examples, a pattern is defined.

### Define a simple pattern

Expand Down
8 changes: 6 additions & 2 deletions data-explorer/kusto/query/print-operator.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: print operator
title: print operator
description: Learn how to use the print operator to output a single row with one or more scalar expression results as columns.
ms.reviewer: alexans
ms.topic: reference
ms.date: 11/20/2024
ms.date: 01/20/2025
---
# print operator

Expand All @@ -30,6 +30,10 @@ A table with one or more columns and a single row. Each column returns the corre

## Examples

The examples in this section show how to use the syntax to help you get started.

[!INCLUDE [help-cluster](../includes/help-cluster-note.md)]

### Print sum and variable value

The following example outputs a row with two columns. One column contains the sum of a series of numbers and the other column contains the value of the variable, `x`.
Expand Down
8 changes: 7 additions & 1 deletion data-explorer/kusto/query/query-parameters-statement.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ To reference query parameters, the query text, or functions it uses, must first
|Name|Type|Required|Description|
|--|--|--|--|
|*Name1*| `string` | :heavy_check_mark:|The name of a query parameter used in the query.|
|*Type1*| `string` | :heavy_check_mark:|The corresponding type, such as `string` or `datetime`. The values provided by the user are encoded as strings. The appropriate parse method is applied to the query parameter to get a strongly-typed value.|
|*Type1*| `string` | :heavy_check_mark:|The corresponding type, such as `string` or `datetime`. The values provided by the user are encoded as strings. The appropriate parse method is applied to the query parameter to get a strongly typed value.|
|*DefaultValue1*| `string` ||A default value for the parameter. This value must be a literal of the appropriate scalar type.|

> [!NOTE]
Expand All @@ -44,8 +44,14 @@ To reference query parameters, the query text, or functions it uses, must first

## Example

The examples in this section show how to use the syntax to help you get started.

[!INCLUDE [help-cluster](../includes/help-cluster-note.md)]

### Declare query parameters

This query retrieves storm events from the *StormEvents* table where the total number of direct and indirect injuries exceeds a specified threshold (default is 90). It then projects the *EpisodeId*, *EventType*, and the total number of injuries for each of these events.

:::moniker range="azure-data-explorer"
> [!div class="nextstepaction"]
> <a href="https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAA4WNuw7CMBAE+0j5hytBpKAFBBUpXEOPrHgFjvzifAEi8fEkQUBJO9qZNWicZtC1A/enpFl7CDjPvH6o0HYMs3YxnGlLq+V8UxYHiezrG4JkKosn3S8Y9GlqkfeW0QgtvkAF80Y7+hVHL3FsR14nm6OBMhVN1WOfUJFE0e7TGL7/9l+H314eyAAAAA==" target="_blank">Run the query</a>
Expand Down
35 changes: 21 additions & 14 deletions data-explorer/kusto/query/range-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: range operator
description: Learn how to use the range operator to generate a single-column table of values.
ms.reviewer: alexans
ms.topic: reference
ms.date: 01/07/2025
ms.date: 01/22/2025
---
# range operator

Expand All @@ -26,7 +26,7 @@ Generates a single-column table of values.
|--|--|--|--|
|*columnName*| `string` | :heavy_check_mark:| The name of the single column in the output table.|
|*start*|int, long, real, datetime, or timespan| :heavy_check_mark:| The smallest value in the output.|
|*stop*|int, long, real, datetime, or timespan| :heavy_check_mark:| The highest value being generated in the output or a bound on the highest value if *step* steps over this value.|
|*stop*|int, long, real, datetime, or timespan| :heavy_check_mark:| The highest value being generated in the output or a bound on the highest value if *step* is over this value.|
|*step*|int, long, real, datetime, or timespan| :heavy_check_mark:| The difference between two consecutive values.|

> [!NOTE]
Expand All @@ -39,6 +39,10 @@ whose values are *start*, *start* `+` *step*, ... up to and until *stop*.

## Examples

The example in this section shows how to use the syntax to help you get started.

[!INCLUDE [help-cluster](../includes/help-cluster-note.md)]

### Range over the past seven days

The following example creates a table with entries for the current time stamp extended over the past seven days, once a day.
Expand Down Expand Up @@ -114,13 +118,13 @@ let MyTimeline = range MyMonthHour from MyMonthStart to now() step StepBy

**Output**

| MyMonthHour | MyMonthHourinUnixTime | DateOnly | TimeOnly |
|--------------|------------------------|---------------|------------------------------|
| 2023-02-01 | 00:00:00.0000000 | 1675209600 | 2023-02-01 00:00:00.0000000 |
| 2023-02-01 | 04:32:02.4000000 | 1675225922.4 | 2023-02-01 00:00:00.0000000 |
| 2023-02-01 | 09:04:04.8000000 | 1675242244.8 | 2023-02-01 00:00:00.0000000 |
| 2023-02-01 | 13:36:07.2000000 | 1675258567.2 | 2023-02-01 00:00:00.0000000 |
| ... | ... | ... | ... |
| MyMonthHour | MyMonthHourinUnixTime | DateOnly | TimeOnly |
|--|--|--|--|
| 2023-02-01 | 00:00:00.0000000 | 1675209600 | 2023-02-01 00:00:00.0000000 |
| 2023-02-01 | 04:32:02.4000000 | 1675225922.4 | 2023-02-01 00:00:00.0000000 |
| 2023-02-01 | 09:04:04.8000000 | 1675242244.8 | 2023-02-01 00:00:00.0000000 |
| 2023-02-01 | 13:36:07.2000000 | 1675258567.2 | 2023-02-01 00:00:00.0000000 |
| ... | ... | ... | ... |

### Incremented steps

Expand All @@ -134,16 +138,19 @@ whose type is `long` and results in values from one to eight incremented by thre

```kusto
range Steps from 1 to 8 step 3
```

**Output**

| Steps |
|-------|
| 1 |
| 4 |
| 7 |
|--|
| 1 |
| 4 |
| 7 |

### Traces over a time range

The following example shows how the `range` operator can be used to create a dimension table that is used to introduce zeros where the source data has no values. It takes timestamps from the last four hours and counts traces for each one minute interval. When there are no traces for a specific interval, the count is zero.
The following example shows how the `range` operator can be used to create a dimension table that is used to introduce zeros where the source data has no values. It takes timestamps from the last four hours and counts traces for each one-minute interval. When there are no traces for a specific interval, the count is zero.

```kusto
range TIMESTAMP from ago(4h) to now() step 1m
Expand Down
Loading