Skip to content

Commit f234a2a

Browse files
Merge pull request #2546 from MicrosoftDocs/main638749800299777558sync_temp
For protected branch, push strategy should use PR and merge to target branch method to work around git push error
2 parents e3f799a + bb82719 commit f234a2a

15 files changed

+146
-86
lines changed

data-explorer/includes/kusto-emulator-limitations.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
ms.service: azure
33
ms.topic: include
4-
ms.date: 08/14/2022
4+
ms.date: 02/05/2025
55
---
66

77
## Limitations
@@ -21,7 +21,7 @@ In general, this offering isn't suited for production workloads.
2121
- No support for ingestion endpoints, including Kusto.Ingest SDKs
2222
- No streaming ingestion
2323
- Although ingested data can be stored externally to the container, we don't recommend persisting data for long periods of time for the following reasons:
24-
- There's no guarantee the [extent](/kusto/management/extents-overview?view=azure-data-explorer&preserve-view=true) format will be compatible between versions of the Kusto emulator
24+
- There's no guarantee the [extent](/kusto/management/extents-overview?view=azure-data-explorer&preserve-view=true) format is compatible between versions of the Kusto emulator
2525
- Extents aren't [merged](/kusto/management/merge-policy?view=azure-data-explorer&preserve-view=true) and therefore can become fragmented as data get ingested
26-
- Retention policies can be set but won't be honored
26+
- Retention and partitioning policies can be set but aren't honored
2727
- The [Python plugin](/kusto/query/python-plugin?view=azure-data-explorer&preserve-view=true?view=azure-data-explorer&preserve-view=true) isn't supported

data-explorer/kusto-emulator-install.md

Lines changed: 55 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,30 @@
11
---
22
title: Install the Azure Data Explorer Kusto emulator
3-
description: In this article, you'll learn how to install the Azure Data Explorer Kusto emulator and run your first query.
3+
description: In this article, you learn how to install the Azure Data Explorer Kusto emulator and run your first query.
44
ms.reviewer: vplauzon
55
ms.topic: how-to
6-
ms.date: 12/05/2023
6+
ms.date: 02/09/2025
77
---
88

99
# Install the Azure Data Explorer Kusto emulator
1010

1111
You can install the Azure Data Explorer Kusto emulator in the following ways:
1212

13-
- On your own device: Consider using this option if you need to provision a local development environment
14-
- On a CI/CD agent virtual machine (VM): Use this option if you require a CI/CD pipeline for running automated tests
13+
- **On your own device**: Consider using this option if you need to provision a local development environment
14+
- **On a CI/CD agent virtual machine (VM)**: Use this option if you require a CI/CD pipeline for running automated tests
1515

16-
The emulator is available as a *Windows* or *Linux* Docker container image.
16+
The emulator is available as a *Linux* and *Windows* Docker container image.
1717

18-
In this article, you'll learn how to:
18+
In this article, you learn how to:
1919

2020
- [Install the Kusto emulator](#install-the-kusto-emulator)
2121
- [Connect to the emulator](#connect-to-the-emulator)
2222
- [Create a database](#create-a-database)
2323
- [Ingest data](#ingest-data)
2424
- [Query data](#query-data)
2525

26+
This article focuses on how to install the Linux Docker container on a Windows client.
27+
2628
## Prerequisites
2729

2830
- The host operating system must be one of:
@@ -35,25 +37,35 @@ In this article, you'll learn how to:
3537
> Linux distros only support Linux container images.
3638
3739
- 2 gigabytes (GB) of RAM minimum; we recommend using 4 GB or more
38-
- Docker Client for [Windows](https://docs.docker.com/desktop/windows/install/) or [Linux](https://docs.docker.com/desktop/install/linux-install/)
40+
- Docker Client for [Linux](https://docs.docker.com/desktop/install/linux-install/) or [Windows](https://docs.docker.com/desktop/windows/install/)
3941

4042
## Install the Kusto emulator
4143

4244
The following steps are for using PowerShell to start the emulator using the [Kusto emulator container image](https://aka.ms/adx.emulator.image). For other options, see [Run emulator options](#run-emulator-options).
4345

44-
1. For Windows only, switch Docker to run with Windows containers. You may need to enable the feature in the Docker settings.
46+
1. **For Windows container only**: Switch Docker to run with Windows containers. You might need to enable the feature in the Docker settings.
4547

4648
:::image type="content" source="media/kusto-emulator/kusto-emulator-docker-windows-container.png" alt-text="Screenshot of the Docker settings, showing the Switch to Windows containers option.":::
4749

4850
1. Run the following command to start the emulator.
4951

5052
> [!IMPORTANT]
51-
> The Kusto emulator container image is a free offering under the [Microsoft Software License Terms](https://aka.ms/adx.emulator.license). Since the emulator runs in a container, you must accept the license terms by passing the `ACCEPT_EULA` environment variable to the container with its value set to `Y` indicating.
53+
> The Kusto emulator container image is a free offering under the [Microsoft Software License Terms](https://aka.ms/adx.emulator.license). Since the emulator runs in a container, you must accept the license terms by passing the `ACCEPT_EULA` environment variable to the container with its value set to `Y`.
5254
5355
> [!NOTE]
5456
>
55-
> - The first time this command is run, Docker pulls the container image which is several GBs in size and may take several minutes to download. Once downloaded, the image is cached and available for subsequent runs without having to download it again.
56-
> - (For Windows container only) The container must be run in process-isolation mode. This is the default on some versions of Docker. For other versions, you can start the container in Hyper-V isolation mode by adding `--isolation=hyperv` to the run command.
57+
> - The first time this command is run, Docker pulls the container image which is several GBs in size and might take several minutes to download. Once downloaded, the image is cached and available for subsequent runs without having to download it again.
58+
> - **For Windows container only**: The container must run in process-isolation mode. This is the default on some versions of Docker. For other versions, you can start the container in Hyper-V isolation mode by adding `--isolation=hyperv` to the run command.
59+
60+
# [Linux container](#tab/linux-container)
61+
62+
To start the Linux container, make sure you use the `latest` or `stable` tag:
63+
64+
```powershell
65+
docker run -e ACCEPT_EULA=Y -m 4G -d -p 8080:8080 -t mcr.microsoft.com/azuredataexplorer/kustainer-linux:latest
66+
```
67+
68+
# [Windows container](#tab/windowscontainer)
5769
5870
- To start the Windows container on Windows Server operating system, make sure you use the `latest` or `stable` tag:
5971
@@ -67,11 +79,7 @@ The following steps are for using PowerShell to start the emulator using the [Ku
6779
docker run -e ACCEPT_EULA=Y -m 4G -d -p 8080:8080 -t mcr.microsoft.com/azuredataexplorer/kustainer:windows11
6880
```
6981
70-
- To start the Linux container:
71-
72-
```powershell
73-
docker run -e ACCEPT_EULA=Y -m 4G -d -p 8080:8080 -t mcr.microsoft.com/azuredataexplorer/kustainer-linux:latest
74-
```
82+
---
7583
7684
1. Run the following command to verify that the container is running.
7785
@@ -114,24 +122,35 @@ The following steps are for using PowerShell to start the emulator using the [Ku
114122
RawContentLength : 988
115123
```
116124
117-
### Run emulator options
125+
## Run emulator options
118126
119127
You can use any of the following options when running the emulator:
120128
121129
- Mount a local folder to the container: Use this option to mount a folder in the host environment into the container. Mounting a host folder enables your queries to interact with local files, which is useful for [creating a database persistent between container runs](#create-a-database) and [ingesting data](#ingest-data).
122130
123-
For example, to mount the folder "D:\host\local" on the host to the folder "c:\kustodatadata" in the container, use the next command on Windows Server:
131+
# [Linux container](#tab/linux-container)
132+
133+
For example, to mount the folder "D:\host\local" on the host to the folder "/kustodatadata" in the container, use the following command on Windows Server:
134+
135+
```powershell
136+
docker run -v d:\host\local:/kustodata -e ACCEPT_EULA=Y -m 4G -d -p 8080:8080 -t mcr.microsoft.com/azuredataexplorer/kustainer-linux:latest
137+
```
138+
139+
# [Windows container](#tab/windowscontainer)
140+
For example, to mount the folder "D:\host\local" on the host to the folder "c:\kustodata" in the container, use the following command on Windows Server:
124141
125142
```powershell
126143
docker run -v d:\host\local:c:\kustodata -e ACCEPT_EULA=Y -m 4G -d -p 8080:8080 -t mcr.microsoft.com/azuredataexplorer/kustainer:latest
127144
```
128145
146+
---
147+
129148
- Run on a different port: The Kusto emulator exposes access to the Kusto Query Engine on port 8080; hence in other examples you mapped the host port 8080 to the emulator port 8080. You can use this option to map a different host to the engine.
130149
131150
For example, to map port 9000 on the host to the engine, use the following command on Windows Server:
132151
133152
```powershell
134-
docker run -e ACCEPT_EULA=Y -m 4G -d -p 9000:8080 -t mcr.microsoft.com/azuredataexplorer/kustainer:latest
153+
docker run -e ACCEPT_EULA=Y -m 4G -d -p 9000:8080 -t mcr.microsoft.com/azuredataexplorer/kustainer-linux:latest
135154
```
136155
137156
## Connect to the emulator
@@ -142,55 +161,58 @@ You can use any of the following tools to connect to and interact with the emula
142161
- [Kusto.CLI](/kusto/tools/kusto-cli?view=azure-data-explorer&preserve-view=true)
143162
- [Kusto.Data SDKs](/kusto/api/netfx/about-kusto-data?view=azure-data-explorer&preserve-view=true)
144163
145-
In the following sections, you'll use Kusto.Explorer to create a database, ingest data, and query it. To learn more, see [Using Kusto.Explorer](/kusto/tools/kusto-explorer-using?view=azure-data-explorer&preserve-view=true).
164+
In the following sections, you use Kusto.Explorer to create a database, ingest data, and query it. To learn more, see [Using Kusto.Explorer](/kusto/tools/kusto-explorer-using?view=azure-data-explorer&preserve-view=true).
146165
147166
> [!IMPORTANT]
148167
> The Kusto Emulator doesn't support HTTPS or Microsoft Entra authentication.
149168
>
150169
> You must first allow unsafe connections in **Tools** > **Options** > **Connections** > **Allow unsafe connections**.
151-
>
170+
>
152171
> :::image type="content" source="media/kusto-emulator/unsafe-connections.png" alt-text="Screenshot of Kusto Explorer connection options.":::
153-
>
172+
>
154173
> The following image highlights the affected fields in the **Add connection** properties.
155174
>
156175
> - The **Cluster connection** must begin with `http://` and not `https://`.
157-
> - In **Security** > **Advanced: Connection String**, you'll need to remove the `AAD Federated Security=True` portion of the connection string to disable Microsoft Entra authentication.
176+
> - In **Security** > **Advanced: Connection String**, you need to remove the `AAD Federated Security=True` portion of the connection string to disable Microsoft Entra authentication.
158177
>
159178
> :::image type="content" source="media/kusto-emulator/kusto-emulator-connection.png" alt-text="Screenshot of Kusto Explorer single connection setup.":::
160179
161180
## Create a database
162181
163182
To store data and run queries, create a database or attach the emulator to an existing database.
164183
165-
A database can be persisted in a container folder or on a [mounted folder](#run-emulator-options). The former's lifetime is bound to the container, so restarting the container loses any changes. Also, the container virtual storage is less efficient than native one. Mounted folder enables you to keep the data between container runs.
184+
A database can be persisted in a container folder or on a [mounted folder](#run-emulator-options). The former's lifetime is bound to the container, so restarting the container loses any changes. Also, the container virtual storage is less efficient than native one. Mounted folder enables you to keep the data between container runs.
166185
167186
In this example, we keep the data on the container.
168187
169-
In the [Kusto.Explorer Query mode](/kusto/tools/kusto-explorer-using?view=azure-data-explorer&preserve-view=true#query-mode), run the following command to create a persistent database if using a Windows container image:
188+
In the [Kusto.Explorer Query mode](/kusto/tools/kusto-explorer-using?view=azure-data-explorer&preserve-view=true#query-mode), run the following command to create a persistent database:
189+
190+
# [Linux container](#tab/linux-container)
170191
171192
```kusto
172193
.create database <YourDatabaseName> persist (
173-
@"c:\kustodata\dbs\<YourDatabaseName>\md",
174-
@"c:\kustodata\dbs\<YourDatabaseName>\data"
194+
@"/kustodata/dbs/<YourDatabaseName>/md",
195+
@"/kustodata/dbs/<YourDatabaseName>/data"
175196
)
176197
```
177198

178-
The equivalent command for a Linux container image is:
199+
# [Windows container](#tab/windowscontainer)
179200

180201
```kusto
181202
.create database <YourDatabaseName> persist (
182-
@"/kustodata/dbs/<YourDatabaseName>/md",
183-
@"/kustodata/dbs/<YourDatabaseName>/data"
203+
@"c:\kustodata\dbs\<YourDatabaseName>\md",
204+
@"c:\kustodata\dbs\<YourDatabaseName>\data"
184205
)
185206
```
207+
---
186208

187209
This command requires that the folders don't already exist, to prevent over-writing existing information. To attach to an existing database, use the following command instead, specifying the path that ends with `md`:
188210

189211
```kusto
190212
.attach database <YourDatabaseName> from @"<PathToMdFolder>"
191213
```
192214

193-
It's also possible to detach the database from the emulator, which will keep all the database metadata and data intact (so you could reattach to it in the future):
215+
It's also possible to detach the database from the emulator, which will keep all the database metadata and data intact so you can reattach to it in the future:
194216

195217
```kusto
196218
.detach database <YourDatabaseName>
@@ -200,7 +222,7 @@ It's also possible to detach the database from the emulator, which will keep all
200222

201223
To ingest data, you must first create an external table linked to a file and then ingest the data into a table in the database.
202224

203-
Use the steps in the following example to create an external table and ingest data into it. For the example, in the local folder *c:\kustodata*, create a file called `sample.csv` with the following data:
225+
Use the steps in the following example to create an external table and ingest data into it. As part of the example, create a file called `sample.csv` in the local folder *d:\host\local*, with the following data:
204226

205227
```text
206228
Alice, 1
@@ -217,7 +239,7 @@ Carl, 3
217239
1. Run the following command to [ingest the file into the table](/kusto/management/data-ingestion/ingest-from-storage?view=azure-data-explorer&preserve-view=true):
218240
219241
```kusto
220-
.ingest into table MyIngestedSample(@"c:\kustodata\sample.csv")
242+
.ingest into table MyIngestedSample(@"/kustodata/sample.csv")
221243
```
222244
223245
## Query data

data-explorer/kusto-emulator-overview.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
---
22
title: Azure Data Explorer Kusto emulator overview
3-
description: In this article, you'll learn about the Azure Data Explorer Kusto emulator
3+
description: In this article, you learn about the Azure Data Explorer Kusto emulator.
44
ms.reviewer: vplauzon
55
ms.topic: conceptual
6-
ms.date: 05/08/2023
6+
ms.date: 02/06/2025
77
---
88
# Azure Data Explorer Kusto emulator
99

@@ -13,11 +13,11 @@ The Kusto emulator is a local environment that encapsulates the query engine. Yo
1313
>
1414
> - The emulator is provided *as-is*, without any support or warranties.
1515
> - The emulator isn't intended for use in production environments.
16-
> - The [license terms](https://aka.ms/adx.emulator.license) explicitly prohibit benchmark tests using the emulator as it is not intended or optimized for that purpose. The emulator has a very different performance profile compared to the Azure Data Explorer service.
16+
> - The [license terms](https://aka.ms/adx.emulator.license) explicitly prohibit benchmark tests using the emulator as it isn't intended or optimized for that purpose. The emulator has a different performance profile compared to the Azure Data Explorer service.
1717
1818
## Architecture
1919

20-
The emulator is available as a *Windows* or *Linux* Docker container image. It exposes a query endpoint over HTTP that can be used with any client including [Kusto.Explorer](/kusto/tools/kusto-explorer-using?view=azure-data-explorer&preserve-view=true), [Kusto.CLI](/kusto/tools/kusto-cli?view=azure-data-explorer&preserve-view=true), or [Kusto.Data SDKs](/kusto/api/netfx/about-kusto-data?view=azure-data-explorer&preserve-view=true).
20+
The emulator is available as a *Linux* and *Windows* Docker container image. It exposes a query endpoint over HTTP that can be used with any client including [Kusto.Explorer](/kusto/tools/kusto-explorer-using?view=azure-data-explorer&preserve-view=true), [Kusto.CLI](/kusto/tools/kusto-cli?view=azure-data-explorer&preserve-view=true), or [Kusto.Data SDKs](/kusto/api/netfx/about-kusto-data?view=azure-data-explorer&preserve-view=true).
2121

2222
![Schematic representation of Kusto emulator architecture.](media/kusto-emulator/kusto-emulator-architecture.png)
2323

@@ -34,7 +34,7 @@ The main scenarios for the emulator are:
3434

3535
## Kusto emulator vs. free cluster
3636

37-
There are some overlaps between the Kusto emulator and the [free cluster](start-for-free.md) offerings. The following table contrasts the features of the offerings.
37+
There are some overlaps between the Kusto emulator and the [free cluster](start-for-free.md) offerings. The following table contrasts the features of the offerings.
3838

3939
| Item | Kusto emulator | Free cluster |
4040
|--|--|--|
@@ -49,10 +49,10 @@ There are some overlaps between the Kusto emulator and the [free cluster](start-
4949
| Long term data management | No extent merge capability | Full support |
5050
| Internet requirement | No connectivity required | Cloud service |
5151

52-
When choosing the best option for your use case, you should consider the following:
52+
When choosing the best option for your use case, you should consider:
5353

5454
- **Local development**: The best fit for local development depends on the features you require. For instance, if your scenario requires the use of managed pipelines, the free cluster offering works best. On the other hand, if your scenario requires local development in a disconnected environment, the Kusto emulator would be a better fit.
55-
- **Automated testing**: In general, the Kusto emulator is more suited to automated testing than the free cluster offering. It's faster to provision and doesn't require any Azure principal to set up.
55+
- **Automated testing**: In general, the Kusto emulator is more suited to automated testing than the free cluster offering. It's faster to provision and doesn't require any Azure principal to set up.
5656

5757
[!INCLUDE [kusto-emulator-limitations](includes/kusto-emulator-limitations.md)]
5858

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
ms.topic: include
3+
ms.date: 01/22/2025
4+
---
5+
6+
:::moniker range="azure-data-explorer"
7+
> The examples in this article use publicly available tables in the [help cluster](https://dataexplorer.azure.com/clusters/help/), such as the `StormEvents` table in the *Samples* database.
8+
::: moniker-end

0 commit comments

Comments
 (0)