diff --git a/data-explorer/create-cluster-and-database.md b/data-explorer/create-cluster-and-database.md index 520da94aad..8fd9d0e090 100644 --- a/data-explorer/create-cluster-and-database.md +++ b/data-explorer/create-cluster-and-database.md @@ -3,7 +3,7 @@ title: "Quickstart: Create an Azure Data Explorer cluster and database" description: In this quickstart, you learn how to create an Azure Data Explorer cluster and database. ms.reviewer: mblythe ms.topic: quickstart -ms.date: 06/03/2025 +ms.date: 06/05/2025 ms.custom: mode-portal adobe-target: true @@ -69,7 +69,7 @@ Create an Azure Data Explorer cluster with a defined set of compute and storage | Region | *West US* or *West US 2* | Select *West US* or *West US 2* (if using availability zones) for this quickstart. For a production system, select the region that best meets your needs. | Workload | *Dev/Test* | Select *Dev/Test* for this quickstart. For a production system, select the specification that best meets your needs. | Compute specifications | *Dev(No SLA)_Standard_E2a_v4* | Select *Dev(No SLA)_Standard_E2a_v4* for this quickstart. For a production system, select the specification that best meets your needs. - | Availability zones | On | Turning on this feature distributes the cluster storage and compute resources across multiple physical zones within a region for added protection and availability. By default, this feature is turned on if zones are supported in the region. If fewer than 3 zones are available for the compute instances, the portal displays the number of supported zones. Deployment to availability zones is possible when creating the cluster or [can be migrated later](migrate-cluster-to-multiple-availability-zone.md). Read more about [Azure Availability Zones](/azure/availability-zones/az-overview).| + | Availability zones | On | Turning on this feature distributes the cluster storage and compute resources across multiple physical zones within a region for added protection and availability. By default, this feature is turned on if zones are supported in the region. If fewer than three zones are available for the compute instances, the portal displays the number of supported zones. Deployment to availability zones is possible when creating the cluster or [can be migrated later](migrate-cluster-to-multiple-availability-zone.md). Read more about [Azure Availability Zones](/azure/availability-zones/az-overview).| 1. Select **Review + create** to review your cluster details, and on the next screen select **Create** to provision the cluster. Provisioning typically takes about 10 minutes. @@ -120,7 +120,7 @@ To create a database in your full cluster: |---|---|---| | Admin | *Default selected* | The admin field is disabled. New admins can be added after database creation. | | Database name | *TestDatabase* | The name of database to create. The name must be unique within the cluster. | - | Retention period | *365* | The number of days that data is guaranteed to be kept available for querying. The period is measured from the time data is ingested. | + | Retention period | *365* | The number of days that data is available for querying. The period is measured from the time data is ingested. | | Cache period | *31* | The number of days to keep frequently queried data available in SSD storage or RAM to optimize querying. | :::image type="content" source="media/create-cluster-and-database/create-test-database.png" alt-text="Create database form." lightbox="media/create-cluster-and-database/create-test-database.png"::: @@ -170,7 +170,7 @@ You can stop and restart a cluster depending on business needs. Stopping the clu :::image type="content" source="media/create-cluster-and-database/stop-button.png" alt-text="Screenshot of the Azure portal and cluster stop button." lightbox="media/create-cluster-and-database/stop-button.png"::: > [!NOTE] - > When the cluster is stopped, data is not available for queries, and you can't ingest new data. + > When the cluster is stopped, data isn't available for queries, and you can't ingest new data. 1. To restart the cluster, at the top of the **Overview** tab, select **Start**. diff --git a/data-explorer/dashboard-customize-visuals.md b/data-explorer/dashboard-customize-visuals.md index 5bb18012d1..f0b14c53db 100644 --- a/data-explorer/dashboard-customize-visuals.md +++ b/data-explorer/dashboard-customize-visuals.md @@ -3,7 +3,7 @@ title: Customize Azure Data Explorer dashboard visuals description: Easily customize your Azure Data Explorer dashboard visuals ms.reviewer: gabil ms.topic: how-to -ms.date: 01/31/2024 +ms.date: 06/10/2025 --- # Customize Azure Data Explorer dashboard visuals @@ -14,6 +14,9 @@ All visualizations that are created in the context of the [render operator](/kus ## Prerequisites +Review Visualize data with Azure Data Explorer dashboards. + +* Ensure that you have the editor permission on an Azure Data Explorer dashboard. * [Visualize data with Azure Data Explorer dashboards](azure-data-explorer-dashboards.md) * Editor permissions on an Azure Data Explorer dashboard diff --git a/data-explorer/dashboard-parameters.md b/data-explorer/dashboard-parameters.md index 093340821f..29e98fc90a 100644 --- a/data-explorer/dashboard-parameters.md +++ b/data-explorer/dashboard-parameters.md @@ -3,7 +3,7 @@ title: Use parameters in Azure Data Explorer dashboards description: Learn how to use parameters as a building block for dashboard filters. ms.reviewer: gabil ms.topic: how-to -ms.date: 10/19/2023 +ms.date: 06/05/2025 --- # Use parameters in Azure Data Explorer dashboards @@ -52,12 +52,12 @@ In the **Add parameter** pane, configure the following properties: |Field |Description | |---------|---------| |**Label**|The name of the parameter shown on the dashboard or the edit card.| -|**Parameter type**|One of the following parameters:| +|**Parameter type**|One of the following parameters:| |**Variable name**|The name of the parameter to be used in the query. Tip: Prefixing variable names with a distinctive character, such as an underscore character (_), makes them easier to find when writing queries. | |**Data type**|The data type of the parameter values.| -|**Show on pages**|Select the pages where this parameter will be displayed. The **Select all** option shows the parameter on all pages.| +|**Show on pages**|Select the pages where this parameter is displayed. The **Select all** option shows the parameter on all pages.| |**Source**|The source of the parameter values: | -|**Default value**|The default value of the filter. The filter will start always with the default value upon initial rendering of the dashboard.| +|**Default value**|The default value of the filter. The filter always starts with the default value upon initial rendering of the dashboard.| |**Add a "Select all" value**|Applicable only to single selection and multiple selection parameter types. Used to retrieve data for all the parameter values. This value should be built into the query to provide the functionality. For more examples of building such queries, see [Use the multiple-selection query-based parameter](#multiple-selection-fixed-values-parameters).| ## Manage parameters @@ -365,7 +365,7 @@ To create a cross-filter, you must turn on the option in the visual, and then sp ## Interact with your data using cross-filter -Once the cross-filter is defined, you can use it to interact with your data. In visuals where you've defined cross-filters, you can select data points and use their values to filter the current dashboard page. For table visuals, select data points by right-clicking on the relevant cell and then in the context menu, select **Cross-filter**. +Once the cross-filter is defined, you can use it to interact with your data. In visuals where you define cross-filters, you can select data points and use their values to filter the current dashboard page. For table visuals, select data points by right-clicking on the relevant cell and then in the context menu, select **Cross-filter**. :::image type="content" source="media/dashboard-parameters/cross-filter.jpg" alt-text="Screenshot of a table visual, showing the cross-filter context menu option."::: @@ -379,13 +379,13 @@ With drillthrough, you can select a value in a visual and use it to filter the v ### Define a drillthrough -To create a drillthrough, you must first turn it on in the visual and then add one or more drillthroughs as required. +To create a drillthrough, you must first turn on drillthroughs in the visual and then add one or more drillthroughs as required. 1. Edit the visual where you want to add a drillthrough. Make sure that the **Visual** tab is selected. 1. In the right pane, select **Interactions**, then turn on drillthrough. - :::image type="content" source="media/dashboard-parameters/drillthrough-visual-create.png" alt-text="Screenshot of drillthrough page, showing the turn on option."::: + :::image type="content" source="media/dashboard-parameters/drillthrough-visual-create.png" alt-text="Screenshot of drillthrough page, showing the turn-on option."::: 1. Under **Drillthrough**, select **Create new**, and specify the following information. Optionally select **Add another pair** to add multiple parameters for a target page. Repeat this step to add drillthroughs to other pages in the current dashboard using different parameters. @@ -401,11 +401,11 @@ To create a drillthrough, you must first turn it on in the visual and then add o ## Interact with your data using drillthroughs -Once drillthroughs are defined, you can use them to interact with your data. To do so, in visuals or tables where you've defined a drillthrough, right-click on a data point, and then select **Drill through to** > *destination page*. The values from the data point will be used as the parameters to filter the visuals on the target page. +Once drillthroughs are defined, you can use them to interact with your data. To do so, in visuals or tables where you define a drillthrough, right-click on a data point, and then select **Drill through to** > *destination page*. The values from the data point are used as the parameters to filter the visuals on the target page. :::image type="content" source="media/dashboard-parameters/drillthrough-combined.jpg" alt-text="Screenshot of a dashboard visual, showing the drillthrough interaction."::: -To return to the source page, in the top-right of the destination page, select **Reset**. All filters assigned by the drillthrough will be reset. +To return to the source page, in the top-right of the destination page, select **Reset**. All filters assigned by the drillthrough are reset. ## Use filter search for single and multiple selection filters diff --git a/data-explorer/data-explorer-overview.md b/data-explorer/data-explorer-overview.md index a9cb766e80..b898374a92 100644 --- a/data-explorer/data-explorer-overview.md +++ b/data-explorer/data-explorer-overview.md @@ -3,7 +3,7 @@ title: What is Azure Data Explorer? description: Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data. ms.reviewer: mblythe ms.topic: overview -ms.date: 06/03/2025 +ms.date: 06/10/2025 adobe-target: true # Customer intent: As a data analyst, I want to understand Azure Data Explorer, so I can decide if it's suitable for my analytics workloads. @@ -13,7 +13,7 @@ adobe-target: true Azure Data Explorer is a fully managed, high-performance, big data analytics platform that makes it easy to analyze high volumes of data in near real time. The Azure Data Explorer toolbox gives you an end-to-end solution for data ingestion, query, visualization, and management. -By analyzing structured, semi-structured, and unstructured data across time series, and by using Machine Learning, Azure Data Explorer makes it simple to extract key insights, spot patterns and trends, and create forecasting models. Azure Data Explorer uses a traditional relational model, organizing data into tables with strongly-typed schemas. Tables are stored within databases, and a cluster can manage multiple databases. Azure Data Explorer is scalable, secure, robust, and enterprise-ready, and is useful for log analytics, time series analytics, IoT, and general-purpose exploratory analytics. +By analyzing structured, semi-structured, and unstructured data across time series, and by using Machine Learning, Azure Data Explorer makes it simple to extract key insights, spot patterns and trends, and create forecasting models. Azure Data Explorer uses a traditional relational model, organizing data into tables with strongly typed schemas. Tables are stored within databases, and a cluster can manage multiple databases. Azure Data Explorer is scalable, secure, robust, and enterprise-ready, and is useful for log analytics, time series analytics, IoT, and general-purpose exploratory analytics. Azure Data Explorer capabilities are extended by other services built on its query language: [Kusto Query Language (KQL)](/kusto/query/index?view=azure-data-explorer&preserve-view=true). These services include [Azure Monitor logs](/azure/log-analytics/), [Application Insights](/azure/application-insights/), [Time Series Insights](/azure/time-series-insights/), and [Microsoft Defender for Endpoint](/microsoft-365/security/defender-endpoint/microsoft-defender-endpoint). @@ -68,7 +68,7 @@ Generally speaking, when you interact with Azure Data Explorer, you're going to > [!NOTE] > You can access your Azure Data Explorer resources either in the [Azure Data Explorer web UI](web-query-data.md) or by using [SDKs](/kusto/api/index?view=azure-data-explorer&preserve-view=true). -1. **Create database:** Create a *cluster* and then create one or more *databases* in that cluster. Each Azure Data Explorer cluster can hold up to 10,000 databases and each database up to 10,000 tables. The data in each table is stored in data shards also called "extents". All data is automatically indexed and partitioned based on the ingestion time. This means you can store a large ammount of varied data and because of the way it's stored, you get fast access to querying it. [Quickstart: Create an Azure Data Explorer cluster and database](create-cluster-and-database.md) +1. **Create database:** Create a *cluster* and then create one or more *databases* in that cluster. Each Azure Data Explorer cluster can hold up to 10,000 databases and each database up to 10,000 tables. The data in each table is stored in data shards also called "extents". All data is automatically indexed and partitioned based on the ingestion time. This means you can store a large amount of varied data and because of the way it's stored, you get fast access to querying it. [Quickstart: Create an Azure Data Explorer cluster and database](create-cluster-and-database.md) 1. **Ingest data:** Load data into database tables so that you can run queries against it. Azure Data Explorer supports several [ingestion methods](ingest-data-overview.md), each with its own target scenarios. These methods include ingestion tools, connectors and plugins to diverse services, managed pipelines, programmatic ingestion using SDKs, and direct access to ingestion. Get started with the [get data experience](ingest-data-overview.md). diff --git a/data-explorer/data-lake-query-data.md b/data-explorer/data-lake-query-data.md index c5f9714e8b..836b56a687 100644 --- a/data-explorer/data-lake-query-data.md +++ b/data-explorer/data-lake-query-data.md @@ -3,7 +3,7 @@ title: Query data in Azure Data Lake using Azure Data Explorer description: Learn how to query data in Azure Data Lake using Azure Data Explorer. ms.reviewer: orspodek ms.topic: how-to -ms.date: 08/25/2022 +ms.date: 06/10/2025 --- # Query data in Azure Data Lake using Azure Data Explorer @@ -26,7 +26,7 @@ Let's say you have lots of CSV files containing historical info on products stor The files are stored in Azure Blob storage `mycompanystorage` under a container named `archivedproducts`, partitioned by date: -``` +```text https://mycompanystorage.blob.core.windows.net/archivedproducts/2019/01/01/part-00000-7e967c99-cf2b-4dbb-8c53-ce388389470d.csv.gz https://mycompanystorage.blob.core.windows.net/archivedproducts/2019/01/01/part-00001-ba356fa4-f85f-430a-8b5a-afd64f128ca4.csv.gz https://mycompanystorage.blob.core.windows.net/archivedproducts/2019/01/01/part-00002-acb644dc-2fc6-467c-ab80-d1590b23fc31.csv.gz @@ -51,7 +51,9 @@ The external table is now visible in the left pane of the Azure Data Explorer we :::image type="content" source="media/data-lake-query-data/external-tables-web-ui.png" alt-text="Screenshot that shows external table in Azure Data Explorer web UI."::: -### External table permissions +## External table permissions + +Review the following table permissions: * The database user can create an external table. The table creator automatically becomes the table administrator. * The cluster, database, or table administrator can edit an existing table. @@ -72,7 +74,7 @@ external_table("ArchivedProducts") You can query both external tables and ingested data tables within the same query. You can [`join`](/kusto/query/join-operator?view=azure-data-explorer&preserve-view=true) or [`union`](/kusto/query/union-operator?view=azure-data-explorer&preserve-view=true) the external table with other data from Azure Data Explorer, SQL servers, or other sources. Use a [`let( ) statement`](/kusto/query/let-statement?view=azure-data-explorer&preserve-view=true) to assign a shorthand name to an external table reference. -In the example below, *Products* is an ingested data table and *ArchivedProducts* is an external table that we've defined previously: +In the example below, *Products* is an ingested data table and *ArchivedProducts* is an external table that we've defined: ```kusto let T1 = external_table("ArchivedProducts") | where TimeStamp > ago(100d); @@ -80,7 +82,7 @@ let T = Products; //T is an internal table T1 | join T on ProductId | take 10 ``` -## Querying hierarchical data formats +### Querying hierarchical data formats Azure Data Explorer allows querying hierarchical formats, such as `JSON`, `Parquet`, `Avro`, and `ORC`. To map hierarchical data schema to an external table schema (if it's different), use [external table mappings commands](/kusto/management/external-table-mapping-create?view=azure-data-explorer&preserve-view=true). For instance, if you want to query JSON log files with the following format: @@ -88,14 +90,14 @@ Azure Data Explorer allows querying hierarchical formats, such as `JSON`, `Parqu { "timestamp": "2019-01-01 10:00:00.238521", "data": { - "tenant": "e1ef54a6-c6f2-4389-836e-d289b37bcfe0", + "tenant": "aaaabbbb-0000-cccc-1111-dddd2222eeee", "method": "RefreshTableMetadata" } } { "timestamp": "2019-01-01 10:00:01.845423", "data": { - "tenant": "9b49d0d7-b3e6-4467-bb35-fa420a25d324", + "tenant": "bbbbcccc-1111-dddd-2222-eeee3333ffff", "method": "GetFileList" } } @@ -119,7 +121,7 @@ Define a JSON mapping that maps data fields to external table definition fields: .create external table ApiCalls json mapping 'MyMapping' '[{"Column":"Timestamp","Properties":{"Path":"$.timestamp"}},{"Column":"TenantId","Properties":{"Path":"$.data.tenant"}},{"Column":"MethodName","Properties":{"Path":"$.data.method"}}]' ``` -When you query the external table, the mapping will be invoked, and relevant data will be mapped to the external table columns: +When you query the external table, the mapping is invoked, and relevant data mapped to the external table columns: ```kusto external_table('ApiCalls') | take 10 @@ -127,13 +129,13 @@ external_table('ApiCalls') | take 10 For more info on mapping syntax, see [data mappings](/kusto/management/mappings?view=azure-data-explorer&preserve-view=true). -## Query *TaxiRides* external table in the help cluster +### Query *TaxiRides* external table in the help cluster Use the test cluster called *help* to try out different Azure Data Explorer capabilities. The *help* cluster contains an external table definition for a [New York City taxi dataset](https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page) containing billions of taxi rides. ### Create external table *TaxiRides* -This section shows the query used to create the *TaxiRides* external table in the *help* cluster. Since this table has already been created, you can skip this section and go directly to [query *TaxiRides* external table data](#query-taxirides-external-table-data). +This section shows the query used to create the *TaxiRides* external table in the *help* cluster. Since you already created this table, you can skip this section and go directly to [query *TaxiRides* external table data](#query-taxirides-external-table-data). ```kusto .create external table TaxiRides @@ -206,7 +208,7 @@ You can find the created **TaxiRides** table by looking at the left pane of the Sign in to [https://dataexplorer.azure.com/clusters/help/databases/Samples](https://dataexplorer.azure.com/clusters/help/databases/Samples). -#### Query *TaxiRides* external table without partitioning +### Query *TaxiRides* external table without partitioning [Run this query](https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAAx3LSwqAMAwFwL3gHYKreh1xL7F9YrCtElP84OEV9zM4DZo5DsZjhGt6PqWTgL1p6+qhvaTEKjeI/FqyuZbGiwJf63QAi9vEL2UbAhtMEv6jyAH6+VhS9jOr1dULfUgAm2cAAAA=) on the external table *TaxiRides* to show rides for each day of the week, across the entire dataset. @@ -220,7 +222,7 @@ This query shows the busiest day of the week. Since the data isn't partitioned, :::image type="content" source="media/data-lake-query-data/taxirides-no-partition.png" alt-text="Graph representation to render non-partitioned query."::: -#### Query TaxiRides external table with partitioning +### Query TaxiRides external table with partitioning [Run this query](https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAA13NQQqDMBQE0L3gHT6ukkVF3fQepXv5SQYMNWmIP6ilh68WuinM6jHMYBPkyPMobGao5s6bv3mHpdF19aZ1QgYlbx8ljY4F4gPIQFYgkvqJGrr+eun6I5ralv58OP27t5QQOPsXiOyzRFGazE6WzSh7wtnIiA75uISdOEtdfQDLWmP+ogAAAA==) on the external table *TaxiRides* to show taxi cab types (yellow or green) used in January of 2017. @@ -237,7 +239,7 @@ This query uses partitioning, which optimizes query time and performance. The qu You can write other queries to run on the external table *TaxiRides* and learn more about the data. -## Optimize your query performance +### Optimize your query performance Optimize your query performance in the lake by using the following best practices for querying external data. @@ -268,6 +270,6 @@ Organize your data using "folder" partitions that enable the query to skip irrel Select VM SKUs with more cores and higher network throughput (memory is less important). For more information, see [Select the correct VM SKU for your Azure Data Explorer cluster](manage-cluster-choose-sku.md). -## Related content +### Related content * [Tutorial: Learn common KQL operators](/kusto/query/tutorials/learn-common-operators?view=azure-data-explorer&preserve-view=true) diff --git a/data-explorer/follower.md b/data-explorer/follower.md index d0b75c8d0e..6713a4df8e 100644 --- a/data-explorer/follower.md +++ b/data-explorer/follower.md @@ -4,19 +4,22 @@ description: Learn about how to attach databases in Azure Data Explorer using th ms.reviewer: gabilehner ms.topic: how-to ms.custom: devx-track-azurepowershell -ms.date: 05/17/2023 +ms.date: 06/10/2025 --- # Use follower databases -The **follower database** feature allows you to attach a database located in a different cluster to your Azure Data Explorer cluster. The **follower database** is attached in *read-only* mode, making it possible to view the data and run queries on the data that was ingested into the **leader database**. The follower database synchronizes changes in the leader databases. Because of the synchronization, there's a data lag of a few seconds to a few minutes in data availability. The length of the time lag depends on the overall size of the leader database metadata. The leader and follower databases use the same storage account to fetch the data. The storage is owned by the leader database. The follower database views the data without needing to ingest it. Since the attached database is a read-only database, the data, tables, and policies in the database can't be modified except for [caching policy](#configure-caching-policy), [principals](#manage-principals), and [permissions](#manage-permissions). Attached databases can't be deleted. They must be detached by the leader or follower and only then they can be deleted. +The **follower database** feature allows you to attach a database located in a different cluster to your Azure Data Explorer cluster. The **follower database** is attached in *read-only* mode, making it possible to view the data and run queries on the data that was ingested into the **leader database**. The follower database synchronizes changes in the leader databases. Because of the synchronization, there's a data lag of a few seconds to a few minutes in data availability. The length of the time lag depends on the overall size of the leader database metadata. The leader and follower databases use the same storage account to fetch the data. The leader database owns the storage. The follower database views the data without needing to ingest it. Since the attached database is a read-only database, the data, tables, and policies in the database can't be modified except for [caching policy](#configure-caching-policy), [principals](#manage-principals), and [permissions](#manage-permissions). Attached databases can't be deleted. The leader or follower must detach databases before deletion. Attaching a database to a different cluster using the follower capability is used as the infrastructure to share data between organizations and teams. The feature is useful to segregate compute resources to protect a production environment from non-production use cases. Follower can also be used to associate the cost of Azure Data Explorer cluster to the party that runs queries on the data. +>[!NOTE] > For code samples based on previous SDK versions, see the [archived article](/previous-versions/azure/data-explorer/follower). ## Which databases are followed? +The following apply to clusters: + * A cluster can follow one database, several databases, or all databases of a leader cluster. * A single cluster can follow databases from multiple leader clusters. * A cluster can contain both follower databases and leader databases. @@ -37,13 +40,13 @@ To attach a database, you must have user, group, service principal, or managed i ### Table level sharing -When attaching the database all tables, external tables and materialized views are followed as well. You can share specific tables/external tables/materialized views by configuring the '*TableLevelSharingProperties*'. +When you attach databases, all tables, external tables, and materialized views are followed as well. You can share specific tables/external tables/materialized views by configuring the '*TableLevelSharingProperties*'. '*TableLevelSharingProperties*' contains eight arrays of strings: `tablesToInclude`, `tablesToExclude`, `externalTablesToInclude`, `externalTablesToExclude`, `materializedViewsToInclude`, `materializedViewsToExclude`, `functionsToInclude`, and `functionsToExclude`. The maximum number of entries in all arrays together is 100. > [!NOTE] > -> * Table level sharing is not supported when using '*' all databases notation. +> * Table level sharing isn't supported when using '*' all databases notation. > * When materialized views are included, their source tables are included as well. #### Examples @@ -251,7 +254,7 @@ Use the following steps to attach a database: | **Parameter** | **Description** | **Example** | |--|--|--| - | *followerClusterName* | The name of the follower cluster; where the template will be deployed. | | + | *followerClusterName* | The name of the follower cluster; where the template is deployed. | | | *attachedDatabaseConfigurationsName* | The name of the attached database configurations object. The name can be any string that is unique at the cluster level. | | | *databaseName* | The name of the database to be followed. To follow all the leader's databases, use '*'. | | | *leaderClusterResourceId* | The resource ID of the leader cluster. | | @@ -416,7 +419,7 @@ To verify that the database was successfully attached, find your attached databa You can also view this list in the database overview page: - :::image type="content" source="media/follower/read-only-follower-database-overview.png" alt-text="Screenshot of databases overview page with list of follower clusters."::: + :::image type="content" source="media/follower/read-only-follower-database-overview.png" alt-text="Screenshot of databases overview page with list of follower clusters."::: ### Check your leader cluster @@ -430,12 +433,12 @@ To verify that the database was successfully attached, find your attached databa :::image type="content" source="media/follower/read-write-databases-shared-overview.png" alt-text="Screenshot of overview with list of databases shared with others."::: -## Detach the follower database +### Detach the follower database > [!NOTE] -> To detach a database from the follower or leader side, you must have user, group, service principal, or managed identity with at least contributor role on the cluster from which you are detaching the database. In the example below, we use service principal. +> To detach a database from the follower or leader side, you must have user, group, service principal, or managed identity with at least contributor role on the cluster from which you're detaching the database. In the example below, we use service principal. -## [C#](#tab/csharp) +### [C#](#tab/csharp) ### Detach the attached follower database from the follower cluster using C#** @@ -611,21 +614,25 @@ Managing read-only database permission is the same as for all database types. To The follower database administrator can modify the [caching policy](/kusto/management/show-table-cache-policy-command?view=azure-data-explorer&preserve-view=true) of the attached database or any of its tables on the hosting cluster. The default is to combine the source database in the leader cluster database and table-level caching policies with the policies defined in the database and table-level override policies. You can, for example, have a 30 day caching policy on the leader database for running monthly reporting and a three day caching policy on the follower database to query only the recent data for troubleshooting. For more information about using management commands to configure the caching policy on the follower database or table, see [Management commands for managing a follower cluster](/kusto/management/cluster-follower?view=azure-data-explorer&preserve-view=true). -## Notes +### Notes + +Review the following notes: * If there are conflicts between databases of leader/follower clusters, when all databases are followed by the follower cluster, they're resolved as follows: * A database named *DB* created on the follower cluster takes precedence over a database with the same name that was created on the leader cluster. That's why database *DB* in the follower cluster needs to be removed or renamed for the follower cluster to include the leader's database *DB*. * A database named *DB* followed from two or more leader clusters will be arbitrarily chosen from *one* of the leader clusters, and won't be followed more than once. -* Commands for showing [cluster activity log and history](/kusto/management/system-info?view=azure-data-explorer&preserve-view=true) run on a follower cluster will show the activity and history on the follower cluster, and their result sets won't include those results of the leader cluster or clusters. - * For example: a `.show queries` command run on the follower cluster will only show queries run on databases followed by follower cluster, and not queries run against the same database in the leader cluster. +* Commands for showing [cluster activity log and history](/kusto/management/system-info?view=azure-data-explorer&preserve-view=true) run on a follower cluster shows the activity and history on the follower cluster, and their result sets don't include those results of the leader cluster or clusters. + * For example: a `.show queries` command run on the follower cluster shows only queries run on databases followed by follower cluster, and not queries run against the same database in the leader cluster. + +### Limitations -## Limitations +Review the following limitations: * The follower and the leader clusters must be in the same region. * If [Streaming ingestion](ingest-data-streaming.md) is used on a database that is being followed, the follower cluster should be enabled for Streaming Ingestion to allow following of streaming ingestion data. * Following a cluster with data encryption using [customer managed keys](security.md#customer-managed-keys-with-azure-key-vault) (CMK) is supported with the following limitations: - * Neither the follower cluster nor the leader cluster is following other clusters. - * If a follower cluster is following a leader cluster with CMK enabled, and the leader's access to the key is revoked, both the leader and the follower clusters will be suspended. In this situation, you can either resolve the CMK issue and then resume the follower cluster, or you can detach the follower databases from the follower cluster and resume independent of the leader cluster. + * The follower cluster or the leader cluster isn't following other clusters. + * If a follower cluster is following a leader cluster with CMK enabled, and the leader's access to the key is revoked, both the leader and the follower clusters will be suspended. In this situation, you can either resolve the CMK issue and then resume the follower cluster, or you can detach the follower databases from the follower cluster and resume independent of the leader cluster. * You can't delete a database that is attached to a different cluster before detaching it. * You can't delete a cluster that has a database attached to a different cluster before detaching it. * Table level sharing properties aren't supported when following all databases. diff --git a/data-explorer/how-it-works.md b/data-explorer/how-it-works.md index 8323328de1..58bc7e446b 100644 --- a/data-explorer/how-it-works.md +++ b/data-explorer/how-it-works.md @@ -3,7 +3,7 @@ title: How Azure Data Explorer works description: 'Learn more about how Azure Data Explorer works.' ms.reviewer: avnera ms.topic: conceptual -ms.date: 09/26/2023 +ms.date: 06/10/2025 --- # How Azure Data Explorer works diff --git a/data-explorer/ingest-data-streaming.md b/data-explorer/ingest-data-streaming.md index 004945b97e..5f46a8ce8e 100644 --- a/data-explorer/ingest-data-streaming.md +++ b/data-explorer/ingest-data-streaming.md @@ -3,7 +3,7 @@ title: Configure streaming ingestion on your Azure Data Explorer cluster description: Learn how to configure your Azure Data Explorer cluster and start loading data with streaming ingestion. ms.reviewer: alexefro ms.topic: how-to -ms.date: 05/17/2023 +ms.date: 06/10/2025 --- # Configure streaming ingestion on your Azure Data Explorer cluster @@ -25,8 +25,8 @@ Two streaming ingestion types are supported: | Ingestion type | Description | | -- | -- | -| Data connection | Event Hubs, IoT Hub, and Event Grid data connections can use streaming ingestion, provided it is enabled on the cluster level. The decision to use streaming ingestion is done according to the streaming ingestion policy configured on the target table.
For information on managing data connections, see [**Event Hub**](create-event-hubs-connection.md?tabs=portalADX), [**IoT Hub**](create-iot-hub-connection.md?tabs=portal) and [**Event Grid**](create-event-grid-connection.md). | -| Custom ingestion | Custom ingestion requires you to write an application that uses one of the Azure Data Explorer [client libraries](/kusto/api/client-libraries).
Use the information in this topic to configure custom ingestion. You may also find the [C?view=azure-data-explorer&preserve-view=true# streaming ingestion sample application](https://github.com/Azure/azure-kusto-samples-dotnet/tree/master/client/StreamingIngestionSample) helpful. | +| Data connection | Event Hubs, IoT Hub, and Event Grid data connections can use streaming ingestion, provided it's enabled on the cluster level. The decision to use streaming ingestion is done according to the streaming ingestion policy configured on the target table.
For information on managing data connections, see [**Event Hub**](create-event-hubs-connection.md?tabs=portalADX), [**IoT Hub**](create-iot-hub-connection.md?tabs=portal) and [**Event Grid**](create-event-grid-connection.md). | +| Custom ingestion | Custom ingestion requires you to write an application that uses one of the Azure Data Explorer [client libraries](/kusto/api/client-libraries).
Use the information in this article to configure custom ingestion. You may also find the [C?view=azure-data-explorer&preserve-view=true# streaming ingestion sample application](https://github.com/Azure/azure-kusto-samples-dotnet/tree/master/client/StreamingIngestionSample) helpful. | Use the following table to help you choose the ingestion type that's appropriate for your environment: @@ -36,7 +36,7 @@ Use the following table to help you choose the ingestion type that's appropriate |Development overhead | Fast and easy setup, no development overhead | High development overhead to create an application ingest the data, handle errors, and ensure data consistency | > [!NOTE] -> You can manage the process to [enable](#enable-streaming-ingestion-on-your-cluster) and [disable](#disable-streaming-ingestion-on-your-cluster) streaming ingestion on your cluster using the Azure portal or programmatically in C\#. If you are using C\# for your [custom application](#create-a-streaming-ingestion-application-to-ingest-data-to-your-cluster), you may find it more convenient using the programmatic approach. +> You can manage the process to [enable](#enable-streaming-ingestion-on-your-cluster) and [disable](#disable-streaming-ingestion-on-your-cluster) streaming ingestion on your cluster using the Azure portal or programmatically in C\#. If you're using C\# for your [custom application](#create-a-streaming-ingestion-application-to-ingest-data-to-your-cluster), you may find it more convenient using the programmatic approach. ## Prerequisites @@ -179,7 +179,7 @@ Create a table to receive the streaming ingestion data and define its related po 1. Copy one of the following commands into the **Query pane** and select **Run**. This defines the [streaming ingestion policy](/kusto/management/streaming-ingestion-policy?view=azure-data-explorer&preserve-view=true) on the table you created or on the database that contains the table. > [!TIP] - > A policy that is defined at the database level applies to all existing and future tables in the database. When you enable the policy at the database level, there is no need to enable it per table. + > A policy that is defined at the database level applies to all existing and future tables in the database. When you enable the policy at the database level, there's no need to enable it per table. * To define the policy on the table you created, use: @@ -557,7 +557,7 @@ class Program * [Data mappings](/kusto/management/mappings?view=azure-data-explorer&preserve-view=true) must be [pre-created](/kusto/management/create-ingestion-mapping-command?view=azure-data-explorer&preserve-view=true) for use in streaming ingestion. Individual streaming ingestion requests don't accommodate inline data mappings. * [Extent tags](/kusto/management/extent-tags?view=azure-data-explorer&preserve-view=true) can't be set on the streaming ingestion data. * [Update policy](/kusto/management/update-policy?view=azure-data-explorer&preserve-view=true). The update policy can reference only the newly ingested data in the source table and not any other data or tables in the database. -* When an update policy with a [transactional policy](/kusto/management/update-policy?view=azure-data-explorer&preserve-view=true#handling-failures) fails, the retries will fall back to batch ingestion. +* When an update policy with a [transactional policy](/kusto/management/update-policy?view=azure-data-explorer&preserve-view=true#handling-failures) fails, the retries fall back to batch ingestion. * If streaming ingestion is enabled on a cluster used as a leader for [follower databases](follower.md), streaming ingestion must be enabled on the following clusters as well to follow streaming ingestion data. Same applies whether the cluster data is shared via [Data Share](data-share.md). ## Related content diff --git a/data-explorer/kql-learning-resources.md b/data-explorer/kql-learning-resources.md index ae311b13b5..1e6e7e1829 100644 --- a/data-explorer/kql-learning-resources.md +++ b/data-explorer/kql-learning-resources.md @@ -3,7 +3,7 @@ title: Kusto Query Language learning resources description: This article provides a list of different types of learning resources for ramping up on KQL. ms.reviewer: alexans ms.topic: conceptual -ms.date: 09/28/2023 +ms.date: 06/10/2025 --- # Kusto Query Language learning resources @@ -12,7 +12,7 @@ For more information on KQL, see [KQL overview](/kusto/query/index?view=azure-da ## General training -This training is not product-specific, and provides basic KQL learning: +This training isn't product-specific, and provides basic KQL learning: * [Pluralsight: KQL from scratch](https://www.pluralsight.com/courses/kusto-query-language-kql-from-scratch) * [Kusto Detective Agency](https://detective.kusto.io/) diff --git a/data-explorer/manage-database-permissions.md b/data-explorer/manage-database-permissions.md index bf1a7455f0..66e1c1a289 100644 --- a/data-explorer/manage-database-permissions.md +++ b/data-explorer/manage-database-permissions.md @@ -3,7 +3,7 @@ title: Manage database permissions in Azure Data Explorer description: This article describes role-based access controls for databases and tables in Azure Data Explorer. ms.reviewer: mblythe ms.topic: how-to -ms.date: 05/28/2023 +ms.date: 06/06/2025 --- # Manage Azure Data Explorer database permissions in the Azure portal diff --git a/data-explorer/media/manage-database-permissions/new-principals.png b/data-explorer/media/manage-database-permissions/new-principals.png index 5bcae8d73a..83001050f6 100644 Binary files a/data-explorer/media/manage-database-permissions/new-principals.png and b/data-explorer/media/manage-database-permissions/new-principals.png differ diff --git a/data-explorer/power-bi-data-connector.md b/data-explorer/power-bi-data-connector.md index 0f579ceabb..5435f719ae 100644 --- a/data-explorer/power-bi-data-connector.md +++ b/data-explorer/power-bi-data-connector.md @@ -3,7 +3,7 @@ title: Use data from Azure Data Explorer in Power BI description: Learn how to use Azure Data Explorer data in Power BI. ms.reviewer: danyhoter ms.topic: how-to -ms.date: 12/29/2022 +ms.date: 06/05/2025 #Customer intent: As a data analyst, I want to understand connection options in Power BI so I can choose the option most appropriate to my scenario. --- diff --git a/data-explorer/start-for-free-web-ui.md b/data-explorer/start-for-free-web-ui.md index 6d3a46b360..afa4b0ea51 100644 --- a/data-explorer/start-for-free-web-ui.md +++ b/data-explorer/start-for-free-web-ui.md @@ -1,20 +1,20 @@ --- title: Create a free Azure Data Explorer cluster. -description: This article you'll learn how to create a free cluster, ingest data, and run queries to gain insights into your data using your free cluster. +description: This article you learn how to create a free cluster, ingest data, and run queries to gain insights into your data using your free cluster. ms.reviewer: avnera ms.topic: how-to -ms.date: 09/11/2022 +ms.date: 06/10/2025 --- # Create a free Azure Data Explorer cluster Creating your own [free cluster](start-for-free.md) gives you the opportunity to explore some of the incredible capabilities of Azure Data Explorer. In particular, it's a great way to experience our fast and highly scalable data exploration service for log and telemetry data, and use the powerful and intuitive Kusto Query Language to gain business insights into your data. -In this article, we'll show you how to create a free cluster, ingest data, and run queries using your free cluster. +In this article, we show you how to create a free cluster, ingest data, and run queries using your free cluster. ## Prerequisites -A Microsoft account or a Microsoft Entra user identity to create a free cluster. You do not require an Azure subscription or a credit card. +A Microsoft account or a Microsoft Entra user identity to create a free cluster. You don't require an Azure subscription or a credit card. ## Create your free cluster @@ -24,7 +24,7 @@ A Microsoft account or a Microsoft Entra user identity to create a free cluster. :::image type="content" source="media/start-for-free-web-ui/start-for-free-overview-page.png" alt-text="Screenshot of a free cluster, showing the overview page." lightbox="media/start-for-free-web-ui/start-for-free-overview-page.png"::: -On your cluster's overview page, you'll see the following: +On your cluster's overview page, you see the following: 1. Your cluster's name, the option to upgrade to a full cluster, and the option to delete the cluster. 1. Your cluster's location, policies, and URI links for connecting to your cluster via our [APIs](/kusto/api/index?view=azure-data-explorer&preserve-view=true) or other tools.