diff --git a/data-explorer/.openpublishing.redirection.json b/data-explorer/.openpublishing.redirection.json index 3380d7608a..4fe514c58d 100644 --- a/data-explorer/.openpublishing.redirection.json +++ b/data-explorer/.openpublishing.redirection.json @@ -459,8 +459,7 @@ "source_path": "query-exported-azure-monitor-data.md", "redirect_url": "/azure/data-explorer/query-monitor-data", "redirect_document_id": false - }, - { + }, { "source_path": "using-metrics.md", "redirect_url": "/azure/data-explorer/monitor-data-explorer", "redirect_document_id": true @@ -469,6 +468,26 @@ "source_path": "using-diagnostic-logs.md", "redirect_url": "/azure/data-explorer/monitor-data-explorer", "redirect_document_id": false + }, + { + "source_path": "vnet-create-cluster-portal.md", + "redirect_url": "/azure/data-explorer/security-network-private-endpoint-create", + "redirect_document_id": false + }, + { + "source_path": "vnet-deploy-troubleshoot.md", + "redirect_url": "/azure/data-explorer/security-network-private-endpoint-troubleshoot", + "redirect_document_id": false + }, + { + "source_path": "vnet-deployment.md", + "redirect_url": "/azure/data-explorer/security-network-overview", + "redirect_document_id": false + }, + { + "source_path": "vnet-endpoint-storage-event-hub.md", + "redirect_url": "/azure/data-explorer/security-network-managed-private-endpoint-create", + "redirect_document_id": false } ] } diff --git a/data-explorer/azure-advisor.md b/data-explorer/azure-advisor.md index fb6f9243aa..ef5bbc0140 100644 --- a/data-explorer/azure-advisor.md +++ b/data-explorer/azure-advisor.md @@ -138,7 +138,6 @@ Reliability recommendations include the following: * [Cluster uses subnet without delegation](#cluster-uses-subnet-without-delegation) * [Cluster uses subnet with invalid IP configuration](#cluster-uses-subnet-with-invalid-ip-configuration) -* [Cluster failed to install or resume due to virtual network issues](#cluster-failed-to-install-or-resume-due-to-virtual-network-issues) #### Cluster uses subnet without delegation @@ -148,10 +147,6 @@ The strong recommendation is given to a virtual network cluster that uses a subn The recommendation is given to a virtual network cluster where the subnet is also used by other services. The recommendation is to remove all other services from the subnet and only use it for your cluster. -#### Cluster failed to install or resume due to virtual network issues - -The recommendation is given to a cluster that failed to install or resume due to virtual network issues. The recommendation is to use the [virtual network troubleshooting guide](vnet-deploy-troubleshoot.md) to resolve the issue. - ## Related content * [Manage cluster horizontal scaling (scale out) in Azure Data Explorer to accommodate changing demand](manage-cluster-horizontal-scaling.md) diff --git a/data-explorer/media/vnet-create-cluster-portal/create-cluster-form-vnet.png b/data-explorer/media/vnet-create-cluster-portal/create-cluster-form-vnet.png deleted file mode 100644 index 2ed564cdca..0000000000 Binary files a/data-explorer/media/vnet-create-cluster-portal/create-cluster-form-vnet.png and /dev/null differ diff --git a/data-explorer/media/vnet-create-cluster-portal/subnets.png b/data-explorer/media/vnet-create-cluster-portal/subnets.png deleted file mode 100644 index 5f2b9349ab..0000000000 Binary files a/data-explorer/media/vnet-create-cluster-portal/subnets.png and /dev/null differ diff --git a/data-explorer/media/vnet-create-cluster-portal/vnet-blade.png b/data-explorer/media/vnet-create-cluster-portal/vnet-blade.png deleted file mode 100644 index 36151e831d..0000000000 Binary files a/data-explorer/media/vnet-create-cluster-portal/vnet-blade.png and /dev/null differ diff --git a/data-explorer/media/vnet-create-cluster-portal/vnet-create-public-ip.png b/data-explorer/media/vnet-create-cluster-portal/vnet-create-public-ip.png deleted file mode 100644 index f63f9ad9a0..0000000000 Binary files a/data-explorer/media/vnet-create-cluster-portal/vnet-create-public-ip.png and /dev/null differ diff --git a/data-explorer/media/vnet-deployment/vnet-diagram.png b/data-explorer/media/vnet-deployment/vnet-diagram.png deleted file mode 100644 index a9896fe230..0000000000 Binary files a/data-explorer/media/vnet-deployment/vnet-diagram.png and /dev/null differ diff --git a/data-explorer/media/vnet-private-link-storage-event-hub/add-networks.png b/data-explorer/media/vnet-private-link-storage-event-hub/add-networks.png deleted file mode 100644 index 697c80a080..0000000000 Binary files a/data-explorer/media/vnet-private-link-storage-event-hub/add-networks.png and /dev/null differ diff --git a/data-explorer/media/vnet-private-link-storage-event-hub/event-hub-add-existing-vnet.png b/data-explorer/media/vnet-private-link-storage-event-hub/event-hub-add-existing-vnet.png deleted file mode 100644 index 48e5890c24..0000000000 Binary files a/data-explorer/media/vnet-private-link-storage-event-hub/event-hub-add-existing-vnet.png and /dev/null differ diff --git a/data-explorer/media/vnet-private-link-storage-event-hub/event-hub-firewalls-and-vnet.png b/data-explorer/media/vnet-private-link-storage-event-hub/event-hub-firewalls-and-vnet.png deleted file mode 100644 index a86199a8c5..0000000000 Binary files a/data-explorer/media/vnet-private-link-storage-event-hub/event-hub-firewalls-and-vnet.png and /dev/null differ diff --git a/data-explorer/media/vnet-private-link-storage-event-hub/networking.png b/data-explorer/media/vnet-private-link-storage-event-hub/networking.png deleted file mode 100644 index 28e40d9600..0000000000 Binary files a/data-explorer/media/vnet-private-link-storage-event-hub/networking.png and /dev/null differ diff --git a/data-explorer/media/vnet-private-link-storage-event-hub/storage-add-existing-vnet.png b/data-explorer/media/vnet-private-link-storage-event-hub/storage-add-existing-vnet.png deleted file mode 100644 index fb80c71867..0000000000 Binary files a/data-explorer/media/vnet-private-link-storage-event-hub/storage-add-existing-vnet.png and /dev/null differ diff --git a/data-explorer/media/vnet-private-link-storage-event-hub/storage-add-networks.png b/data-explorer/media/vnet-private-link-storage-event-hub/storage-add-networks.png deleted file mode 100644 index 4d3dc7d415..0000000000 Binary files a/data-explorer/media/vnet-private-link-storage-event-hub/storage-add-networks.png and /dev/null differ diff --git a/data-explorer/media/vnet-private-link-storage-event-hub/storage-virtual-network.png b/data-explorer/media/vnet-private-link-storage-event-hub/storage-virtual-network.png deleted file mode 100644 index b8aacf75c5..0000000000 Binary files a/data-explorer/media/vnet-private-link-storage-event-hub/storage-virtual-network.png and /dev/null differ diff --git a/data-explorer/policy-reference.md b/data-explorer/policy-reference.md index 4f3ef135d9..eca592d79b 100644 --- a/data-explorer/policy-reference.md +++ b/data-explorer/policy-reference.md @@ -22,7 +22,6 @@ the link in the **Version** column to view the source on the |[Azure Data Explorer encryption at rest should use a customer-managed key](https://portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F81e74cea-30fd-40d5-802f-d72103c2aaaa) |Enabling encryption at rest using a customer-managed key on your Azure Data Explorer cluster provides additional control over the key being used by the encryption at rest. This feature is oftentimes applicable to customers with special compliance requirements and requires a Key Vault to managing the keys. |Audit, Deny, Disabled |[1.0.0](https://github.com/Azure/azure-policy/blob/master/built-in-policies/policyDefinitions/Azure%20Data%20Explorer/ADX_CMK.json) | |[Disk encryption should be enabled on Azure Data Explorer](https://portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2Ff4b53539-8df9-40e4-86c6-6b607703bd4e) |Enabling disk encryption helps protect and safeguard your data to meet your organizational security and compliance commitments. |Audit, Deny, Disabled |[2.0.0](https://github.com/Azure/azure-policy/blob/master/built-in-policies/policyDefinitions/Azure%20Data%20Explorer/ADX_disk_encrypted.json) | |[Double encryption should be enabled on Azure Data Explorer](https://portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2Fec068d99-e9c7-401f-8cef-5bdde4e6ccf1) |Enabling double encryption helps protect and safeguard your data to meet your organizational security and compliance commitments. When double encryption has been enabled, data in the storage account is encrypted twice, once at the service level and once at the infrastructure level, using two different encryption algorithms and two different keys. |Audit, Deny, Disabled |[2.0.0](https://github.com/Azure/azure-policy/blob/master/built-in-policies/policyDefinitions/Azure%20Data%20Explorer/ADX_doubleEncryption.json) | -|[Virtual network injection should be enabled for Azure Data Explorer](https://portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F9ad2fd1f-b25f-47a2-aa01-1a5a779e6413) |Secure your network perimeter with virtual network injection which allows you to enforce network security group rules, connect on-premises and secure your data connection sources with service endpoints. |Audit, Deny, Disabled |[1.0.0](https://github.com/Azure/azure-policy/blob/master/built-in-policies/policyDefinitions/Azure%20Data%20Explorer/ADX_VNET_configured.json) | |[Azure Data Explorer should use a SKU that supports private link](https://ms.portal.azure.com/#view/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F1fec9658-933f-4b3e-bc95-913ed22d012b) |With supported SKUs, Azure Private Link lets you connect your virtual network to Azure services without a public IP address at the source or destination. The Private Link platform handles the connectivity between the consumer and services over the Azure backbone network. By mapping private endpoints to apps, you can reduce data leakage risks. |Audit, Deny, Disabled |[1.0.0](https://github.com/Azure/azure-policy/blob/master/built-in-policies/policyDefinitions/Azure%20Data%20Explorer/ADX_PrivateEndpoint_NonPeSku_Deny.json) | |[Public network access should be disabled on Azure Data Explorer](https://portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F43bc7be6-5e69-4b0d-a2bb-e815557ca673) |Disabling the public network access property improves security by ensuring Azure Data Explorer can only be accessed from a private endpoint. This configuration denies the creation of Azure Data Explorer clusters with public network access enabled. |Audit, Deny, Disabled |[1.0.0](https://github.com/Azure/azure-policy/blob/master/built-in-policies/policyDefinitions/Azure%20Data%20Explorer/ADX_PublicAccess_Deny.json) | |[Configure Azure Data Explorer to disable public network access](https://ms.portal.azure.com/#view/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F7b32f193-cb28-4e15-9a98-b9556db0bafa) |Disabling the public network access property shuts down public connectivity such that Azure Data Explorer can only be accessed from a private endpoint. This configuration disables the public network access for all Azure Data Explorer clusters. |Modify, Disabled |[1.0.0](https://github.com/Azure/azure-policy/blob/master/built-in-policies/policyDefinitions/Azure%20Data%20Explorer/ADX_DisablePublicAccess_Modify.json) | diff --git a/data-explorer/security-network-migrate-vnet-to-private-endpoint.md b/data-explorer/security-network-migrate-vnet-to-private-endpoint.md index e9646de1ea..d8aa4c077e 100644 --- a/data-explorer/security-network-migrate-vnet-to-private-endpoint.md +++ b/data-explorer/security-network-migrate-vnet-to-private-endpoint.md @@ -8,7 +8,10 @@ ms.date: 10/07/2024 # Migrate a Virtual Network injected cluster to private endpoints -This article describes the migration of a Microsoft Azure Virtual Network injected Azure Data Explorer cluster to an Azure Private Endpoints network security model. For a detailed comparison, see [Private endpoint vs. virtual network injection](security-network-overview.md#private-endpoint-vs-virtual-network-injection). +> [!WARNING] +> Virtual Network Injection was retired for Azure Data Explorer by 1 February 2025. + +This article describes the migration of a Microsoft Azure Virtual Network injected Azure Data Explorer cluster to an Azure Private Endpoints network security model. The process of the migration takes several minutes. The migration creates a new cluster for the engine and data management services, which reside in a virtual network managed by Microsoft. The connection is switched to the newly created services for you. This process results in a minimal downtime for querying the cluster. diff --git a/data-explorer/security-network-overview.md b/data-explorer/security-network-overview.md index f98c75ecc3..164a072f2b 100644 --- a/data-explorer/security-network-overview.md +++ b/data-explorer/security-network-overview.md @@ -3,19 +3,16 @@ title: Network security for Azure Data Explorer cluster description: 'Learn about the different options to secure your Azure Data Explorer cluster applying network security measures.' ms.reviewer: basaba ms.topic: reference -ms.date: 07/23/2024 +ms.date: 06/02/2025 --- # Network security for Azure Data Explorer -Azure Data Explorer clusters are designed to be accessible using public URLs. Anyone with valid identity on a cluster can access it from any location. As an organization, securing data may be one your highest priority tasks. As such, you may want to limit and secure access to your cluster, or even only allow access to your cluster through your private virtual network. You can use one of the following options to achieve this goal: +Azure Data Explorer clusters are designed to be accessible using public URLs. Anyone with valid identity on a cluster can access it from any location. As an organization, securing data may be one your highest priority tasks. As such, you may want to limit and secure access to your cluster, or even only allow access to your cluster through your private virtual network. To achieve this goal, use: -* [Private endpoint](#private-endpoint) (recommended) -* [Virtual network (VNet) injection](#virtual-network-injection) +* [Private endpoint](#private-endpoint) -We highly recommended using *private endpoints* to secure network access to your cluster. This option has many advantages over *virtual network injection* that results in lower maintenance overhead, including a simpler deployment process and being more robust to virtual network changes. - -The following section explains how to secure your cluster using private endpoints and virtual network injection. +The following section explains how to secure your cluster using private endpoints. ## Private endpoint @@ -25,38 +22,16 @@ A private endpoint is a network interface that uses private IP addresses from yo To successfully deploy your cluster into a private endpoint, you only require a set of private IP addresses. -> [!NOTE] -> Private endpoints aren't supported for a cluster that's injected into a virtual network. - -## Virtual network injection - -> [!WARNING] -> Virtual Network Injection will be retired for Azure Data Explorer by 1 February 2025. For more information on the deprecation, see [Deprecation of Virtual Network Injection for Azure Data Explorer](https://aka.ms/adx.security.vnet.deprecation). - -Virtual network injection allows you to directly deploy your cluster into a virtual network. The cluster can be privately accessed from within the virtual network and over a VPN gateway, or Azure ExpressRoute from on-premises networks. Injecting a cluster into a virtual network enables you to manage all of its traffic. This includes the traffic to access the cluster and all of its data ingestion or exports. Additionally, you're responsible to allow Microsoft to access the cluster for management and health monitoring. - -:::image type="content" source="media/vnet-deployment/vnet-diagram.png" alt-text="Diagram showing the schema of the virtual network injection architecture."::: - -To successfully inject your cluster into a virtual network, you must configure your virtual network to meet the following requirements: - -* You must delegate the subnet to *Microsoft.Kusto/clusters* to enable the service and to define its preconditions for deployment in the form of *network intent policies* -* The subnet needs to be well scaled to support future growth of the cluster's usage -* Two public IP addresses are required to manage the cluster and ensure that it's healthy -* Optionally, if you're using an additional firewall appliance to secure your network, you must allow your cluster to connect to a set of Fully Qualified Domain Names (FQDNs) for outgoing traffic - -## Private endpoint vs. virtual network injection - -Virtual network injection can lead to a high maintenance overhead, as a result of implementation details such as maintaining FQDN lists in firewalls or deploying public IP addresses in a restricted environment. Therefore, we recommend using a private endpoint to connect to your cluster. +## Network security features with private endpoints -The following table shows how network security related features could be implemented based on a cluster injected into a virtual network or secured using a private endpoint. +The following table shows how network security related features can be implemented using a private endpoint: -| Feature | Private endpoint | Virtual network injection | -|--- |--- |--- | -| Inbound IP address filtering | [Manage public access](security-network-restrict-public-access.md) | [Create an inbound Network Security Group rule](/azure/virtual-network/network-security-groups-overview) | -| Transitive access to other services (Storage, Event Hubs, etc.) | [Create a managed private endpoint](security-network-managed-private-endpoint-create.md) | [Create a private endpoint to the resource](vnet-endpoint-storage-event-hub.md) | -| Restricting outbound access | Use [Callout policies or the AllowedFQDNList](security-network-restrict-outbound-access.md) | Use a [virtual appliance](/azure/firewall/tutorial-firewall-deploy-portal) to the subnet's filter outgoing traffic | +| Feature | Private endpoint | +|--- |--- | +| Inbound IP address filtering | [Manage public access](security-network-restrict-public-access.md) | +| Transitive access to other services (Storage, Event Hubs, etc.) | [Create a managed private endpoint](security-network-managed-private-endpoint-create.md) | +| Restricting outbound access | Use [Callout policies or the AllowedFQDNList](security-network-restrict-outbound-access.md) | ## Related content -* [Private Endpoints for Azure Data Explorer](security-network-private-endpoint.md) -* [Deploy Azure Data Explorer into your Virtual Network](vnet-deployment.md) \ No newline at end of file +* [Private Endpoints for Azure Data Explorer](security-network-private-endpoint.md) \ No newline at end of file diff --git a/data-explorer/security.md b/data-explorer/security.md index 1d49ae7a46..58527390c2 100644 --- a/data-explorer/security.md +++ b/data-explorer/security.md @@ -14,12 +14,11 @@ For more resources regarding compliance for your business or organization, see t ## Network security -Network security is a requirement shared by many of our security-conscious enterprise customers. The intent is to isolate the network traffic and limit the attack surface for Azure Data Explorer and corresponding communications. You can therefore block traffic originating from non-Azure Data Explorer network segments and assure that only traffic from known sources reach Azure Data Explorer end points. This includes traffic originating on-premises or outside of Azure, with an Azure destination and vice versa. Azure Data Explorer supports the following features to achieve this goal: +Network security is a requirement shared by many of our security-conscious enterprise customers. The intent is to isolate the network traffic and limit the attack surface for Azure Data Explorer and corresponding communications. You can therefore block traffic originating from non-Azure Data Explorer network segments and assure that only traffic from known sources reach Azure Data Explorer end points. This includes traffic originating on-premises or outside of Azure, with an Azure destination and vice versa. -* [Private endpoint](security-network-overview.md#private-endpoint) (recommended) -* [Virtual network (VNet) injection](security-network-overview.md#virtual-network-injection) +Azure Data Explorer supports private endpoints to achieve network isolation and security. Private endpoints provide a secure way to connect to your Azure Data Explorer cluster by using a private IP address from your virtual network, effectively bringing the service into your VNet. This ensures that traffic between your VNet and the service travels over the Microsoft backbone network, eliminating exposure from the public internet. -We highly recommended using private endpoints to secure network access to your cluster. This option has many advantages over virtual network injection that results in lower maintenance overhead, including a simpler deployment process and being more robust to virtual network changes. +For more information about configuring private endpoints for your cluster, see [Private endpoint](security-network-overview.md#private-endpoint). ## Identity and access control diff --git a/data-explorer/toc.yml b/data-explorer/toc.yml index de47ac75c9..1a463ee8aa 100644 --- a/data-explorer/toc.yml +++ b/data-explorer/toc.yml @@ -79,7 +79,7 @@ items: href: cross-tenant-query-and-commands.md - name: Isolated compute href: isolated-compute.md - - name: Network isolation (VNet) + - name: Network security items: - name: Network security overview href: security-network-overview.md @@ -96,19 +96,8 @@ items: displayName: restrict, prevent, limit, IP addresses, allowlist, whitelist, VNET, private endpoint, virtual network - name: Troubleshoot private endpoints href: security-network-private-endpoint-troubleshoot.md - - name: Virtual network injection - items: - - name: Migrate VNet injected cluster to private endpoints - href: security-network-migrate-vnet-to-private-endpoint.md - - name: Create a cluster in your VNet - Portal - href: vnet-create-cluster-portal.md - - name: Deploy your cluster to your VNet - href: vnet-deployment.md - - name: Create a private or service endpoint to resources used by data connections - displayName: VNet, private endpoint, Azure Storage, event hub - href: vnet-endpoint-storage-event-hub.md - - name: Troubleshoot VNet cluster creation, connectivity, and operation - href: vnet-deploy-troubleshoot.md + - name: Migrate VNet injected cluster to private endpoints + href: security-network-migrate-vnet-to-private-endpoint.md - name: Restrict outbound requests href: security-network-restrict-outbound-access.md - name: Encryption diff --git a/data-explorer/vnet-create-cluster-portal.md b/data-explorer/vnet-create-cluster-portal.md deleted file mode 100644 index 16b5743fed..0000000000 --- a/data-explorer/vnet-create-cluster-portal.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -title: Create an Azure Data Explorer cluster & DB in your virtual network -description: 'In this article, you learn how to create an Azure Data Explorer cluster in your virtual network.' -ms.reviewer: basaba -ms.topic: how-to -ms.date: 07/23/2024 - -#Customer intent: As a database administrator, I want to create an Azure Data Explorer cluster and database in my virtual network. ---- - -# Create an Azure Data Explorer cluster in your virtual network - -Azure Data Explorer supports deploying a cluster into a subnet in your virtual network (VNet). This capability enables you to access the cluster privately from your Azure virtual network or on-premises, access resource such as Event Hubs and Azure Storage inside your virtual network, and restrict inbound and outbound traffic. - -> [!WARNING] -> Virtual Network Injection will be retired for Azure Data Explorer by 1 February 2025. For more information on the deprecation, see [Deprecation of Virtual Network Injection for Azure Data Explorer](https://aka.ms/adx.security.vnet.deprecation). - -**See also** - -> [!div class="nextstepaction"] -> [Deploy Azure Data Explorer into your Virtual Network](vnet-deployment.md) diff --git a/data-explorer/vnet-deploy-troubleshoot.md b/data-explorer/vnet-deploy-troubleshoot.md deleted file mode 100644 index 30f633b77a..0000000000 --- a/data-explorer/vnet-deploy-troubleshoot.md +++ /dev/null @@ -1,162 +0,0 @@ ---- -title: Troubleshoot access, ingestion, and operation of your Azure Data Explorer cluster in your virtual network -description: Troubleshoot connectivity, ingestion, cluster creation, and operation of your Azure Data Explorer cluster in your virtual network -ms.reviewer: basaba -ms.topic: how-to -ms.date: 07/23/2024 ---- - -# Troubleshoot access, ingestion, and operation of your Azure Data Explorer cluster in your virtual network - -> [!WARNING] -> Virtual Network Injection will be retired for Azure Data Explorer by 1 February 2025. For more information on the deprecation, see [Deprecation of Virtual Network Injection for Azure Data Explorer](https://aka.ms/adx.security.vnet.deprecation). - -In this section you learn how to troubleshoot connectivity, operational, and cluster creation issues for a cluster that is deployed into your [Virtual Network](/azure/virtual-network/virtual-networks-overview). - -## Access issues - -If you have an issue while accessing cluster using the public (cluster.region.kusto.windows.net) or private (private-cluster.region.kusto.windows.net) endpoint and you suspect it's related to virtual network setup, perform the following steps to troubleshoot the issue. - -### Check TCP connectivity - -The first step includes checking TCP connectivity using Windows or Linux OS. - -#### [Windows](#tab/windows) - -1. Download [TCping](https://www.elifulkerson.com/projects/tcping.php) to the machine connecting to the cluster. -1. Ping the destination from the source machine by using the following command: - - ```cmd - C:\> tcping -t yourcluster.kusto.windows.net 443 - ** Pinging continuously. Press control-c to stop ** - Probing 1.2.3.4:443/tcp - Port is open - time=100.00ms - ``` - -#### [Linux](#tab/linux) - -1. Install *netcat* in the machine connecting to the cluster - - ```bash - apt-get install netcat - ``` - -1. Ping the destination from the source machine by using the following command: - - ```bash - $ netcat -z -v yourcluster.kusto.windows.net 443 - Connection to yourcluster.kusto.windows.net 443 port [tcp/https] succeeded! - ``` - ---- - -If the test isn't successful, proceed with the following steps. If the test is successful, the issue isn't due to a TCP connectivity issue. Go to [operational issues](#cluster-creation-and-operations-issues) to troubleshoot further. - -### Check Network Security Group (NSG) rules - -Check that the [NSG](/azure/virtual-network/security-overview) attached to the cluster's subnet, has an inbound rule that allows access from the client machine's IP for port 443. - -### Check the route table is configured to prevent access issues - -If the cluster's subnet is configured to force tunnel all internet-bound traffic back to your firewall (subnet with a [route table](/azure/virtual-network/virtual-networks-udr-overview) that contains the default route '0.0.0.0/0'), make sure that the machine IP address has a route with [next hop type](/azure/virtual-network/virtual-networks-udr-overview) to VirtualNetwork/Internet. This route is required to prevent asymmetric route issues. - -## Ingestion issues - -If you're experiencing ingestion issues and you suspect it's related to virtual network setup, perform the following steps. - -### Check ingestion health - -Check that the [cluster ingestion metrics](using-metrics.md#ingestion-metrics) indicate a healthy state. - -### Check security rules on data source resources - -If the metrics indicate that no events were processed from the data source (*Events processed* metric for Event/IoT Hubs), make sure that the data source resources (Event Hubs or Storage) allow access from cluster's subnet in the firewall rules or service endpoints. - -### Check security rules configured on cluster's subnet - -Make sure cluster's subnet has NSG, UDR, and firewall rules are properly configured. In addition, test network connectivity for all dependent endpoints. - -## Cluster creation and operations issues - -If you're experiencing cluster creation or operation issues and you suspect it's related to virtual network setup, follow these steps to troubleshoot the issue. - -### Check the "DNS servers" configuration - -Setting up Private Endpoint requires configuring DNS, We support Azure Private DNS zone setup only. Custom DNS server setup isn't support, check that the records that were created as part of private endpoint are registered to Azure Private DNS zone. - -### Diagnose the virtual network with the REST API - -The [ARMClient](https://chocolatey.org/packages/ARMClient) is used to call the REST API using PowerShell. - -1. Sign in with ARMClient - - ```powerShell - armclient login - ``` - -1. Invoke diagnose operation - - ```powershell - $subscriptionId = '' - $clusterName = '' - $resourceGroupName = '' - $apiversion = '2019-11-09' - - armclient post "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Kusto/clusters/$clusterName/diagnoseVirtualNetwork?api-version=$apiversion" - verbose - ``` - -1. Check the response - - ```powershell - HTTP/1.1 202 Accepted - ... - Azure-AsyncOperation: https://management.azure.com/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationResults/{operation-id}?api-version=2019-11-09 - ... - ``` - -1. Wait for operation completion - - ```powershell - armclient get https://management.azure.com/subscriptions/$subscriptionId/providers/Microsoft.Kusto/locations/{location}/operationResults/{operation-id}?api-version=2019-11-09 - - { - "id": "/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationresults/{operation-id}", - "name": "{operation-name}", - "status": "[Running/Failed/Completed]", - "startTime": "{start-time}", - "endTime": "{end-time}", - "properties": {...} - } - ``` - - Wait until the *status* property shows *Completed*, then the *properties* field should show: - - ```powershell - { - "id": "/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationresults/{operation-id}", - "name": "{operation-name}", - "status": "Completed", - "startTime": "{start-time}", - "endTime": "{end-time}", - "properties": { - "Findings": [...] - } - } - ``` - -If the *Findings* property shows an empty result, it means that all network tests passed and no connections are broken. If the following error is shown, *Outbound dependency '{dependencyName}:{port}' might be not satisfied (Outbound)*, the cluster can't reach the dependent service endpoints. Proceed with the following steps. - -### Check NSG rules - -Make sure that the [NSG](/azure/virtual-network/security-overview) is configured properly per the instructions in [Configure Network Security Group rules](vnet-deployment.md#configure-network-security-group-rules). - -### Check the route table is configured to prevent ingestion issues - -If the cluster's subnet is configured to force tunnel all internet-bound traffic back to your firewall (subnet with a [route table](/azure/virtual-network/virtual-networks-udr-overview) that contains the default route '0.0.0.0/0') make sure that the [management IP addresses](vnet-deployment.md#azure-data-explorer-management-ip-addresses)) and [health monitoring IP addresses](vnet-deployment.md#health-monitoring-addresses) have a route with [next hop type](/azure/virtual-network/virtual-networks-udr-overview##next-hop-types-across-azure-tools) *Internet*, and [source address prefix](/azure/virtual-network/virtual-networks-udr-overview#how-azure-selects-a-route) to *'management-ip/32'* and *'health-monitoring-ip/32'*. This route required to prevent asymmetric route issues. - -### Check firewall rules - -If you force tunnel subnet outbound traffic to a firewall, make sure all dependencies FQDN (for example, *.blob.core.windows.net*) are allowed in the firewall configuration as described in [securing outbound traffic with firewall](vnet-deployment.md#securing-outbound-traffic-with-a-firewall). - -## Cluster suspension issues - -If the cluster fails to suspend, confirm that there aren't any locks on the networking resources in your subscription. diff --git a/data-explorer/vnet-deployment.md b/data-explorer/vnet-deployment.md deleted file mode 100644 index acb67301c4..0000000000 --- a/data-explorer/vnet-deployment.md +++ /dev/null @@ -1,405 +0,0 @@ ---- -title: Deploy Azure Data Explorer into your Virtual Network -description: Learn how to deploy Azure Data Explorer into your Virtual Network -ms.reviewer: basaba -ms.topic: how-to -ms.custom: devx-track-arm-template -ms.date: 07/23/2024 ---- - -# Deploy Azure Data Explorer cluster into your Virtual Network - -> [!WARNING] -> Virtual Network Injection will be retired for Azure Data Explorer by 1 February 2025. For more information on the deprecation, see [Deprecation of Virtual Network Injection for Azure Data Explorer](https://aka.ms/adx.security.vnet.deprecation). - -This article explains the resources that are present when you deploy an Azure Data Explorer cluster into a custom Azure Virtual Network. This information will help you deploy a cluster into a subnet in your Virtual Network (VNet). For more information on Azure Virtual Networks, see [What is Azure Virtual Network?](/azure/virtual-network/virtual-networks-overview) - -:::image type="content" source="media/vnet-deployment/vnet-diagram.png" alt-text="diagram showing schematic virtual network architecture."::: - -Azure Data Explorer supports deploying a cluster into a subnet in your Virtual Network (VNet). This capability enables you to: - -* Enforce [Network Security Group](/azure/virtual-network/security-overview) (NSG) rules on your Azure Data Explorer cluster traffic. -* Connect your on-premises network to Azure Data Explorer cluster's subnet. -* Secure your data connection sources ([Event Hubs](/azure/event-hubs/event-hubs-about) and [Event Grid](/azure/event-grid/overview)) with [service endpoints](/azure/virtual-network/virtual-network-service-endpoints-overview). - -## Access your Azure Data Explorer cluster in your virtual network - -You can access your Azure Data Explorer cluster using the following IP addresses for each service (engine and data management services): - -* **Private IP**: Used for accessing the cluster inside the virtual network. -* **Public IP**: Used for accessing the cluster from outside the virtual network for management and monitoring, and as a source address for outbound connections started from the cluster. - -> [!IMPORTANT] -> The default NSG rules block access to public IP addresses outside the virtual network. To reach a public endpoint, you must add an exception for your public IP addresses in the NSG. - -The following DNS records are created to access the service: - -* `[clustername].[geo-region].kusto.windows.net` (engine) `ingest-[clustername].[geo-region].kusto.windows.net` (data management) are mapped to the public IP for each service. - -* `private-[clustername].[geo-region].kusto.windows.net` (engine) `ingest-private-[clustername].[geo-region].kusto.windows.net`\\`private-ingest-[clustername].[geo-region].kusto.windows.net` (data management) are mapped to the private IP for each service. - -## Plan subnet size in your virtual network - -The size of the subnet used to host an Azure Data Explorer cluster can't be altered after the subnet is deployed. In your virtual network, Azure Data Explorer uses one private IP address for each VM and two private IP addresses for the internal load balancers (engine and data management). Azure networking also uses five IP addresses for each subnet. Azure Data Explorer provisions two VMs for the data management service. Engine service VMs are provisioned per user configuration scale capacity. - -The total number of IP addresses: - -| Use | Number of addresses | -|--|--| -| Engine service | 1 per instance | -| Data management service | 2 | -| Internal load balancers | 2 | -| Azure reserved addresses | 5 | -| **Total** | **#engine_instances + 9** | - -> [!IMPORTANT] -> -> - Make sure that you plan the subnet size before deploying Azure Data Explorer. Once deployed, the subnet size cannot be changed. -> - Make sure that you don't deploy any other Azure resources or services in the Subnet where you plan to deploy Azure Data Explorer. Doing so will prevent Azure Data Explorer starting when resuming from a suspended state. - -## Service endpoints for connecting to Azure Data Explorer - -[Azure Service Endpoints](/azure/virtual-network/virtual-network-service-endpoints-overview) enables you to secure your Azure multitenant resources to your virtual network. -Deploying the cluster into your subnet allows you to set up data connections with [Event Hubs](/azure/event-hubs/event-hubs-about) or [Event Grid](/azure/event-grid/overview) while restricting the underlying resources for Azure Data Explorer subnet. - -## Private Endpoints - -[Private Endpoints](/azure/private-link/private-endpoint-overview) allow private access to Azure resources (such as [storage/event hub](vnet-endpoint-storage-event-hub.md)/Data Lake Gen 2), and use private IP from your Virtual Network, effectively bringing the resource into your virtual network. -Create a [private endpoint](/azure/private-link/private-endpoint-overview) to resources used by data connections, such as event hub and storage, and external tables such as Storage, Data Lake Gen 2, and SQL Database from your virtual network to access the underlying resources privately. - - > [!NOTE] - > Setting up Private Endpoint requires [configuring DNS](/azure/private-link/private-endpoint-dns), We support [Azure Private DNS zone](/azure/dns/private-dns-privatednszone) setup only. Custom DNS server isn't supported. - -## Configure Network Security Group rules - -[NSGs](/azure/virtual-network/security-overview) give you the ability to control network access within a virtual network. You must configure NSGs for your Azure Data Explorer cluster to work in your virtual network. - -### Configure Network Security Group rules using subnet delegation - -[Subnet delegation](/azure/virtual-network/subnet-delegation-overview) is the default method for configuring Network Security Group rules for Azure Data Explorer clusters deployed into a subnet in your virtual network. When using subnet delegation, you must delegate the subnet to *Microsoft.Kusto/clusters* before creating the cluster in the subnet. - -By enabling subnet delegation on the cluster's subnet, you enable the service to define its pre-conditions for deployment in the form of Network Intent Policies. When creating the cluster in the subnet, the NSG configurations mentioned in the following sections are automatically created for you. - -> [!WARNING] -> Changing your subnet delegation configuration will eventually disrupt the normal operation of your cluster. For example, after stopping the cluster you may not be able to start your cluster, run management commands, or apply health monitoring on your cluster. - -### Configure Network Security Group rules manually - -Alternatively, you can manually configure your NSG. By default, deploying a cluster into a virtual network enforces subnet delegation for "Microsoft.Kusto/clusters" to be configured. Opting out of this requirement is possible using the [Preview features](https://portal.azure.com/#blade/Microsoft_Azure_Resources/PreviewFeaturesBlade) pane. - - > [!WARNING] - > Manually configuring NSG rules for your cluster is not trivial and requires you to constantly monitor this article for changes. We highly recommended using subnet delegation for your cluster or, if you prefer, consider using a [Private Endpoint](security-network-private-endpoint.md) based solution. - -#### Inbound NSG configuration - -| **Use** | **From** | **To** | **Protocol** | -|--|--|--|--| -| Management | [Azure Data Explorer management addresses](#azure-data-explorer-management-ip-addresses)/AzureDataExplorerManagement(ServiceTag) | *YourAzureDataExplorerSubnet*:443 | TCP | -| Health monitoring | [Azure Data Explorer health monitoring addresses](#health-monitoring-addresses) | *YourAzureDataExplorerSubnet*:443 | TCP | -| Azure Data Explorer internal communication | *YourAzureDataExplorerSubnet*: All ports | *YourAzureDataExplorerSubnet*:All ports | All | -| Allow Azure load balancer inbound (health probe) | AzureLoadBalancer | *YourAzureDataExplorerSubnet*:80,443 | TCP | - -#### Outbound NSG configuration - -| **Use** | **From** | **To** | **Protocol** | -|--|--|--|--| -| Dependency on Azure Storage | *YourAzureDataExplorerSubnet* | Storage:443 | TCP | -| Dependency on Azure Data Lake | *YourAzureDataExplorerSubnet* | AzureDataLake:443 | TCP | -| Event Hubs ingestion and service monitoring | *YourAzureDataExplorerSubnet* | EventHub:443,5671 | TCP | -| Publish Metrics | *YourAzureDataExplorerSubnet* | AzureMonitor:443 | TCP | -| Active Directory (if applicable) | *YourAzureDataExplorerSubnet* | AzureActiveDirectory:443 | TCP | -| Dependency on KeyVault | *YourAzureDataExplorerSubnet* | AzureKeyVault:443 | TCP | -| Certificate authority | *YourAzureDataExplorerSubnet* | Internet:80 | TCP | -| Internal communication | *YourAzureDataExplorerSubnet* | Azure Data Explorer Subnet:All Ports | All | -| Ports that are used for `sql\_request` and `http\_request` plugins | *YourAzureDataExplorerSubnet* | Internet:Custom | TCP | - -The following sections list the relevant IP addresses for management and health monitoring. - -> [!NOTE] -> You can disregard the following lists if your subnet is delegated to *Microsoft.Kusto/clusters* as described in [Configure Network Security Group rules using subnet delegation](#configure-network-security-group-rules-using-subnet-delegation). In this scenario, IP addresses may be not be up to date but will be automatically updated when the required NSG rules are assigned to the cluster. - -#### Azure Data Explorer management IP addresses - -> [!NOTE] -> For future deployments, use AzureDataExplorer Service Tag - -| Region | Addresses | -| --- | --- | -| Australia Central | 20.37.26.134 | -| Australia Central 2 | 20.39.99.177 | -| Australia East | 40.82.217.84 | -| Australia Southeast | 20.40.161.39 | -| Brazil South | 191.233.25.183 | -| Brazil Southeast | 191.232.16.14 | -| Canada Central | 40.82.188.208 | -| Canada East | 40.80.255.12 | -| Central India | 40.81.249.251, 104.211.98.159 | -| Central US | 40.67.188.68 | -| Central US EUAP | 40.89.56.69 | -| China East 2 | 139.217.184.92 | -| China North 2 | 139.217.60.6 | -| East Asia | 20.189.74.103 | -| East US | 52.224.146.56 | -| East US 2 | 52.232.230.201 | -| East US 2 EUAP | 52.253.226.110 | -| France Central | 40.66.57.91 | -| France South | 40.82.236.24 | -| Germany West Central | 51.116.98.150 | -| Japan East | 20.43.89.90 | -| Japan West | 40.81.184.86 | -| Korea Central | 40.82.156.149 | -| Korea South | 40.80.234.9 | -| North Central US | 40.81.43.47 | -| North Europe | 52.142.91.221 | -| Norway East | 51.120.49.100 | -| Norway West | 51.120.133.5 | -| Poland Central | 20.215.208.177 | -| South Africa North | 102.133.129.138 | -| South Africa West | 102.133.0.97 | -| South Central US | 20.45.3.60 | -| Southeast Asia | 40.119.203.252 | -| South India | 40.81.72.110, 104.211.224.189 | -| Switzerland North | 20.203.198.33 | -| Switzerland West | 51.107.98.201 | -| UAE Central | 20.37.82.194 | -| UAE North | 20.46.146.7 | -| UK South | 40.81.154.254 | -| UK West | 40.81.122.39 | -| USDoD Central | 52.182.33.66 | -| USDoD East | 52.181.33.69 | -| USGov Arizona | 52.244.33.193 | -| USGov Texas | 52.243.157.34 | -| USGov Virginia | 52.227.228.88 | -| West Central US | 52.159.55.120 | -| West Europe | 51.145.176.215 | -| West India | 40.81.88.112 | -| West US | 13.64.38.225 | -| West US 2 | 40.90.219.23 | -| West US 3 | 20.40.24.116 | - -#### Health monitoring addresses - -| Region | Addresses | -| --- | --- | -| Australia Central | 52.163.244.128, 20.36.43.207, 20.36.44.186, 20.36.45.105, 20.36.45.34, 20.36.44.177, 20.36.45.33, 20.36.45.9 | -| Australia Central 2 | 52.163.244.128 | -| Australia East | 52.163.244.128, 13.70.72.44, 52.187.248.59, 52.156.177.51, 52.237.211.110, 52.237.213.135, 104.210.70.186, 104.210.88.184, 13.75.183.192, 52.147.30.27, 13.72.245.57 | -| Australia Southeast | 52.163.244.128, 13.77.50.98, 52.189.213.18, 52.243.76.73, 52.189.194.173, 13.77.43.81, 52.189.213.33, 52.189.216.81, 52.189.233.66, 52.189.212.69, 52.189.248.147 | -| Brazil South | 23.101.115.123, 191.233.203.34, 191.232.48.69, 191.232.169.24, 191.232.52.16, 191.239.251.52, 191.237.252.188, 191.234.162.82, 191.232.49.124, 191.232.55.149, 191.232.49.236 | -| Canada Central | 23.101.115.123, 52.228.121.143, 52.228.121.146, 52.228.121.147, 52.228.121.149, 52.228.121.150, 52.228.121.151, 20.39.136.152, 20.39.136.155, 20.39.136.180, 20.39.136.185, 20.39.136.187, 20.39.136.193, 52.228.121.152, 52.228.121.153, 52.228.121.31, 52.228.118.139, 20.48.136.29, 52.228.119.222, 52.228.121.123 | -| Canada East | 23.101.115.123, 40.86.225.89, 40.86.226.148, 40.86.227.81, 40.86.225.159, 40.86.226.43, 40.86.227.75, 40.86.231.40, 40.86.225.81 | -| Central India | 52.163.244.128, 52.172.204.196, 52.172.218.144, 52.172.198.236, 52.172.187.93, 52.172.213.78, 52.172.202.195, 52.172.210.146 | -| Central US | 23.101.115.123, 13.89.172.11, 40.78.130.218, 40.78.131.170, 40.122.52.191, 40.122.27.37, 40.113.224.199, 40.122.118.225, 40.122.116.133, 40.122.126.193, 40.122.104.60 | -| Central US EUAP | 23.101.115.123 | -| China East 2 | 40.73.96.39 | -| China North 2 | 40.73.33.105 | -| East Asia | 52.163.244.128, 13.75.34.175, 168.63.220.81, 207.46.136.220, 168.63.210.90, 23.101.15.21, 23.101.7.253, 207.46.136.152, 65.52.180.140, 23.101.13.231, 23.101.3.51 | -| East US | 52.249.253.174, 52.149.248.192, 52.226.98.175, 52.226.98.216, 52.149.184.133, 52.226.99.54, 52.226.99.58, 52.226.99.65, 52.186.38.56, 40.88.16.66, 40.88.23.108, 52.224.135.234, 52.151.240.130, 52.226.99.68, 52.226.99.110, 52.226.99.115, 52.226.99.127, 52.226.99.153, 52.226.99.207, 52.226.100.84, 52.226.100.121, 52.226.100.138, 52.226.100.176, 52.226.101.50, 52.226.101.81, 52.191.99.133, 52.226.96.208, 52.226.101.102, 52.147.211.11, 52.147.211.97, 52.147.211.226, 20.49.104.10 | -| East US 2 | 104.46.110.170, 40.70.147.14, 40.84.38.74, 52.247.116.27, 52.247.117.99, 52.177.182.76, 52.247.117.144, 52.247.116.99, 52.247.67.200, 52.247.119.96, 52.247.70.70 | -| East US 2 EUAP | 104.46.110.170 | -| France Central | 40.127.194.147, 40.79.130.130, 40.89.166.214, 40.89.172.87, 20.188.45.116, 40.89.133.143, 40.89.148.203, 20.188.44.60, 20.188.45.105, 20.188.44.152, 20.188.43.156 | -| France South | 40.127.194.147 | -| Japan East | 52.163.244.128, 40.79.195.2, 40.115.138.201, 104.46.217.37, 40.115.140.98, 40.115.141.134, 40.115.142.61, 40.115.137.9, 40.115.137.124, 40.115.140.218, 40.115.137.189 | -| Japan West | 52.163.244.128, 40.74.100.129, 40.74.85.64, 40.74.126.115, 40.74.125.67, 40.74.128.17, 40.74.127.201, 40.74.128.130, 23.100.108.106, 40.74.128.122, 40.74.128.53 | -| Korea Central | 52.163.244.128, 52.231.77.58, 52.231.73.183, 52.231.71.204, 52.231.66.104, 52.231.77.171, 52.231.69.238, 52.231.78.172, 52.231.69.251 | -| Korea South | 52.163.244.128, 52.231.200.180, 52.231.200.181, 52.231.200.182, 52.231.200.183, 52.231.153.175, 52.231.164.160, 52.231.195.85, 52.231.195.86, 52.231.195.129, 52.231.200.179, 52.231.146.96 | -| North Central US | 23.101.115.123 | -| North Europe | 40.127.194.147, 40.85.74.227, 40.115.100.46, 40.115.100.121, 40.115.105.188, 40.115.103.43, 40.115.109.52, 40.112.77.214, 40.115.99.5 | -| South Africa North | 52.163.244.128 | -| South Africa West | 52.163.244.128 | -| South Central US | 104.215.116.88, 13.65.241.130, 40.74.240.52, 40.74.249.17, 40.74.244.211, 40.74.244.204, 40.84.214.51, 52.171.57.210, 13.65.159.231 | -| South India | 52.163.244.128 | -| Southeast Asia | 52.163.244.128, 20.44.192.205, 20.44.193.4, 20.44.193.56, 20.44.193.98, 20.44.193.147, 20.44.193.175, 20.44.194.249, 20.44.196.82, 20.44.196.95, 20.44.196.104, 20.44.196.115, 20.44.197.158, 20.195.36.24, 20.195.36.25, 20.195.36.27, 20.195.36.37, 20.195.36.39, 20.195.36.40, 20.195.36.41, 20.195.36.42, 20.195.36.43, 20.195.36.44, 20.195.36.45, 20.195.36.46, 20.44.197.160, 20.44.197.162, 20.44.197.219, 20.195.58.80, 20.195.58.185, 20.195.59.60, 20.43.132.128 | -| Switzerland North | 51.107.58.160, 51.107.87.163, 51.107.87.173, 51.107.83.216, 51.107.68.81, 51.107.87.174, 51.107.87.170, 51.107.87.164, 51.107.87.186, 51.107.87.171 | -| UK South | 40.127.194.147, 51.11.174.122, 51.11.173.237, 51.11.174.192, 51.11.174.206, 51.11.175.74, 51.11.175.129, 20.49.216.23, 20.49.216.160, 20.49.217.16, 20.49.217.92, 20.49.217.127, 20.49.217.151, 20.49.166.84, 20.49.166.178, 20.49.166.237, 20.49.167.84, 20.49.232.77, 20.49.232.113, 20.49.232.121, 20.49.232.130, 20.49.232.140, 20.49.232.169, 20.49.165.24, 20.49.232.240, 20.49.217.152, 20.49.217.164, 20.49.217.181, 51.145.125.189, 51.145.126.43, 51.145.126.48, 51.104.28.64 | -| UK West | 40.127.194.147, 51.140.245.89, 51.140.246.238, 51.140.248.127, 51.141.48.137, 51.140.250.127, 51.140.231.20, 51.141.48.238, 51.140.243.38 | -| USDoD Central | 52.126.176.221, 52.126.177.43, 52.126.177.89, 52.126.177.90, 52.126.177.171, 52.126.177.233, 52.126.177.245, 52.126.177.150, 52.126.178.37, 52.126.178.44, 52.126.178.56, 52.126.178.59, 52.126.178.68, 52.126.178.70, 52.126.178.97, 52.126.178.98, 52.126.178.93, 52.126.177.54, 52.126.178.94, 52.126.178.129, 52.126.178.130, 52.126.178.142, 52.126.178.144, 52.126.178.151, 52.126.178.172, 52.126.178.179, 52.126.178.182, 52.126.178.187, 52.126.178.189, 52.126.178.154, 52.127.34.97 | -| USDoD East | 52.127.161.3, 52.127.163.115, 52.127.163.124, 52.127.163.125, 52.127.163.130, 52.127.163.131, 52.127.163.152, 20.140.189.226, 20.140.191.106, 20.140.191.107, 20.140.191.128, 52.127.161.234, 52.245.216.185, 52.245.216.186, 52.245.216.187, 52.245.216.160, 52.245.216.161, 52.245.216.162, 52.245.216.163, 52.245.216.164, 52.245.216.165, 52.245.216.166, 52.245.216.167, 52.245.216.168, 20.140.191.129, 20.140.191.144, 20.140.191.170, 52.245.214.70, 52.245.214.164, 52.245.214.189, 52.127.50.128 | -| USGov Arizona |52.244.204.5, 52.244.204.137, 52.244.204.158, 52.244.204.184, 52.244.204.225, 52.244.205.3, 52.244.50.212, 52.244.55.231, 52.244.205.91, 52.244.205.238, 52.244.201.244, 52.244.201.250, 52.244.200.92, 52.244.206.12, 52.244.206.58, 52.244.206.69, 52.244.206.83, 52.244.207.78, 52.244.203.11, 52.244.203.159, 52.244.203.238, 52.244.200.31, 52.244.202.155, 52.244.206.225, 52.244.218.1, 52.244.218.34, 52.244.218.38, 52.244.218.47, 52.244.202.7, 52.244.203.6, 52.127.2.97 | -| USGov Texas | 52.126.176.221, 52.126.177.43, 52.126.177.89, 52.126.177.90, 52.126.177.171, 52.126.177.233, 52.126.177.245, 52.126.177.150, 52.126.178.37, 52.126.178.44, 52.126.178.56, 52.126.178.59, 52.126.178.68, 52.126.178.70, 52.126.178.97, 52.126.178.98, 52.126.178.93, 52.126.177.54, 52.126.178.94, 52.126.178.129, 52.126.178.130, 52.126.178.142, 52.126.178.144, 52.126.178.151, 52.126.178.172, 52.126.178.179, 52.126.178.182, 52.126.178.187, 52.126.178.189, 52.126.178.154, 52.127.34.97 | -| USGov Virginia | 52.127.161.3, 52.127.163.115, 52.127.163.124, 52.127.163.125, 52.127.163.130, 52.127.163.131, 52.127.163.152, 20.140.189.226, 20.140.191.106, 20.140.191.107, 20.140.191.128, 52.127.161.234, 52.245.216.185, 52.245.216.186, 52.245.216.187, 52.245.216.160, 52.245.216.161, 52.245.216.162, 52.245.216.163, 52.245.216.164, 52.245.216.165, 52.245.216.166, 52.245.216.167, 52.245.216.168, 20.140.191.129, 20.140.191.144, 20.140.191.170, 52.245.214.70, 52.245.214.164, 52.245.214.189, 52.127.50.128 | -| West Central US | 23.101.115.123, 13.71.194.194, 13.78.151.73, 13.77.204.92, 13.78.144.31, 13.78.139.92, 13.77.206.206, 13.78.140.98, 13.78.145.207, 52.161.88.172, 13.77.200.169 | -| West Europe | 213.199.136.176, 51.124.88.159, 20.50.253.190, 20.50.254.255, 52.143.5.71, 20.50.255.137, 20.50.255.176, 52.143.5.148, 20.50.255.211, 20.54.216.1, 20.54.216.113, 20.54.216.236, 20.54.216.244, 20.54.217.89, 20.54.217.102, 20.54.217.162, 20.50.255.109, 20.54.217.184, 20.54.217.197, 20.54.218.36, 20.54.218.66, 51.124.139.38, 20.54.218.71, 20.54.218.104, 52.143.0.117, 20.54.218.240, 20.54.219.47, 20.54.219.75, 20.76.10.82, 20.76.10.95, 20.76.10.139, 20.50.2.13 | -| West India | 52.163.244.128 | -| West US | 13.88.13.50, 40.80.156.205, 40.80.152.218, 104.42.156.123, 104.42.216.21, 40.78.63.47, 40.80.156.103, 40.78.62.97, 40.80.153.6 | -| West US 2 | 52.183.35.124, 40.64.73.23, 40.64.73.121, 40.64.75.111, 40.64.75.125, 40.64.75.227, 40.64.76.236, 40.64.76.240, 40.64.76.242, 40.64.77.87, 40.64.77.111, 40.64.77.122, 40.64.77.131, 40.91.83.189, 52.250.74.132, 52.250.76.69, 52.250.76.130, 52.250.76.137, 52.250.76.145, 52.250.76.146, 52.250.76.153, 52.250.76.177, 52.250.76.180, 52.250.76.191, 52.250.76.192, 40.64.77.143, 40.64.77.159, 40.64.77.195, 20.64.184.243, 20.64.184.249, 20.64.185.9, 20.42.128.97 | - -## ExpressRoute setup - -Use ExpressRoute to connect on premises network to the Azure Virtual Network. A common setup is to advertise the default route (0.0.0.0/0) through the Border Gateway Protocol (BGP) session. This forces traffic coming out of the Virtual Network to be forwarded to the customer's premise network that may drop the traffic, causing outbound flows to break. To overcome this default, [User Defined Route (UDR)](/azure/virtual-network/virtual-networks-udr-overview#user-defined) (0.0.0.0/0) can be configured and next hop will be *Internet*. Since the UDR takes precedence over BGP, the traffic will be destined to the Internet. - -## Securing outbound traffic with a firewall - -If you want to secure outbound traffic using [Azure Firewall](/azure/firewall/overview) or any virtual appliance to limit domain names, the following Fully Qualified Domain Names (FQDN) must be allowed in the firewall. - -```ini -prod.warmpath.msftcloudes.com:443 -gcs.prod.monitoring.core.windows.net:443 -production.diagnostics.monitoring.core.windows.net:443 -graph.windows.net:443 -graph.microsoft.com:443 -*.login.microsoft.com :443 -*.update.microsoft.com:443 -login.live.com:443 -wdcp.microsoft.com:443 -login.microsoftonline.com:443 -azureprofilerfrontdoor.cloudapp.net:443 -*.core.windows.net:443 -*.servicebus.windows.net:443,5671 -shoebox2.metrics.nsatc.net:443 -prod-dsts.dsts.core.windows.net:443 -*.vault.azure.net -ocsp.msocsp.com:80 -*.windowsupdate.com:80 -ocsp.digicert.com:80 -go.microsoft.com:80 -dmd.metaservices.microsoft.com:80 -www.msftconnecttest.com:80 -crl.microsoft.com:80 -www.microsoft.com:80 -adl.windows.com:80 -crl3.digicert.com:80 -``` - -> [!NOTE] -> -> * To restrict access for dependencies with a wildcard (*), use the API described in [How to discover dependencies automatically](vnet-deployment.md#how-to-discover-dependencies-automatically). -> * If you're using [Azure Firewall](/azure/firewall/overview), add **Network Rule** with the following properties: -> -> **Protocol**: TCP -> **Source Type**: IP Address -> **Source**: \* -> **Service Tags**: AzureMonitor -> **Destination Ports**: 443 - -### Configure the route table - -You must configure the [route table](/azure/virtual-network/virtual-networks-udr-overview) of your cluster's subnet with next hop *Internet* to prevent asymmetric routes issues. - -#### Configure the route table using subnet delegation - -We recommend using subnet delegation to configure the route table for your cluster's deployment, similarly to how it was done for [NSG rules](#configure-network-security-group-rules-using-subnet-delegation). By enabling subnet delegation on the cluster's subnet, you enable the service to configure and update the route table for you. - -#### Configure the route table manually - -Alternatively, you can manually configure the route table. By default, deploying a cluster into a virtual network enforces subnet delegation for "Microsoft.Kusto/clusters" to be configured. Opting out of this requirement is possible using the [Preview features](https://portal.azure.com/#blade/Microsoft_Azure_Resources/PreviewFeaturesBlade) pane. - - > [!WARNING] - > Manually configuring the route table for your cluster is not trivial and requires you to constantly monitor this article for changes. We highly recommended using subnet delegation for your cluster or, if you prefer, consider using a [Private Endpoint](security-network-private-endpoint.md) based solution. - -To manually configure the [route table](/azure/virtual-network/virtual-networks-udr-overview) you must define it on the subnet. You need to add the [management](vnet-deployment.md#azure-data-explorer-management-ip-addresses) and [health monitoring](vnet-deployment.md#health-monitoring-addresses) addresses with next hop *Internet*. - -For example, for **West US** region, the following UDRs must be defined: - -| Name | Address Prefix | Next Hop | -| --- | --- | --- | -| ADX_Management | 13.64.38.225/32 | Internet | -| ADX_Monitoring | 23.99.5.162/32 | Internet | -| ADX_Monitoring_1 | 40.80.156.205/32 | Internet | -| ADX_Monitoring_2 | 40.80.152.218/32 | Internet | -| ADX_Monitoring_3 | 104.42.156.123/32 | Internet | -| ADX_Monitoring_4 | 104.42.216.21/32 | Internet | -| ADX_Monitoring_5 | 40.78.63.47/32 | Internet | -| ADX_Monitoring_6 | 40.80.156.103/32 | Internet | -| ADX_Monitoring_7 | 40.78.62.97/32 | Internet | -| ADX_Monitoring_8 | 40.80.153.6/32 | Internet | - -## How to discover dependencies automatically - -Azure Data Explorer provides an API that allows customers to discover all external outbound dependencies (FQDNs) programmatically. -These outbound dependencies will allow customers to set up a Firewall at their end to allow management traffic through the dependent FQDNs. Customers can have these firewall appliances either in Azure or on-premises. The latter might cause additional latency and might impact the service performance. Service teams will need to test out this scenario to evaluate impact on the service performance. - -The [ARMClient](https://chocolatey.org/packages/ARMClient) is used to demonstrate the REST API using PowerShell. - -1. Sign in with ARMClient - - ```powerShell - armclient login - ``` - -2. Invoke diagnose operation - - ```powershell - $subscriptionId = '' - $clusterName = '' - $resourceGroupName = '' - $apiversion = '2021-01-01' - - armclient get /subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Kusto/clusters/$clusterName/OutboundNetworkDependenciesEndpoints?api-version=$apiversion - ``` - -3. Check the response - - ```javascript - { - "value": - [ - ... - { - "id": "/subscriptions//resourceGroups//providers/Microsoft.Kusto/Clusters//OutboundNetworkDependenciesEndpoints/AzureActiveDirectory", - "name": "/AzureActiveDirectory", - "type": "Microsoft.Kusto/Clusters/OutboundNetworkDependenciesEndpoints", - "etag": "\"\"", - "location": "", - "properties": { - "category": "Azure Active Directory", - "endpoints": [ - { - "domainName": "login.microsoftonline.com", - "endpointDetails": [ - { - "port": 443 - } - ] - }, - { - "domainName": "graph.windows.net", - "endpointDetails": [ - { - "port": 443 - } - ] - } - ], - "provisioningState": "Succeeded" - } - }, - { - "id": "/subscriptions//resourceGroups//providers/Microsoft.Kusto/Clusters//OutboundNetworkDependenciesEndpoints/InternalTracing", - "name": "/InternalTracing", - "type": "Microsoft.Kusto/Clusters/OutboundNetworkDependenciesEndpoints", - "location": "Australia Central", - "properties": { - "category": "Internal Tracing", - "endpoints": [ - { - "domainName": "ingest-..kusto.windows.net", - "endpointDetails": [ - { - "port": 443, - "ipAddress": "25.24.23.22" - } - ] - } - ], - "provisioningState": "Succeeded" - } - } - ... - ] - } - ``` - -The outbound dependencies cover categories such as *Microsoft Entra ID*, *Azure Monitor*, *Certificate Authority*, *Azure Storage*, and *Internal Tracing*. In each category, there's a list of domain names and ports that are needed to run the service. They can be used to programmatically configure the firewall appliance of choice. - -## Deploy Azure Data Explorer cluster into your virtual network using an Azure Resource Manager template - -To deploy Azure Data Explorer cluster into your virtual network, use the [Deploy Azure Data Explorer cluster into your virtual network](https://azure.microsoft.com/resources/templates/kusto-vnet/) Azure Resource Manager template. - -This template creates the cluster, virtual network, subnet, network security group, and public IP addresses. - -## Known limitations - -* Virtual network resources with deployed clusters don't support the [move to a new resource group or subscription](/azure/azure-resource-manager/management/move-resource-group-and-subscription) operation. -* Public IP address resources used for the cluster engine or the data management service don't support the move to a new resource group or subscription operation. -* It's not possible to use the "private-" DNS prefix of virtual network injected Azure Data Explorer clusters as part of your query diff --git a/data-explorer/vnet-endpoint-storage-event-hub.md b/data-explorer/vnet-endpoint-storage-event-hub.md deleted file mode 100644 index 898095303d..0000000000 --- a/data-explorer/vnet-endpoint-storage-event-hub.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -title: Create a private or service endpoint to resources used by data connections, such as event hub and Azure Storage -description: Learn how to enable a private or service endpoint to resources used by data connections, such as event hub and Storage -ms.reviewer: gunjand -ms.topic: how-to -ms.date: 07/23/2024 ---- -# Create a private or service endpoint to event hub and Azure Storage - -> [!WARNING] -> Virtual Network Injection will be retired for Azure Data Explorer by 1 February 2025. For more information on the deprecation, see [Deprecation of Virtual Network Injection for Azure Data Explorer](https://aka.ms/adx.security.vnet.deprecation). - -[Azure Virtual Network (VNet)](/azure/virtual-network/virtual-networks-overview) enables many types of Azure resources to securely communicate with each other. [Azure Private Link](/azure/private-link/) enables you to access Azure Services and Azure hosted customer-owned/partner services over a Private Endpoint in your virtual network. A [Private Endpoint](/azure/private-link/private-endpoint-overview) uses an IP address from your virtual network’s address space for the Azure service to securely connect between Azure Data Explorer and Azure services such as Azure Storage and event hub. Azure Data Explorer accesses the Private Endpoint of the storage accounts or event hubs over the Microsoft backbone, and all communication, for example, data export, external tables, and data ingestion, takes place over the private IP address. - -In contrast to a Private Endpoint, a [service endpoint](/azure/virtual-network/virtual-network-service-endpoints-overview) remains a publicly routable IP address. A Virtual Network (VNet) service endpoint provides secure and direct connectivity to Azure services over an optimized route over the Azure backbone network. - -This article shows you how to create a connection between Azure Data Explorer and [event hub](ingest-data-event-hub-overview.md) or [Azure Storage](/azure/storage/). - -## Prerequisites - -* [Azure Virtual Network](/azure/virtual-network/virtual-networks-overview) -* [Azure Data Explorer subnet](vnet-deployment.md) - -## Private Endpoint - -Azure [Private Endpoint](/azure/private-link/private-endpoint-overview) is a network interface that connects you privately and securely to a service powered by Azure [Private Link](/azure/private-link/). Private Endpoint uses a private IP address from your virtual network, effectively bringing the service into your virtual network. - -# [Azure Storage - Private Endpoint](#tab/storage-account) - -### Allow access to Azure Storage Account from Azure Data Explorer Subnets using a Private Endpoint - -For a tutorial on how to create a Private Endpoint in your Azure Storage account, see [Tutorial: Connect to a storage account using an Azure Private Endpoint](/azure/private-link/tutorial-private-endpoint-storage-portal). - -Within this tutorial, select the virtual network where the Azure Data Explorer subnet exists, and the Azure Data Explorer subnet. - -# [Event hub - private endpoint](#tab/event-hub) - -### Allow access to Azure Event Hubs from Azure Data Explorer Subnets using a Private Endpoint - -For a tutorial on how to create a Private Endpoint in your event hub, see [Add a Private Endpoint using Azure portal](/azure/event-hubs/private-link-service#add-a-private-endpoint-using-azure-portal). - -Within this tutorial, select the virtual network where the Azure Data Explorer subnet exists, and the Azure Data Explorer subnet. - ---- - -## Service Endpoint - -Virtual Network (VNet) [service endpoint](/azure/virtual-network/virtual-network-service-endpoints-overview) provides secure and direct connectivity to Azure services over an optimized route over the Azure backbone network. Endpoints allow you to secure your critical Azure service resources to only your virtual networks. - -# [Azure Storage - service endpoint](#tab/storage-account) - -### Allow access to Azure Storage account from Azure Data Explorer subnets using a service endpoint - -This section shows you how to use Azure portal to add a virtual network service endpoint. To limit access, integrate the virtual network service endpoint for this Azure Storage account. - -### Add a virtual network - -1. Navigate to the storage account you want to secure. -1. In the left-hand menu, select **Firewalls and virtual networks**. -1. Enable access from **Selected networks**. -1. Under **Virtual Networks**, select **+ Add existing virtual network**. - - :::image type="content" source="media/vnet-private-link-storage-event-hub/storage-add-existing-vnet.png" alt-text="Add existing virtual network connection Azure Storage to Azure Data Explorer."::: - -### Add networks pane - -:::image type="content" source="media/vnet-private-link-storage-event-hub/storage-add-networks.png" alt-text="Add virtual network to Azure Storage Account to connect to Azure Data Explorer."::: - -1. In the right-hand **Add networks** pane, select your Azure subscription. - -1. Select the virtual network from the list of virtual networks, and then pick the subnet. - - > [!NOTE] - > Enable the service endpoint before adding the virtual network to the list. If the service endpoint is not enabled, the portal will prompt you to enable it. - -1. Select **Add**. - -### Save and verify virtual network settings - -1. Select **Save** on the toolbar to save the settings. - - :::image type="content" source="media/vnet-private-link-storage-event-hub/storage-virtual-network.png" alt-text="Vnet to connect storage account to Azure Data Explorer."::: - - Wait for a few minutes for confirmation to appear on the portal notifications. - -# [Event hub - service endpoint](#tab/event-hub) - -### Allow access to Azure Event Hubs from Azure Data Explorer subnets using a service endpoint - -> [!IMPORTANT] -> Virtual networks are supported in **standard** and **dedicated** tiers of event hubs, and aren't supported in the basic tier. - -### Add a virtual network - -1. In the Azure portal, navigate to the **Event hubs namespace** you want to secure. -1. On the left-hand menu, select **Networking**. This tab appears only in **standard** or **dedicated** namespaces. -1. Select the **Firewalls and virtual networks** tab. - - :::image type="content" source="media/vnet-private-link-storage-event-hub/networking.png" alt-text="Networking in event hub."::: - -1. Enable access from **Selected Networks**. -1. Under **Virtual Networks**, select **+ Add existing virtual network**. - - :::image type="content" source="media/vnet-private-link-storage-event-hub/event-hub-add-existing-vnet.png" alt-text="Add existing virtual network in Azure Data Explorer."::: - -### Add networks pane - -:::image type="content" source="media/vnet-private-link-storage-event-hub/add-networks.png" alt-text="Add networks fields to connect Virtual Network to Azure Data Explorer."::: - -1. In the right-hand **Add networks** pane, select your Azure subscription. - -1. Select the virtual network from the list of virtual networks, and then pick the subnet. You must enable the service endpoint before adding the virtual network to the list. - > [!NOTE] - > Enable the service endpoint before adding the virtual network to the list. If the service endpoint is not enabled, the portal will prompt you to enable it. -1. Select **Add**. - -### Save and verify virtual network settings - -1. Select **Save** on the toolbar to save the settings. Wait for a few minutes for the confirmation to appear on the portal notifications - - :::image type="content" source="media/vnet-private-link-storage-event-hub/event-hub-firewalls-and-vnet.png" alt-text="Add virtual network and subnet in event hub to connect to Azure Data Explorer."::: - ---- - -## Related content - -* [Export data to storage](/kusto/management/data-export/export-data-to-storage?view=azure-data-explorer&preserve-view=true) -* [Ingest data from event hub into Azure Data Explorer](create-event-hubs-connection.md?tabs=portalADX)