Skip to content

Commit b411640

Browse files
orspodorspod
committed
Add troubleshooting topic
1 parent 9e1a648 commit b411640

File tree

3 files changed

+166
-152
lines changed

3 files changed

+166
-152
lines changed

‎articles/data-explorer/toc.yml

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,14 @@
164164
href: k2bridge.md
165165
- name: Secure
166166
items:
167+
- name: Network isolation (VNet)
168+
items:
169+
- name: Create a cluster in your VNet - Portal
170+
href: vnet-create-cluster-portal.md
171+
- name: Deploy your cluster to your VNet
172+
href: vnet-deployment.md
173+
- name: Troubleshoot VNet cluster creation, connectivity, and operation
174+
href: vnet-deploy-troubleshoot.md
167175
- name: Manage cluster security
168176
href: manage-cluster-security.md
169177
- name: Configure managed identities
@@ -220,10 +228,6 @@
220228
href: delete-data.md
221229
- name: Deal with duplicate data
222230
href: dealing-with-duplicates.md
223-
- name: Deploy
224-
items:
225-
- name: Deploy your cluster to your virtual network
226-
href: vnet-deployment.md
227231
- name: Monitor
228232
items:
229233
- name: Use diagnostic logs to monitor ingestion

‎articles/data-explorer/vnet-deploy-troubleshoot.md

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
---
2+
title: Troubleshooting while deploy Azure Data Explorer cluster into your Virtual Network
3+
description: Troubleshoot connectivity, operational and cluster creation issues when deploying cluster into your virtual network
4+
author: basaba
5+
ms.author: basaba
6+
ms.reviewer: orspodek
7+
ms.service: data-explorer
8+
ms.topic: conceptual
9+
ms.date: 03/24/2020
10+
---
11+
12+
# Troubleshoot connectivity, operational and cluster creation when deploying a cluster into your virtual network
13+
14+
In this section you learn how to troubleshoot connectivity, operational, and cluster creation issues for a cluster that is deployed into your [Virtual Network](/azure/virtual-network/virtual-networks-overview).
15+
16+
## Access issues
17+
18+
If you have an issue while accessing cluster using the public (cluster.region.kusto.windows.net) or private (private-cluster.region.kusto.windows.net) endpoint and you suspect it's related to virtual network setup, perform the following steps to troubleshoot the issue.
19+
20+
### Check TCP connectivity
21+
22+
The first step includes checking TCP connectivity using Windows or Linux OS.
23+
24+
# [Windows](#tab/windows)
25+
26+
1. Download [TCping](https://www.elifulkerson.com/projects/tcping.php) to the machine connecting to the cluster.
27+
2. Ping the destination from the source machine by using the following command:
28+
29+
```cmd
30+
C:\> tcping -t yourcluster.kusto.windows.net 443
31+
32+
** Pinging continuously. Press control-c to stop **
33+
34+
Probing 1.2.3.4:443/tcp - Port is open - time=100.00ms
35+
```
36+
37+
# [Linux](#tab/linux)
38+
39+
1. Install *netcat* in the machine connecting to the cluster
40+
41+
```bash
42+
$ apt-get install netcat
43+
```
44+
45+
2. Ping the destination from the source machine by using the following command:
46+
47+
```bash
48+
$ netcat -z -v yourcluster.kusto.windows.net 443
49+
50+
Connection to yourcluster.kusto.windows.net 443 port [tcp/https] succeeded!
51+
```
52+
---
53+
54+
If the test isn't successful, proceed with the following steps. If the test is successful, the issue isn't due to a TCP connectivity issue. Go to [operational issues](#cluster-creation-and-operations-issues) to troubleshoot further.
55+
56+
### Check the Network Security Group (NSG)
57+
58+
Check that the [Network Security Group](/azure/virtual-network/security-overview) (NSG) attached to the cluster's subnet, has an inbound rule that allows access from the client machine's IP for port 443.
59+
60+
### Check route table
61+
62+
If the cluster's subnet has force-tunneling setup to firewall (subnet with a [route table](/azure/virtual-network/virtual-networks-udr-overview) that contains the default route '0.0.0.0/0'), make sure that the machine IP address has a route with [next hop type](/azure/virtual-network/virtual-networks-udr-overview) to VirtualNetwork/Internet. This is required to prevent asymmetric route issues.
63+
64+
## Ingestion issues
65+
66+
If you're experiencing ingestion issues and you suspect it's related to virtual network setup, perform the following steps.
67+
68+
### Check ingestion health
69+
70+
Check that the [cluster ingestion metrics](/azure/data-explorer/using-metrics#ingestion-health-and-performance-metrics) indicate a healthy state.
71+
72+
### Check security rules on data source resources
73+
74+
If the metrics indicate that no events were processed from the data source (*Events processed* (for Event/IoT Hubs) metric), make sure that the data source resources (Event Hub or Storage) allow access from cluster's subnet in the firewall rules or service endpoints.
75+
76+
### Check security rules configured on cluster's subnet
77+
78+
Make sure cluster's subnet has NSG, UDR, and firewall rules are properly configured. In addition, test network connectivity for all dependent endpoints.
79+
80+
## Cluster creation and operations issues
81+
82+
If you're experiencing cluster creation or operation issues and you suspect it's related to virtual network setup, follow these steps to troubleshoot the issue.
83+
84+
### Diagnose the virtual network with the REST API
85+
86+
The [ARMClient](https://chocolatey.org/packages/ARMClient) is used to call the REST API using PowerShell.
87+
88+
1. Log in with ARMClient
89+
90+
```powerShell
91+
armclient login
92+
```
93+
94+
1. Invoke diagnose operation
95+
96+
```powershell
97+
$subscriptionId = '<subscription id>'
98+
$clusterName = '<name of cluster>'
99+
$resourceGroupName = '<resource group name>'
100+
$apiversion = '2019-11-09'
101+
102+
armclient post "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Kusto/clusters/$clusterName/diagnoseVirtualNetwork?api-version=$apiversion" -verbose
103+
```
104+
105+
1. Check the response
106+
107+
```powershell
108+
HTTP/1.1 202 Accepted
109+
...
110+
Azure-AsyncOperation: https://management.azure.com/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationResults/{operation-id}?api-version=2019-11-09
111+
...
112+
```
113+
114+
1. Wait for operation completion
115+
116+
```powershell
117+
armclient get https://management.azure.com/subscriptions/$subscriptionId/providers/Microsoft.Kusto/locations/{location}/operationResults/{operation-id}?api-version=2019-11-09
118+
119+
{
120+
"id": "/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationresults/{operation-id}",
121+
"name": "{operation-name}",
122+
"status": "[Running/Failed/Completed]",
123+
"startTime": "{start-time}",
124+
"endTime": "{end-time}",
125+
"properties": {...}
126+
}
127+
```
128+
129+
Wait until the *status* property shows *Completed*, then the *properties* field should show:
130+
131+
```powershell
132+
{
133+
"id": "/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationresults/{operation-id}",
134+
"name": "{operation-name}",
135+
"status": "Completed",
136+
"startTime": "{start-time}",
137+
"endTime": "{end-time}",
138+
"properties": {
139+
"Findings": [...]
140+
}
141+
}
142+
```
143+
144+
If the *Findings* property shows an empty result, it means that all network tests passed and no connections are broken. If it shows an error as follows: *Outbound dependency '{dependencyName}:{port}' might be not satisfied (Outbound)*, the cluster can't reach the dependent service endpoints. Proceed with the following steps to troubleshoot.
145+
146+
### Check Network Security Group (NSG)
147+
148+
Make sure that the [Network Security Group](/azure/virtual-network/security-overview) is configured properly per the instructions in [Dependencies for VNet deployment](/azure/data-explorer/vnet-deployment#dependencies-for-vnet-deployment)
149+
150+
### Check route table
151+
152+
If the cluster's subnet has force-tunneling set up to firewall (subnet with a [route table](/azure/virtual-network/virtual-networks-udr-overview) that contains the default route '0.0.0.0/0') make sure that the [management IP addresses](#azure-data-explorer-management-ip-addresses) and [health monitoring IP addresses](#health-monitoring-addresses) have a route with [next hop type](/azure/virtual-network/virtual-networks-udr-overview##next-hop-types-across-azure-tools) *Internet*, and [source address prefix](/azure/virtual-network/virtual-networks-udr-overview#how-azure-selects-a-route) to *'management-ip/32'* and *'health-monitoring-ip/32'*. This is required to prevent asymmetric route issues.
153+
154+
### Check firewall rules
155+
156+
If you force tunnel subnet outbound traffic to a firewall, make sure all dependencies FQDN (for example, *.blob.core.windows.net*) are allowed in the firewall configuration as described in [securing outbound traffic with firewall](/azure/data-explorer/vnet-deployment#securing-outbound-traffic-with-firewall).

‎articles/data-explorer/vnet-deployment.md

Lines changed: 2 additions & 148 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.topic: conceptual
99
ms.date: 10/31/2019
1010
---
1111

12-
# Deploy Azure Data Explorer into your Virtual Network
12+
# Deploy Azure Data Explorer cluster into your Virtual Network
1313

1414
This article explains the resources that are present when you deploy an Azure Data Explorer cluster into a custom Azure Virtual Network. This information will help you deploy a cluster into a subnet in your Virtual Network (VNet). For more information on Azure Virtual Networks, see [What is Azure Virtual Network?](/azure/virtual-network/virtual-networks-overview)
1515

@@ -254,150 +254,4 @@ For example, for **West US** region, the following UDRs must be defined:
254254

255255
To deploy Azure Data Explorer cluster into your virtual network, use the [Deploy Azure Data Explorer cluster into your VNet](https://azure.microsoft.com/resources/templates/101-kusto-vnet/) Azure Resource Manager template.
256256

257-
This template creates the cluster, virtual network, subnet, network security group, and public IP addresses.
258-
259-
## Troubleshooting
260-
261-
In this section you learn how to troubleshoot connectivity, operational, and cluster creation issues for a cluster that is deployed into your [Virtual Network](/azure/virtual-network/virtual-networks-overview).
262-
263-
### Access issues
264-
265-
If you have an issue while accessing cluster using the public (cluster.region.kusto.windows.net) or private (private-cluster.region.kusto.windows.net) endpoint and you suspect it's related to virtual network setup, perform the following steps to troubleshoot the issue.
266-
267-
#### Check TCP connectivity
268-
269-
The first step includes checking TCP connectivity using Windows or Linux OS.
270-
271-
# [Windows](#tab/windows)
272-
273-
1. Download [TCping](https://www.elifulkerson.com/projects/tcping.php) to the machine connecting to the cluster.
274-
2. Ping the destination from the source machine by using the following command:
275-
276-
```cmd
277-
C:\> tcping -t yourcluster.kusto.windows.net 443
278-
279-
** Pinging continuously. Press control-c to stop **
280-
281-
Probing 1.2.3.4:443/tcp - Port is open - time=100.00ms
282-
```
283-
284-
# [Linux](#tab/linux)
285-
286-
1. Install *netcat* in the machine connecting to the cluster
287-
288-
```bash
289-
$ apt-get install netcat
290-
```
291-
292-
2. Ping the destination from the source machine by using the following command:
293-
294-
```bash
295-
$ netcat -z -v yourcluster.kusto.windows.net 443
296-
297-
Connection to yourcluster.kusto.windows.net 443 port [tcp/https] succeeded!
298-
```
299-
---
300-
301-
If the test isn't successful, proceed with the following steps. If the test is successful, the issue isn't due to a TCP connectivity issue. Go to [operational issues](#cluster-creation-and-operations-issues) to troubleshoot further.
302-
303-
#### Check the Network Security Group (NSG)
304-
305-
Check that the [Network Security Group](/azure/virtual-network/security-overview) (NSG) attached to the cluster's subnet, has an inbound rule that allows access from the client machine's IP for port 443.
306-
307-
#### Check route table
308-
309-
If the cluster's subnet has force-tunneling setup to firewall (subnet with a [route table](/azure/virtual-network/virtual-networks-udr-overview) that contains the default route '0.0.0.0/0'), make sure that the machine IP address has a route with [next hop type](/azure/virtual-network/virtual-networks-udr-overview) to VirtualNetwork/Internet. This is required to prevent asymmetric route issues.
310-
311-
### Ingestion issues
312-
313-
If you're experiencing ingestion issues and you suspect it's related to virtual network setup, perform the following steps.
314-
315-
#### Check ingestion health
316-
317-
Check that the [cluster ingestion metrics](/azure/data-explorer/using-metrics#ingestion-health-and-performance-metrics) indicate a healthy state.
318-
319-
#### Check security rules on data source resources
320-
321-
If the metrics indicate that no events were processed from the data source (*Events processed* (for Event/IoT Hubs) metric), make sure that the data source resources (Event Hub or Storage) allow access from cluster's subnet in the firewall rules or service endpoints.
322-
323-
#### Check security rules configured on cluster's subnet
324-
325-
Make sure cluster's subnet has NSG, UDR, and firewall rules are properly configured. In addition, test network connectivity for all dependent endpoints.
326-
327-
### Cluster creation and operations issues
328-
329-
If you're experiencing cluster creation or operation issues and you suspect it's related to virtual network setup, follow these steps to troubleshoot the issue.
330-
331-
#### Diagnose the virtual network with the REST API
332-
333-
The [ARMClient](https://chocolatey.org/packages/ARMClient) is used to call the REST API using PowerShell.
334-
335-
1. Log in with ARMClient
336-
337-
```powerShell
338-
armclient login
339-
```
340-
341-
1. Invoke diagnose operation
342-
343-
```powershell
344-
$subscriptionId = '<subscription id>'
345-
$clusterName = '<name of cluster>'
346-
$resourceGroupName = '<resource group name>'
347-
$apiversion = '2019-11-09'
348-
349-
armclient post "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Kusto/clusters/$clusterName/diagnoseVirtualNetwork?api-version=$apiversion" -verbose
350-
```
351-
352-
1. Check the response
353-
354-
```powershell
355-
HTTP/1.1 202 Accepted
356-
...
357-
Azure-AsyncOperation: https://management.azure.com/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationResults/{operation-id}?api-version=2019-11-09
358-
...
359-
```
360-
361-
1. Wait for operation completion
362-
363-
```powershell
364-
armclient get https://management.azure.com/subscriptions/$subscriptionId/providers/Microsoft.Kusto/locations/{location}/operationResults/{operation-id}?api-version=2019-11-09
365-
366-
{
367-
"id": "/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationresults/{operation-id}",
368-
"name": "{operation-name}",
369-
"status": "[Running/Failed/Completed]",
370-
"startTime": "{start-time}",
371-
"endTime": "{end-time}",
372-
"properties": {...}
373-
}
374-
```
375-
376-
Wait until the *status* property shows *Completed*, then the *properties* field should show:
377-
378-
```powershell
379-
{
380-
"id": "/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationresults/{operation-id}",
381-
"name": "{operation-name}",
382-
"status": "Completed",
383-
"startTime": "{start-time}",
384-
"endTime": "{end-time}",
385-
"properties": {
386-
"Findings": [...]
387-
}
388-
}
389-
```
390-
391-
If the *Findings* property shows an empty result, it means that all network tests passed and no connections are broken. If it shows an error as follows: *Outbound dependency '{dependencyName}:{port}' might be not satisfied (Outbound)*, the cluster can't reach the dependent service endpoints. Proceed with the following steps to troubleshoot.
392-
393-
#### Check Network Security Group (NSG)
394-
395-
Make sure that the [Network Security Group](/azure/virtual-network/security-overview) is configured properly per the instructions in [Dependencies for VNet deployment](/azure/data-explorer/vnet-deployment#dependencies-for-vnet-deployment)
396-
397-
#### Check route table
398-
399-
If the cluster's subnet has force-tunneling set up to firewall (subnet with a [route table](/azure/virtual-network/virtual-networks-udr-overview) that contains the default route '0.0.0.0/0') make sure that the [management IP addresses](#azure-data-explorer-management-ip-addresses) and [health monitoring IP addresses](#health-monitoring-addresses) have a route with [next hop type](/azure/virtual-network/virtual-networks-udr-overview##next-hop-types-across-azure-tools) *Internet*, and [source address prefix](/azure/virtual-network/virtual-networks-udr-overview#how-azure-selects-a-route) to *'management-ip/32'* and *'health-monitoring-ip/32'*. This is required to prevent asymmetric route issues.
400-
401-
#### Check firewall rules
402-
403-
If you force tunnel subnet outbound traffic to a firewall, make sure all dependencies FQDN (for example, *.blob.core.windows.net*) are allowed in the firewall configuration as described in [securing outbound traffic with firewall](/azure/data-explorer/vnet-deployment#securing-outbound-traffic-with-firewall).
257+
This template creates the cluster, virtual network, subnet, network security group, and public IP addresses.

0 commit comments

Comments
 (0)