Skip to content

Commit fa11865

Browse files
orspodorspod
committed
vnet troubleshooting
1 parent d0c9f0a commit fa11865

File tree

1 file changed

+99
-93
lines changed

1 file changed

+99
-93
lines changed

‎articles/data-explorer/vnet-deployment.md

Lines changed: 99 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,7 @@ Deploying Azure Data Explorer cluster into your subnet allows you to setup data
169169
| West Europe | 23.97.212.5 |
170170
| West India | 23.99.5.162 |
171171
| West US | 23.99.5.162 |
172-
| West US 2 | 23.99.5.162 |
172+
| West US 2 | 23.99.5.162 |
173173

174174
#### Azure Monitor configuration endpoint addresses
175175

@@ -187,7 +187,7 @@ Deploying Azure Data Explorer cluster into your subnet allows you to setup data
187187
| Central US EUAP | 13.90.43.231 |
188188
| East Asia | 13.75.117.221 |
189189
| East US | 13.90.43.231 |
190-
| East US 2 | 13.68.89.19 |
190+
| East US 2 | 13.68.89.19 |
191191
| East US 2 EUAP | 13.68.89.19 |
192192
| France Central | 52.174.4.112 |
193193
| France South | 52.174.4.112 |
@@ -212,7 +212,7 @@ Deploying Azure Data Explorer cluster into your subnet allows you to setup data
212212

213213
## ExpressRoute setup
214214

215-
Use ExpressRoute to connect on premises network to the Azure Virtual Network. A common setup is to advertise the default route (0.0.0.0/0) through the Border Gateway Protocol (BGP) session. This forces traffic coming out of the Virtual Network to be forwarded to the customer’s premise network that may drop the traffic, causing outbound flows to break. To overcome this default, [User Defined Route (UDR)](/azure/virtual-network/virtual-networks-udr-overview#user-defined) (0.0.0.0/0) can be configured and next hop will be *Internet*. Since the UDR takes precedence over BGP, the traffic will be destined to the Internet.
215+
Use ExpressRoute to connect on premises network to the Azure Virtual Network. A common setup is to advertise the default route (0.0.0.0/0) through the Border Gateway Protocol (BGP) session. This forces traffic coming out of the Virtual Network to be forwarded to the customer's premise network that may drop the traffic, causing outbound flows to break. To overcome this default, [User Defined Route (UDR)](/azure/virtual-network/virtual-networks-udr-overview#user-defined) (0.0.0.0/0) can be configured and next hop will be *Internet*. Since the UDR takes precedence over BGP, the traffic will be destined to the Internet.
216216

217217
## Securing outbound traffic with firewall
218218

@@ -261,139 +261,145 @@ This template creates the cluster, virtual network, subnet, network security gro
261261

262262
## Troubleshooting
263263

264-
Learn how to troubleshoot connectivity, operational, cluster creation issues for a cluster that is deployed into your [Virtual Network](/azure/virtual-network/virtual-networks-overview).
264+
In this section you learn how to troubleshoot connectivity, operational, and cluster creation issues for a cluster that is deployed into your [Virtual Network](/azure/virtual-network/virtual-networks-overview).
265265

266-
## Access issues
266+
### Access issues
267267

268-
If you have an issue while accessing cluster using the public (cluster.region.kusto.windows.net) or private (private-cluster.region.kusto.windows.net) endpoint and you suspect it is related to virtual network setup, follow these steps to troubleshoot the issue.
268+
If you have an issue while accessing cluster using the public (cluster.region.kusto.windows.net) or private (private-cluster.region.kusto.windows.net) endpoint and you suspect it's related to virtual network setup, perform the following steps to troubleshoot the issue.
269269

270-
### Step 1: Check TCP connectivity
270+
1. Check TCP connectivity
271271

272-
#### Check using Windows OS
272+
The first step includes checking TCP connectivity using Windows or Linux OS.
273+
274+
**Check using Windows OS**
275+
276+
1. Download [TCping](https://www.elifulkerson.com/projects/tcping.php) to the machine connecting to the cluster.
277+
2. Ping the destination from the source machine by using the following command:
273278

274-
1. Download [TCping](https://www.elifulkerson.com/projects/tcping.php) to the machine that is connecting to the cluster.
275-
2. Ping the destination from the source machine by using the following command:
276279
```cmd
277-
C:\> tcping -t yourcluster.kusto.windows.net 443
280+
C:\> tcping -t yourcluster.kusto.windows.net 443
278281
279-
** Pinging continuously. Press control-c to stop **
282+
** Pinging continuously. Press control-c to stop **
280283
281-
Probing 1.2.3.4:443/tcp - Port is open - time=100.00ms
282-
```
283-
#### Check using Linux OS
284+
Probing 1.2.3.4:443/tcp - Port is open - time=100.00ms
285+
```
284286
285-
1. Install *netcat* to the machine that is connecting to the cluster
286-
```bash
287-
$ apt-get install netcat
288-
```
289-
2. Ping the destination from the source machine by using the following command:
287+
**Check using Linux OS**
288+
289+
1. Install *netcat* in the machine connecting to the cluster
290290
291291
```bash
292-
$ netcat -z -v yourcluster.kusto.windows.net 443
292+
$ apt-get install netcat
293+
```
294+
295+
2. Ping the destination from the source machine by using the following command:
296+
297+
```bash
298+
$ netcat -z -v yourcluster.kusto.windows.net 443
293299
294-
Connection to yourcluster.kusto.windows.net 443 port [tcp/https] succeeded!
295-
```
300+
Connection to yourcluster.kusto.windows.net 443 port [tcp/https] succeeded!
301+
```
296302
297-
If the test is successful, that means that the issue is due to other issue or operational issue, please go to [operational issues](#operational-issues) section to troubleshoot, otherwise proceed with the following steps.
303+
If the test isn't successful, proceed with the following steps. If the test is successful, the issue isn't due to a TCP connectivity issue. Go to [operational issues](#operational-issues) to troubleshoot further.
298304
299-
### Step 2: Check Network Secuirty Group (NSG)
305+
2. Check the Network Security Group (NSG)
300306
301-
Make sure that [Network Security Group](/azure/virtual-network/security-overview) (NSG) attached to the cluster's subnet has an inbound rule that allows the access from the client machine's IP for the port 443.
307+
Check that the [Network Security Group](/azure/virtual-network/security-overview) (NSG) attached to the cluster's subnet, has an inbound rule that allows access from the client machine's IP for port 443.
302308
303-
### Step 3: Check Route Table
309+
3. Check route table
304310
305-
If the cluster's subnet has force-tunneling setup to firewall (subnet with a [route table](/azure/virtual-network/virtual-networks-udr-overview) that contains the default route '0.0.0.0/0') make sure that the machine IP address has route with [next hop type](/azure/virtual-network/virtual-networks-udr-overview) to VirtualNetwork/Internet, this is required to prevent asymmetric route issues.
311+
If the cluster's subnet has force-tunneling setup to firewall (subnet with a [route table](/azure/virtual-network/virtual-networks-udr-overview) that contains the default route '0.0.0.0/0'), make sure that the machine IP address has a route with [next hop type](/azure/virtual-network/virtual-networks-udr-overview) to VirtualNetwork/Internet. This is required to prevent asymmetric route issues.
306312
307-
## Ingestion issues
313+
### Ingestion issues
308314
309-
If you're experiencing ingestion issues and you suspect it is related to virtual network setup, follow these steps to troubleshoot the issue.
315+
If you're experiencing ingestion issues and you suspect it's related to virtual network setup, perform the following steps.
310316
311-
### Step 1: Check ingestion health
317+
1. Check ingestion health
312318
313-
Make sure that the [cluster ingestion metrics](/azure/data-explorer/using-metrics#ingestion-health-and-performance-metrics) indicate healthy state.
319+
Check that the [cluster ingestion metrics](/azure/data-explorer/using-metrics#ingestion-health-and-performance-metrics) indicate a healthy state.
314320
315-
### Step 2: Check security rules on source resources
321+
2. Check security rules on data source resources
316322
317-
If you see that metrics indicate that no events were processed from data source (*'Events processed (for Event/IoT Hubs)'* metric) make sure that the data source resources (EventHub/Storage) allow access from cluster's subnet in the firewall rules or service endpoints.
318-
<br>
323+
If the metrics indicate that no events were processed from the data source (*Events processed* (for Event/IoT Hubs) metric), make sure that the data source resources (Event Hub or Storage) allow access from cluster's subnet in the firewall rules or service endpoints.
319324
320-
### Step 3: Check security rules configured on cluster's subnet
325+
3. Check security rules configured on cluster's subnet
321326
322-
Make sure cluster's subnet has NSG, UDR and firewall rules are properly configured.
323-
<br>
324-
To test network connectivty for all dependent endpoints follow the steps [here](#step-1:-diagnose-virtual-network-with-the-rest-api).
327+
Make sure cluster's subnet has NSG, UDR and firewall rules are properly configured. In addition, [test network connectivity for all dependent endpoints](#diagnose-virtual-network-with-the-rest-api).
325328
326329
## Cluster creation and operations issues
327330
328-
If you're experiencing cluster creation or operateion issues and you suspect it is related to virtual network setup, follow these steps to troubleshoot the issue.
331+
If you're experiencing cluster creation or operation issues and you suspect it's related to virtual network setup, follow these steps to troubleshoot the issue.
329332
330-
### Step 1: Diagnose virtual network with the REST API
333+
1. Diagnose the virtual network with the REST API.
331334
332-
The ARMclient is used to call the REST API using PowerShell. The ARMClient is found on chocolatey at [ARMClient on Chocolatey](https://chocolatey.org/packages/ARMClient)
335+
The [ARMClient](https://chocolatey.org/packages/ARMClient) is used to call the REST API using PowerShell.
333336
334-
### Log in with ARMClient
337+
1. Log in with ARMClient
335338
336-
```powerShell
337-
armclient login
338-
```
339+
```powerShell
340+
armclient login
341+
```
339342

340-
### Invoke diagnose operation.
343+
1. Invoke diagnose operation
341344

342-
```powershell
343-
$subscriptionId = '<subscription id>'
344-
$clusterName = '<name of cluster>'
345-
$resourceGroupName = '<resource group name>'
346-
$apiversion = '2019-11-09'
345+
```powershell
346+
$subscriptionId = '<subscription id>'
347+
$clusterName = '<name of cluster>'
348+
$resourceGroupName = '<resource group name>'
349+
$apiversion = '2019-11-09'
350+
351+
armclient post "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Kusto/clusters/$clusterName/diagnoseVirtualNetwork?api-version=$apiversion" -verbose
352+
```
347353
348-
armclient post "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Kusto/clusters/$clusterName/diagnoseVirtualNetwork?api-version=$apiversion" -verbose
349-
```
354+
1. Check the response
350355
351-
Check the response
356+
```powershell
357+
HTTP/1.1 202 Accepted
358+
...
359+
Azure-AsyncOperation: https://management.azure.com/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationResults/{operation-id}?api-version=2019-11-09
360+
...
361+
```
352362
353-
```powershell
354-
HTTP/1.1 202 Accepted
355-
...
356-
Azure-AsyncOperation: https://management.azure.com/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationResults/{operation-id}?api-version=2019-11-09
357-
...
358-
```
363+
1. Wait for operation completion
359364
360-
### Wait for the operation completion
361-
```powershell
362-
armclient get https://management.azure.com/subscriptions/$subscriptionId/providers/Microsoft.Kusto/locations/{location}/operationResults/{operation-id}?api-version=2019-11-09
363-
364-
{
365-
"id": "/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationresults/{operation-id}",
366-
"name": "{operation-name}",
367-
"status": "[Running/Failed/Completed]",
368-
"startTime": "{start-time}",
369-
"endTime": "{end-time}",
370-
"properties": {...}
371-
}
372-
```
365+
```powershell
366+
armclient get https://management.azure.com/subscriptions/$subscriptionId/providers/Microsoft.Kusto/locations/{location}/operationResults/{operation-id}?api-version=2019-11-09
367+
368+
{
369+
"id": "/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationresults/{operation-id}",
370+
"name": "{operation-name}",
371+
"status": "[Running/Failed/Completed]",
372+
"startTime": "{start-time}",
373+
"endTime": "{end-time}",
374+
"properties": {...}
375+
}
376+
```
377+
378+
Wait until the *status* property shows *Completed*, then the *properties* field should show:
379+
380+
```powershell
381+
{
382+
"id": "/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationresults/{operation-id}",
383+
"name": "{operation-name}",
384+
"status": "Completed",
385+
"startTime": "{start-time}",
386+
"endTime": "{end-time}",
387+
"properties": {
388+
"Findings": [...]
389+
}
390+
}
391+
```
373392
374-
Wait until the *status* property shows *Completed*, then the *properties* field should show:
375-
```powershell
376-
{
377-
"id": "/subscriptions/{subscription-id}/providers/Microsoft.Kusto/locations/{location}/operationresults/{operation-id}",
378-
"name": "{operation-name}",
379-
"status": "Completed",
380-
"startTime": "{start-time}",
381-
"endTime": "{end-time}",
382-
"properties": {
383-
"Findings": [...]
384-
}
385-
}
386-
```
387-
If *Findings* property shows empty result, it means that all network tests passed and no connections are broken, otherwise it will show an error as follows *"Outbound dependency '{dependencyName}:{port}' might be not satisfied (Outbound)"* it means the cluster cannot reach the dependent service's endpoints, proceed with the following steps to troubleshoot.
393+
If the *Findings* property shows an empty result, it means that all network tests passed and no connections are broken. If it shows an error as follows *Outbound dependency '{dependencyName}:{port}' might be not satisfied (Outbound)*, the cluster can't reach the dependent service endpoints. Proceed with the following steps to troubleshoot.
388394
389-
### Step 2: Check Network Security Group (NSG)
395+
2. Check Network Security Group (NSG)
390396
391397
Make sure that the [Network Security Group](/azure/virtual-network/security-overview) is configured properly per the instuctions in [Dependencies for VNet deployment](/azure/data-explorer/vnet-deployment#dependencies-for-vnet-deployment)
392398
393-
### Step 3: Check Route Table
399+
3. Check route table
394400
395-
If the cluster's subnet has force-tunneling setup to firewall (subnet with a [route table](/azure/virtual-network/virtual-networks-udr-overview) that contains the default route '0.0.0.0/0') make sure that the [management IP addresses](/azure/data-explorer/vnet-deployment#azure-data-explorer-management-ip-addresses) and [health monitoring IP addresses](/azure/data-explorer/vnet-deployment#health-monitoring-ip-addresses) has route with [next hop type](/azure/virtual-network/virtual-networks-udr-overview##next-hop-types-across-azure-tools) *Internet*, and [source address prefix](/azure/virtual-network/virtual-networks-udr-overview#how-azure-selects-a-route) to *'management-ip/32'* and *'health-monitoring-ip/32'*, this is required to prevent asymmetric route issues.
401+
If the cluster's subnet has force-tunneling setup to firewall (subnet with a [route table](/azure/virtual-network/virtual-networks-udr-overview) that contains the default route '0.0.0.0/0') make sure that the [management IP addresses](#azure-data-explorer-management-ip-addresses) and [health monitoring IP addresses](#health-monitoring-ip-addresses) have a route with [next hop type](/azure/virtual-network/virtual-networks-udr-overview##next-hop-types-across-azure-tools) *Internet*, and [source address prefix](/azure/virtual-network/virtual-networks-udr-overview#how-azure-selects-a-route) to *'management-ip/32'* and *'health-monitoring-ip/32'*. This is required to prevent asymmetric route issues.
396402
397-
### Step 4: Check firewall rules
403+
4. Check firewall rules
398404
399-
If you force tunnel subnet outbound traffic to a firewall, make sure all dependencies FQDN (e.g. **.blob.core.windows.net*) are allowed in the firewall configuration as described in [Securing outbound traffic with firewall](/azure/data-explorer/vnet-deployment#securing-outbound-traffic-with-firewall).
405+
If you force tunnel subnet outbound traffic to a firewall, make sure all dependencies FQDN (e.g. **.blob.core.windows.net*) are allowed in the firewall configuration as described in [securing outbound traffic with firewall](/azure/data-explorer/vnet-deployment#securing-outbound-traffic-with-firewall).

0 commit comments

Comments
 (0)