Skip to content

Commit 2f5fc13

Browse files
authored
Merge pull request #97665 from v-rihow/self-hosted-integration-runtime-troubleshoot-guide
edit pass: self hosted integration runtime troubleshoot guide
2 parents c53eb8d + 5760aa6 commit 2f5fc13

File tree

1 file changed

+79
-74
lines changed

1 file changed

+79
-74
lines changed
Lines changed: 79 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Troubleshoot self-hosted integration runtime in Azure Data Factory
3-
description: Learn how to troubleshoot Self-hosted integration runtime issues in Azure Data Factory.
3+
description: Learn how to troubleshoot self-hosted integration runtime issues in Azure Data Factory.
44
services: data-factory
55
author: nabhishek
66
ms.service: data-factory
@@ -15,115 +15,120 @@ This article explores common troubleshooting methods for self-hosted integration
1515

1616
## Common errors and resolutions
1717

18-
### Error message: Self-hosted integration runtime is unable to connect to cloud service.
18+
### Error message: Self-hosted integration runtime can't connect to cloud service
1919

20-
- **Symptom**:
20+
![Self-hosted IR connection issue](media/self-hosted-integration-runtime-troubleshoot-guide/unable-to-connect-to-cloud-service.png)
2121

22-
![Self-Hosted IR connection issue](media/self-hosted-integration-runtime-troubleshoot-guide/unable-to-connect-to-cloud-service.png)
22+
#### Cause
2323

24-
- **Cause**: The self-hosted integration runtime isn't able to connect to data factory service (backend). Most often than not it's caused due to network settings in Firewall.
24+
The self-hosted integration runtime can't connect to the Data Factory service (backend). This issue is typically caused by network settings in the firewall.
2525

26-
- **Resolution**:
26+
#### Resolution
2727

28-
1. Check if the windows service "Integration Runtime Service" is running.
28+
1. Check whether the integration runtime service is running.
2929

30-
![Self-Hosted IR service running status](media/self-hosted-integration-runtime-troubleshoot-guide/integration-runtime-service-running-status.png)
30+
![Self-hosted IR service running status](media/self-hosted-integration-runtime-troubleshoot-guide/integration-runtime-service-running-status.png)
3131

32-
2. If the windows service as shown in [1] is running, follow below instructions as appropriate:
32+
1. If the service is running, go on to step 3.
3333

34-
1. If "proxy" is not configured on self-hosted integration runtime (default settings is no proxy configuration), run the below PowerShell command on the machine where self-hosted integration runtime is installed:
35-
36-
```powershell
37-
(New-Object System.Net.WebClient).DownloadString("https://wu2.frontend.clouddatahub.net/")
38-
```
39-
> [!NOTE]
40-
> The service URL may vary based on your data factory location. You can find the service URL under ADF UI -> Connections -> Integration runtimes -> Edit Self-hosted IR -> Nodes -> View Service URLs.
34+
1. If there's no proxy configured on the self-hosted integration runtime (which is the default setting), run the following PowerShell command on the machine where the self-hosted integration runtime is installed:
35+
36+
```powershell
37+
(New-Object System.Net.WebClient).DownloadString("https://wu2.frontend.clouddatahub.net/")
38+
```
39+
40+
> [!NOTE]
41+
> The service URL may vary, depending on your Data Factory location. You can find the service URL under **ADF UI** > **Connections** > **Integration runtimes** > **Edit Self-hosted IR** > **Nodes** > **View Service URLs**.
4142
42-
Below is the expected response:
43+
The following is the expected response:
4344
44-
![Powershell command response](media/self-hosted-integration-runtime-troubleshoot-guide/powershell-command-response.png)
45+
![PowerShell command response](media/self-hosted-integration-runtime-troubleshoot-guide/powershell-command-response.png)
4546
46-
If the response is different, then follow the below instructions as appropriate:
47+
1. If you don't receive the expected response, use one of the following methods as appropriate to your situation:
4748
48-
* If you get error "the remote name could not be resolved", there is an issue with DNS. Please get in touch with network team to get the DNS resolution issue fixed!
49-
* If you get error "ssl/tls cert is not trusted", please check if the Certificate for "https://wu2.frontend.clouddatahub.net/" is trusted on the machine, install the public certificate using cert manager, which should mitigate this issue.
50-
* Check Windows -> Event viewer (logs) -> Applications and Services Logs -> Integration Runtime for any failure, mostly caused by DNS, firewall rule, and network settings of the company (Forcedly close the connection). For this issue, please engage your network team for further troubleshot, because every company has customized network settings.
49+
* If you receive a "Remote name could not be resolved" message, there's a Domain Name System (DNS) issue. Contact your network team to fix this issue.
50+
* If you receive an "ssl/tls cert is not trusted" message, check whether the certificate for https://wu2.frontend.clouddatahub.net/ is trusted on the machine, and then install the public certificate by using Certificate Manager. This action should mitigate the issue.
51+
* Go to **Windows** > **Event viewer (logs)** > **Applications and Services Logs** > **Integration Runtime** and check for any failure that's caused by DNS, a firewall rule, or company network settings. (If you find such a failure, forcibly close the connection.) Because every company has customized network settings, contact your network team to troubleshoot these issues.
5152
52-
2. If "proxy" has been configured on the self-hosted integration runtime, verify whether your proxy server is able to access our service endpoint. For a sample command, refer [this](https://stackoverflow.com/questions/571429/powershell-web-requests-and-proxies).
53+
1. If "proxy" has been configured on the self-hosted integration runtime, verify that your proxy server can access the service endpoint. For a sample command, see [PowerShell, web requests, and proxies](https://stackoverflow.com/questions/571429/powershell-web-requests-and-proxies).
5354
54-
```powershell
55-
$user = $env:username
56-
$webproxy = (get-itemproperty 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet
57-
Settings').ProxyServer
58-
$pwd = Read-Host "Password?" -assecurestring
59-
$proxy = new-object System.Net.WebProxy
60-
$proxy.Address = $webproxy
61-
$account = new-object System.Net.NetworkCredential($user,[Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($pwd)), "")
62-
$proxy.credentials = $account
63-
$url = "https://wu2.frontend.clouddatahub.net/"
64-
$wc = new-object system.net.WebClient
65-
$wc.proxy = $proxy
66-
$webpage = $wc.DownloadData($url)
67-
$string = [System.Text.Encoding]::ASCII.GetString($webpage)
68-
$string
69-
```
70-
71-
Below is the expected response:
55+
```powershell
56+
$user = $env:username
57+
$webproxy = (get-itemproperty 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet
58+
Settings').ProxyServer
59+
$pwd = Read-Host "Password?" -assecurestring
60+
$proxy = new-object System.Net.WebProxy
61+
$proxy.Address = $webproxy
62+
$account = new-object System.Net.NetworkCredential($user,[Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($pwd)), "")
63+
$proxy.credentials = $account
64+
$url = "https://wu2.frontend.clouddatahub.net/"
65+
$wc = new-object system.net.WebClient
66+
$wc.proxy = $proxy
67+
$webpage = $wc.DownloadData($url)
68+
$string = [System.Text.Encoding]::ASCII.GetString($webpage)
69+
$string
70+
```
71+
72+
The following is the expected response:
7273
73-
![Powershell command response 2](media/self-hosted-integration-runtime-troubleshoot-guide/powershell-command-response.png)
74+
![Powershell command response 2](media/self-hosted-integration-runtime-troubleshoot-guide/powershell-command-response.png)
7475
75-
> [!NOTE]
76-
> Proxy considerations:
77-
> * Check if the proxy server requires whitelisting. If so, have [these domains](https://docs.microsoft.com/azure/data-factory/data-movement-security-considerations#firewall-requirements-for-on-premisesprivate-network) whitelisted.
78-
> * Check TLS/SSL cert for "wu2.frontend.clouddatahub.net/" is trusted on proxy server.
79-
> * If you are using active directory authentication in proxy, then change the service account to the user account that can access the proxy as "Integration Runtime Service".
76+
> [!NOTE]
77+
> Proxy considerations:
78+
> * Check whether the proxy server needs to be put on the Safe Recipients list. If so, make sure [these domains](https://docs.microsoft.com/azure/data-factory/data-movement-security-considerations#firewall-requirements-for-on-premisesprivate-network) are on the Safe Recipients list.
79+
> * Check whether the TLS/SSL certificate "wu2.frontend.clouddatahub.net/" is trusted on the proxy server.
80+
> * If you're using Active Directory authentication on the proxy, change the service account to the user account that can access the proxy as "Integration Runtime Service."
8081
8182
### Error message: Self-hosted integration runtime node/ logical SHIR is in Inactive/ "Running (Limited)" state
8283
83-
- **Cause**: You may see Self-hosted IR node in Inactive status as shown in the screenshot below:
84+
#### Cause
8485
85-
![Inactive Self-Hosted IR node](media/self-hosted-integration-runtime-troubleshoot-guide/inactive-self-hosted-ir-node.png)
86+
The self-hosted integrated runtime node might have an **Inactive** status, as shown in the following screenshot:
8687
87-
It happens so when nodes are not able to communicate with each other.
88+
![Inactive Self-Hosted IR node](media/self-hosted-integration-runtime-troubleshoot-guide/inactive-self-hosted-ir-node.png)
8889
89-
- **Resolution**:
90+
This behavior occurs when nodes can't communicate with each other.
9091
91-
Log into the node hosted VM, and open Event View, under the Applications and Services Logs -> Integration Runtime, filter all the error logs.
92+
#### Resolution
9293
93-
1. If the error log contains:
94-
95-
**Error log**: System.ServiceModel.EndpointNotFoundException: Could not connect to net.tcp://xxxxxxx.bwld.com:8060/ExternalService.svc/WorkerManager. The connection attempt lasted for a time span of 00:00:00.9940994. TCP error code 10061: No connection could be made because the target machine actively refused it 10.2.4.10:8060. --->
96-
System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.2.4.10:8060
97-
98-
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
99-
100-
at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
101-
102-
at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout)
94+
1. Log in to the node-hosted VM. Under **Applications and Services Logs** > **Integration Runtime**, open Event Viewer, and filter all the error logs.
95+
96+
1. Check whether an error log contains the following error:
10397
104-
**Solution:** launch the command line: telnet 10.2.4.10 8060
98+
```System.ServiceModel.EndpointNotFoundException: Could not connect to net.tcp://xxxxxxx.bwld.com:8060/ExternalService.svc/WorkerManager. The connection attempt lasted for a time span of 00:00:00.9940994. TCP error code 10061: No connection could be made because the target machine actively refused it 10.2.4.10:8060.
99+
System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it.
100+
10.2.4.10:8060
105101
106-
If you get below error, please contact your IT guys for help with fixing this issue. After you could successfully telnet, contact Microsoft support if you still have issues for the IR node status.
102+
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
103+
104+
at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
105+
106+
at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout)
107+
108+
1. If you see this error, run the following on a command line:
109+
110+
**telnet 10.2.4.10 8060**.
111+
1. If you receive the following error, contact your IT department for help with fixing this issue. After you can successfully telnet, contact Microsoft Support if you still have issues with the integrative runtime node status.
107112
108-
![Command-line error](media/self-hosted-integration-runtime-troubleshoot-guide/command-line-error.png)
113+
![Command-line error](media/self-hosted-integration-runtime-troubleshoot-guide/command-line-error.png)
109114
110-
2. If the error log contains:
111-
112-
**Error log:** Cannot connect to worker manager: net.tcp://xxxxxx:8060/ExternalService.svc/ No DNS entries exist for host azranlcir01r1. No such host is known Exception detail: System.ServiceModel.EndpointNotFoundException: No DNS entries exist for host xxxxx. ---> System.Net.Sockets.SocketException: No such host is known at System.Net.Dns.GetAddrInfo(String name) at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6) at System.Net.Dns.GetHostEntry(String hostNameOrAddress) at System.ServiceModel.Channels.DnsCache.Resolve(Uri uri) --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.DnsCache.Resolve(Uri uri)
115+
1. Check whether the error log contains the following:
116+
117+
```Error log: Cannot connect to worker manager: net.tcp://xxxxxx:8060/ExternalService.svc/ No DNS entries exist for host azranlcir01r1. No such host is known Exception detail: System.ServiceModel.EndpointNotFoundException: No DNS entries exist for host xxxxx. ---> System.Net.Sockets.SocketException: No such host is known at System.Net.Dns.GetAddrInfo(String name) at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6) at System.Net.Dns.GetHostEntry(String hostNameOrAddress) at System.ServiceModel.Channels.DnsCache.Resolve(Uri uri) --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.DnsCache.Resolve(Uri uri)```
113118
114-
**Solution:** One of the below two actions can help resolve the issue:
115-
1. Put all the nodes in the same domain.
116-
2. Add IP to host mapping in all the hosted VM's hosts file.
119+
1. To resolve the issue, try one or both of the following methods:
120+
- Put all the nodes in the same domain.
121+
- Add the IP to host mapping in all the hosted VM's host files.
117122
118123
119124
## Next steps
120125
121-
For more troubleshooting help, try these resources:
126+
For more help with troubleshooting, try the following resources:
122127
123128
* [Data Factory blog](https://azure.microsoft.com/blog/tag/azure-data-factory/)
124129
* [Data Factory feature requests](https://feedback.azure.com/forums/270578-data-factory)
125130
* [Azure videos](https://azure.microsoft.com/resources/videos/index/?sort=newest&services=data-factory)
126131
* [MSDN forum](https://social.msdn.microsoft.com/Forums/home?sort=relevancedesc&brandIgnore=True&searchTerm=data+factory)
127-
* [Stack Overflow forum for Data Factory](https://stackoverflow.com/questions/tagged/azure-data-factory)
132+
* [Stack overflow forum for Data Factory](https://stackoverflow.com/questions/tagged/azure-data-factory)
128133
* [Twitter information about Data Factory](https://twitter.com/hashtag/DataFactory)
129-
* [ADF mapping data flows Performance Guide](concepts-data-flow-performance.md)
134+
* [Mapping data flows performance guide](concepts-data-flow-performance.md)

0 commit comments

Comments
 (0)