|
| 1 | +--- |
| 2 | +title: Troubleshoot self-hosted integration runtime in Azure Data Factory |
| 3 | +description: Learn how to troubleshoot Self-hosted integration runtime issues in Azure Data Factory. |
| 4 | +services: data-factory |
| 5 | +author: nabhishek |
| 6 | +ms.service: data-factory |
| 7 | +ms.topic: troubleshooting |
| 8 | +ms.date: 11/07/2019 |
| 9 | +ms.author: abnarain |
| 10 | +--- |
| 11 | + |
| 12 | +# Troubleshoot self-hosted integration runtime |
| 13 | + |
| 14 | +This article explores common troubleshooting methods for self-hosted integration runtime in Azure Data Factory. |
| 15 | + |
| 16 | +## Common errors and resolutions |
| 17 | + |
| 18 | +### Error message: Self-hosted integration runtime is unable to connect to cloud service. |
| 19 | + |
| 20 | +- **Symptom**: |
| 21 | + |
| 22 | +  |
| 23 | + |
| 24 | +- **Cause**: The self-hosted integration runtime isn't able to connect to data factory service (backend). Most often than not it's caused due to network settings in Firewall. |
| 25 | + |
| 26 | +- **Resolution**: |
| 27 | + |
| 28 | + 1. Check if the windows service "Integration Runtime Service" is running. |
| 29 | + |
| 30 | +  |
| 31 | + |
| 32 | + 2. If the windows service as shown in [1] is running, follow below instructions as appropriate: |
| 33 | + |
| 34 | + 1. If "proxy" is not configured on self-hosted integration runtime (default settings is no proxy configuration), run the below PowerShell command on the machine where self-hosted integration runtime is installed: |
| 35 | + |
| 36 | + ```powershell |
| 37 | + (New-Object System.Net.WebClient).DownloadString("https://wu2.frontend.clouddatahub.net/") |
| 38 | + ``` |
| 39 | + > [!NOTE] |
| 40 | + > The service URL may vary based on your data factory location. You can find the service URL under ADF UI -> Connections -> Integration runtimes -> Edit Self-hosted IR -> Nodes -> View Service URLs. |
| 41 | + |
| 42 | + Below is the expected response: |
| 43 | + |
| 44 | +  |
| 45 | + |
| 46 | + If the response is different, then follow the below instructions as appropriate: |
| 47 | + |
| 48 | + * If you get error "the remote name could not be resolved", there is an issue with DNS. Please get in touch with network team to get the DNS resolution issue fixed! |
| 49 | + * If you get error "ssl/tls cert is not trusted", please check if the Certificate for "https://wu2.frontend.clouddatahub.net/" is trusted on the machine, install the public certificate using cert manager, which should mitigate this issue. |
| 50 | + * Check Windows -> Event viewer (logs) -> Applications and Services Logs -> Integration Runtime for any failure, mostly caused by DNS, firewall rule, and network settings of the company (Forcedly close the connection). For this issue, please engage your network team for further troubleshot, because every company has customized network settings. |
| 51 | + |
| 52 | + 2. If "proxy" has been configured on the self-hosted integration runtime, verify whether your proxy server is able to access our service endpoint. For a sample command, refer [this](https://stackoverflow.com/questions/571429/powershell-web-requests-and-proxies). |
| 53 | + |
| 54 | + ```powershell |
| 55 | + $user = $env:username |
| 56 | + $webproxy = (get-itemproperty 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet |
| 57 | + Settings').ProxyServer |
| 58 | + $pwd = Read-Host "Password?" -assecurestring |
| 59 | + $proxy = new-object System.Net.WebProxy |
| 60 | + $proxy.Address = $webproxy |
| 61 | + $account = new-object System.Net.NetworkCredential($user,[Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($pwd)), "") |
| 62 | + $proxy.credentials = $account |
| 63 | + $url = "https://wu2.frontend.clouddatahub.net/" |
| 64 | + $wc = new-object system.net.WebClient |
| 65 | + $wc.proxy = $proxy |
| 66 | + $webpage = $wc.DownloadData($url) |
| 67 | + $string = [System.Text.Encoding]::ASCII.GetString($webpage) |
| 68 | + $string |
| 69 | + ``` |
| 70 | +
|
| 71 | + Below is the expected response: |
| 72 | + |
| 73 | +  |
| 74 | +
|
| 75 | + > [!NOTE] |
| 76 | + > Proxy considerations: |
| 77 | + > * Check if the proxy server requires whitelisting. If so, have [these domains](https://docs.microsoft.com/azure/data-factory/data-movement-security-considerations#firewall-requirements-for-on-premisesprivate-network) whitelisted. |
| 78 | + > * Check TLS/SSL cert for "wu2.frontend.clouddatahub.net/" is trusted on proxy server. |
| 79 | + > * If you are using active directory authentication in proxy, then change the service account to the user account that can access the proxy as "Integration Runtime Service". |
| 80 | +
|
| 81 | +### Error message: Self-hosted integration runtime node/ logical SHIR is in Inactive/ "Running (Limited)" state |
| 82 | +
|
| 83 | +- **Cause**: You may see Self-hosted IR node in Inactive status as shown in the screenshot below: |
| 84 | +
|
| 85 | +  |
| 86 | +
|
| 87 | + It happens so when nodes are not able to communicate with each other. |
| 88 | +
|
| 89 | +- **Resolution**: |
| 90 | +
|
| 91 | + Log into the node hosted VM, and open Event View, under the Applications and Services Logs -> Integration Runtime, filter all the error logs. |
| 92 | +
|
| 93 | + 1. If the error log contains: |
| 94 | + |
| 95 | + **Error log**: System.ServiceModel.EndpointNotFoundException: Could not connect to net.tcp://xxxxxxx.bwld.com:8060/ExternalService.svc/WorkerManager. The connection attempt lasted for a time span of 00:00:00.9940994. TCP error code 10061: No connection could be made because the target machine actively refused it 10.2.4.10:8060. ---> |
| 96 | + System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.2.4.10:8060 |
| 97 | + |
| 98 | + at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) |
| 99 | + |
| 100 | + at System.Net.Sockets.Socket.Connect(EndPoint remoteEP) |
| 101 | + |
| 102 | + at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout) |
| 103 | + |
| 104 | + **Solution:** launch the command line: telnet 10.2.4.10 8060 |
| 105 | + |
| 106 | + If you get below error, please contact your IT guys for help with fixing this issue. After you could successfully telnet, contact Microsoft support if you still have issues for the IR node status. |
| 107 | + |
| 108 | +  |
| 109 | + |
| 110 | + 2. If the error log contains: |
| 111 | + |
| 112 | + **Error log:** Cannot connect to worker manager: net.tcp://xxxxxx:8060/ExternalService.svc/ No DNS entries exist for host azranlcir01r1. No such host is known Exception detail: System.ServiceModel.EndpointNotFoundException: No DNS entries exist for host xxxxx. ---> System.Net.Sockets.SocketException: No such host is known at System.Net.Dns.GetAddrInfo(String name) at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6) at System.Net.Dns.GetHostEntry(String hostNameOrAddress) at System.ServiceModel.Channels.DnsCache.Resolve(Uri uri) --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.DnsCache.Resolve(Uri uri) |
| 113 | + |
| 114 | + **Solution:** One of the below two actions can help resolve the issue: |
| 115 | + 1. Put all the nodes in the same domain. |
| 116 | + 2. Add IP to host mapping in all the hosted VM's hosts file. |
| 117 | +
|
| 118 | +
|
| 119 | +## Next steps |
| 120 | +
|
| 121 | +For more troubleshooting help, try these resources: |
| 122 | +
|
| 123 | +* [Data Factory blog](https://azure.microsoft.com/blog/tag/azure-data-factory/) |
| 124 | +* [Data Factory feature requests](https://feedback.azure.com/forums/270578-data-factory) |
| 125 | +* [Azure videos](https://azure.microsoft.com/resources/videos/index/?sort=newest&services=data-factory) |
| 126 | +* [MSDN forum](https://social.msdn.microsoft.com/Forums/home?sort=relevancedesc&brandIgnore=True&searchTerm=data+factory) |
| 127 | +* [Stack Overflow forum for Data Factory](https://stackoverflow.com/questions/tagged/azure-data-factory) |
| 128 | +* [Twitter information about Data Factory](https://twitter.com/hashtag/DataFactory) |
| 129 | +* [ADF mapping data flows Performance Guide](concepts-data-flow-performance.md) |
0 commit comments