connect-proxy example isn't working with DNS Over HTTPS

[NghiaTranUIT]

Expected behavior

• Connect-Proxy example should work with DNS Over HTTPS -> Can access the Internet as usual

Actual behavior

• Could not connect to certain domains

2020-12-08T10:18:44+0700 error com.apple.nio-connect-proxy.ConnectHandler : remoteAddress=Optional([IPv4]127.0.0.1/127.0.0.1:51759) localAddress=Optional([IPv4]127.0.0.1/127.0.0.1:9090) channel=ObjectIdentifier(0x000000010840c7a0) Connect failed: NIOConnectionError(host: "in.appcenter.ms", port: 443, dnsAError: Optional(NIO.SocketAddressError.unknown(host: "in.appcenter.ms", port: 443)), dnsAAAAError: Optional(NIO.SocketAddressError.unknown(host: "in.appcenter.ms", port: 443)), connectionErrors: [])

2020-12-08T10:18:44+0700 error com.apple.nio-connect-proxy.ConnectHandler : channel=ObjectIdentifier(0x0000000100c0b770) remoteAddress=Optional([IPv4]127.0.0.1/127.0.0.1:51760) localAddress=Optional([IPv4]127.0.0.1/127.0.0.1:9090) Connect failed: NIOConnectionError(host: "google.com", port: 443, dnsAError: Optional(NIO.SocketAddressError.unknown(host: "google.com", port: 443)), dnsAAAAError: Optional(NIO.SocketAddressError.unknown(host: "google.com", port: 443)), connectionErrors: [])

2020-12-08T10:18:44+0700 error com.apple.nio-connect-proxy.ConnectHandler : remoteAddress=Optional([IPv4]127.0.0.1/127.0.0.1:51855) localAddress=Optional([IPv4]127.0.0.1/127.0.0.1:9090) channel=ObjectIdentifier(0x000000010800a3a0) Connect failed: connectTimeout(NIO.TimeAmount(nanoseconds: 10000000000))

• It happens 50-50 all the time. Sometimes I could connect, but sometimes I couldn't and get a bunch of errors.
• If I delete the DNS Profile -> 100% it's working

Steps to reproduce

1. Download DNS Profile at https://github.com/paulmillr/encrypted-dns
2. Install cloudflare-https.mobileconfig on Big Sur macOS
3. Go to System Preferrrence -> Profile and install it again
4. Verify that you can open any websites from Safari and Google Chrome as usual`
5. Set HTTP/HTTPS Proxy from Wifi -> Advanced -> Proxies to 9090
6. Start connect-proxy at port 9090
7. Quit Safari or Google Chrome if it's opening. Try to access some well-known websites, such as google.com, stackoverflow.com, ...

version/commit hashes from all involved dependencies

• The lasted commit on the main branch (09044ff)

Swift & OS version (output of swift --version && uname -a)

Apple Swift version 5.3.1 (swiftlang-1200.0.41 clang-1200.0.32.8)
Target: x86_64-apple-darwin20.1.0
Darwin Nghias-MacBook-Pro.local 20.1.0 Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:11 PDT 2020; root:xnu-7195.50.7~2/RELEASE_X86_64 x86_64

---

Thank Swift NIO team for developing a powerful tool 👍

[Lukasa]

Thanks @NghiaTranUIT!

This is a surprising behaviour to me, I definitely don't know the macOS DNS configuration well enough to understand this issue. I'll consult with some colleagues and get back to you.

[Lukasa]

Ok, I haven't yet reproduced this today. I'll do some more investigation tomorrow. Please also do file Feedback Assistant reports and note their numbers here.

[Lukasa]

Out of curiosity @NghiaTranUIT, when you run this are you seeing DNS lookups to Cloudflare domains?

[NghiaTranUIT]

Yes, I can see they call CONNECT cloudflare-dns.com:443 HTTP/1.1

-12-09T18:59:58+0700 info com.apple.nio-connect-proxy.ConnectHandler : localAddress=Optional([IPv4]127.0.0.1/127.0.0.1:9090) channel=ObjectIdentifier(0x0000000108b04230) remoteAddress=Optional([IPv4]127.0.0.1/127.0.0.1:57790) CONNECT cloudflare-dns.com:443 HTTP/1.1
2020-12-09T18:59:58+0700 info com.apple.nio-connect-proxy.ConnectHandler : localAddress=Optional([IPv4]127.0.0.1/127.0.0.1:9090) channel=ObjectIdentifier(0x0000000108b04230) remoteAddress=Optional([IPv4]127.0.0.1/127.0.0.1:57790) Connecting to cloudflare-dns.com:443
2020-12-09T18:59:59+0700 info com.apple.nio-connect-proxy.ConnectHandler : remoteAddress=Optional([IPv4]127.0.0.1/127.0.0.1:57791) channel=ObjectIdentifier(0x0000000100e07210) localAddress=Optional([IPv4]127.0.0.1/127.0.0.1:9090) CONNECT p63-caldav.icloud.com:443 HTTP/1.1
2020-12-09T18:59:59+0700 info com.apple.nio-connect-proxy.ConnectHandler : remoteAddress=Optional([IPv4]127.0.0.1/127.0.0.1:57791) channel=ObjectIdentifier(0x0000000100e07210) localAddress=Optional([IPv4]127.0.0.1/127.0.0.1:9090) Connecting to p63-caldav.icloud.com:443
2020-12-09T19:00:00+0700 info com.apple.nio-connect-proxy.ConnectHandler : remoteAddress=Optional([IPv4]127.0.0.1/127.0.0.1:57792) localAddress=Optional([IPv4]127.0.0.1/127.0.0.1:9090) channel=ObjectIdentifier(0x0000000108b06000) CONNECT cloudflare-dns.com:443 HTTP/1.1
2020-12-09T19:00:00+0700 info com.apple.nio-connect-proxy.ConnectHandler : remoteAddress=Optional([IPv4]127.0.0.1/127.0.0.1:57792) localAddress=Optional([IPv4]127.0.0.1/127.0.0.1:9090) channel=ObjectIdentifier(0x0000000108b06000) Connecting to cloudflare-dns.com:443

After a while, I'm still received a bunch of Timeout Error like the first comment. Stop and Start the project again, I'm able to visit those websites as usual. Like 50% reproducible rate.

---

Full log:
log.txt

[Lukasa]

Are you seeing those actually work? That is, do those DNS lookups ever succeed?

[NghiaTranUIT]

I just noticed that the CloudFlare profile is unsiged. Not sure if it's related 🤔

[NghiaTranUIT]

Here is the success log: You can see the app call cloudflare-dns.com and there is no error log. I'm able to access google.com and another website as usual.

success_log.txt

If I start/stop the project, 50% works, 50% doesn't work for some reason 🤔

[Lukasa]

I think we have a recursive DNS issue. I think if the DNS lookup for Cloudflare-dns is cached then we don't have an issue, but if we try to do a new lookup we get stuck.

[Lukasa]

@NghiaTranUIT Can I ask you to try adding cloudflare-dns.com to the excluded hosts list for the HTTPS proxying to see if that resolves the issue?

[NghiaTranUIT]

I added cloudflare-dns.com and www.google.com to the exclusive list, but for some reason, I still can see cloudflare-dns.com from the console log, but not www.google.com

Result:

• Still see cloudflare DNS traffic in the console log -> Look like it does not by-pass the proxy
• I'm able to access www.google.com (since it's in the exclusive list)
• I could not access another website, such as https://news.ycombinator.com (see the error log)

Log:
log.txt

[Lukasa]

That’s really useful information @NghiaTranUIT, thanks!