DNS Debug Option #1725

reggie-mcmurtrey · 2022-10-02T17:46:35Z

reggie-mcmurtrey
Oct 2, 2022

Would it be possible to add a DNS Debug command line option. I would be helpful if you could print what sever dns responses are coming from and what the response is: empty, wrong id or valid id. It would also be helpful if the expected id was printed.

I have been fighting issues with traefik and duckdns. I have installed lego on an ubuntu 22.04 machine to eliminate any config issues with traefik and I'm having problems there too. If I use dig on 8.8.8.8 and 1.1.1.1 I can see the TXT record change, but lego keeps giving errors:

lego -a \
        -m '[email protected]' \
        -s 'https://acme-staging-v02.api.letsencrypt.org/directory' \
        --dns-timeout 1000 \
        --dns duckdns \
        -d something.duckdns.org \
        run

2022/10/02 02:03:51 [INFO] [something.duckdns.org] acme: Obtaining bundled SAN certificate
2022/10/02 02:03:51 [INFO] [something.duckdns.org] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/3820126564
2022/10/02 02:03:51 [INFO] [something.duckdns.org] acme: Could not find solver for: tls-alpn-01
2022/10/02 02:03:51 [INFO] [something.duckdns.org] acme: Could not find solver for: http-01
2022/10/02 02:03:51 [INFO] [something.duckdns.org] acme: use dns-01 solver
2022/10/02 02:03:51 [INFO] [something.duckdns.org] acme: Preparing to solve DNS-01
2022/10/02 02:03:56 [INFO] [something.duckdns.org] acme: Trying to solve DNS-01
2022/10/02 02:03:56 [INFO] [something.duckdns.org] acme: Checking DNS record propagation using [192.168.96.2:53 8.8.8.8:53 71.10.216.1:53]
2022/10/02 02:03:56 [INFO] Wait for propagation [timeout: 2m0s, interval: 5s]
2022/10/02 02:04:26 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 02:04:46 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 02:05:11 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 02:05:33 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 02:06:08 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 02:06:13 [INFO] [something.duckdns.org] acme: Cleaning DNS-01 challenge
2022/10/02 02:06:14 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/3820126564
2022/10/02 02:06:14 Could not obtain certificates:
        acme: Error -> One or more domains had a problem:
[something.duckdns.org] time limit exceeded: last error: could not determine the zone: unexpected response code 'SERVFAIL' for _acme-challenge.something.duckdns.org.

If I add the --dns.resolvers 8.8.8.8 the Checking DNS record propagation line changes, but exact same error even though dig shows:

dig @8.8.8.8 _acme-challenge.something.duckdns.org txt

; <<>> DiG 9.16.1-Ubuntu <<>> @8.8.8.8 _acme-challenge.something.duckdns.org txt
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45760
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;_acme-challenge.something.duckdns.org. IN TXT

;; ANSWER SECTION:
_acme-challenge.something.duckdns.org. 57 IN TXT   "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

;; Query time: 28 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Oct 02 02:35:43 CDT 2022
;; MSG SIZE  rcvd: 119

I just feel like some additional debug info may help people out. Something like:

2022/10/02 02:04:26 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 02:04:26 [INFO] [something.duckdns.org]: Response from 8.8.8.8
2022/10/02 02:04:26 [INFO] [something.duckdns.org]: Empty TXT record!
2022/10/02 02:04:26 [INFO] [something.duckdns.org]: expected "acfalfja;le[ad-ad;lvia"

Also does --dns.resolvers override all local dns config as I am having to use split DNS currently, so I would expect issues if lego is still trying to use any local dns servers.

Thanks,
Reggie

ldez · 2022-10-02T17:55:30Z

ldez
Oct 2, 2022
Maintainer

Hello,

could not determine the zone: unexpected response code 'SERVFAIL' for _acme-challenge.something.duckdns.org.

This message is related to the SOA call to get the "zone" related to the FQDN.
It's not related to the TXT record.

To test it:

drill _acme-challenge.something.duckdns.org. SOA

1 reply

reggie-mcmurtrey Oct 2, 2022
Author

Sorry just realized I replied to myself instead of your response. See info below.

reggie-mcmurtrey · 2022-10-02T23:46:41Z

reggie-mcmurtrey
Oct 2, 2022
Author

Hi ldez,

Thanks for the fast response. I took your advice and did some more testing:

drill -c resolv.conf _acme-challenge.something.duckdns.org. SOA
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 35537
;; flags: qr rd ra ; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION:
;; _acme-challenge.something.duckdns.org.  IN      SOA

;; ANSWER SECTION:

;; AUTHORITY SECTION:
duckdns.org.    600     IN      SOA     ns1.duckdns.org. hostmaster.duckdns.org. 2022061901 6000 120 2419200 600

;; ADDITIONAL SECTION:

;; Query time: 1058 msec
;; SERVER: 8.8.8.8
;; WHEN: Sun Oct  2 18:17:22 2022
;; MSG SIZE  rcvd: 103

This all looks reasonable to me, so I tried lego again a few times with:

lego -a \
-m '[email protected]' \
-s 'https://acme-staging-v02.api.letsencrypt.org/directory' \
--dns duckdns \
--dns.resolvers 8.8.8.8 \
--dns.resolvers 1.1.1.1 \
-d something.duckdns.org \
run

usally all I get is:

2022/10/02 18:18:04 [INFO] [something.duckdns.org] acme: Obtaining bundled SAN certificate
2022/10/02 18:18:05 [INFO] [something.duckdns.org] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/3827960074
2022/10/02 18:18:05 [INFO] [something.duckdns.org] acme: Could not find solver for: tls-alpn-01
2022/10/02 18:18:05 [INFO] [something.duckdns.org] acme: Could not find solver for: http-01
2022/10/02 18:18:05 [INFO] [something.duckdns.org] acme: use dns-01 solver
2022/10/02 18:18:05 [INFO] [something.duckdns.org] acme: Preparing to solve DNS-01
2022/10/02 18:18:05 [INFO] [something.duckdns.org] acme: Trying to solve DNS-01
2022/10/02 18:18:05 [INFO] [something.duckdns.org] acme: Checking DNS record propagation using [8.8.8.8:53 1.1.1.1:53]
2022/10/02 18:18:05 [INFO] Wait for propagation [timeout: 2m0s, interval: 5s]
2022/10/02 18:18:26 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:18:46 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:19:07 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:19:27 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:19:43 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:20:00 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:20:15 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:20:20 [INFO] [something.duckdns.org] acme: Cleaning DNS-01 challenge
2022/10/02 18:20:20 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/3827960074
2022/10/02 18:20:20 Could not obtain certificates:
        acme: Error -> One or more domains had a problem:
[something.duckdns.org] time limit exceeded: last error: dial udp: i/o timeout

But once I got:

2022/10/02 17:58:51 [INFO] [something.duckdns.org] acme: Obtaining bundled SAN certificate
2022/10/02 17:58:51 [INFO] [something.duckdns.org] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/3827768084
2022/10/02 17:58:51 [INFO] [something.duckdns.org] acme: Could not find solver for: tls-alpn-01
2022/10/02 17:58:51 [INFO] [something.duckdns.org] acme: Could not find solver for: http-01
2022/10/02 17:58:51 [INFO] [something.duckdns.org] acme: use dns-01 solver
2022/10/02 17:58:51 [INFO] [something.duckdns.org] acme: Preparing to solve DNS-01
2022/10/02 17:58:51 [INFO] [something.duckdns.org] acme: Trying to solve DNS-01
2022/10/02 17:58:51 [INFO] [something.duckdns.org] acme: Checking DNS record propagation using [8.8.8.8:53 1.1.1.1:53]
2022/10/02 17:58:51 [INFO] Wait for propagation [timeout: 2m0s, interval: 5s]
2022/10/02 17:58:58 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 17:59:32 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 17:59:48 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:00:05 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:00:29 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:00:48 [INFO] [something.duckdns.org] acme: Waiting for DNS record propagation.
2022/10/02 18:00:53 [INFO] [something.duckdns.org] acme: Cleaning DNS-01 challenge
2022/10/02 18:00:54 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/3827768084
2022/10/02 18:00:54 Could not obtain certificates:
        acme: Error -> One or more domains had a problem:
[something.duckdns.org] time limit exceeded: last error: read udp 192.168.96.4:46661->99.79.143.35:53: i/o timeout

tried to find out who 99.79.143.35 is:

dig -x 99.79.16.64

; <<>> DiG 9.16.1-Ubuntu <<>> -x 99.79.16.64
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36470
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 5

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;64.16.79.99.in-addr.arpa.      IN      PTR

;; ANSWER SECTION:
64.16.79.99.in-addr.arpa. 300   IN      PTR     ec2-99-79-16-64.ca-central-1.compute.amazonaws.com.

;; AUTHORITY SECTION:
16.79.99.in-addr.arpa.  300     IN      NS      ns1-24-ca-central-1.ec2-rdns.amazonaws.com.
16.79.99.in-addr.arpa.  300     IN      NS      ns3-24-ca-central-1.ec2-rdns.amazonaws.com.
16.79.99.in-addr.arpa.  300     IN      NS      ns2-24-ca-central-1.ec2-rdns.amazonaws.com.
16.79.99.in-addr.arpa.  300     IN      NS      ns4-24-ca-central-1.ec2-rdns.amazonaws.com.

;; ADDITIONAL SECTION:
ns1-24-ca-central-1.ec2-rdns.amazonaws.com. 300 IN A 205.251.196.243
ns2-24-ca-central-1.ec2-rdns.amazonaws.com. 300 IN A 205.251.199.239
ns3-24-ca-central-1.ec2-rdns.amazonaws.com. 300 IN A 205.251.192.216
ns4-24-ca-central-1.ec2-rdns.amazonaws.com. 300 IN A 205.251.194.81

;; Query time: 504 msec
;; SERVER: 192.168.96.2#53(192.168.96.2)
;; WHEN: Sun Oct 02 18:13:12 CDT 2022
;; MSG SIZE  rcvd: 326

ok, so amazon DNS servers, I think DuckDNS is built on Amazon cloud infrastructure, so that makes sense. I did some googling on the i/o timeout error message and found a post talking about checking connection with netcat so:

Test Command against Google DNS server

nc -zv 8.8.8.8 53
Connection to 8.8.8.8 53 port [tcp/domain] succeeded!

Test Command against DuckDNS/Amazon server?

nc -zv 99.79.16.64 53
(no response, hangs forever)
^C

So does this mean something is broken with DuckDNS/Amazon and I'm just out of luck?

1 reply

ldez Oct 10, 2022
Maintainer

Thanks for the fast response. I took your advice and did some more testing:

drill -c resolv.conf _acme-challenge.something.duckdns.org. SOA
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 35537
;; flags: qr rd ra ; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION:
;; _acme-challenge.something.duckdns.org.  IN      SOA

;; ANSWER SECTION:

;; AUTHORITY SECTION:
duckdns.org.    600     IN      SOA     ns1.duckdns.org. hostmaster.duckdns.org. 2022061901 6000 120 2419200 600

;; ADDITIONAL SECTION:

;; Query time: 1058 msec
;; SERVER: 8.8.8.8
;; WHEN: Sun Oct  2 18:17:22 2022
;; MSG SIZE  rcvd: 103

The answer section is empty, so it's not the expected answer.

The expected answer is something like that:

$ drill example.com. SOA              
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 17755
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; example.com. IN      SOA

;; ANSWER SECTION:
example.com.    2604    IN      SOA     ns.icann.org. noc.dns.icann.org. 2022091119 7200 3600 1209600 3600

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 12 msec
;; SERVER: 2a02:842b:5da:c101::1
;; WHEN: Mon Oct 10 04:26:18 2022
;; MSG SIZE  rcvd: 85

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

DNS Debug Option #1725

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

DNS Debug Option #1725

Uh oh!

Uh oh!

reggie-mcmurtrey Oct 2, 2022

Replies: 2 comments · 2 replies

Uh oh!

ldez Oct 2, 2022 Maintainer

Uh oh!

reggie-mcmurtrey Oct 2, 2022 Author

Uh oh!

reggie-mcmurtrey Oct 2, 2022 Author

Uh oh!

ldez Oct 10, 2022 Maintainer

reggie-mcmurtrey
Oct 2, 2022

Replies: 2 comments 2 replies

ldez
Oct 2, 2022
Maintainer

reggie-mcmurtrey Oct 2, 2022
Author

reggie-mcmurtrey
Oct 2, 2022
Author

ldez Oct 10, 2022
Maintainer