PowerDNS Provider deletes all TXT records, not just the ones it created

[t-schuster]

Welcome

• Yes, I'm using a binary release within 2 latest releases.
• Yes, I've searched similar issues on GitHub and didn't find any.
• Yes, I've included all information below (version, config, etc).

What did you expect to see?

When multiple seperated certificates are requested by Traefik or when multiple Traefik instances attempt to obtain an ACME Cert from a domain at similar points in time, both obtain the certificate and clean up their respective DNS records.

What did you see instead?

When the PowerDNS Lego provider finishes it's challenge, it deletes all TXT records from the zone, leading to spurious failure on other instances of Lego/Traefik who are attempting the same challenge.

Looking at the code at providers/dns/pdns/pdns.go#L195, Lego is simply requesting all TXT records to be deleted, if another instance of Lego is attempting the challenge, it will timeout if it never sees the record or it will fail the challenge when it sees the record but LetsEncrypt cannot find it.

As an additional side effect, Lego will also wipe any other DNS records on the Zone, which in normal circumstances is fine but if one enables LEGO_EXPERIMENTAL_CNAME_SUPPORT=true then a configuration mistake may lead to Lego being pointed at an unrelated domain and ends up wiping unrelated but important records such as SPF or DKIM.

A cursory check in the code reveals that providers like Cloudflare do this properly and delete only the created record but RFC2136 via TSIG has the same issue as PDNS.

How do you use lego?

Through Traefik

Reproduction steps

1. Setup multiple Traefik instances
2. Setup PowerDNS with HTTP API in Traefik
3. Attempt to obtain multiple certificates from the same domain at the same time

Version of lego

Traefik Docker Container 2.6 (traefik:v2.6)
github.com/go-acme/lego/v4 v4.6.0

Logs

exemplary dig output

DNS Setup:

_acme-challenge.example.com CNAME acme.example.com
_acme-challenge.subdomain.example.com CNAME acme.example.com

$ dig acme.example.com ANY +short
acme.example.com A 1.2.3.4
acme.example.com TXT "important TXT record data"
acme.example.com TXT "currently running ACME challenge for www.example.com"
$ # begin validation for www.subdomain.example.com
$ dig acme.example.com ANY +short
acme.example.com A 1.2.3.4
acme.example.com TXT "important TXT record data"
acme.example.com TXT "currently running ACME challenge"
acme.example.com TXT "the new started ACME challenge"
$ # validation of previously running challenge for www.example.com ends
$ dig acme.example.com ANY +short
acme.example.com A 1.2.3.4
$ # newly begun validation for www.subdomain.example.com fails due to missing record
$ # previously existing TXT record is missing

Go environment (if applicable)

No response

[ldez]

Hello,

first, I don't recommend using mulitple Traefik instances to claim certificates on the same domain.

The PowerDNS API is quite restrictive, you focused on the cleanup but the present can produce the same thing because we have to replace the full RRSet.

[t-schuster]

Hi,

I need multiple Traefik instances, not just 2, since I'm working on a geo-redundant setup for a web project and I've not found a way to distribute certificates between Traefik instances easily when DNS resolution is geo-restricted. So I can't exactly avoid doing it this way, I've not found an easy way to handle certificates with Traefik in TLS or HTTP mode. It's also not necessarily the same domain, as I need to handle multiple domains using CNAME's to go to a single DNS zone.

That the PDNS provider behaves this way is also not very well indicated by documentation. If someone has only experienced, for example, the cloudflare provider, this behaviour could easily lead to accidentally wiping an entire zone's TXT records.

Would the alternative recommendation then be to use the HTTP Request provider or exec provider with custom code to handle the DNS updates?

[ldez]

I recommend using the rfc2136 provider, from my memory PowerDNS follows the RFC.

[t-schuster]

Alright I will take a look at that provider instead. I would still think it could be useful if Lego noted which providers work in distributed scenarios, simply to avoid users being confused about failed certificate challenges.

But thanks anyway!