Error determining zone identifier - DNS Common Lexicon Error

[dblanque]

Hey, I seem to be getting a DNS Lexicon error when renewing a sub-domain for our LDAP Server that didn't happen before. Possibly due to a Python or PDNS Version Change?

PowerDNS Version: 4.8.3
Python Version: 3.11

requirements.txt

acme==2.7.1
beautifulsoup4==4.12.2
certbot==2.7.1
certbot-dns-powerdns==0.2.1
certifi==2023.7.22
cffi==1.16.0
charset-normalizer==3.3.0
ConfigArgParse==1.7
configobj==5.0.8
cryptography==3.4.8
distro==1.8.0
dns-lexicon==3.5.6
dnspython==2.4.2
filelock==3.12.4
future==0.18.3
idna==3.4
josepy==1.13.0
mock==5.1.0
parsedatetime==2.6
pycparser==2.21
pyOpenSSL==21.0.0
pyRFC3339==1.1
pytz==2023.3.post1
PyYAML==5.3.1
requests==2.31.0
requests-file==1.5.1
six==1.16.0
soupsieve==2.5
tldextract==3.5.0
urllib3==2.0.6
watchdog==3.0.0
zope.interface==6.1

If there is any further info I can give let me know.

Error Dump with Debug:

Exiting abnormally:
Traceback (most recent call last):
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/plugins/dns_common_lexicon.py", line 108, in _find_domain_id
    self.provider.authenticate()
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/lexicon/providers/base.py", line 74, in authenticate
    return self._authenticate()
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/lexicon/providers/powerdns.py", line 108, in _authenticate
    self.zone_data()
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/lexicon/providers/powerdns.py", line 102, in zone_data
    self._zone_data = self._get(
                      ^^^^^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/lexicon/providers/base.py", line 159, in _get
    return self._request("GET", url, query_params=query_params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/lexicon/providers/powerdns.py", line 263, in _request
    response.raise_for_status()
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: NOT FOUND for url: https://ct-pdns-1.brconsulting.info/api/v1/servers/localhost/zones/vm1001-ldap.brconsulting.info.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/ispc-pdns-certbot/shared/../bin/certbot", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1873, in main
    return config.func(config, plugins)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1600, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 131, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/_internal/renewal.py", line 396, in renew_cert
    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/_internal/client.py", line 428, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/_internal/client.py", line 496, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/_internal/auth_handler.py", line 88, in handle_authorizations
    resps = self.auth.perform(achalls)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/plugins/dns_common.py", line 76, in perform
    self._perform(domain, validation_domain_name, validation)
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot_dns_powerdns/dns_powerdns.py", line 54, in _perform
    self._get_powerdns_client().add_txt_record(
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/plugins/dns_common_lexicon.py", line 60, in add_txt_record
    self._find_domain_id(domain)
  File "/opt/ispc-pdns-certbot/lib/python3.11/site-packages/certbot/plugins/dns_common_lexicon.py", line 115, in _find_domain_id
    raise result1
certbot.errors.PluginError: Error determining zone identifier for vm1001-ldap.brconsulting.info: 404 Client Error: NOT FOUND for url: https://ct-pdns-1.brconsulting.info/api/v1/servers/localhost/zones/vm1001-ldap.brconsulting.info..

[dblanque]

Fixed it manually on my lib/python3.11/site-packages/certbot_dns_powerdns/dns_powerdns.py by adding the following code to the _handle_http_error function.

# 4.8.3
str(e).startswith('404 Client Error: NOT FOUND for url:')

    def _handle_http_error(self, e, domain_name):
        if domain_name in str(e) and (
            # 4.0 and 4.1 compatibility
            str(e).startswith('422 Client Error: Unprocessable Entity for url:') or
            # 4.2
            str(e).startswith('404 Client Error: Not Found for url:') or
            # 4.8.3
            str(e).startswith('404 Client Error: NOT FOUND for url:')
            ):
            return  # Expected errors when zone name guess is wrong
        return super(_PowerDNSLexiconClient, self)._handle_http_error(e, domain_name)

[dtryba]

After upgrading the machine that runs certbot from older versions to latest (Debian/10 with pip versions I guess about 2 or 3 years ago) to currect on Debian/12, I was experiencing comparable issues.

requests.exceptions.HTTPError: 422 Client Error: Unknown Status for url: https://ns1.xxx.nl/api/v1/servers/localhost/zones/hostname.domain.nl.

The version of powerdns wasn't affected (4.0.3) with the mentioned upgrade. Seeing the patch for pdns 4.8.3 I tried to add the error code to certbot_dns_powerdns/dns_powerdns.py and that fixed my issue. So maybe this can be amended for those few that still tun ancient pdns versions and use certbot with the pdns authenticator?

+++ /usr/local/lib/python3.11/dist-packages/certbot_dns_powerdns/dns_powerdns.py        2023-11-22 13:47:26.940731065 +0100
@@ -88,7 +88,9 @@
             # 4.0 and 4.1 compatibility
             str(e).startswith('422 Client Error: Unprocessable Entity for url:') or
             # 4.2
-            str(e).startswith('404 Client Error: Not Found for url:')
+            str(e).startswith('404 Client Error: Not Found for url:') or
+            # 4.0
+            str(e).startswith('422 Client Error: Unknown Status for url')
             ):
             return  # Expected errors when zone name guess is wrong
         return super(_PowerDNSLexiconClient, self)._handle_http_error(e, domain_name)

[jmouette]

I’ve encountered the same error by using a custom authentication proxy for the PowerDNS API: I discovered the client fails depending on the exact, case-sensitive text of the HTTP status line.
This is definitely a non-conformity to the HTTP standard: https://www.rfc-editor.org/rfc/rfc2616#section-6.1 specifies you can change the status text.

Parsing exceptions is a Bad Idea™. Please find some other way to get the HTTP status code. At the very least, please make this case-insensitive so that it works with default status lines from other web servers (capitalized status lines are hardcoded in Werkzeug, for example).
Something like:

str(e).lower().startswith('404 Client Error: Not Found for url:'.lower())