"RFC2136 update failed: bad return code: NOTAUTH

[survivant]

I need help to complete my BIND9 with Kubernetes setup. I'm on premise. I want to add a domain name for my cluster like

dev.clusterxxx.local
qa.clusterxxx.local

I try to follow that guide : https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/rfc2136.md

here my named.conf.local file

root@test-pcl4014:/etc/bind# cat named.conf.local
zone    "cluster114.local"   {
        type master;
        file    "/etc/bind/forward.cluster114.local";
 };

zone   "0.1.10.in-addr.arpa"        {
       type master;
       file    "/etc/bind/reverse.cluster114.local";
 };

zone    "cluster111.local"   {
        type master;
        file    "/etc/bind/forward.cluster111.local";
 };

zone "k8s.clusterxxx.local" {
    type master;
    file "/etc/bind/zone.clusterxxx.local";
};

my file zone.clusterxxx.local

root@test-pcl4014:/etc/bind# cat zone.clusterxxx.local
$TTL    60 ; 1 minute

@       IN      SOA     primary.clusterxxx.local. root.primary.clusterxxx.local. (
                             16         ; Serial
                             60         ; Refresh
                             60         ; Retry
                             60         ; Expire
                             60 )       ; Negative Cache TTL

;Name Server Information
@       IN      NS      primary.clusterxxx.local.

;IP address of Your Domain Name Server(DNS)
primary IN       A      10.1.34.14
ns      IN       A       10.1.34.55
root@test-pcl4014:/etc/bind#

I have my ingress-controller here (10.1.34.55)

root@test-pcl4014:/etc/bind# kubectl -n nginx-ingress get svc
NAME                                 TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.99.95.158    10.1.34.55    80:31224/TCP,443:31751/TCP   10d
ingress-nginx-controller-admission   ClusterIP      10.96.133.146   <none>        443/TCP                      10d
root@test-pcl4014:/etc/bind#

I installed external-dns using Helm

My Bind server is on the IP 10.1.34.14

helm install my-release bitnami/external-dns --set provider=rfc2136 --set rfc2136.host="10.1.34.14" --set rfc2136.zone="clusterxxx.local" --set rfc2136.tsigKeyname=""

In my journal logs, I found this

client @0x7ffb20007640 192.168.178.115#51682: update failed: not authoritative for update zone (NOTAUTH)

Here the logs of external-dns pod

root@test-pcl4014:/etc/bind# kubectl logs my-release-external-dns-85dcd4dfdd-8j4xt --follow
time="2021-07-01T19:21:10Z" level=info msg="config: {APIServerURL: KubeConfig: RequestTimeout:30s ContourLoadBalancerService:heptio-contour/contour GlooNamespace:gloo-system SkipperRouteGroupVersion:zalando.org/v1 Sources:[service ingress] Namespace: AnnotationFilter: LabelFilter: FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false IgnoreIngressTLSSpec:false Compatibility: PublishInternal:false PublishHostIP:false AlwaysPublishNotReadyAddresses:false ConnectorSourceServer:localhost:8080 Provider:rfc2136 GoogleProject: GoogleBatchChangeSize:1000 GoogleBatchChangeInterval:1s DomainFilter:[] ExcludeDomains:[] RegexDomainFilter: RegexDomainExclusion: ZoneNameFilter:[] ZoneIDFilter:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType: AWSZoneTagFilter:[] AWSAssumeRole: AWSBatchChangeSize:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AWSZoneCacheDuration:0s AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: AzureSubscriptionID: AzureUserAssignedIdentityClientID: BluecatConfigFile:/etc/kubernetes/bluecat.json CloudflareProxied:false CloudflareZonesPerPage:50 CoreDNSPrefix:/skydns/ RcodezeroTXTEncrypt:false AkamaiServiceConsumerDomain: AkamaiClientToken: AkamaiClientSecret: AkamaiAccessToken: AkamaiEdgercPath: AkamaiEdgercSection: InfobloxGridHost: InfobloxWapiPort:443 InfobloxWapiUsername:admin InfobloxWapiPassword: InfobloxWapiVersion:2.3.1 InfobloxSSLVerify:true InfobloxView: InfobloxMaxResults:0 DynCustomerName: DynUsername: DynPassword: DynMinTTLSeconds:0 OCIConfigFile:/etc/kubernetes/oci.yaml InMemoryZones:[] OVHEndpoint:ovh-eu OVHApiRateLimit:20 PDNSServer:http://localhost:8081 PDNSAPIKey: PDNSTLSEnabled:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:upsert-only Registry:txt TXTOwnerID:default TXTPrefix: TXTSuffix: Interval:1m0s MinEventSyncInterval:5s Once:false DryRun:false UpdateEvents:false LogFormat:text MetricsAddress::7979 LogLevel:info TXTCacheInterval:0s TXTWildcardReplacement: ExoscaleEndpoint:https://api.exoscale.ch/dns ExoscaleAPIKey: ExoscaleAPISecret: CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: RFC2136Host:10.1.34.14 RFC2136Port:53 RFC2136Zone:clusterxxx.local RFC2136Insecure:true RFC2136GSSTSIG:false RFC2136KerberosRealm: RFC2136KerberosUsername: RFC2136KerberosPassword: RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false RFC2136MinTTL:0s NS1Endpoint: NS1IgnoreSSL:false NS1MinTTLSeconds:0 TransIPAccountName: TransIPPrivateKeyFile: DigitalOceanAPIPageSize:50 ManagedDNSRecordTypes:[A CNAME] GoDaddyAPIKey: GoDaddySecretKey: GoDaddyTTL:0 GoDaddyOTE:false}"
time="2021-07-01T19:21:10Z" level=info msg="Instantiating new Kubernetes client"
time="2021-07-01T19:21:10Z" level=info msg="Using inCluster-config based on serviceaccount-token"
time="2021-07-01T19:21:10Z" level=info msg="Created Kubernetes client https://10.96.0.1:443"
time="2021-07-01T19:21:12Z" level=info msg="Configured RFC2136 with zone 'clusterxxx.local.' and nameserver '10.1.34.14:53'"
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: ingress.clusterxxx.local 0 A 10.1.34.55"
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: ingress.k8s.clusterxxx.local 0 A 10.1.34.55"
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: qa.cluster114.local 0 A 10.1.34.55"
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: qa.cluster111.local 0 A 10.1.34.55"
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: qa.ks8-ns1.cluster111.local 0 A 10.1.34.55"
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: ks8-ns1.cluster111.local 0 A 10.1.34.55"
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: svc.clusterxxx.local 0 A 10.1.34.57"
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: dev.cluster114.local 0 A 10.1.34.55"
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: ingress.clusterxxx.local 0 TXT \"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/default/my-ingress\""
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: ingress.k8s.clusterxxx.local 0 TXT \"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/default/my-ingress\""
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: qa.cluster114.local 0 TXT \"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/qa/crc\""
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: qa.cluster111.local 0 TXT \"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/qa/production-wui\""
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: qa.ks8-ns1.cluster111.local 0 TXT \"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/qa/production-wui\""
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: ks8-ns1.cluster111.local 0 TXT \"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/qa/production-wui\""
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: svc.clusterxxx.local 0 TXT \"heritage=external-dns,external-dns/owner=default,external-dns/resource=service/default/nginx\""
time="2021-07-01T19:21:17Z" level=info msg="Adding RR: dev.cluster114.local 0 TXT \"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/dev/vault\""
time="2021-07-01T19:21:17Z" level=info msg="Bad dns.Client.Exchange response: ;; opcode: UPDATE, status: NOTAUTH, id: 34836\n;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0\n\n;; QUESTION SECTION:\n;clusterxxx.local.\tIN\t SOA\n"
time="2021-07-01T19:21:17Z" level=error msg="RFC2136 update failed: bad return code: NOTAUTH"

[survivant]

I started from scratch on a vagrant VM. I reinstalled Bind9. My node VM is : 192.68.50.180. I have a loadbalancer (metallb) 192.68.50.200

Here my file named.conf.local

vagrant@enroute-master:/etc/bind$ cat named.conf.local
key "externaldns-key" {
        algorithm hmac-sha256;
        secret "JTbGJiqcd3habjGcLbJEHGBt1KQlZfZasBvteM1IlnU=";
};

zone "example.com" {
      type master;
      file "/etc/bind/db.example.com";
      allow-transfer {
        key "externaldns-key";
      };
      update-policy {
        grant externaldns-key subdomain example.com. ANY;
      };
};

vagrant@enroute-master:/etc/bind$ cat named.conf.options
acl "trusted" {
        127.0.0.1;
        192.168.50.180;
        192.168.50.200;
};

options {
        directory "/var/cache/bind";
        dnssec-validation auto;

        listen-on { any; };
        listen-on port 53 { any; };
        allow-transfer { any; };

};
vagrant@enroute-master:/etc/bind$

vagrant@enroute-master:/etc/bind$ cat db.example.com
;
; BIND data file for local loopback interface
;
$TTL    604800
@       IN      SOA     ns.example.com. root.example.com. (
                              1         ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                         604800 )       ; Negative Cache TTL
;
@       IN      NS      ns.example.com.
ns      IN      A       192.168.50.200

vagrant@enroute-master:/etc/bind$

My ingress hosts are : dev.ns.example.com,dev.example.com

My external-dns pod's logs are

vagrant@enroute-master:/etc/bind$ kubectl logs my-release-external-dns-586d8df54f-wl5fp
time="2021-07-02T18:00:42Z" level=info msg="config: {APIServerURL: KubeConfig: RequestTimeout:30s ContourLoadBalancerService:heptio-contour/contour GlooNamespace:gloo-system SkipperRouteGroupVersion:zalando.org/v1 Sources:[service ingress] Namespace: AnnotationFilter: LabelFilter: FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false IgnoreIngressTLSSpec:false Compatibility: PublishInternal:false PublishHostIP:false AlwaysPublishNotReadyAddresses:false ConnectorSourceServer:localhost:8080 Provider:rfc2136 GoogleProject: GoogleBatchChangeSize:1000 GoogleBatchChangeInterval:1s DomainFilter:[] ExcludeDomains:[] RegexDomainFilter: RegexDomainExclusion: ZoneNameFilter:[] ZoneIDFilter:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType: AWSZoneTagFilter:[] AWSAssumeRole: AWSBatchChangeSize:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AWSZoneCacheDuration:0s AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: AzureSubscriptionID: AzureUserAssignedIdentityClientID: BluecatConfigFile:/etc/kubernetes/bluecat.json CloudflareProxied:false CloudflareZonesPerPage:50 CoreDNSPrefix:/skydns/ RcodezeroTXTEncrypt:false AkamaiServiceConsumerDomain: AkamaiClientToken: AkamaiClientSecret: AkamaiAccessToken: AkamaiEdgercPath: AkamaiEdgercSection: InfobloxGridHost: InfobloxWapiPort:443 InfobloxWapiUsername:admin InfobloxWapiPassword: InfobloxWapiVersion:2.3.1 InfobloxSSLVerify:true InfobloxView: InfobloxMaxResults:0 DynCustomerName: DynUsername: DynPassword: DynMinTTLSeconds:0 OCIConfigFile:/etc/kubernetes/oci.yaml InMemoryZones:[] OVHEndpoint:ovh-eu OVHApiRateLimit:20 PDNSServer:http://localhost:8081 PDNSAPIKey: PDNSTLSEnabled:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:upsert-only Registry:txt TXTOwnerID:default TXTPrefix: TXTSuffix: Interval:1m0s MinEventSyncInterval:5s Once:false DryRun:false UpdateEvents:false LogFormat:text MetricsAddress::7979 LogLevel:info TXTCacheInterval:0s TXTWildcardReplacement: ExoscaleEndpoint:https://api.exoscale.ch/dns ExoscaleAPIKey: ExoscaleAPISecret: CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: RFC2136Host:192.168.50.180 RFC2136Port:53 RFC2136Zone:example.com RFC2136Insecure:false RFC2136GSSTSIG:false RFC2136KerberosRealm: RFC2136KerberosUsername: RFC2136KerberosPassword: RFC2136TSIGKeyName:externaldns-key RFC2136TSIGSecret: RFC2136TSIGSecretAlg:hmac-sha256 RFC2136TAXFR:false RFC2136MinTTL:0s NS1Endpoint: NS1IgnoreSSL:false NS1MinTTLSeconds:0 TransIPAccountName: TransIPPrivateKeyFile: DigitalOceanAPIPageSize:50 ManagedDNSRecordTypes:[A CNAME] GoDaddyAPIKey: GoDaddySecretKey: GoDaddyTTL:0 GoDaddyOTE:false}"
time="2021-07-02T18:00:42Z" level=info msg="Instantiating new Kubernetes client"
time="2021-07-02T18:00:42Z" level=info msg="Using inCluster-config based on serviceaccount-token"
time="2021-07-02T18:00:42Z" level=info msg="Created Kubernetes client https://10.96.0.1:443"
time="2021-07-02T18:00:44Z" level=info msg="Configured RFC2136 with zone 'example.com.' and nameserver '192.168.50.180:53'"
vagrant@enroute-master:/etc/bind$ kubectl

the only entry that is reaching my ingress-controler is ns.example.com, look like external-dns is not able to update my BIND server

vagrant@enroute-master:/etc/bind$ curl http://dev.ns.example.com/chuck
curl: (6) Could not resolve host: dev.ns.example.com
vagrant@enroute-master:/etc/bind$ curl http://dev.example.com/chuck
curl: (6) Could not resolve host: dev.example.com
vagrant@enroute-master:/etc/bind$ curl http://ns.example.com
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
vagrant@enroute-master:/etc/bind$

I installed external-dns with this :

helm install my-release bitnami/external-dns --set provider=rfc2136 --set rfc2136.host="192.168.50.180" --set rfc2136.zone="example.com" --set rfc2136.tsigAxfr=false

if I dont put the param : tsigAxfr, I'll obtain this error in the logs

time="2021-07-02T18:16:35Z" level=error msg="AXFR error: dns: bad authentication"

and in bind9 logs I have this :

02-Jul-2021 18:16:35.833 security: error: client @0x7f58cc021750 192.168.123.73#58048: request has invalid signature: TSIG externaldns-key: tsig verify failure (BADSIG)
02-Jul-2021 18:17:36.833 security: error: client @0x7f58c802bf10 192.168.123.73#58162: request has invalid signature: TSIG externaldns-key: tsig verify failure (BADSIG)

[k0da]

@survivant I'm running bind in k8s in my test setup and use external-dns to create NS records. See example manifests here: https://github.com/k0da/k8gb/tree/edge/deploy/edge