DNS Resolution Fails with .cluster.local on macOS Due to Local Network Permission Requirements

[tmoschou]

What Happened?

Summary

On macOS, DNS resolution for services using the default .cluster.local domain is unreliable when accessed from the host, due to:

• Apple/MacOS reserving .local TLD for Multicast DNS (mDNS) and link-local networking
• Every application (browser, terminal, IDE, etc.) requiring "Local Network" permission in System Settings

This creates a poor user experience as DNS resolution "sometimes" works depending on which application is making the request and whether it has been granted the necessary permissions.

Using --dns-domain=cluster.internal works reliably as a workaround (.internal is an ICANN-reserved TLD for private use) without going through MacOS's Multicast DNS (mDNS) resolver and without requiring special permissions.

Environment

• macOS Version: macOS Tahoe 26.2
• minikube Version: v1.38.0
• Driver: vfkit
• Network: vmnet-shared (via homebrew)
• Device enrolled in MDM (Mobile Device Management)

Steps to Reproduce

1. Start minikube with default DNS domain:
   minikube start --driver=vfkit --network=vmnet-shared

2. Create a test service:

kubectl create deployment hello-minikube --image=kicbase/echo-server:1.0
kubectl expose deployment hello-minikube --type=LoadBalancer --port=8080

3. Start the tunnel: sudo minikube tunnel

4. Verify DNS resolver is configured:

cat /etc/resolver/cluster.local
# Output:
# nameserver 10.96.0.10
# search_order 1

# Or check via scutil
scutil --dns

5. Verify IP route is configured:
   netstat -nr | grep '^10.96'

6. Verify can service by IP address
   open "http://$(kubectl get svc/hello-minikube -o go-template='{{ .spec.clusterIP }}'):8080/"

Request served by hello-minikube-58f7c595dd-zxb7f

HTTP/1.1 GET /

Host: 10.108.100.65:8080
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36

7. Attempt to access the service by DNS address: open http://hello-minikube.default.svc.cluster.local:8080/

Actual Behavior

• DNS is unreliable,
• Success depends on whether each application (Browser/IDE/Terminal/etc.) has been granted "Local Network" permission in:
  • macOS Settings → Privacy & Security → Local Network
• Applications without this permission cannot resolve .cluster.local domains
• Users experience intermittent failures with no clear indication of the cause

Expected Behavior

DNS resolution should work reliably for all applications without requiring manual permission grants.

Workaround

Use --dns-domain=cluster.internal to minikube start instead of relying on the kubernetes default .cluster.local

minikube delete
minikube start --driver=vfkit --network=vmnet-shared --dns-domain=cluster.internal

kubectl create deployment hello-minikube --image=kicbase/echo-server:1.0
kubectl expose deployment hello-minikube --type=LoadBalancer --port=8080

sudo minikube tunnel

# This now works reliably without Local Network permission
open http://hello-minikube.default.svc.cluster.internal:8080/

Root Cause

The .local TLD is reserved by the IETF in https://datatracker.ietf.org/doc/html/rfc6762 for Multicast DNS (mDNS) and link-local networking. According to the specification:

• Any fully qualified name ending in .local is considered link-local only (same network segment)
• DNS queries for .local domains must be sent to the mDNS IPv4 multicast address (224.0.0.251)
• The .local domain is intended for zero-configuration networking without traditional DNS infrastructure

Kubernetes' use of .cluster.local conflicts with this specification because:

1. macOS treats .cluster.local as falling under the .local TLD, expecting it to be resolved via mDNS/Bonjour
2. minikube's vmnet networking doesn't fit the link-local model - it uses unicast DNS resolution, not mDNS multicast
3. Apple explicitly warns against using .local in unicast DNS contexts, noting it can prevent proper DNS resolution and cause conflicts (https://support.apple.com/en-us/101903)

The .internal TLD, by contrast, is ICANN-reserved for private use and works reliably with traditional unicast DNS resolution, making it a better fit for Kubernetes internal networking.

Proposed Solution

Rather than changing minikube's default DNS domain (which would break compatibility with many tools and Helm charts that assume cluster.local), I propose simply updating documentation with note that users on MacOS may wish to consider configuring --dns-domain=cluster.internal to minikube start in the following sections

• https://minikube.sigs.k8s.io/docs/drivers/vfkit/
• https://minikube.sigs.k8s.io/docs/commands/tunnel/
• https://minikube.sigs.k8s.io/docs/handbook/accessing/#dns-resolution-experimental

Attach the log file

N/A

Operating System

macOS (Default)

Driver

vfkit

[nirs]

@tmoschou Thanks for opening this, awesome work!

I never used the minikube tunnel command, mainly because the documentation is very terse, unclear, and lacking examples.

Can you explain why you want to access a service in a cluster using:

http://servicen-name.defalt.src.cluster.local

We use NodePort service and access services as:

http://{cluster-ip}:{service-port}/

Maybe the tunnel command was needed for drivers that do not have a real IP address for the the VM, but with vfkit we have a real IP on the vmnet network. Why not use it to access the service, just like a real cluster?

Our main use case is adding minio s3 endpoint to the mc tool which is used for debugging by tools, so we don't care about a "nice" URL.

Also using sudo minikube tunnel is alarming - if minikube is install via brew or download binary in you home directory it is unsafe to run it as root.

Also running the tunnel command as root is blocker for automated tests, unless you add passwordless sudo configuration for this command.

Do yo know how the tunnel command fix the host DNS to work with cluster.internal names?

Finally, do you have such integration with real Kubernetes clusters? Being able to access a service in the real cluster on external host using the internal k8s DNS name?

[wt]

@tmoschou you said that the tld .internal. is reserved by rfc6761. I'm assuming you were actually referring to rfc6762 (as linked above that reference in your initial post. It appears that the text that "reserves" that tld though is only informational at best. The section is labelled as "Appendix G". However, I don't see any reference in the normative text to that appendix.

In fact, the errata for rfc6762, specifically addresses that section saying that it should be removed or at least edited to unambiguously not recommend use to any private TLDs.

I think it might be better to use a domain that is subordinate to a domain that you control. For example, if I control example.com, I could use users.corp.example.com to house all domains that are controlled by my corp users. Then I could allow users to create something like ANYTHING.{username}.users.corp.example.com. Then, I could use cluster.wt.users.corp.example.com as the dns domain for my minikube cluster.

FWIW, I have an outstanding change for minikube that allows one to set the default dns name for new clusters @ #21064.

[tmoschou]

Can you explain why you want to access a service in a cluster using:

Our main use-case for using minikube tunnel is it simplifies local development, configuration and workflows on the host.
The application is developed on the host machine (not as a dev-container), with dependent services (e.g. DB, Minio, etc) deployed in minikube. While developing, its handy to be be able deploy the same stack into isolated seperate namespaces concurrently, for when working on different branches / features / git worktrees - so as to keep the state seperate and swap between them.

E.g.

postgres.main.svc.cluster.internal:5432
postgres.branch-123.svc.cluster.internal:5432
postgres.feature-xyz.svc.cluster.internal:5432

By simply changing a single namespace variable, we can point our application to a different stack/state, without having to allocate and resequence NodePorts manually, or lookup dynamically creating much complexity.

E.g.

# local-dev-minikube Config Profile defaults
k8s:
  namespace: main 
  domain: cluster.internal

service1: https://service1.${k8s.namespace}.svc.${k8s.domain}:8080
service2: http://service2.${k8s.namespace}.svc.${k8s.domain}:8080

java -jar App.jar --config-profile local-dev-minikube --set k8s.namespace=feature-xyz

The Gateway API may help here using HTTPRoute alternatively, though doesn't work so well for non HTTP protocols (e.g. postgres), UDP etc, and is a bit heavy handed and unnecessary complexity, since this tunnelling is only for local development.

Also running the tunnel command as root is blocker for automated tests, unless you add passwordless sudo configuration for this command.

Our CI and deployment architecture doesn't use minikube or require any tunnelling - only for local development. App would live adjacent to services. I agree running sudo for minikube tunnel to setup IP route and create /etc/resolver/<domain> resolver is not ideal, but another issue.

Do yo know how the tunnel command fix the host DNS to work with cluster.internal names?

On MacOS, I believe its just create a resolver at /etc/resolver/<domain>, same as cluster.local

cat /etc/resolver/cluster.internal
nameserver 10.96.0.10
search_order 1

Finally, do you have such integration with real Kubernetes clusters? Being able to access a service in the real cluster on external host using the internal k8s DNS name?

No, would be going through Gateway (or Ingress) / Load Balancer, not tunnelling or accessing internal services via their cluster DNS.

@wt - Yep my mistake, RFC 6761 doesn't specifically mention .internal as Special-Use Domain Names. There was no RFC yet, but the ICANN board did reserve it in Resolution 2024.07.29.06

References:

• https://www.icann.org/en/board-activities-and-meetings/materials/approved-resolutions-special-meeting-of-the-icann-board-29-07-2024-en#section2.a
• https://en.wikipedia.org/wiki/.internal
• https://en.wikipedia.org/wiki/.local