Skip to content

fix: Use FQDN for backend service to avoid DNS search domain issues#2487

Merged
ldming merged 1 commit intoapecloud:mainfrom
jelipo:fix/alpine-dns-search
Mar 9, 2026
Merged

fix: Use FQDN for backend service to avoid DNS search domain issues#2487
ldming merged 1 commit intoapecloud:mainfrom
jelipo:fix/alpine-dns-search

Conversation

@jelipo
Copy link
Contributor

@jelipo jelipo commented Mar 6, 2026

Problem

Loki init containers fail to resolve Kubernetes service DNS on Alpine Linux (musl libc):

Symptoms

Waiting for index gateway ring to be ready...
Backend service: logs-775b7c67d9-backend.kb-system.svc.cluster.local:3100
Max wait time: 300 seconds
Waiting for index gateway ring... (0/300 seconds)
Waiting for index gateway ring... (5/300 seconds)
Waiting for index gateway ring... (10/300 seconds)
Waiting for index gateway ring... (15/300 seconds)
Waiting for index gateway ring... (20/300 seconds)
Waiting for index gateway ring... (25/300 seconds)
Waiting for index gateway ring... (30/300 seconds)
Waiting for index gateway ring... (35/300 seconds)
Waiting for index gateway ring... (40/300 seconds)
me@k8s-master:~$ kubectl exec -it   logs-775b7c67d9-read-0 -n kb-system -c wait-index-gateway -- sh
/ # curl logs-775b7c67d9-backend.kb-system.svc.cluster.local:3100/ready
curl: (6) Could not resolve host: logs-775b7c67d9-backend.kb-system.svc.cluster.local

Reproduction

Tested on two different Kubernetes clusters with the same issue:

  1. k3d cluster (v1.31.5+k3s1)
  2. kubeadm cluster (v1.35.2)
    Both clusters exhibited the same DNS resolution failure in Alpine-based init containers.

Solution

Append a trailing dot (.) to service DNS names to use Fully Qualified Domain Names (FQDNs):

me@k8s-master:~$ kubectl exec -it   logs-775b7c67d9-read-0 -n kb-system -c wait-index-gateway -- sh
/ # curl logs-775b7c67d9-backend.kb-system.svc.cluster.local:3100/ready
curl: (6) Could not resolve host: logs-775b7c67d9-backend.kb-system.svc.cluster.local
/ # curl logs-775b7c67d9-backend.kb-system.svc.cluster.local.:3100/ready
ready

@jelipo jelipo requested review from a team as code owners March 6, 2026 08:14
@apecloud-bot apecloud-bot added the chart-release Auto release chart when PR merged main. label Mar 6, 2026
@ldming ldming requested a review from heng4fun March 9, 2026 05:45
@ldming ldming added pick-1.0 Auto cherry-pick to release-1.0 when PR merged pick-1.1 Auto cherry-pick to release-1.1 when PR merged labels Mar 9, 2026
@apecloud-bot apecloud-bot added chart-release-1.0 Auto release chart when PR merged release-1.0. chart-release-1.1 Auto release chart when PR merged release-1.1. labels Mar 9, 2026
Copy link
Contributor

@heng4fun heng4fun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ldming ldming merged commit f5fd266 into apecloud:main Mar 9, 2026
22 of 32 checks passed
@apecloud-bot
Copy link
Collaborator

/cherry-pick release-1.1

@apecloud-bot
Copy link
Collaborator

/cherry-pick release-1.0

@apecloud-bot
Copy link
Collaborator

🤖 says: cherry pick action finished successfully 🎉!
See: https://github.com/apecloud/kubeblocks-addons/actions/runs/22852603669

apecloud-bot pushed a commit that referenced this pull request Mar 9, 2026
@apecloud-bot
Copy link
Collaborator

🤖 says: cherry pick action finished successfully 🎉!
See: https://github.com/apecloud/kubeblocks-addons/actions/runs/22852602548

apecloud-bot pushed a commit that referenced this pull request Mar 9, 2026
@jelipo jelipo deleted the fix/alpine-dns-search branch March 9, 2026 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chart-release Auto release chart when PR merged main. chart-release-1.0 Auto release chart when PR merged release-1.0. chart-release-1.1 Auto release chart when PR merged release-1.1. pick-1.0 Auto cherry-pick to release-1.0 when PR merged pick-1.1 Auto cherry-pick to release-1.1 when PR merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants