Skip to content

Commit cec4045

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #11 from npintaux/np-coredns-mc-plugin
Added instructions to integrate the multicluster plugin in coredns
2 parents 32c49f3 + 0cf2f66 commit cec4045

File tree

1 file changed

+301
-0
lines changed

1 file changed

+301
-0
lines changed

‎site-src/guides/coredns.md

Lines changed: 301 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,301 @@
1+
# Configuring CoreDNS with the multicluster option
2+
3+
There is a ready made [CoreDNS multicluster plugin](https://coredns.io/explugins/multicluster/) you can use as the DNS component against MCS API implementations using EndpointSlices. By default, [CoreDNS](https://coredns.io/) does not include this plugin and it is therefore necessary to recompile CoreDNS to build a container image enabling this functionality.
4+
The following paragraphs highlight how one can re-deploy CoreDNS with the [multicluster plugin](https://coredns.io/explugins/multicluster/). To illustrate the concepts, it has been chosen to use a Kind cluster with its default configuration.
5+
6+
## Step 1: Deploying a Kind cluster and checking its CoreDNS configuration
7+
8+
Kind provides an easy way to deploy a small cluster locally on your computer. To do so, you only need to deploy the kind CLI, and type:
9+
10+
```
11+
kind create cluster --name cluster-1
12+
```
13+
14+
**Output (Do Not Copy)**
15+
```
16+
Creating cluster "cluster-1" ...
17+
✓ Ensuring node image (kindest/node:v1.25.3) 🖼
18+
✓ Preparing nodes 📦
19+
✓ Writing configuration 📜
20+
✓ Starting control-plane 🕹️
21+
✓ Installing CNI 🔌
22+
✓ Installing StorageClass 💾
23+
Set kubectl context to "kind-cluster-1"
24+
You can now use your cluster with:
25+
26+
kubectl cluster-info --context kind-cluster-1
27+
28+
Not sure what to do next? 😅 Check out https://kind.sigs.k8s.io/docs/user/quick-start/
29+
```
30+
31+
You can then verify that CoreDNS is indeed well deployed:
32+
33+
```
34+
kubectl get pods -n kube-system | grep coredns
35+
```
36+
**Output (Do Not Copy)**
37+
```
38+
coredns-565d847f94-brvlb 1/1 Running 0 5m20s
39+
coredns-565d847f94-zzdrd 1/1 Running 0 5m20s
40+
```
41+
42+
Before proceding with the re-compilation of CoreDNS, let's retrieve its current configuration by executing the command `coredns -plugins` in one of those pods.
43+
44+
```
45+
kubectl exec -ti coredns-565d847f94-brvlb -n kube-system -- /coredns -plugins
46+
```
47+
**Output (Do Not Copy)**
48+
```
49+
Server types:
50+
dns
51+
52+
Caddyfile loaders:
53+
flag
54+
default
55+
56+
Other plugins:
57+
dns.acl
58+
dns.any
59+
dns.auto
60+
dns.autopath
61+
dns.azure
62+
dns.bind
63+
dns.bufsize
64+
dns.cache
65+
dns.cancel
66+
dns.chaos
67+
dns.clouddns
68+
dns.debug
69+
dns.dns64
70+
dns.dnssec
71+
dns.dnstap
72+
dns.erratic
73+
dns.errors
74+
dns.etcd
75+
dns.file
76+
dns.forward
77+
dns.geoip
78+
dns.grpc
79+
dns.header
80+
dns.health
81+
dns.hosts
82+
dns.k8s_external
83+
dns.kubernetes
84+
dns.loadbalance
85+
dns.local
86+
dns.log
87+
dns.loop
88+
dns.metadata
89+
dns.minimal
90+
dns.nsid
91+
dns.pprof
92+
dns.prometheus
93+
dns.ready
94+
dns.reload
95+
dns.rewrite
96+
dns.root
97+
dns.route53
98+
dns.secondary
99+
dns.sign
100+
dns.template
101+
dns.tls
102+
dns.trace
103+
dns.transfer
104+
dns.whoami
105+
on
106+
```
107+
108+
This list is important as we will use it and just add the multicluster plugin to recompile CoreDNS.
109+
110+
## Re-compiling CoreDNS
111+
The easiest way to recompile CoreDNS and generate a new container image is to use the instructions on the [official CoreDNS source repository](https://github.com/coredns/coredns#compilation-from-source). We specifically recommend the compilation with Docker which includes a ready-to-use Go environment.
112+
113+
Prior to triggering the compilation, it is necessary to modify the `plugin.cfg` file. First, it is highly recommended to keep the same plugins as the ones found in the previous paragraph.
114+
Second, it is necessary to add the following instruction right **after** the `kubernetes:kubernetes` line:
115+
```
116+
...
117+
kubernetes:kubernetes
118+
multicluster:github.com/coredns/multicluster
119+
...
120+
```
121+
Then trigger the recompilation:
122+
123+
```
124+
$ docker run --rm -i -t -v $PWD:/v -w /v golang:1.18 make
125+
```
126+
127+
Once you have recompiled CoreDNS with the multicluster plugin, you can then build the new container image.
128+
129+
```
130+
sudo docker buildx build --platform linux/amd64 . -t [your-image-registry-path]:with-mcs-plugin
131+
```
132+
133+
Then push the image to your registry.
134+
135+
```
136+
docker push [your-image-registry-path]:with-mcs-plugin
137+
```
138+
139+
## Deploying the multicluster-enabled CoreDNS image
140+
The new CoreDNS with multicluster plugin enabled is now ready to be deployed. However, this new component will require some extra RBAC roles in order to query the Kubernetes API to discover ServiceImport objects, and a modification of its configuration file (Corefile) stored in the `coredns` ConfigMap object.
141+
142+
### Deploy the Multicluster-related CRDs
143+
144+
```
145+
kubectl apply -f https://github.com/kubernetes-sigs/mcs-api/blob/master/config/crd/multicluster.x-k8s.io_serviceexports.yaml
146+
kubectl apply -f https://github.com/kubernetes-sigs/mcs-api/blob/master/config/crd/multicluster.x-k8s.io_serviceimports.yaml
147+
```
148+
149+
### Setting up RBAC roles for CoreDNS
150+
Create a new ClusterRole and bind it to the `coredns` Service Account.
151+
152+
```
153+
cat <<EOF > coredns-multicluster-rbac.yaml
154+
apiVersion: rbac.authorization.k8s.io/v1
155+
kind: ClusterRole
156+
metadata:
157+
name: system:coredns-multicluster
158+
rules:
159+
- apiGroups:
160+
- "multicluster.x-k8s.io"
161+
resources:
162+
- serviceimports
163+
verbs: ["*"]
164+
---
165+
apiVersion: rbac.authorization.k8s.io/v1
166+
kind: ClusterRoleBinding
167+
metadata:
168+
name: system:coredns-multicluster
169+
roleRef:
170+
apiGroup: rbac.authorization.k8s.io
171+
kind: ClusterRole
172+
name: system:coredns-multicluster
173+
subjects:
174+
- kind: ServiceAccount
175+
name: coredns
176+
namespace: kube-system
177+
EOF
178+
```
179+
then apply the manifest.
180+
181+
```
182+
kubectl apply -f ./coredns-multicluster-rbac.yaml
183+
```
184+
185+
### Update the ConfigMap with the CoreDNS Corefile
186+
The CoreDNS Corefile includes the configuration of CoreDNS running on the cluster.
187+
188+
You need to edit the ConfigMap to add the `multicluster clusterset.local` line before the `kubernetes cluster.local...` line.
189+
190+
```
191+
...
192+
multicluster clusterset.local
193+
kubernetes cluster.local in-addr.arpa ip6.arpa {
194+
pods insecure
195+
fallthrough in-addr.arpa ip6.arpa
196+
ttl 30
197+
}
198+
...
199+
```
200+
201+
then perform a `rollout restart` of the `coredns` deployment so that CoreDNS can take this new configuration into account.
202+
203+
```
204+
kubectl rollout restart deploy coredns -n kube-system
205+
```
206+
207+
### Re-deploy CoreDNS
208+
Save the current configuration of the CoreDNS deployment, and delete it from the cluster.
209+
210+
```
211+
kubectl get deploy coredns -n kube-system -o yaml > coredns-deploy.yaml
212+
kubectl delete deploy coredns
213+
```
214+
215+
Replace the image path in the deployment manifest with the one you previously pushed to your image registry, and re-deploy the manifest. The new pods will restart with the new configuration stored in the Config Map previously modified.
216+
217+
### Verify that CoreDNS is healthy
218+
219+
```
220+
kubectl get pods -n kube-system
221+
```
222+
223+
**Output (Do Not Copy)**
224+
```
225+
kubectl get pods -n kube-system
226+
NAME READY STATUS RESTARTS AGE
227+
coredns-55fcfcb54f-lg25c 1/1 Running 0 25s
228+
coredns-55fcfcb54f-ts9sp 1/1 Running 0 25s
229+
etcd-cluster-1-control-plane 1/1 Running 0 38m
230+
kindnet-wd847 1/1 Running 0 38m
231+
kube-apiserver-cluster-1-control-plane 1/1 Running 0 39m
232+
kube-controller-manager-cluster-1-control-plane 1/1 Running 0 38m
233+
kube-proxy-n9ds4 1/1 Running 0 38m
234+
kube-scheduler-cluster-1-control-plane 1/1 Running 0 39m
235+
```
236+
237+
## More steps when you want to check that the multicluster plugin works
238+
239+
Create a demo namespace and deploy a fake ServiceImport.
240+
241+
```
242+
cat <<EOF > demo-service-import.yaml
243+
apiVersion: multicluster.x-k8s.io/v1alpha1
244+
kind: ServiceImport
245+
metadata:
246+
name: myservice
247+
namespace: demo
248+
spec:
249+
type: ClusterSetIP
250+
ips:
251+
- 1.2.3.4
252+
ports:
253+
- port: 80
254+
protocol: TCP
255+
EOF
256+
```
257+
```
258+
kubectl apply -f demo-service-import.yaml
259+
```
260+
261+
Then deploy a `dnsutils` pod in the demo namespace.
262+
263+
```
264+
cat <<EOF > dnsutils.yaml
265+
apiVersion: v1
266+
kind: Pod
267+
metadata:
268+
name: dnsutils
269+
namespace: demo
270+
spec:
271+
containers:
272+
- name: dnsutils
273+
image: k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.3
274+
command:
275+
- sleep
276+
- "3600"
277+
imagePullPolicy: IfNotPresent
278+
restartPolicy: Always
279+
EOF
280+
```
281+
282+
```
283+
kubectl apply -f dnsutils.yaml
284+
```
285+
286+
You can then use the `dnsutils` pod to confirm that the DNS query for the ServiceImport responds with the IP set in the fake ServiceImport previously defined.
287+
288+
```
289+
kubectl exec -it dnsutils -n demo -- bash
290+
```
291+
```
292+
root@dnsutils:/# nslookup myservice.demo.svc.clusterset.local
293+
```
294+
**Output (Do Not Copy)**
295+
```
296+
Server: 10.96.0.10
297+
Address: 10.96.0.10#53
298+
299+
Name: myservice.demo.svc.clusterset.local
300+
Address: 1.2.3.4
301+
```

0 commit comments

Comments
 (0)