Skip to content

Commit e96d0fb

Browse files
cameronsenesecameronsenese
authored
Update deployment samples and README.md (#127)
* Update deployment samples and README.md * modify updates to deployment samples and README.md * include example- config files in /samples * update README.md - standardise kubectl command format and minor grammar/wording changes
1 parent ce584c5 commit e96d0fb

File tree

4 files changed

+267
-16
lines changed

4 files changed

+267
-16
lines changed

‎README.md

Lines changed: 46 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -13,27 +13,56 @@
1313
[![Go Report Card](https://goreportcard.com/badge/github.com/aws/aws-cloud-map-mcs-controller-for-k8s)](https://goreportcard.com/report/github.com/aws/aws-cloud-map-mcs-controller-for-k8s)
1414

1515
## Introduction
16-
AWS Cloud Map multi-cluster service discovery for Kubernetes (K8s) is a controller that implements existing multi-cluster services API that allows services to communicate across multiple clusters. The implementation relies on [AWS Cloud Map](https://aws.amazon.com/cloud-map/) for enabling cross-cluster service discovery.
16+
The AWS Cloud Map Multi-cluster Service Discovery Controller for Kubernetes (K8s) implements the Kubernetes [multi-cluster services API](https://github.com/kubernetes/enhancements/tree/master/keps/sig-multicluster/1645-multi-cluster-services-api) specification, which allows services to communicate across multiple clusters. The implementation relies on [AWS Cloud Map](https://aws.amazon.com/cloud-map/) for enabling cross-cluster service discovery.
1717

1818
See the demo from AWS Container Day x KubeCon!
1919

2020
[![Watch the video](https://img.youtube.com/vi/3f0Tv7IiQQw/0.jpg)](https://youtu.be/3f0Tv7IiQQw?t=24458)
2121

22-
## Usage
23-
> ⚠ **There must exist network connectivity (i.e. VPC peering, security group rules, ACLs, etc.) between clusters**: Undefined behavior may occur if controller is set up without network connectivity between clusters.
22+
## Installation
23+
24+
Perform the following installation steps on each participating cluster.
25+
26+
- For multi-cluster service discovery and consumption, the controller should be installed on a minimum of 2 EKS clusters.
27+
- Participating clusters should be provisioned into a single AWS account, within a single AWS region.
28+
29+
### Dependencies
30+
31+
#### Network
32+
33+
> ⚠ **The AWS Cloud Map MCS Controller for K8s provides service discovery and communication across multiple clusters, therefore implementations depend on end-end network connectivity between workloads provisioned within each participating cluster.**
34+
35+
- In deployment scenarios where participating clusters are provisioned into separate VPCs, connectivity will depend on correctly configured [VPC Peering](https://docs.aws.amazon.com/vpc/latest/peering/create-vpc-peering-connection.html), [inter-VPC routing](https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-routing.html), and Security Group configuration. The [VPC Reachability Analyzer](https://docs.aws.amazon.com/vpc/latest/reachability/getting-started.html) can be used to test and validate end-end connectivity between worker nodes within each cluster.
36+
- Undefined behavior may occur if controllers are deployed without the required network connectivity between clusters.
37+
38+
#### Configure CoreDNS
2439

25-
### Setup clusters
40+
Install the The CoreDNS multicluster plugin into each participating cluster. The multicluster plugin enables CoreDNS to lifecycle manage DNS records for `ServiceImport` objects.
2641

27-
First, install the controller with latest release on at least 2 AWS EKS clusters. Nodes must have sufficient IAM permissions to perform CloudMap operations.
42+
To install the plugin, run the following commands.
2843

29-
> **_NOTE:_** AWS region environment variable can be _optionaly_ set like `export AWS_REGION=us-west-2` Otherwise controller will infer region in the order `AWS_REGION` environment variable, ~/.aws/config file, then EC2 metadata (for EKS environment)
44+
```bash
45+
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/samples/coredns-clusterrole.yaml
46+
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/samples/coredns-configmap.yaml
47+
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/samples/coredns-deployment.yaml
48+
```
49+
50+
### Install Controller
51+
52+
To install the latest release of the controller, run the following commands.
53+
54+
> **_NOTE:_** AWS region environment variable can be _optionaly_ set like `export AWS_REGION=us-west-2` Otherwise the controller will infer region in the order `AWS_REGION` environment variable, ~/.aws/config file, then EC2 metadata (for EKS environment)
3055
3156
```sh
3257
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/config/controller_install_release"
3358
```
3459
3560
> 📌 See [Releases](#Releases) section for details on how to install other versions.
3661
62+
The controller must have sufficient IAM permissions to perform required Cloud Map operations. Grant IAM access rights `AWSCloudMapFullAccess` to the controller Service Account to enable the controller to manage Cloud Map resources.
63+
64+
## Usage
65+
3766
### Export services
3867
3968
Then assuming you already have a Service installed, apply a `ServiceExport` yaml to the cluster in which you want to export a service. This can be done for each service you want to export.
@@ -55,17 +84,18 @@ metadata:
5584
name: my-amazing-service
5685
```
5786
58-
*See the `samples` directory for a set of example yaml files to set up a service and export it. To apply the sample files run*
87+
*See the `samples` directory for a set of example yaml files to set up a service and export it. To apply the sample files run the following commands.*
88+
5989
```sh
6090
kubectl create namespace example
61-
kubectl apply -f https://raw.githubusercontent.com/aws/aws-cloud-map-mcs-controller-for-k8s/main/samples/example-deployment.yaml
62-
kubectl apply -f https://raw.githubusercontent.com/aws/aws-cloud-map-mcs-controller-for-k8s/main/samples/example-service.yaml
63-
kubectl apply -f https://raw.githubusercontent.com/aws/aws-cloud-map-mcs-controller-for-k8s/main/samples/example-serviceexport.yaml
91+
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/samples/example-deployment.yaml
92+
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/samples/example-service.yaml
93+
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/samples/example-serviceexport.yaml
6494
```
6595
6696
### Import services
6797
68-
In your other cluster, the controller will automatically sync services registered in AWS Cloud Map by applying the appropriate `ServiceImport`. To list them all, run
98+
In your other cluster, the controller will automatically sync services registered in AWS Cloud Map by applying the appropriate `ServiceImport`. To list them all, run the following command.
6999
```sh
70100
kubectl get ServiceImport -A
71101
```
@@ -76,24 +106,24 @@ AWS Cloud Map MCS Controller for K8s adheres to the [SemVer](https://semver.org/
76106
77107
> **_NOTE:_** AWS region environment variable can be _optionally_ set like `export AWS_REGION=us-west-2` Otherwise controller will infer region in the order `AWS_REGION` environment variable, ~/.aws/config file, then EC2 metadata (for EKS environment)
78108
79-
To install from a release run
109+
The following command format is used to install from a particular release.
80110
```sh
81111
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/config/controller_install_release[?ref=*git version tag*]"
82112
```
83113
84-
Example to install latest release
114+
Run the following command to install the latest release.
85115
```sh
86116
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/config/controller_install_release"
87117
```
88118
89-
Example to install v0.1.0
119+
The following example will install release v0.1.0.
90120
```sh
91121
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/config/controller_install_release?ref=v0.1.0"
92122
```
93123
94124
We also maintain a `latest` tag, which is updated to stay in line with the `main` branch. We **do not** recommend installing this on any production cluster, as any new major versions updated on the `main` branch will introduce breaking changes.
95125
96-
To install from `latest` tag run
126+
To install from `latest` tag run the following command.
97127
```sh
98128
kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/config/controller_install_latest"
99129
```
@@ -109,4 +139,4 @@ Join the channel with this [invite](https://join.slack.com/t/awsappmesh/shared_i
109139
110140
This project is distributed under the
111141
[Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0),
112-
see [LICENSE](https://github.com/aws/aws-cloud-map-mcs-controller-for-k8s/blob/main/LICENSE) and [NOTICE](https://github.com/aws/aws-cloud-map-mcs-controller-for-k8s/blob/main/NOTICE) for more information.
142+
see [LICENSE](https://github.com/aws/aws-cloud-map-mcs-controller-for-k8s/blob/main/LICENSE) and [NOTICE](https://github.com/aws/aws-cloud-map-mcs-controller-for-k8s/blob/main/NOTICE) for more information.

‎samples/coredns-clusterrole.yaml

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
apiVersion: rbac.authorization.k8s.io/v1
2+
kind: ClusterRole
3+
metadata:
4+
labels:
5+
eks.amazonaws.com/component: coredns
6+
k8s-app: kube-dns
7+
kubernetes.io/bootstrapping: rbac-defaults
8+
name: system:coredns
9+
rules:
10+
- apiGroups:
11+
- ""
12+
resources:
13+
- endpoints
14+
- services
15+
- pods
16+
- namespaces
17+
verbs:
18+
- list
19+
- watch
20+
- apiGroups:
21+
- ""
22+
resources:
23+
- nodes
24+
verbs:
25+
- get
26+
- apiGroups:
27+
- discovery.k8s.io
28+
resources:
29+
- endpointslices
30+
verbs:
31+
- create
32+
- get
33+
- list
34+
- patch
35+
- update
36+
- watch
37+
- apiGroups:
38+
- multicluster.x-k8s.io
39+
resources:
40+
- serviceimports
41+
verbs:
42+
- create
43+
- get
44+
- list
45+
- patch
46+
- update
47+
- watch
48+
- apiGroups:
49+
- multicluster.x-k8s.io
50+
resources:
51+
- serviceexports
52+
verbs:
53+
- create
54+
- get
55+
- list
56+
- patch
57+
- update
58+
- watch

‎samples/coredns-configmap.yaml

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
apiVersion: v1
2+
data:
3+
Corefile: |
4+
.:53 {
5+
errors
6+
health
7+
multicluster clusterset.local
8+
kubernetes cluster.local in-addr.arpa ip6.arpa {
9+
pods insecure
10+
fallthrough in-addr.arpa ip6.arpa
11+
}
12+
prometheus :9153
13+
forward . /etc/resolv.conf
14+
cache 30
15+
loop
16+
reload
17+
loadbalance
18+
}
19+
kind: ConfigMap
20+
metadata:
21+
annotations:
22+
labels:
23+
eks.amazonaws.com/component: coredns
24+
k8s-app: kube-dns
25+
name: coredns
26+
namespace: kube-system

‎samples/coredns-deployment.yaml

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
labels:
5+
eks.amazonaws.com/component: coredns
6+
k8s-app: kube-dns
7+
kubernetes.io/name: CoreDNS
8+
name: coredns
9+
namespace: kube-system
10+
spec:
11+
progressDeadlineSeconds: 600
12+
replicas: 2
13+
revisionHistoryLimit: 10
14+
selector:
15+
matchLabels:
16+
eks.amazonaws.com/component: coredns
17+
k8s-app: kube-dns
18+
strategy:
19+
rollingUpdate:
20+
maxSurge: 25%
21+
maxUnavailable: 1
22+
type: RollingUpdate
23+
template:
24+
metadata:
25+
annotations:
26+
eks.amazonaws.com/compute-type: ec2
27+
creationTimestamp: null
28+
labels:
29+
eks.amazonaws.com/component: coredns
30+
k8s-app: kube-dns
31+
spec:
32+
affinity:
33+
nodeAffinity:
34+
requiredDuringSchedulingIgnoredDuringExecution:
35+
nodeSelectorTerms:
36+
- matchExpressions:
37+
- key: beta.kubernetes.io/os
38+
operator: In
39+
values:
40+
- linux
41+
- key: beta.kubernetes.io/arch
42+
operator: In
43+
values:
44+
- amd64
45+
- arm64
46+
podAntiAffinity:
47+
preferredDuringSchedulingIgnoredDuringExecution:
48+
- podAffinityTerm:
49+
labelSelector:
50+
matchExpressions:
51+
- key: k8s-app
52+
operator: In
53+
values:
54+
- kube-dns
55+
topologyKey: kubernetes.io/hostname
56+
weight: 100
57+
containers:
58+
- args:
59+
- -conf
60+
- /etc/coredns/Corefile
61+
image: ghcr.io/aws/aws-cloud-map-mcs-controller-for-k8s/coredns-multicluster/coredns:v1.8.4
62+
imagePullPolicy: IfNotPresent
63+
livenessProbe:
64+
failureThreshold: 5
65+
httpGet:
66+
path: /health
67+
port: 8080
68+
scheme: HTTP
69+
initialDelaySeconds: 60
70+
periodSeconds: 10
71+
successThreshold: 1
72+
timeoutSeconds: 5
73+
name: coredns
74+
ports:
75+
- containerPort: 53
76+
name: dns
77+
protocol: UDP
78+
- containerPort: 53
79+
name: dns-tcp
80+
protocol: TCP
81+
- containerPort: 9153
82+
name: metrics
83+
protocol: TCP
84+
readinessProbe:
85+
failureThreshold: 3
86+
httpGet:
87+
path: /health
88+
port: 8080
89+
scheme: HTTP
90+
periodSeconds: 10
91+
successThreshold: 1
92+
timeoutSeconds: 1
93+
resources:
94+
limits:
95+
memory: 170Mi
96+
requests:
97+
cpu: 100m
98+
memory: 70Mi
99+
securityContext:
100+
allowPrivilegeEscalation: false
101+
capabilities:
102+
add:
103+
- NET_BIND_SERVICE
104+
drop:
105+
- all
106+
readOnlyRootFilesystem: true
107+
terminationMessagePath: /dev/termination-log
108+
terminationMessagePolicy: File
109+
volumeMounts:
110+
- mountPath: /etc/coredns
111+
name: config-volume
112+
readOnly: true
113+
- mountPath: /tmp
114+
name: tmp
115+
dnsPolicy: Default
116+
priorityClassName: system-cluster-critical
117+
restartPolicy: Always
118+
schedulerName: default-scheduler
119+
securityContext: {}
120+
serviceAccount: coredns
121+
serviceAccountName: coredns
122+
terminationGracePeriodSeconds: 30
123+
tolerations:
124+
- effect: NoSchedule
125+
key: node-role.kubernetes.io/master
126+
- key: CriticalAddonsOnly
127+
operator: Exists
128+
volumes:
129+
- emptyDir: {}
130+
name: tmp
131+
- configMap:
132+
defaultMode: 420
133+
items:
134+
- key: Corefile
135+
path: Corefile
136+
name: coredns
137+
name: config-volume

0 commit comments

Comments
 (0)