You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Custom certificate authority (CA) in Azure Kubernetes Service (AKS) (preview)
12
12
13
-
Custom certificate authorities (CAs) allow you to establish trust between your Azure Kubernetes Service (AKS) clusters and workloads, such as private registries, proxies, and firewalls. A Kubernetes secret stores the certificate authority's information, and then it's passed to all nodes in the cluster.
13
+
AKS generates and uses the following certificates, Certificate Authorities (CAs), and Service Accounts (SAs):
14
14
15
-
This feature is applied per node pool, so new and existing node pools must be configured to enable this feature.
15
+
* The AKS API server creates a CA called the Cluster CA.
16
+
* The API server has a Cluster CA, which signs certificates for one-way communication from the API server to kubelets.
17
+
* Each kubelet also creates a Certificate Signing Request (CSR), which is signed by the Cluster CA, for communication from the kubelet to the API server.
18
+
* The API aggregator uses the Cluster CA to issue certificates for communication with other APIs. The API aggregator can also have its own CA for issuing those certificates, but it currently uses the Cluster CA.
19
+
* Each node uses an SA token, which is signed by the Cluster CA.
20
+
* The `kubectl` client has a certificate for communicating with the AKS cluster.
21
+
22
+
You can also create custom certificate authorities, which allow you to establish trust between your Azure Kubernetes Service (AKS) clusters and workloads, such as private registries, proxies, and firewalls. A Kubernetes secret stores the certificate authority's information, and then it's passed to all nodes in the cluster. This feature is applied per node pool, so you need to enable it on new and existing node pools.
23
+
24
+
This article shows you how to create custom CAs and apply them to your AKS clusters.
16
25
17
26
## Prerequisites
18
27
@@ -28,44 +37,45 @@ This feature is applied per node pool, so new and existing node pools must be co
28
37
29
38
[!INCLUDE [preview features callout](includes/preview/preview-callout.md)]
30
39
31
-
Install the aks-preview extension using the [`az extension add`][az-extension-add] command.
40
+
1.Install the aks-preview extension using the [`az extension add`][az-extension-add] command.
32
41
33
-
```azurecli
34
-
az extension add --name aks-preview
35
-
```
42
+
```azurecli
43
+
az extension add --name aks-preview
44
+
```
36
45
37
-
Update to the latest version of the extension using the [`az extension update`][az-extension-update] command.
46
+
2. Update to the latest version of the extension using the [`az extension update`][az-extension-update] command.
38
47
39
-
```azurecli
40
-
az extension update --name aks-preview
41
-
```
48
+
```azurecli
49
+
az extension update --name aks-preview
50
+
```
42
51
43
-
## Register the 'CustomCATrustPreview' feature flag
52
+
## Register the `CustomCATrustPreview` feature flag
44
53
45
-
Register the `CustomCATrustPreview` feature flag using the [`az feature register`][az-feature-register] command. It takes a few minutes for the status to show *Registered*.
54
+
1. Register the `CustomCATrustPreview` feature flag using the [`az feature register`][az-feature-register] command.
46
55
47
-
```azurecli
48
-
az feature register --namespace "Microsoft.ContainerService" --name "CustomCATrustPreview"
49
-
```
56
+
```azurecli
57
+
az feature register --namespace "Microsoft.ContainerService" --name "CustomCATrustPreview"
58
+
```
50
59
51
-
Verify the registration status using the [`az feature show`][az-feature-show] command.
60
+
It takes a few minutes for the status to show *Registered*.
52
61
53
-
```azurecli
54
-
az feature show --namespace "Microsoft.ContainerService" --name "CustomCATrustPreview"
55
-
```
62
+
2. Verify the registration status using the [`az feature show`][az-feature-show] command.
56
63
57
-
When the status reflects *Registered*, refresh the registration of the *Microsoft.ContainerService* resource provider using the [`az provider register`][az-provider-register] command.
64
+
```azurecli
65
+
az feature show --namespace "Microsoft.ContainerService" --name "CustomCATrustPreview"
66
+
```
58
67
59
-
```azurecli
60
-
az provider register --namespace Microsoft.ContainerService
61
-
```
68
+
3. When the status reflects *Registered*, refresh the registration of the *Microsoft.ContainerService* resource provider using the [`az provider register`][az-provider-register] command.
69
+
70
+
```azurecli
71
+
az provider register --namespace Microsoft.ContainerService
72
+
```
62
73
63
74
## Custom CA installation on AKS node pools
64
75
65
-
### Install CAs during node pool creation
76
+
### Install CAs on AKS node pools
66
77
67
-
If your environment requires your custom CAs to be added to node trust store for correct provisioning, you need to pass a text file containing up to 10 blank line separated certificates during
68
-
[`az aks create`][az-aks-create] or [`az aks update`][az-aks-update] operations.
78
+
If your environment requires your custom CAs to be added to node trust store for correct provisioning, you need to pass a text file containing up to 10 blank line separated certificates during [`az aks create`][az-aks-create] or [`az aks update`][az-aks-update] operations.
69
79
70
80
Example text file:
71
81
@@ -79,128 +89,125 @@ cert2
79
89
-----END CERTIFICATE-----
80
90
```
81
91
82
-
```azurecli
83
-
az aks create \
84
-
--resource-group myResourceGroup \
85
-
--name myAKSCluster \
86
-
--node-count 2 \
87
-
--enable-custom-ca-trust \
88
-
--custom-ca-trust-certificates pathToFileWithCAs
89
-
```
92
+
#### Install CAs during node pool creation
90
93
91
-
#### CA rotation for availability during node pool boot up
94
+
* Install CAs during node pool creation using the [`az aks create][az-aks-create] command and specifying your text file for the `--custom-ca-trust-certificates` parameter.
92
95
93
-
You can update CAs passed to your cluster during boot up using the [`az aks update`][az-aks-update] command and specifying your text file.
96
+
```azurecli
97
+
az aks create \
98
+
--resource-group myResourceGroup \
99
+
--name myAKSCluster \
100
+
--node-count 2 \
101
+
--enable-custom-ca-trust \
102
+
--custom-ca-trust-certificates pathToFileWithCAs
103
+
```
94
104
95
-
```azurecli
96
-
az aks update \
97
-
--resource-group myResourceGroup \
98
-
--name myAKSCluster \
99
-
--custom-ca-trust-certificates pathToFileWithCAs
100
-
```
105
+
#### CA rotation for availability during node pool boot up
101
106
102
-
> [!NOTE]
103
-
> This operation triggers a model update, ensuring new nodes have the newest CAs required for correct provisioning. AKS creates additional nodes, drains existing ones, deletes them, and replaces them with nodes that have the new set of CAs installed.
107
+
* Update CAs passed to your cluster during boot up using the [`az aks update`][az-aks-update] command and specifying your text file for the `--custom-ca-trust-certificates` parameter.
104
108
105
-
### Install CAs once node pool is up and running
109
+
```azurecli
110
+
az aks update \
111
+
--resource-group myResourceGroup \
112
+
--name myAKSCluster \
113
+
--custom-ca-trust-certificates pathToFileWithCAs
114
+
```
106
115
107
-
If your environment can be successfully provisioned without your custom CAs, you can provide the CAs by deploying a secret in the `kube-system` namespace. This approach allows for certificate rotation without the need for node recreation.
116
+
> [!NOTE]
117
+
> This operation triggers a model update, ensuring new nodes have the newest CAs required for correct provisioning. AKS creates additional nodes, drains existing ones, deletes them, and replaces them with nodes that have the new set of CAs installed.
108
118
109
-
Create a [Kubernetes secret][kubernetes-secrets] YAML manifest with your base64 encoded certificate string in the `data` field. Data from this secret is used to update CAs on all nodes. Make sure the secret is named `custom-ca-trust-secret` and is created in the `kube-system` namespace. Installing CAs using the secret in the `kube-system` namespace allows for CA rotation without the need for node recreation.
110
-
111
-
```yaml
112
-
apiVersion: v1
113
-
kind: Secret
114
-
metadata:
115
-
name: custom-ca-trust-secret
116
-
namespace: kube-system
117
-
type: Opaque
118
-
data:
119
-
ca1.crt: |
120
-
{base64EncodedCertStringHere}
121
-
ca2.crt: |
122
-
{anotherBase64EncodedCertStringHere}
123
-
```
119
+
### Install CAs after node pool creation
124
120
125
-
To update or remove a CA, you can edit and apply the YAML manifest. The cluster polls for changes and updates the nodes accordingly. It may take a couple minutes before changes are applied.
121
+
If your environment can be successfully provisioned without your custom CAs, you can provide the CAs by deploying a secret in the `kube-system` namespace. This approach allows for certificate rotation without the need for node recreation.
126
122
127
-
> [!NOTE]
128
-
> Sometimes, containerd restart on the node might be required for the CAs to be picked up properly. If it appears like CAs aren't correctly added to your node's trust store, you can trigger a restart using the following command from node's shell:
129
-
>
130
-
> ```systemctl restart containerd```
123
+
* Create a [Kubernetes secret][kubernetes-secrets] YAML manifest with your base64 encoded certificate string in the `data` field.
124
+
125
+
```yaml
126
+
apiVersion: v1
127
+
kind: Secret
128
+
metadata:
129
+
name: custom-ca-trust-secret
130
+
namespace: kube-system
131
+
type: Opaque
132
+
data:
133
+
ca1.crt: |
134
+
{base64EncodedCertStringHere}
135
+
ca2.crt: |
136
+
{anotherBase64EncodedCertStringHere}
137
+
```
138
+
139
+
Data from this secret is used to update CAs on all nodes. Make sure the secret is named `custom-ca-trust-secret` and is created in the `kube-system` namespace. Installing CAs using the secret in the `kube-system` namespace allows for CA rotation without the need for node recreation. To update or remove a CA, you can edit and apply the YAML manifest. The cluster polls for changes and updates the nodes accordingly. It may take a couple minutes before changes are applied.
140
+
141
+
> [!NOTE]
142
+
>
143
+
> containerd restart on the node might be required for the CAs to be picked up properly. If it appears like CAs aren't correctly added to your node's trust store, you can trigger a restart using the following command from node's shell:
144
+
>
145
+
> ```systemctl restart containerd```
131
146
132
147
## Configure a new AKS cluster to use a custom CA
133
148
134
-
Configure a new AKS cluster to use a custom CA using the [`az aks create`][az-aks-create] command with the `--enable-custom-ca-trust` parameter.
149
+
* Configure a new AKS cluster to use a custom CA using the [`az aks create`][az-aks-create] command with the `--enable-custom-ca-trust` parameter.
135
150
136
-
```azurecli
137
-
az aks create \
138
-
--resource-group myResourceGroup \
139
-
--name myAKSCluster \
140
-
--node-count 2 \
141
-
--enable-custom-ca-trust
142
-
```
151
+
```azurecli
152
+
az aks create \
153
+
--resource-group myResourceGroup \
154
+
--name myAKSCluster \
155
+
--node-count 2 \
156
+
--enable-custom-ca-trust
157
+
```
143
158
144
-
You can also configure a new AKS cluster to use custom CA with CAs installed before the node boots up using the [`az aks create`][az-aks-create] command with the `--enable-custom-ca-trust` and `--custom-ca-trust-certificates` parameters.
159
+
## Configure a new AKS cluster to use a custom CA with CAs installed before node boots up
145
160
146
-
```azurecli
147
-
az aks create \
148
-
--resource-group myResourceGroup \
149
-
--name myAKSCluster \
150
-
--node-count 2 \
151
-
--enable-custom-ca-trust \
152
-
--custom-ca-trust-certificates pathToFileWithCAs
153
-
```
161
+
* Configure a new AKS cluster to use custom CA with CAs installed before the node boots up using the [`az aks create`][az-aks-create] command with the `--enable-custom-ca-trust` and `--custom-ca-trust-certificates` parameters.
162
+
163
+
```azurecli
164
+
az aks create \
165
+
--resource-group myResourceGroup \
166
+
--name myAKSCluster \
167
+
--node-count 2 \
168
+
--enable-custom-ca-trust \
169
+
--custom-ca-trust-certificates pathToFileWithCAs
170
+
```
154
171
155
172
## Configure an existing AKS cluster to have custom CAs installed before node boots up
156
173
157
-
Configure an existing AKS cluster to have your custom CAs added to node's trust store before it boots up using the [`az aks update`][az-aks-update] command with the `--custom-ca-trust-certificates` parameter.
174
+
* Configure an existing AKS cluster to have your custom CAs added to node's trust store before it boots up using the [`az aks update`][az-aks-update] command with the `--custom-ca-trust-certificates` parameter.
158
175
159
-
```azurecli
160
-
az aks update \
161
-
--resource-group myResourceGroup \
162
-
--name myAKSCluster \
163
-
--custom-ca-trust-certificates pathToFileWithCAs
164
-
```
176
+
```azurecli
177
+
az aks update \
178
+
--resource-group myResourceGroup \
179
+
--name myAKSCluster \
180
+
--custom-ca-trust-certificates pathToFileWithCAs
181
+
```
165
182
166
183
## Configure a new node pool to use a custom CA
167
184
168
-
Configure a new node pool to use a custom CA using the [`az aks nodepool add`][az-aks-nodepool-add] command with the `--enable-custom-ca-trust` parameter.
185
+
* Configure a new node pool to use a custom CA using the [`az aks nodepool add`][az-aks-nodepool-add] command with the `--enable-custom-ca-trust` parameter.
169
186
170
-
```azurecli
171
-
az aks nodepool add \
172
-
--cluster-name myAKSCluster \
173
-
--resource-group myResourceGroup \
174
-
--name myNodepool \
175
-
--enable-custom-ca-trust \
176
-
--os-type Linux
177
-
```
187
+
```azurecli
188
+
az aks nodepool add \
189
+
--cluster-name myAKSCluster \
190
+
--resource-group myResourceGroup \
191
+
--name myNodepool \
192
+
--enable-custom-ca-trust \
193
+
--os-type Linux
194
+
```
178
195
179
-
If no other node pools with the feature enabled exist, the cluster has to reconcile its settings for the changes to take effect. This operation happens automatically as a part of AKS's reconcile loop. Before the operation, the daemon set and pods don't appear on the cluster. You can trigger an immediate reconcile operation using the [`az aks update`][az-aks-update] command. The daemon set and pods appear after the update completes.
180
-
181
-
```azurecli
182
-
az aks update \
183
-
--resource-group myResourceGroup \
184
-
--name cluster-name
185
-
```
196
+
If no other node pools with the feature enabled exist, the cluster has to reconcile its settings for the changes to take effect. This operation happens automatically as a part of AKS's reconcile loop. Before the operation, the daemon set and pods don't appear on the cluster. You can trigger an immediate reconcile operation using the [`az aks update`][az-aks-update] command. The daemon set and pods appear after the update completes.
186
197
187
198
## Configure an existing node pool to use a custom CA
188
199
189
-
Configure an existing node pool to use a custom CA using the [`az aks nodepool update`][az-aks-nodepool-update] command with the `--enable-custom-trust-ca` parameter.
190
-
191
-
```azurecli
192
-
az aks nodepool update \
193
-
--resource-group myResourceGroup \
194
-
--cluster-name myAKSCluster \
195
-
--name myNodepool \
196
-
--enable-custom-ca-trust
197
-
```
200
+
* Configure an existing node pool to use a custom CA using the [`az aks nodepool update`][az-aks-nodepool-update] command with the `--enable-custom-trust-ca` parameter.
198
201
199
-
If no other node pools with the feature enabled exist, the cluster has to reconcile its settings for the changes to take effect. This operation happens automatically as a part of AKS's reconcile loop. Before the operation, the daemon set and pods don't appear on the cluster. You can trigger an immediate reconcile operation using the [`az aks update`][az-aks-update] command. The daemon set and pods appear after the update completes.
202
+
```azurecli
203
+
az aks nodepool update \
204
+
--resource-group myResourceGroup \
205
+
--cluster-name myAKSCluster \
206
+
--name myNodepool \
207
+
--enable-custom-ca-trust
208
+
```
200
209
201
-
```azurecli
202
-
az aks update -g myResourceGroup --name cluster-name
203
-
```
210
+
If no other node pools with the feature enabled exist, the cluster has to reconcile its settings for the changes to take effect. This operation happens automatically as a part of AKS's reconcile loop. Before the operation, the daemon set and pods don't appear on the cluster. You can trigger an immediate reconcile operation using the [`az aks update`][az-aks-update] command. The daemon set and pods appear after the update completes.
204
211
205
212
## Troubleshooting
206
213
@@ -219,9 +226,6 @@ From the node's shell, run ```systemctl restart containerd```. Once containerd i
219
226
220
227
For more information on AKS security best practices, see [Best practices for cluster security and upgrades in Azure Kubernetes Service (AKS)][aks-best-practices-security-upgrades].
0 commit comments