You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/aks/custom-certificate-authority.md
+51-67Lines changed: 51 additions & 67 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,82 +5,71 @@ author: rayoef
5
5
ms.author: rayoflores
6
6
ms.topic: article
7
7
ms.custom: devx-track-azurecli
8
-
ms.date: 4/12/2022
8
+
ms.date: 4/24/2022
9
9
---
10
10
11
11
# Custom certificate authority (CA) in Azure Kubernetes Service (AKS) (preview)
12
12
13
-
Custom certificate authorities (CAs) allow you to establish trust between your Azure Kubernetes Service (AKS) cluster and your workloads, such as private registries, proxies, and firewalls. A Kubernetes secret is used to store the certificate authority's information, then it's passed to all nodes in the cluster.
13
+
Custom certificate authorities (CAs) allow you to establish trust between your Azure Kubernetes Service (AKS) clusters and workloads, such as private registries, proxies, and firewalls. A Kubernetes secret stores the certificate authority's information, and then it's passed to all nodes in the cluster.
14
14
15
-
This feature is applied per nodepool, so new and existing node pools must be configured to enable this feature.
15
+
This feature is applied per node pool, so new and existing node pools must be configured to enable this feature.
16
16
17
17
## Prerequisites
18
18
19
-
* An Azure subscription. If you don't have an Azure subscription, you can create a [free account](https://azure.microsoft.com/free).
19
+
* An Azure subscription. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free).
20
20
*[Azure CLI installed][azure-cli-install] (version 2.43.0 or greater).
21
21
* A base64 encoded certificate string or a text file with certificate.
22
22
23
23
## Limitations
24
24
25
-
This feature isn't currently supported for Windows node pools.
25
+
*This feature currently isn't supported for Windows node pools.
26
26
27
-
## Install the aks-preview Azure CLI extension
27
+
## Install the `aks-preview` Azure CLI extension
28
28
29
29
[!INCLUDE [preview features callout](includes/preview/preview-callout.md)]
30
30
31
-
To install the aks-preview extension, run the following command:
31
+
Install the aks-preview extension using the [`az extension add`][az-extension-add]command.
32
32
33
33
```azurecli
34
34
az extension add --name aks-preview
35
35
```
36
36
37
-
Run the following command to update to the latest version of the extension released:
37
+
Update to the latest version of the extension using the [`az extension update`][az-extension-update] command.
38
38
39
39
```azurecli
40
40
az extension update --name aks-preview
41
41
```
42
42
43
43
## Register the 'CustomCATrustPreview' feature flag
44
44
45
-
Register the `CustomCATrustPreview` feature flag by using the [az feature register][az-feature-register] command, as shown in the following example:
45
+
Register the `CustomCATrustPreview` feature flag using the [`az feature register`][az-feature-register] command. It takes a few minutes for the status to show *Registered*.
46
46
47
47
```azurecli
48
48
az feature register --namespace "Microsoft.ContainerService" --name "CustomCATrustPreview"
49
49
```
50
50
51
-
It takes a few minutes for the status to show *Registered*. Verify the registration status by using the [az feature show][az-feature-show] command:
51
+
Verify the registration status using the [`az feature show`][az-feature-show] command.
52
52
53
-
```azurecli-interactive
53
+
```azurecli
54
54
az feature show --namespace "Microsoft.ContainerService" --name "CustomCATrustPreview"
55
55
```
56
56
57
-
When the status reflects *Registered*, refresh the registration of the *Microsoft.ContainerService* resource provider by using the [az provider register][az-provider-register] command:
57
+
When the status reflects *Registered*, refresh the registration of the *Microsoft.ContainerService* resource provider using the [`az provider register`][az-provider-register] command.
58
58
59
-
```azurecli-interactive
59
+
```azurecli
60
60
az provider register --namespace Microsoft.ContainerService
61
61
```
62
62
63
-
## Two ways for custom CA installation on AKS node pools
63
+
## Custom CA installation on AKS node pools
64
64
65
-
Two ways of installing custom CAs on your AKS cluster are available. They're intended for different use cases, which are outlined below.
65
+
### Install CAs during node pool creation
66
66
67
-
### Install CAs during node pool boot up
68
-
If your environment requires your custom CAs to be added to node trust store for correct provisioning,
69
-
text file containing up to 10 blank line separated certificates needs to be passed during
70
-
[az aks create][az-aks-create] or [az aks update][az-aks-update] operations.
67
+
If your environment requires your custom CAs to be added to node trust store for correct provisioning, you need to pass a text file containing up to 10 blank line separated certificates during
68
+
[`az aks create`][az-aks-create] or [`az aks update`][az-aks-update] operations.
71
69
72
-
Example command:
73
-
```azurecli
74
-
az aks create \
75
-
--resource-group myResourceGroup \
76
-
--name myAKSCluster \
77
-
--node-count 2 \
78
-
--enable-custom-ca-trust \
79
-
--custom-ca-trust-certificates pathToFileWithCAs
80
-
```
70
+
Example text file:
81
71
82
-
Example file:
83
-
```
72
+
```txt
84
73
-----BEGIN CERTIFICATE-----
85
74
cert1
86
75
-----END CERTIFICATE-----
@@ -90,10 +79,19 @@ cert2
90
79
-----END CERTIFICATE-----
91
80
```
92
81
93
-
CAs will be added to node's trust store during node boot up process, allowing the node to, for example access a private registry.
82
+
```azurecli
83
+
az aks create \
84
+
--resource-group myResourceGroup \
85
+
--name myAKSCluster \
86
+
--node-count 2 \
87
+
--enable-custom-ca-trust \
88
+
--custom-ca-trust-certificates pathToFileWithCAs
89
+
```
94
90
95
91
#### CA rotation for availability during node pool boot up
96
-
To update CAs passed to cluster during boot up [az aks update][az-aks-update] operation has to be used.
92
+
93
+
You can update CAs passed to your cluster during boot up using the [`az aks update`][az-aks-update] command and specifying your text file.
94
+
97
95
```azurecli
98
96
az aks update \
99
97
--resource-group myResourceGroup \
@@ -102,19 +100,13 @@ az aks update \
102
100
```
103
101
104
102
> [!NOTE]
105
-
> Running this operation will trigger a model update, to ensure that new nodes added during for example scale up operation have the newest CAs required for correct provisioning.
106
-
> This means that AKS will create additional nodes, drain currently existing ones, delete them and then replace them with nodes that have the new set of CAs installed.
107
-
103
+
> This operation triggers a model update, ensuring new nodes have the newest CAs required for correct provisioning. AKS creates additional nodes, drains existing ones, deletes them, and replaces them with nodes that have the new set of CAs installed.
108
104
109
105
### Install CAs once node pool is up and running
110
-
If your environment can be successfully provisioned without your custom CAs, you can provide the CAs using a secret deployed in the kube-system namespace.
111
-
This approach allows for certificate rotation without the need for node recreation.
112
106
113
-
Create a [Kubernetes secret][kubernetes-secrets] YAML manifest with your base64 encoded certificate string in the `data` field. Data from this secret is used to update CAs on all nodes.
107
+
If your environment can be successfully provisioned without your custom CAs, you can provide the CAs by deploying a secret in the `kube-system` namespace. This approach allows for certificate rotation without the need for node recreation.
114
108
115
-
You must ensure that:
116
-
* The secret is named `custom-ca-trust-secret`.
117
-
* The secret is created in the `kube-system` namespace.
109
+
Create a [Kubernetes secret][kubernetes-secrets] YAML manifest with your base64 encoded certificate string in the `data` field. Data from this secret is used to update CAs on all nodes. Make sure the secret is named `custom-ca-trust-secret` and is created in the `kube-system` namespace. Installing CAs using the secret in the `kube-system` namespace allows for CA rotation without the need for node recreation.
118
110
119
111
```yaml
120
112
apiVersion: v1
@@ -130,18 +122,16 @@ data:
130
122
{anotherBase64EncodedCertStringHere}
131
123
```
132
124
133
-
To update or remove a CA, edit and apply the secret's YAML manifest. The cluster will poll for changes and update the nodes accordingly. This process may take a couple of minutes before changes are applied.
134
-
135
-
Sometimes containerd restart on the node might be required for the CAs to be picked up properly. If it appears like CAs aren't added correctly to your node's trust store, you can trigger such restart using the following command from node's shell:
136
-
137
-
```systemctl restart containerd```
125
+
To update or remove a CA, you can edit and apply the YAML manifest. The cluster polls for changes and updates the nodes accordingly. It may take a couple minutes before changes are applied.
138
126
139
127
> [!NOTE]
140
-
> Installing CAs using the secret in the kube-system namespace will allow for CA rotation without need for node recreation.
128
+
> Sometimes, containerd restart on the node might be required for the CAs to be picked up properly. If it appears like CAs aren't correctly added to your node's trust store, you can trigger a restart using the following command from node's shell:
129
+
>
130
+
> ```systemctl restart containerd```
141
131
142
132
## Configure a new AKS cluster to use a custom CA
143
133
144
-
To configure a new AKS cluster to use a custom CA, run the [az aks create][az-aks-create] command with the `--enable-custom-ca-trust` parameter.
134
+
Configure a new AKS cluster to use a custom CA using the [`az aks create`][az-aks-create] command with the `--enable-custom-ca-trust` parameter.
145
135
146
136
```azurecli
147
137
az aks create \
@@ -151,7 +141,7 @@ az aks create \
151
141
--enable-custom-ca-trust
152
142
```
153
143
154
-
To configure a new AKS cluster to use custom CA with CAs installed before node boots up, run the [az aks create][az-aks-create] command with the `--enable-custom-ca-trust` and `--custom-ca-trust-certificates` parameters.
144
+
You can also configure a new AKS cluster to use custom CA with CAs installed before the node boots up using the [`az aks create`][az-aks-create] command with the `--enable-custom-ca-trust` and `--custom-ca-trust-certificates` parameters.
155
145
156
146
```azurecli
157
147
az aks create \
@@ -164,7 +154,7 @@ az aks create \
164
154
165
155
## Configure an existing AKS cluster to have custom CAs installed before node boots up
166
156
167
-
To configure an existing AKS cluster to have your custom CAs added to node's trust store before it boots up, run [az aks update][az-aks-update] command with the `--custom-ca-trust-certificates` parameter.
157
+
Configure an existing AKS cluster to have your custom CAs added to node's trust store before it boots up using the [`az aks update`][az-aks-update] command with the `--custom-ca-trust-certificates` parameter.
168
158
169
159
```azurecli
170
160
az aks update \
@@ -175,7 +165,7 @@ az aks update \
175
165
176
166
## Configure a new node pool to use a custom CA
177
167
178
-
To configure a new node pool to use a custom CA, run the [az aks nodepool add][az-aks-nodepool-add] command with the `--enable-custom-ca-trust` parameter.
168
+
Configure a new node pool to use a custom CA using the [`az aks nodepool add`][az-aks-nodepool-add] command with the `--enable-custom-ca-trust` parameter.
179
169
180
170
```azurecli
181
171
az aks nodepool add \
@@ -186,22 +176,17 @@ az aks nodepool add \
186
176
--os-type Linux
187
177
```
188
178
189
-
If there are currently no other node pools with the feature enabled, cluster will have to reconcile its settings for
190
-
the changes to take effect. Before that happens, daemonset and pods, which install CAs won't appear on the cluster.
191
-
This operation will happen automatically as a part of AKS's reconcile loop.
192
-
You can trigger reconcile operation immediately by running the [az aks update][az-aks-update] command:
179
+
If no other node pools with the feature enabled exist, the cluster has to reconcile its settings for the changes to take effect. This operation happens automatically as a part of AKS's reconcile loop. Before the operation, the daemonset and pods won't appear on the cluster. You can trigger an immediate reconcile operation using the [`az aks update`][az-aks-update] command. The daemonset and pods will appear after the update completes.
193
180
194
181
```azurecli
195
182
az aks update \
196
183
--resource-group myResourceGroup \
197
184
--name cluster-name
198
185
```
199
186
200
-
Once completed, the daemonset and pods will appear in the cluster.
201
-
202
187
## Configure an existing node pool to use a custom CA
203
188
204
-
To configure an existing node pool to use a custom CA, run the [az aks nodepool update][az-aks-nodepool-update] command with the `--enable-custom-trust-ca` parameter.
189
+
Configure an existing node pool to use a custom CA using the [`az aks nodepool update`][az-aks-nodepool-update] command with the `--enable-custom-trust-ca` parameter.
205
190
206
191
```azurecli
207
192
az aks nodepool update \
@@ -211,25 +196,24 @@ az aks nodepool update \
211
196
--enable-custom-ca-trust
212
197
```
213
198
214
-
If there are currently no other node pools with the feature enabled, cluster will have to reconcile its settings for
215
-
the changes to take effect. Before that happens, daemon set and pods, which install CAs won't appear on the cluster.
216
-
This operation will happen automatically as a part of AKS's reconcile loop.
217
-
You can trigger reconcile operation by running the following command:
199
+
If no other node pools with the feature enabled exist, the cluster has to reconcile its settings for the changes to take effect. This operation happens automatically as a part of AKS's reconcile loop. Before the operation, the daemonset and pods won't appear on the cluster. You can trigger an immediate reconcile operation using the [`az aks update`][az-aks-update] command. The daemonset and pods will appear after the update completes.
218
200
219
201
```azurecli
220
202
az aks update -g myResourceGroup --name cluster-name
221
203
```
222
204
223
-
Once complete, the daemonset and pods will appear in the cluster.
224
-
225
205
## Troubleshooting
226
206
227
207
### Feature is enabled and secret with CAs is added, but operations are failing with X.509 Certificate Signed by Unknown Authority error
208
+
228
209
#### Incorrectly formatted certs passed in the secret
210
+
229
211
AKS requires certs passed in the user-created secret to be properly formatted and base64 encoded. Make sure the CAs you passed are properly base64 encoded and that files with CAs don't have CRLF line breaks.
230
-
Certificates passed to ```--custom-ca-trust-certificates``` option shouldn't be base64 encoded.
231
-
#### Containerd hasn't picked up new certs
232
-
From node's shell, run ```systemctl restart containerd```, once containerd is restarted, new certs will be properly picked up by the container runtime.
212
+
Certificates passed to ```--custom-ca-trust-certificates``` shouldn't be base64 encoded.
213
+
214
+
#### containerd hasn't picked up new certs
215
+
216
+
From the node's shell, run ```systemctl restart containerd```. Once containerd is restarts, the new certs will be properly picked up by the container runtime.
0 commit comments