Skip to content

Commit 77c3fa3

Browse files
v-regandownerv-regandowner
authored
Merge pull request #255409 from MGoedtel/task124926
added disable SSH preview feature
2 parents 110b9ba + fcc6a08 commit 77c3fa3

File tree

1 file changed

+244
-16
lines changed

1 file changed

+244
-16
lines changed

‎articles/aks/manage-ssh-node-access.md

Lines changed: 244 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,32 @@
11
---
22
title: Manage SSH access on Azure Kubernetes Service cluster nodes
33
titleSuffix: Azure Kubernetes Service
4-
description: Learn how to configure SSH on Azure Kubernetes Service (AKS) cluster nodes.
4+
description: Learn how to configure SSH and manage SSH keys on Azure Kubernetes Service (AKS) cluster nodes.
55
ms.topic: article
66
ms.custom: devx-track-azurecli
7-
ms.date: 12/15/2023
7+
ms.date: 02/12/2024
88
---
99

1010
# Manage SSH for secure access to Azure Kubernetes Service (AKS) nodes
1111

12-
This article describes how to configure the SSH key (preview) on your AKS clusters or node pools, during initial deployment or at a later time.
12+
This article describes how to configure the SSH keys (preview) on your AKS clusters or node pools, during initial deployment or at a later time.
13+
14+
AKS supports the following configuration options to manage SSH keys on cluster nodes:
15+
16+
* Create a cluster with SSH keys
17+
* Update the SSH keys on an existing AKS cluster
18+
* Disable and enable the SSH service
1319

1420
[!INCLUDE [preview features callout](./includes/preview/preview-callout.md)]
1521

1622
## Before you begin
1723

18-
* You need the Azure CLI version 2.46.0 or later installed and configured. If you need to install or upgrade, see [Install Azure CLI][install-azure-cli].
19-
* This feature supports Linux, Mariner, and CBLMariner node pools on existing clusters.
24+
* You need `aks-preview` version 0.5.116 or later to use **Update**.
25+
* You need `aks-preview` version 1.0.0b6 or later to use **Disable**.
26+
* The **Create** and **Update** SSH feature supports Linux, Windows, and Azure Linux node pools on existing clusters.
27+
* The **Disable** SSH feature isn't supported in this preview release on node pools running the Windows Server operating system.
2028

21-
## Install the `aks-preview` Azure CLI extension
29+
### Install the `aks-preview` Azure CLI extension
2230

2331
1. Install the aks-preview extension using the [`az extension add`][az-extension-add] command.
2432

@@ -32,18 +40,40 @@ This article describes how to configure the SSH key (preview) on your AKS cluste
3240
az extension update --name aks-preview
3341
```
3442
35-
## Create an AKS cluster with SSH key (preview)
43+
### Register the `DisableSSHPreview` feature flag
44+
45+
1. Register the `DisableSSHPreview` feature flag using the [`az feature register`][az-feature-register] command.
46+
47+
```azurecli-interactive
48+
az feature register --namespace "Microsoft.ContainerService" --name "DisableSSHPreview"
49+
```
50+
51+
It takes a few minutes for the status to show *Registered*.
52+
53+
2. Verify the registration status using the [`az feature show`][az-feature-show] command.
54+
55+
```azurecli-interactive
56+
az feature show --namespace "Microsoft.ContainerService" --name "DisableSSHPreview"
57+
```
58+
59+
3. When the status reflects *Registered*, refresh the registration of the *Microsoft.ContainerService* resource provider using the [`az provider register`][az-provider-register] command.
60+
61+
```azurecli-interactive
62+
az provider register --namespace Microsoft.ContainerService
63+
```
64+
65+
## Create an AKS cluster with SSH keys
3666
3767
Use the [az aks create][az-aks-create] command to deploy an AKS cluster with an SSH public key. You can either specify the key or a key file using the `--ssh-key-value` argument.
3868
3969
|SSH parameter |Description |Default value |
4070
|-----|-----|-----|
41-
|--generate-ssh-key |If you don't have your own SSH key, specify `--generate-ssh-key`. The Azure CLI first looks for the key in the `~/.ssh/` directory. If the key exists, it's used. If the key doesn't exist, the Azure CLI automatically generates a set of SSH keys and saves them in the specified or default directory.||
71+
|`--generate-ssh-key` |If you don't have your own SSH keys, specify `--generate-ssh-key`. The Azure CLI automatically generates a set of SSH keys and saves them in the default directory `~/.ssh/`.||
4272
|--ssh-key-value |Public key path or key contents to install on node VMs for SSH access. For example, `ssh-rsa AAAAB...snip...UcyupgH azureuser@linuxvm`.|`~/.ssh/id_rsa.pub` |
43-
|--no-ssh-key | If you don't require an SSH key, specify this argument. However, AKS automatically generates a set of SSH keys because the Azure Virtual Machine resource dependency doesn’t support an empty SSH key file. As a result, the keys aren't returned and can't be used to SSH into the node VMs. ||
73+
|`--no-ssh-key` | If you don't require SSH keys, specify this argument. However, AKS automatically generates a set of SSH keys because the Azure Virtual Machine resource dependency doesn't support an empty SSH keys file. As a result, the keys aren't returned and can't be used to SSH into the node VMs. The private key is discarded and not saved.||
4474
4575
>[!NOTE]
46-
>If no parameters are specified, the Azure CLI defaults to referencing the SSH keys stored in the `~/.ssh/` directory. If the keys aren't found in the directory, the command returns a `key not found` error message.
76+
>If no parameters are specified, the Azure CLI defaults to referencing the SSH keys stored in the `~/.ssh/id_rsa.pub` file. If the keys aren't found, the command returns the message `An RSA key file or key value must be supplied to SSH Key Value`.
4777
4878
The following are examples of this command:
4979
@@ -53,18 +83,18 @@ The following are examples of this command:
5383
az aks create --name myAKSCluster --resource-group MyResourceGroup --generate-ssh-key
5484
```
5585
56-
* To specify an SSH public key file, specify it with the `--ssh-key-value` argument:
86+
* To specify an SSH public key file, include the `--ssh-key-value` argument:
5787
5888
```azurecli
5989
az aks create --name myAKSCluster --resource-group MyResourceGroup --ssh-key-value ~/.ssh/id_rsa.pub
6090
```
6191
62-
## Update SSH public key (preview) on an existing AKS cluster
92+
## Update SSH public key on an existing AKS cluster
6393
64-
Use the [az aks update][az-aks-update] command to update the SSH public key on your cluster. This operation updates the key on all node pools. You can either specify the key or a key file using the `--ssh-key-value` argument.
94+
Use the [`az aks update`][az-aks-update] command to update the SSH public key (preview) on your cluster. This operation updates the key on all node pools. You can either specify a key or a key file using the `--ssh-key-value` argument.
6595
6696
> [!NOTE]
67-
> Updating of the SSH key is supported on Azure virtual machine scale sets with AKS clusters.
97+
> Updating the SSH keys is supported on Azure virtual machine scale sets with AKS clusters.
6898
6999
The following are examples of this command:
70100
@@ -81,7 +111,202 @@ The following are examples of this command:
81111
```
82112
83113
> [!IMPORTANT]
84-
> After you update the SSH key, AKS doesn't automatically update your node pool. At anytime you can choose to perform a [nodepool update operation][node-image-upgrade]. Only after a node image update is complete does the update SSH key operation take effect.
114+
> After you update the SSH key, AKS doesn't automatically update your node pool. At any time, you can choose to perform a [nodepool update operation][node-image-upgrade]. The update SSH keys operation takes effect after a node image update is complete.
115+
116+
## Disable SSH overview
117+
118+
To improve security and support your corporate security requirements or strategy, AKS supports disabling SSH (preview) both on the cluster and at the node pool level. Disable SSH introduces a simplified approach compared to the only supported solution, which requires configuring [network security group rules][network-security-group-rules-overview] on the AKS subnet/node network interface card (NIC).
119+
120+
When you disable SSH at cluster creation time, it takes effect after the cluster is created. However, when you disable SSH on an existing cluster or node pool, AKS doesn't automatically disable SSH. At any time, you can choose to perform a nodepool upgrade operation. The disable/enable SSH keys operation takes effect after the node image update is complete.
121+
122+
|SSH parameter |Description |
123+
|-----|-----|
124+
|`disabled` |The SSH service is disabled. |
125+
|`localuser` |The SSH service is enabled and users with SSH keys can securely access the node. |
126+
127+
>[!NOTE]
128+
>[kubectl debug node][kubelet-debug-node-access] continues to work after you disable SSH because it doesn't depend on the SSH service.
129+
130+
### Disable SSH on a new cluster deployment
131+
132+
By default, the SSH service on AKS cluster nodes is open to all users and pods running on the cluster. You can prevent direct SSH access from any network to cluster nodes to help limit the attack vector if a container in a pod becomes compromised.
133+
Use the [`az aks create`][az-aks-create] command to create a new cluster, and include the `--ssh-access disabled` argument to disable SSH (preview) on all the node pools during cluster creation.
134+
135+
> [!IMPORTANT]
136+
> After you disable the SSH service, you can't SSH into the cluster to perform administrative tasks or to troubleshoot.
137+
138+
```azurecli-interactive
139+
az aks create -g myResourceGroup -n myManagedCluster --ssh-access disabled
140+
```
141+
142+
After a few minutes, the command completes and returns JSON-formatted information about the cluster. The following example resembles the output and the results related to disabling SSH:
143+
144+
```output
145+
"securityProfile": {
146+
"sshAccess": "Disabled"
147+
},
148+
```
149+
150+
### Disable SSH on an existing cluster
151+
152+
Use the [`az aks update`][az-aks-update] command to update an existing cluster, and include the `--ssh-access disabled` argument to disable SSH (preview) on all the node pools in the cluster.
153+
154+
```azurecli-interactive
155+
az aks update -g myResourceGroup -n myManagedCluster --ssh-access disabled
156+
```
157+
158+
After a few minutes, the command completes and returns JSON-formatted information about the cluster. The following example resembles the output and the results related to disabling SSH:
159+
160+
```output
161+
"securityProfile": {
162+
"sshAccess": "Disabled"
163+
},
164+
```
165+
166+
For the change to take effect, you need to reimage all node pools by using the [`az aks nodepool upgrade`][az-aks-nodepool-upgrade] command.
167+
168+
```azurecli-interactive
169+
az aks nodepool upgrade --cluster-name myManagedCluster --name mynodepool --resource-group myResourceGroup --node-image-only
170+
```
171+
172+
> [!IMPORTANT]
173+
> During this operation, all Virtual Machine Scale Set instances are upgraded and reimaged to use the new SSH configuration.
174+
175+
### Disable SSH for a new node pool
176+
177+
Use the [`az aks nodepool add`][az-aks-nodepool-add] command to add a node pool, and include the `--ssh-access disabled` argument to disable SSH during node pool creation.
178+
179+
```azurecli-interactive
180+
az aks nodepool add --cluster-name myManagedCluster --name mynodepool --resource-group myResourceGroup --ssh-access disabled
181+
```
182+
183+
After a few minutes, the command completes and returns JSON-formatted information about the cluster indicating *mynodepool* was successfully created. The following example resembles the output and the results related to disabling SSH:
184+
185+
```output
186+
"securityProfile": {
187+
"sshAccess": "Disabled"
188+
},
189+
```
190+
191+
### Disable SSH for an existing node pool
192+
193+
Use the [`az aks nodepool update][az-aks-nodepool-update] command with the `--ssh-access disabled` argument to disable SSH (preview) on an existing node pool.
194+
195+
```azurecli-interactive
196+
az aks nodepool update --cluster-name myManagedCluster --name mynodepool --resource-group myResourceGroup --ssh-access disabled
197+
```
198+
199+
After a few minutes, the command completes and returns JSON-formatted information about the cluster indicating *mynodepool* was successfully created. The following example resembles the output and the results related to disabling SSH:
200+
201+
```output
202+
"securityProfile": {
203+
"sshAccess": "Disabled"
204+
},
205+
```
206+
207+
For the change to take effect, you need to reimage the node pool by using the [`az aks nodepool upgrade`][az-aks-nodepool-upgrade] command.
208+
209+
```azurecli-interactive
210+
az aks nodepool upgrade --cluster-name myManagedCluster --name mynodepool --resource-group myResourceGroup --node-image-only
211+
```
212+
213+
### Re-enable SSH on an existing cluster
214+
215+
Use the [`az aks update`][az-aks-update] command to update an existing cluster, and include the `--ssh-access localuser` argument to re-enable SSH (preview) on all the node pools in the cluster.
216+
217+
```azurecli-interactive
218+
az aks update -g myResourceGroup -n myManagedCluster --ssh-access localuser
219+
```
220+
221+
The following message is returned while the process is performed:
222+
223+
```output
224+
Only after all the nodes are reimaged, does the disable/enable SSH Access operation take effect."
225+
```
226+
227+
After re-enabling SSH, the nodes won't be reimaged automatically. At any time, you can choose to perform a [reimage operation][node-image-upgrade].
228+
229+
>[!IMPORTANT]
230+
>During this operation, all Virtual Machine Scale Set instances are upgraded and reimaged to use the new SSH public key.
231+
232+
### Re-enable SSH for a specific node pool
233+
234+
Use the [`az aks update`][az-aks-update] command to update a specific node pool, and include the `--ssh-access localuser` argument to re-enable SSH (preview) on that node pool in the cluster. In the following example, *nodepool1* is the target node pool.
235+
236+
```azurecli-interactive
237+
az aks nodepool update --cluster-name myManagedCluster --name nodepool1 --resource-group myResourceGroup --ssh-access localuser
238+
```
239+
240+
The following message is returned when the process is performed:
241+
242+
```output
243+
Only after all the nodes are reimaged, does the disable/enable SSH Access operation take effect.
244+
```
245+
246+
>[!IMPORTANT]
247+
>During this operation, all Virtual Machine Scale Set instances are upgraded and reimaged to use the new SSH public key.
248+
249+
## SSH service status
250+
251+
#### [Node-shell](#tab/node-shell)
252+
253+
Perform the following steps to use node-shell onto one node and inspect SSH service status using `systemctl`.
254+
255+
1. Get standard bash shell by running the command `kubectl node-shell <node>` command.
256+
257+
```bash
258+
kubectl node-shell aks-nodepool1-20785627-vmss000001
259+
```
260+
261+
2. Run the `systemctl` command to check the status of the SSH service.
262+
263+
```bash
264+
systemctl status ssh
265+
```
266+
267+
If SSH is disabled, the following sample output shows the results:
268+
269+
```output
270+
ssh.service - OpenBSD Secure Shell server
271+
Loaded: loaded (/lib/systemd/system/ssh.service; disabled; vendor preset: enabled)
272+
Active: inactive (dead) since Wed 2024-01-03 15:36:57 UTC; 20min ago
273+
```
274+
275+
If SSH is enabled, the following sample output shows the results:
276+
277+
```output
278+
ssh.service - OpenBSD Secure Shell server
279+
Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
280+
Active: active (running) since Wed 2024-01-03 15:40:20 UTC; 19min ago
281+
```
282+
283+
#### [Using run-command](#tab/run-command)
284+
285+
If node-shell isn't available, you can use the Virtual Machine Scale Set [`az vmss run-command invoke`][run-command-invoke] to check SSH service status.
286+
287+
```azurecli-interactive
288+
az vmss run-command invoke --resource-group myResourceGroup --name myVMSS --command-id RunShellScript --instance-id 0 --scripts "systemctl status ssh"
289+
```
290+
291+
The following sample output shows the json message returned:
292+
293+
```output
294+
{
295+
"value": [
296+
{
297+
"code": "ProvisioningState/succeeded",
298+
"displayStatus": "Provisioning succeeded",
299+
"level": "Info",
300+
"message": "Enable succeeded: \n[stdout]\n○ ssh.service - OpenBSD Secure Shell server\n Loaded: loaded (/lib/systemd/system/ssh.service; disabled; vendor preset: enabled)\n Active: inactive (dead) since Wed 2024-01-03 15:36:53 UTC; 25min ago\n Docs: man:sshd(8)\n man:sshd_config(5)\n Main PID: 827 (code=exited, status=0/SUCCESS)\n CPU: 22ms\n\nJan 03 15:36:44 aks-nodepool1-20785627-vmss000000 systemd[1]: Starting OpenBSD Secure Shell server...\nJan 03 15:36:44 aks-nodepool1-20785627-vmss000000 sshd[827]: Server listening on 0.0.0.0 port 22.\nJan 03 15:36:44 aks-nodepool1-20785627-vmss000000 sshd[827]: Server listening on :: port 22.\nJan 03 15:36:44 aks-nodepool1-20785627-vmss000000 systemd[1]: Started OpenBSD Secure Shell server.\nJan 03 15:36:53 aks-nodepool1-20785627-vmss000000 systemd[1]: Stopping OpenBSD Secure Shell server...\nJan 03 15:36:53 aks-nodepool1-20785627-vmss000000 sshd[827]: Received signal 15; terminating.\nJan 03 15:36:53 aks-nodepool1-20785627-vmss000000 systemd[1]: ssh.service: Deactivated successfully.\nJan 03 15:36:53 aks-nodepool1-20785627-vmss000000 systemd[1]: Stopped OpenBSD Secure Shell server.\n\n[stderr]\n",
301+
"time": null
302+
}
303+
]
304+
}
305+
```
306+
307+
Search for the word **Active** and its value should be `Active: inactive (dead)`, which indicates SSH is disabled on the node.
308+
309+
---
85310
86311
## Next steps
87312
@@ -90,16 +315,19 @@ To help troubleshoot any issues with SSH connectivity to your clusters nodes, yo
90315
<!-- LINKS - external -->
91316
92317
<!-- LINKS - internal -->
93-
[install-azure-cli]: /cli/azure/install-azure-cli
94318
[az-feature-register]: /cli/azure/feature#az_feature_register
95319
[az-feature-show]: /cli/azure/feature#az-feature-show
96320
[az-extension-add]: /cli/azure/extension#az_extension_add
97321
[az-extension-update]: /cli/azure/extension#az_extension_update
98322
[az-provider-register]: /cli/azure/provider#az_provider_register
99323
[az-aks-update]: /cli/azure/aks#az-aks-update
100324
[az-aks-create]: /cli/azure/aks#az-aks-create
325+
[az-aks-nodepool-update]: /cli/azure/aks/nodepool#az-aks-nodepool-update
326+
[az-aks-nodepool-add]: /cli/azure/aks/nodepool#az-aks-nodepool-add
101327
[view-kubelet-logs]: kubelet-logs.md
102328
[view-master-logs]: monitor-aks-reference.md#resource-logs
103329
[node-image-upgrade]: node-image-upgrade.md
104330
[az-aks-nodepool-upgrade]: /cli/azure/aks/nodepool#az-aks-nodepool-upgrade
105331
[network-security-group-rules-overview]: concepts-security.md#azure-network-security-groups
332+
[kubelet-debug-node-access]: node-access.md
333+
[run-command-invoke]: /cli/azure/vmss/run-command#az-vmss-run-command-invoke

0 commit comments

Comments
 (0)