Merge pull request #264007 from schaffererin/aksclusterconfigurationoptionsrewrite

cluster-configuration.md refactor

Author: JamesJBarnett

.openpublishing.redirection.json

@@ -3849,6 +3849,11 @@
             "redirect_url": "/azure/reliability/reliability-guidance-overview",
             "redirect_document_id": false
         },
+        {
+            "source_path_from_root": "/articles/aks/cluster-configuration.md",
+            "redirect_url": "/azure/aks/concepts-clusters-workloads.md",
+            "redirect_document_id": false
+        },
         {
             "source_path_from_root": "/articles/orbital/overview-analytics.md",
             "redirect_url": "/azure/orbital/overview",

articles/aks/TOC.yml

@@ -240,9 +240,9 @@
       items:
         - name: Deploy a cluster in an Edge Zone
           href: edge-zones.md
-        - name: Cluster configuration options
-          href: cluster-configuration.md 
-        - name: Manually scale nodes in an AKS cluster
+        - name: Deploy a cluster with a fully managed resource group
+          href: node-resource-group-lockdown.md
+        - name: Scale an AKS cluster
           href: scale-cluster.md
         - name: Stop and start an AKS cluster
           href: start-stop-cluster.md
@@ -749,6 +749,8 @@
           href: use-windows-hpc.md
         - name: Upgrade from Windows Server 2019 to 2022
           href: upgrade-windows-2019-2022.md
+        - name: Use generation 2 VMs
+          href: generation-2-vm-windows.md
         - name: Create Dockerfiles for Windows Server containers
           href: /virtualization/windowscontainers/manage-docker/manage-windows-dockerfile?context=/azure/aks/context/aks-context
         - name: Optimize Dockerfiles for Windows Server containers

articles/aks/cluster-configuration.md

@@ -49,15 +49,15 @@ The following limitations apply when you create AKS clusters that support multip
 1. Create an Azure resource group using the [`az group create`][az-group-create] command.
 
     ```azurecli-interactive
-    az group create --name myResourceGroup --location eastus
+    az group create --name $RESOURCE_GROUP_NAME --location $LOCATION
     ```
 
 2. Create an AKS cluster with a single node pool using the [`az aks create`][az-aks-create] command.
 
     ```azurecli-interactive
     az aks create \
-        --resource-group myResourceGroup \
-        --name myAKSCluster \
+        --resource-group $RESOURCE_GROUP_NAME \
+        --name $CLUSTER_NAME \
         --vm-set-type VirtualMachineScaleSets \
         --node-count 2 \
         --generate-ssh-keys \
@@ -69,7 +69,7 @@ The following limitations apply when you create AKS clusters that support multip
 3. When the cluster is ready, get the cluster credentials using the [`az aks get-credentials`][az-aks-get-credentials] command.
 
     ```azurecli-interactive
-    az aks get-credentials --resource-group myResourceGroup --name myAKSCluster
+    az aks get-credentials --resource-group $RESOURCE_GROUP_NAME --name $CLUSTER_NAME
     ```
 
 ## Add a node pool
@@ -80,16 +80,16 @@ The cluster created in the previous step has a single node pool. In this section
 
     ```azurecli-interactive
     az aks nodepool add \
-        --resource-group myResourceGroup \
-        --cluster-name myAKSCluster \
-        --name mynodepool \
+        --resource-group $RESOURCE_GROUP_NAME \
+        --cluster-name $CLUSTER_NAME \
+        --name $NODE_POOL_NAME \
         --node-count 3
     ```
 
 2. Check the status of your node pools using the [`az aks node pool list`][az-aks-nodepool-list] command and specify your resource group and cluster name.
 
     ```azurecli-interactive
-    az aks nodepool list --resource-group myResourceGroup --cluster-name myAKSCluster
+    az aks nodepool list --resource-group $RESOURCE_GROUP_NAME --cluster-name $CLUSTER_NAME
     ```
 
     The following example output shows *mynodepool* has been successfully created with three nodes. When the AKS cluster was created in the previous step, a default *nodepool1* was created with a node count of *2*.
@@ -134,9 +134,9 @@ The ARM64 processor provides low power compute for your Kubernetes workloads. To
 
     ```azurecli-interactive
     az aks nodepool add \
-        --resource-group myResourceGroup \
-        --cluster-name myAKSCluster \
-        --name armpool \
+        --resource-group $RESOURCE_GROUP_NAME \
+        --cluster-name $CLUSTER_NAME \
+        --name $ARM_NODE_POOL_NAME \
         --node-count 3 \
         --node-vm-size Standard_D2pds_v5
     ```
@@ -151,9 +151,9 @@ The Azure Linux container host for AKS is an open-source Linux distribution avai
 
     ```azurecli-interactive
     az aks nodepool add \
-        --resource-group myResourceGroup \
-        --cluster-name myAKSCluster \
-        --name azlinuxpool \
+        --resource-group $RESOURCE_GROUP_NAME \
+        --cluster-name $CLUSTER_NAME \
+        --name $AZ_LINUX_NODE_POOL_NAME \
         --os-sku AzureLinux
     ```
 
@@ -186,11 +186,11 @@ A workload may require splitting cluster nodes into separate pools for logical i
 
     ```azurecli-interactive
     az aks nodepool add \
-        --resource-group myResourceGroup \
-        --cluster-name myAKSCluster \
-        --name mynodepool \
+        --resource-group $RESOURCE_GROUP_NAME \
+        --cluster-name $CLUSTER_NAME \
+        --name $NODE_POOL_NAME \
         --node-count 3 \
-        --vnet-subnet-id <YOUR_SUBNET_RESOURCE_ID>
+        --vnet-subnet-id $SUBNET_RESOURCE_ID
     ```
 
 ## FIPS-enabled node pools
@@ -217,10 +217,10 @@ Beginning in Kubernetes version 1.20 and higher, you can specify `containerd` as
 
     ```azurecli-interactive
     az aks nodepool add \
-        --resource-group myResourceGroup \
-        --cluster-name myAKSCluster \
+        --resource-group $RESOURCE_GROUP_NAME \
+        --cluster-name $CLUSTER_NAME \
         --os-type Windows \
-        --name npwcd \
+        --name $CONTAINER_D_NODE_POOL_NAME \
         --node-vm-size Standard_D4s_v3 \
         --kubernetes-version 1.20.5 \
         --aks-custom-headers WindowsContainerRuntime=containerd \
@@ -233,9 +233,9 @@ Beginning in Kubernetes version 1.20 and higher, you can specify `containerd` as
 
     ```azurecli-interactive
     az aks nodepool upgrade \
-        --resource-group myResourceGroup \
-        --cluster-name myAKSCluster \
-        --name npwd \
+        --resource-group $RESOURCE_GROUP_NAME \
+        --cluster-name $CLUSTER_NAME \
+        --name $CONTAINER_D_NODE_POOL_NAME \
         --kubernetes-version 1.20.7 \
         --aks-custom-headers WindowsContainerRuntime=containerd
     ```
@@ -246,12 +246,29 @@ Beginning in Kubernetes version 1.20 and higher, you can specify `containerd` as
 
     ```azurecli-interactive
     az aks nodepool upgrade \
-        --resource-group myResourceGroup \
-        --cluster-name myAKSCluster \
+        --resource-group $RESOURCE_GROUP_NAME \
+        --cluster-name $CLUSTER_NAME \
         --kubernetes-version 1.20.7 \
         --aks-custom-headers WindowsContainerRuntime=containerd
     ```
 
+## Node pools with Ephemeral OS disks
+
+* Add a node pool that uses Ephemeral OS disks to an existing cluster using the [`az aks nodepool add`][az-aks-nodepool-add] command with the `--node-osdisk-type` flag set to `Ephemeral`.
+
+    > [!NOTE]
+    >
+    > * You can specify Ephemeral OS disks during cluster creation using the `--node-osdisk-type` flag with the [`az aks create`][az-aks-create] command.
+    > * If you want to create node pools with network-attached OS disks, you can do so by specifying `--node-osdisk-type Managed`.
+    >
+
+    ```azurecli-interactive
+    az aks nodepool add --name $EPHEMERAL_NODE_POOL_NAME --cluster-name $CLUSTER_NAME --resource-group $RESOURCE_GROUP_NAME -s Standard_DS3_v2 --node-osdisk-type Ephemeral
+    ```
+
+> [!IMPORTANT]
+> With Ephemeral OS, you can deploy VMs and instance images up to the size of the VM cache. The default node OS disk configuration in AKS uses 128 GB, which means that you need a VM size that has a cache larger than 128 GB. The default Standard_DS2_v2 has a cache size of 86 GB, which isn't large enough. The Standard_DS3_v2 VM SKU has a cache size of 172 GB, which is large enough. You can also reduce the default size of the OS disk by using `--node-osdisk-size`, but keep in mind the minimum size for AKS images is 30 GB.
+
 ## Delete a node pool
 
 If you no longer need a node pool, you can delete it and remove the underlying VM nodes.
@@ -262,7 +279,7 @@ If you no longer need a node pool, you can delete it and remove the underlying V
 * Delete a node pool using the [`az aks nodepool delete`][az-aks-nodepool-delete] command and specify the node pool name.
 
     ```azurecli-interactive
-    az aks nodepool delete -g myResourceGroup --cluster-name myAKSCluster --name mynodepool --no-wait
+    az aks nodepool delete --resource-group $RESOURCE_GROUP_NAME --cluster-name $CLUSTER_NAME --name $NODE_POOL_NAME --no-wait
     ```
 
     It takes a few minutes to delete the nodes and the node pool.
@@ -279,7 +296,6 @@ In this article, you learned how to create multiple node pools in an AKS cluster
 [az-aks-get-credentials]: /cli/azure/aks#az_aks_get_credentials
 [az-aks-create]: /cli/azure/aks#az_aks_create
 [az-aks-update]: /cli/azure/aks#az_aks_update
-[az-aks-delete]: /cli/azure/aks#az_aks_delete
 [az-aks-nodepool]: /cli/azure/aks/nodepool
 [az-aks-nodepool-add]: /cli/azure/aks/nodepool#az_aks_nodepool_add
 [az-aks-nodepool-list]: /cli/azure/aks/nodepool#az_aks_nodepool_list

articles/aks/concepts-clusters-workloads.md

@@ -0,0 +1,119 @@
+---
+title: Use generation 2 virtual machines on Windows in Azure Kubernetes Service (AKS)
+description: Learn how to use generation 2 virtual machines on Windows in Azure Kubernetes Service (AKS).
+ms.topic: article
+ms.custom: azure-kubernetes-service
+ms.date: 01/23/2024
+ms.author: schaffererin
+author: schaffererin
+---
+
+# Use generation 2 virtual machines on Windows in Azure Kubernetes Service (AKS) (preview)
+
+Azure supports [Generation 2 (Gen 2) virtual machines (VMs)](../virtual-machines/generation-2.md). Gen 2 VMs support key features not supported in Generation 1 (Gen 1) VMs, including increased memory, Intel Software Guard Extensions (Intel SGX), and virtualized persistent memory (vPMEM).
+
+Gen 2 VMs use the new UEFI-based boot architecture rather than the BIOS-based architecture used by Gen 1 VMs. Only specific SKUs and sizes support Gen 2 VMs. Check the [list of supported sizes](../virtual-machines/generation-2.md#generation-2-vm-sizes) to see if your SKU supports or requires Gen 2.
+
+Additionally, not all VM images support Gen 2 VMs. On AKS, Gen 2 VMs use the AKS Ubuntu 22.04 or 18.04 image or the AKS Windows Server 2022 image. These images support all Gen 2 SKUs and sizes.
+
+[!INCLUDE [preview features callout](includes/preview/preview-callout.md)]
+
+## Before you begin
+
+Before you begin, you need the following resources installed and configured:
+
+* The Azure CLI version 2.44.0 or later. Run `az --version` to find the current version. If you need to install or upgrade, see [Install Azure CLI][azure-cli-install].
+* The `aks-preview` extension version 0.5.126 or later.
+* The `AKSWindows2022Gen2Preview` feature flag registered on your subscription.
+* Generation 2 VMs are supported on Windows for Windows Server 2022 (WS2022) only.
+* Generation 2 VMs are default for Windows clusters running Kubernetes 1.25 or later.
+
+### Install the `aks-preview` Azure CLI extension
+
+* Install or update the aks-preview Azure CLI extension using the [`az extension add`][az-extension-add] or the [`az extension update`][az-extension-update] command.
+
+    ```azurecli-interactive
+    # Install the aks-preview extension
+    az extension add --name aks-preview
+
+    # Update to the latest version of the aks-preview extension
+    az extension update --name aks-preview
+    ```
+
+### Register the `AKSWindows2022Gen2Preview` feature flag
+
+1. Register the `AKSWindows2022Gen2Preview` feature flag using the [`az feature register`][az-feature-register] command.
+
+    ```azurecli-interactive
+    az feature register --namespace "Microsoft.ContainerService" --name "AKSWindows2022Gen2Preview"
+    ```
+
+    It takes a few minutes for the status to show *Registered*.
+
+2. Verify the registration using the [`az feature show`][az-feature-show] command.
+
+    ```azurecli-interactive
+    az feature show --namespace "Microsoft.ContainerService" --name "AKSWindows2022Gen2Preview"
+    ```
+
+3. When the status reflects *Registered*, refresh the registration of the `Microsoft.ContainerService` resource provider using the [`az provider register`][az-provider-register] command.
+
+    ```azurecli-interactive
+    az provider register --namespace "Microsoft.ContainerService"
+    ```
+
+## Create a Windows node pool with a Generation 2 VM
+
+1. Check available Generation 2 VM sizes using the [`az vm list`][az-vm-list] command.
+
+    ```azurecli-interactive
+    az vm list -skus --location <location> --size <vm-size> --output table
+    ```
+
+2. Create a Windows node pool with a Generation 2 VM using the [`az aks nodepool add`][az-aks-nodepool-add] command.
+
+    ```azurecli-interactive
+    az aks nodepool add --resource-group <resource-group-name> --cluster-name <cluster-name> --name <node-pool-name> --os-type Windows --os-sku Windows2022
+    ```
+
+3. Verify a successful node pool creation using the [`az aks nodepool show`][az-aks-nodepool-show] command and check that the `nodeImageVersion` contains `gen2` in the output.
+
+    ```azurecli-interactive
+    az aks nodepool show --resource-group <resource-group-name> --cluster-name <cluster-name> --name <node-pool-name>
+    ```
+
+## Update a Windows node pool to a Generation 2 VM
+
+1. Check available Generation 2 VM sizes using the [`az vm list`][az-vm-list] command.
+
+    ```azurecli-interactive
+    az vm list -skus --location <location> --size <vm-size> --output table
+    ```
+
+2. Update a Windows node pool to a Generation 2 VM using the [`az aks nodepool update`][az-aks-nodepool-update] command.
+
+    ```azurecli-interactive
+    az aks nodepool update --resource-group <resource-group-name> --cluster-name <cluster-name> --name <node-pool-name> --os-type Windows --os-sku Windows2022
+    ```
+
+3. Verify a successful node pool update using the [`az aks nodepool show`][az-aks-nodepool-show] command and check that the `nodeImageVersion` contains `gen2` in the output.
+
+    ```azurecli-interactive
+    az aks nodepool show --resource-group <resource-group-name> --cluster-name <cluster-name> --name <node-pool-name>
+    ```
+
+## Next steps
+
+To learn more about Generation 2 VMs, see [Support for Generation 2 VMs on Azure](../virtual-machines/generation-2.md).
+
+<!-- LINKS -->
+[azure-cli-install]: /cli/azure/install-azure-cli
+[az-aks-nodepool-add]: /cli/azure/aks/nodepool#az_aks_nodepool_add
+[az-aks-nodepool-show]: /cli/azure/aks/nodepool#az_aks_nodepool_show
+[az-aks-nodepool-update]: /cli/azure/aks/nodepool#az_aks_nodepool_update
+[az-extension-add]: /cli/azure/extension#az_extension_add
+[az-extension-update]: /cli/azure/extension#az_extension_update
+[az-feature-register]: /cli/azure/feature#az_feature_register
+[az-feature-show]: /cli/azure/feature#az_feature_show
+[az-provider-register]: /cli/azure/provider#az_provider_register
+[az-vm-list]: /cli/azure/vm#az_vm_list

articles/aks/create-node-pools.md

@@ -59,7 +59,7 @@ An [Azure resource group](../../azure-resource-manager/management/overview.md) i
 
 In this section, we create an AKS cluster with the following configuration:
 
-- The cluster is configured with two nodes to ensure it operates reliably. A [node](../concepts-clusters-workloads.md#nodes-and-node-pools) is an Azure virtual machine (VM) that runs the Kubernetes node components and container runtime.
+- The cluster is configured with two nodes to ensure it operates reliably. A [node](../concepts-clusters-workloads.md#nodes) is an Azure virtual machine (VM) that runs the Kubernetes node components and container runtime.
 - The `--windows-admin-password` and `--windows-admin-username` parameters set the administrator credentials for any Windows Server nodes on the cluster and must meet [Windows Server password requirements][windows-server-password].
 - The node pool uses `VirtualMachineScaleSets`.
 
@@ -71,13 +71,13 @@ To create the AKS cluster with Azure CLI, follow these steps:
     echo "Please enter the username to use as administrator credentials for Windows Server nodes on your cluster: " && read WINDOWS_USERNAME
     ```
 
-1. Create a password for the administrator username you created in the previous step. The password must be a minimum of 14 characters and meet the [Windows Server password complexity requirements][windows-server-password].
+2. Create a password for the administrator username you created in the previous step. The password must be a minimum of 14 characters and meet the [Windows Server password complexity requirements][windows-server-password].
 
     ```azurecli
     echo "Please enter the password to use as administrator credentials for Windows Server nodes on your cluster: " && read WINDOWS_PASSWORD
     ```
 
-1. Create your cluster using the [az aks create][az-aks-create] command and specify the `--windows-admin-username` and `--windows-admin-password` parameters. The following example command creates a cluster using the value from *WINDOWS_USERNAME* you set in the previous command. Alternatively, you can provide a different username directly in the parameter instead of using *WINDOWS_USERNAME*.
+3. Create your cluster using the [az aks create][az-aks-create] command and specify the `--windows-admin-username` and `--windows-admin-password` parameters. The following example command creates a cluster using the value from *WINDOWS_USERNAME* you set in the previous command. Alternatively, you can provide a different username directly in the parameter instead of using *WINDOWS_USERNAME*.
 
     ```azurecli
     az aks create \

articles/aks/generation-2-vm-windows.md

@@ -53,7 +53,7 @@ ResourceId        : /subscriptions/00000000-0000-0000-0000-000000000000/resource
 
 In this section, we create an AKS cluster with the following configuration:
 
-- The cluster is configured with two nodes to ensure it operates reliably. A [node](../concepts-clusters-workloads.md#nodes-and-node-pools) is an Azure virtual machine (VM) that runs the Kubernetes node components and container runtime.
+- The cluster is configured with two nodes to ensure it operates reliably. A [node](../concepts-clusters-workloads.md#nodes) is an Azure virtual machine (VM) that runs the Kubernetes node components and container runtime.
 - The `-WindowsProfileAdminUserName` and `-WindowsProfileAdminUserPassword` parameters set the administrator credentials for any Windows Server nodes on the cluster and must meet the [Windows Server password complexity requirements][windows-server-password].
 - The node pool uses `VirtualMachineScaleSets`.
 
@@ -66,7 +66,7 @@ To create the AKS cluster with Azure PowerShell, follow these steps:
         -Message 'Please create the administrator credentials for your Windows Server containers'
     ```
 
-1. Create your cluster using the [New-AzAksCluster][new-azakscluster] cmdlet and specify the `WindowsProfileAdminUserName` and `WindowsProfileAdminUserPassword` parameters.
+2. Create your cluster using the [New-AzAksCluster][new-azakscluster] cmdlet and specify the `WindowsProfileAdminUserName` and `WindowsProfileAdminUserPassword` parameters.
 
     ```azurepowershell
     New-AzAksCluster -ResourceGroupName myResourceGroup `