Merge pull request #206388 from zr-msft/zr-aks-cvm

[AKS] add CVM doc

Author: BCS2022

articles/aks/TOC.yml

@@ -221,6 +221,8 @@
               href: use-multiple-node-pools.md
             - name: Use spot node pools
               href: spot-node-pool.md
+            - name: Use CVM
+              href: use-cvm.md
             - name: Use system node pools
               href: use-system-pools.md
             - name: Use WebAssembly System Interface (WASI) node pools

articles/aks/index.yml

@@ -26,6 +26,8 @@ landingContent:
             url: intro-kubernetes.md
       - linkListType: whats-new
         links:
+          - text: Use CVM (Preview)
+            url: use-cvm.md
           - text: Automatically upgrade an AKS cluster
             url: auto-upgrade-cluster.md
           - text: Start/stop node pools

articles/aks/use-cvm.md

@@ -0,0 +1,89 @@
+---
+title: Use Confidential Virtual Machines (CVM) in Azure Kubernetes Service (AKS) (Preview)
+description: Learn how to create Confidential Virtual Machines (CVM) node pools with Azure Kubernetes Service (AKS)
+services: container-service
+ms.topic: article
+ms.date: 08/01/2022
+
+---
+
+# Use Confidential Virtual Machines (CVM) in Azure Kubernetes Service (AKS) cluster (Preview)
+
+You can use the generally available [confidential VM sizes (DCav5/ECav5)][cvm-announce] to add a node pool to your AKS cluster with CVM. Confidential VMs with AMD SEV-SNP support bring a new set of security features to protect date-in-use with full VM memory encryption. These features enable node pools with CVM to target the migration of highly sensitive container workloads to AKS without any code refactoring while benefiting from the features of AKS. The nodes in a node pool created with CVM use a customized Ubuntu 20.04 image specially configured for CVM. For more details on CVM, see [Confidential VM node pools support on AKS with AMD SEV-SNP confidential VMs][cvm].
+
+Adding a node pool with CVM to your AKS cluster is currently in preview.
+
+[!INCLUDE [preview features callout](./includes/preview/preview-callout.md)]
+
+## Before you begin
+
+- An Azure subscription. If you don't have an Azure subscription, you can create a [free account](https://azure.microsoft.com/free).
+- [Azure CLI installed](/cli/azure/install-azure-cli).
+- An existing AKS cluster in the *westus*, *eastus*, *westeurope*, or *northeurope* region.
+- The [DCasv5 and DCadsv5-series][cvm-subs-dc] or [ECasv5 and ECadsv5-series][cvm-subs-ec] SKUs available for your subscription.
+
+## Limitations
+
+The following limitations apply when adding a node pool with CVM to AKS:
+
+- You can't use `--enable-fips-image`, ARM64, or Mariner.
+- You can't upgrade an existing node pool to use CVM.
+- The [DCasv5 and DCadsv5-series][cvm-subs-dc] or [ECasv5 and ECadsv5-series][cvm-subs-ec] SKUs must be available for your subscription in the region where the cluster is created.
+
+## Add a node pool with the CVM to AKS
+
+To add a node pool with the CVM to AKS, use `az aks nodepool add` and set `node-vm-size` to `Standard_DCa4_v5`. For example:
+
+```azurecli-interactive
+az aks nodepool add \
+    --resource-group myResourceGroup \
+    --cluster-name myAKSCluster \
+    --name cvmnodepool \
+    --node-count 3 \
+    --node-vm-size Standard_DC4as_v5 
+```
+
+## Verify the node pool uses CVM
+
+To verify a node pool uses CVM, use `az aks nodepool show` and verify the `vmSize` is `Standard_DCa4_v5`. For example:
+
+```azurecli-interactive
+az aks nodepool show \
+    --resource-group myResourceGroup \
+    --cluster-name myAKSCluster \
+    --name cvmnodepool \
+    --query 'vmSize'
+```
+
+The following example command and output shows the node pool uses CVM:
+
+```output
+az aks nodepool show \
+    --resource-group myResourceGroup \
+    --cluster-name myAKSCluster \
+    --name cvmnodepool \
+    --query 'vmSize'
+
+"Standard_DC4as_v5"
+```
+
+## Remove a node pool with CVM from an AKS cluster
+
+To remove a node pool with CVM from an AKS cluster, use `az aks nodepool delete`. For example:
+
+```azurecli-interactive
+az aks nodepool delete \
+    --resource-group myResourceGroup \
+    --cluster-name myAKSCluster \
+    --name cvmnodepool
+```
+
+## Next steps
+
+In this article, you learned how to add a node pool with CVM to an AKS cluster. For more information about CVM, see [Confidential VM node pools support on AKS with AMD SEV-SNP confidential VMs][cvm].
+
+<!-- LINKS - Internal -->
+[cvm]: ../confidential-computing/confidential-node-pool-aks.md
+[cvm-announce]: https://techcommunity.microsoft.com/t5/azure-confidential-computing/azure-confidential-vms-using-sev-snp-dcasv5-ecasv5-are-now/ba-p/3573747
+[cvm-subs-dc]: ../virtual-machines/dcasv5-dcadsv5-series.md
+[cvm-subs-ec]: ../virtual-machines/ecasv5-ecadsv5-series.md

articles/aks/use-multiple-node-pools.md

@@ -170,22 +170,6 @@ az aks nodepool add \
     --node-vm-size Standard_Dpds_v5
 ```
 
-### Add a confidential VM (with AMD SEV-SNP support) node pool (preview)
-AKS node pools now support the generally available [confidential VM sizes (DCav5/ECav5)](https://aka.ms/AMD-ACC-VMs-GA-Inspire-2022) to create confidential VM node pools. Confidential VMs with AMD SEV-SNP support bring a new set of security features to protect date-in-use with full VM memory encryption. This enables confidential VM node pools to target the migration of highly sensitive container workloads to AKS without any code refactoring while benefiting from the full AKS feature support. To learn more, check out our [latest offering](../confidential-computing/confidential-node-pool-aks.md).
-
-[!INCLUDE [preview features callout](./includes/preview/preview-callout.md)]
-
-Add a confidential node pool using the [az aks nodepool add][az-aks-nodepool-add] command. Specify the name *cvmnodepool*, and use the `--node-vm-size` parameter to specify the *Standard_DC2as_v5* size:
-
-```azurecli-interactive
-az aks nodepool add \
-    --resource-group myResourceGroup \
-    --cluster-name myAKSCluster \
-    --name cvmnodepool \
-    --node-count 3 \
-    --node-vm-size Standard_DC2as_v5 \
-```
-
 ### Add a node pool with a unique subnet
 
 A workload may require splitting a cluster's nodes into separate pools for logical isolation. This isolation can be supported with separate subnets dedicated to each node pool in the cluster. This can address requirements such as having non-contiguous virtual network address space to split across node pools.
@@ -197,7 +181,7 @@ A workload may require splitting a cluster's nodes into separate pools for logic
 
 * All subnets assigned to node pools must belong to the same virtual network.
 * System pods must have access to all nodes/pods in the cluster to provide critical functionality such as DNS resolution and tunneling kubectl logs/exec/port-forward proxy.
-* If you expand your VNET after creating the cluster you must update your cluster (perform any managed cluster operation but node pool operations don't count) before adding a subnet outside the original cidr. AKS will error out on the agent pool add now though we originally allowed it. The `aks-preview` Azure CLI extension (version 0.5.66+) now supports running `az aks update -g <resourceGroup> -n <clusterName>` without any optional arguments. This command will perform an update operation without making any changes, which can recover a cluster stuck in a failed state.
+* If you expand your VNET after creating the cluster you must update your cluster (perform any managed cluster operation but node pool operations don't count) before adding a subnet outside the original cidr. AKS will error-out on the agent pool add now though we originally allowed it. The `aks-preview` Azure CLI extension (version 0.5.66+) now supports running `az aks update -g <resourceGroup> -n <clusterName>` without any optional arguments. This command will perform an update operation without making any changes, which can recover a cluster stuck in a failed state.
 * In clusters with Kubernetes version < 1.23.3, kube-proxy will SNAT traffic from new subnets, which can cause Azure Network Policy to drop the packets.
 * Windows nodes will SNAT traffic to the new subnets until the node pool is reimaged.
 * Internal load balancers default to one of the node pool subnets (usually the first subnet of the node pool at cluster creation). To override this behavior, you can [specify the load balancer's subnet explicitly using an annotation][internal-lb-different-subnet].

articles/confidential-computing/confidential-node-pool-aks.md

@@ -26,13 +26,13 @@ In addition to the hardened security profile, confidential node pools on AKS als
 
 :::image type="content" source="media/confidential-vm-node-pools-on-aks/snp-on-aks-architecture-image.png" alt-text="Graphic of VM nodes in AKS with encrypted code and data in confidential VM node pools 1 and 2, on top of the hypervisor":::
 
-Get started and add confidential node pools to existing AKS cluster with [this quick start guide](../aks/use-multiple-node-pools.md#add-a-confidential-vm-with-amd-sev-snp-support-node-pool-preview).
+Get started and add confidential node pools to existing AKS cluster with [this quick start guide](../aks/use-cvm.md).
 
 ## Questions?
 
 If you have questions about container offerings, please reach out to <[email protected]>.
 
 ## Next steps
 
-- [Deploy a confidential node pool in your AKS cluster](../aks/use-multiple-node-pools.md#add-a-confidential-vm-with-amd-sev-snp-support-node-pool-preview)
+- [Deploy a confidential node pool in your AKS cluster](../aks/use-cvm.md)
 - Learn more about sizes and specs for [general purpose](../virtual-machines/dcasv5-dcadsv5-series.md) and [memory-optimized](../virtual-machines/ecasv5-ecadsv5-series.md) confidential VMs.