Merge pull request #279547 from schaffererin/new-userstory261623

Core concepts for Azure Kubernetes Service (AKS) - Doc rewrite

Author: v-shils

articles/aks/.openpublishing.redirection.aks.json

@@ -21,6 +21,11 @@
             "redirect_url": "/azure/aks/ha-dr-overview",
             "redirect_document_id": false
         },
+        {
+            "source_path_from_root": "/articles/aks/concepts-clusters-workloads.md",
+            "redirect_url": "/azure/aks/core-aks-concepts",
+            "redirect_document_id": false
+        },
         {
             "source_path_from_root": "/articles/aks/upgrade-windows-2019-2022.md",
             "redirect_url": "/azure/aks/upgrade-windows-os",

articles/aks/TOC.yml

@@ -92,11 +92,13 @@
 - name: Concepts
   items:
     - name: Core concepts
-      href: concepts-clusters-workloads.md
+      href: core-aks-concepts.md
     - name: Pricing tiers for AKS
       href: free-standard-pricing-tiers.md
     - name: Maintain and upgrade an AKS cluster
       href: upgrade.md
+    - name: Resource reservations
+      href: node-resource-reservations.md
     - name: Backup and recover cluster
       href: ../backup/azure-kubernetes-service-backup-overview.md?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json
     - name: Observability

articles/aks/concepts-clusters-workloads.md

@@ -28,6 +28,20 @@ This article introduces the core concepts that provide storage to your applicati
 
 ![Diagram of storage options for applications in an Azure Kubernetes Services (AKS) cluster.](media/concepts-storage/aks-storage-concept.png)
 
+## Default OS disk sizing
+
+When you create a new cluster or add a new node pool to an existing cluster, the number for vCPUs by default determines the OS disk size. The number of vCPUs is based on the VM SKU. The following table lists the default OS disk size for each VM SKU:
+
+|VM SKU Cores (vCPUs)| Default OS Disk Tier | Provisioned IOPS | Provisioned Throughput (Mbps) |
+|--|--|--|--|
+| 1 - 7 | P10/128G | 500 | 100 |
+| 8 - 15 | P15/256G | 1100 | 125 |
+| 16 - 63 | P20/512G | 2300 | 150 |
+| 64+ | P30/1024G | 5000 | 200 |
+
+> [!IMPORTANT]
+> Default OS disk sizing is only used on new clusters or node pools when Ephemeral OS disks aren't supported and a default OS disk size isn't specified. The default OS disk size might impact the performance or cost of your cluster. You can't change the OS disk size after cluster or node pool creation. This default disk sizing affects clusters or node pools created on July 2022 or later.
+
 ## Ephemeral OS disk
 
 By default, Azure automatically replicates the operating system disk for a virtual machine to Azure Storage to avoid data loss when the VM is relocated to another host. However, since containers aren't designed to have local state persisted, this behavior offers limited value while providing some drawbacks. These drawbacks include, but aren't limited to, slower node provisioning and higher read/write latency.

articles/aks/concepts-storage.md

@@ -0,0 +1,177 @@
+---
+title: Azure Kubernetes Services (AKS) core concepts
+description: Learn about the core concepts of Azure Kubernetes Service (AKS).
+ms.topic: conceptual
+ms.date: 07/10/2024
+author: schaffererin
+ms.author: schaffererin
+---
+
+# Core concepts for Azure Kubernetes Service (AKS)
+
+This article describes core concepts of Azure Kubernetes Service (AKS), a managed Kubernetes service that you can use to deploy and operate containerized applications at scale on Azure.
+
+## What is Kubernetes?
+
+Kubernetes is an open-source container orchestration platform for automating the deployment, scaling, and management of containerized applications. For more information, see the official [Kubernetes documentation][kubernetes-docs].
+
+## What is AKS?
+
+AKS is a managed Kubernetes service that simplifies deploying, managing, and scaling containerized applications using Kubernetes. For more information, see [What is Azure Kubernetes Service (AKS)?][aks-overview]
+
+## Cluster components
+
+An AKS cluster is divided into two main components:
+
+* **Control plane**: The control plane provides the core Kubernetes services and orchestration of application workloads.
+* **Nodes**: Nodes are the underlying virtual machines (VMs) that run your applications.
+
+![Screenshot of Kubernetes control plane and node components](media/concepts-clusters-workloads/control-plane-and-nodes.png)
+
+### Control plane
+
+The Azure managed control plane is comprised of several components that help manage the cluster:
+
+| Component | Description |  
+| --------- | ----------- |  
+| *kube-apiserver* | The API server ([kube-apiserver][kube-apiserver]) exposes the Kubernetes API to enable requests to the cluster from inside and outside of the cluster. |  
+| *etcd* | [etcd][etcd] is a highly available key-value store that helps maintain the state of your Kubernetes cluster and configuration. |  
+| *kube-scheduler* | The scheduler ([kube-scheduler][kube-scheduler]) helps make scheduling decisions, watching for new pods with no assigned node and selecting a node for them to run on. |  
+| *kube-controller-manager* | The controller manager ([kube-controller-manager][kube-controller-manager]) runs controller processes, such as noticing and responding when nodes go down. |
+| *cloud-controller-manager* | The cloud controller manager ([cloud-controller-manager][cloud-controller-manager]) embeds cloud-specific control logic to run controllers specific to the cloud provider. |
+
+### Nodes
+
+Each AKS cluster has at least one node, which is an Azure virtual machine (VM) that runs Kubernetes node components. The following components run on each node:
+
+| Component | Description |
+| --------- | ----------- |
+| *kubelet* | The [kubelet][kubelet] ensures that containers are running in a pod. |
+| *kube-proxy* | The [kube-proxy][kube-proxy] is a network proxy that maintains network rules on nodes. |
+| *container runtime* | The [container runtime][container-runtime] manages the execution and lifecycle of containers. |
+
+![Screenshot of Azure virtual machine and supporting resources for a Kubernetes node](media/concepts-clusters-workloads/aks-node-resource-interactions.png)
+
+## Node configuration
+
+### VM size and image
+
+The **Azure VM size** for your nodes defines CPUs, memory, size, and the storage type available, such as high-performance SSD or regular HDD. The VM size you choose depends on the workload requirements and the number of pods you plan to run on each node. For more information, see [Supported VM sizes in Azure Kubernetes Service (AKS)][aks-vm-sizes].
+
+In AKS, the **VM image** for your cluster's nodes is based on Ubuntu Linux, [Azure Linux](use-azure-linux.md), or Windows Server 2022. When you create an AKS cluster or scale out the number of nodes, the Azure platform automatically creates and configures the requested number of VMs. Agent nodes are billed as standard VMs, so any VM size discounts, including [Azure reservations][reservation-discounts], are automatically applied.
+
+### OS disks
+
+Default OS disk sizing is only used on new clusters or node pools when Ephemeral OS disks aren't supported and a default OS disk size isn't specified. For more information, see [Default OS disk sizing][default-os-disk] and [Ephemeral OS disks][ephemeral-os-disks].
+
+### Resource reservations
+
+AKS uses node resources to help the nodes function as part of the cluster. This usage can cause a discrepancy between the node's total resources and the allocatable resources in AKS. To maintain node performance and functionality, AKS reserves two types of resources, **CPU** and **memory**, on each node. For more information, see [Resource reservations in AKS][resource-reservations].
+
+### OS
+
+AKS supports Ubuntu 22.04 and Azure Linux 2.0 as the node OS for Linux node pools. For Windows node pools, AKS supports Windows Server 2022 as the default OS. Windows Server 2019 is being retired after Kubernetes version 1.32 reaches end of life and isn't supported in future releases. If you need to upgrade your Windows OS version, see [Upgrade from Windows Server 2019 to Windows Server 2022][upgrade-2019-2022]. For more information on using Windows Server on AKS, see [Windows container considerations in Azure Kubernetes Service (AKS)][windows-considerations].
+
+### Container runtime
+
+A container runtime is software that executes containers and manages container images on a node. The runtime helps abstract away sys-calls or OS-specific functionality to run containers on Linux or Windows. For Linux node pools, [`containerd`][containerd] is used on Kubernetes version 1.19 and higher. For Windows Server 2019 and 2022 node pools, [`containerd`][containerd] is generally available and is the only runtime option on Kubernetes version 1.23 and higher.
+
+## Pods
+
+A *pod* is a group of one or more containers that share the same network and storage resources and a specification for how to run the containers. Pods typically have a 1:1 mapping with a container, but you can run multiple containers in a pod.
+
+## Node pools
+
+In AKS, nodes of the same configuration are grouped together into *node pools*. These node pools contain the underlying virtual machine scale sets and virtual machines (VMs) that run your applications. When you create an AKS cluster, you define the initial number of nodes and their size (SKU), which creates a [*system node pool*][use-system-pool]. System node pools serve the primary purpose of hosting critical system pods, such as CoreDNS and `konnectivity`. To support applications that have different compute or storage demands, you can create *user node pools*. User node pools serve the primary purpose of hosting your application pods.
+
+For more information, see [Create node pools in AKS][create-node-pools] and [Manage node pools in AKS][manage-node-pools].
+
+## Node resource group
+
+When you create an AKS cluster in an Azure resource group, the AKS resource provider automatically creates a second resource group called the *node resource group*. This resource group contains all the infrastructure resources associated with the cluster, including virtual machines (VMs), virtual machine scale sets, and storage.
+
+For more information, see the following resources:
+
+* [Why are two resource groups created with AKS?][node-resource-group]
+* [Can I provide my own name for the AKS node resource group?][custom-nrg]
+* [Can I modify tags and other properties of the resources in the AKS node resource group?][modify-nrg-resources]
+
+## Namespaces
+
+Kubernetes resources, such as pods and deployments, are logically grouped into *namespaces* to divide an AKS cluster and create, view, or manage access to resources.
+
+The following namespaces are created by default in an AKS cluster:
+
+| Namespace | Description |  
+| --------- | ----------- |  
+| *default* | The [default][kubernetes-namespaces] namespace allows you to start using cluster resources without creating a new namespace. |
+| *kube-node-lease* | The [kube-node-lease][kubernetes-namespaces] namespace enables nodes to communicate their availability to the control plane. |
+| *kube-public* | The [kube-public][kubernetes-namespaces] namespace isn't typically used, but can be used for resources to be visible across the whole cluster by any user. |
+| *kube-system* | The [kube-system][kubernetes-namespaces] namespace is used by Kubernetes to manage cluster resources, such as `coredns`, `konnectivity-agent`, and `metrics-server`. |
+
+![Screenshot of Kubernetes namespaces to logically divide resources and applications](media/concepts-clusters-workloads/namespaces.png)
+
+## Cluster modes
+
+In AKS, you can create a cluster with the **Automatic (preview)** or **Standard** mode. AKS Automatic provides a more fully managed experience, managing cluster configuration, including nodes, scaling, security, and other preconfigured settings. AKS Standard provides more control over the cluster configuration, including the ability to manage node pools, scaling, and other settings.
+
+For more information, see [AKS Automatic and Standard feature comparison][automatic-standard].
+
+## Pricing tiers
+
+AKS offers three pricing tiers for cluster management: **Free**, **Standard**, and **Premium**. The pricing tier you choose determines the features available for managing your cluster.
+
+For more information, see [Pricing tiers for AKS cluster management][pricing-tiers].
+
+## Supported Kubernetes versions
+
+For more information, see [Supported Kubernetes versions in AKS][supported-kubernetes-versions].
+
+## Next steps
+
+For information on more core concepts for AKS, see the following resources:
+
+* [AKS access and identity][access-identity]
+* [AKS security][security]
+* [AKS networking][networking]
+* [AKS storage][storage]
+* [AKS scaling][scaling]
+* [AKS monitoring][monitoring]
+* [AKS backup and recovery][backup-recovery]
+
+<!---LINKS--->
+[kube-apiserver]: https://kubernetes.io/docs/concepts/overview/components/#kube-apiserver
+[etcd]: https://kubernetes.io/docs/concepts/overview/components/#etcd
+[kube-scheduler]: https://kubernetes.io/docs/concepts/overview/components/#kube-scheduler
+[kube-controller-manager]: https://kubernetes.io/docs/concepts/overview/components/#kube-controller-manager
+[cloud-controller-manager]: https://kubernetes.io/docs/concepts/overview/components/#cloud-controller-manager
+[kubelet]: https://kubernetes.io/docs/concepts/overview/components/#kubelet
+[kube-proxy]: https://kubernetes.io/docs/concepts/overview/components/#kube-proxy
+[container-runtime]: https://kubernetes.io/docs/concepts/overview/components/#container-runtime
+[create-node-pools]: ./create-node-pools.md
+[manage-node-pools]: ./manage-node-pools.md
+[node-resource-group]: ./faq.md#why-are-two-resource-groups-created-with-aks
+[custom-nrg]: ./faq.md#can-i-provide-my-own-name-for-the-aks-node-resource-group
+[modify-nrg-resources]: ./faq.md#can-i-modify-tags-and-other-properties-of-the-aks-resources-in-the-node-resource-group
+[kubernetes-namespaces]: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/#initial-namespaces
+[use-system-pool]: ./use-system-pools.md
+[automatic-standard]: ./intro-aks-automatic.md#aks-automatic-and-standard-feature-comparison
+[pricing-tiers]: ./free-standard-pricing-tiers.md
+[access-identity]: ./concepts-identity.md
+[security]: ./concepts-security.md
+[networking]: ./concepts-network.md
+[storage]: ./concepts-storage.md
+[scaling]: ./concepts-scale.md
+[monitoring]: ./monitor-aks.md
+[backup-recovery]: ../backup/azure-kubernetes-service-backup-overview.md
+[kubernetes-docs]: https://kubernetes.io/docs/home/
+[resource-reservations]: ./node-resource-reservations.md
+[reservation-discounts]: ../cost-management-billing/reservations/save-compute-costs-reservations.md
+[supported-kubernetes-versions]: ./supported-kubernetes-versions.md
+[default-os-disk]: ./concepts-storage.md#default-os-disk-sizing
+[ephemeral-os-disks]: ./concepts-storage.md#ephemeral-os-disk
+[aks-overview]: ./what-is-aks.md
+[containerd]: https://containerd.io/
+[aks-vm-sizes]: ./quotas-skus-regions.md#supported-vm-sizes
+[windows-considerations]: ./windows-vs-linux-containers.md
+[upgrade-2019-2022]: ./upgrade-windows-os.md

articles/aks/core-aks-concepts.md

@@ -0,0 +1,74 @@
+---
+title: Node resource reservations in Azure Kubernetes Service (AKS)
+description: Learn about node resource reservations in Azure Kubernetes Service (AKS).
+ms.topic: how-to
+ms.service: azure-kubernetes-service
+ms.date: 04/16/2024
+ms.author: schaffererin
+author: schaffererin
+---
+
+# Node resource reservations in Azure Kubernetes Service (AKS)
+
+In this article, you learn about node resource reservations in Azure Kubernetes Service (AKS).
+
+## Resource reservations
+
+AKS uses node resources to help the nodes function as part of the cluster. This usage can cause a discrepancy between the node's total resources and the allocatable resources in AKS.
+
+AKS reserves two types of resources, **CPU** and **memory**, on each node to maintain node performance and functionality. As a node grows larger in resources, the resource reservations also grow due to a higher need for management of user-deployed pods. Keep in mind that you can't change resource reservations on a node.
+
+### CPU reservations
+
+Reserved CPU is dependent on node type and cluster configuration, which might result in less allocatable CPU due to running extra features. The following table shows CPU reservations in millicores:
+
+| CPU cores on host | 1 core | 2 cores | 4 cores | 8 cores | 16 cores | 32 cores | 64 cores |
+| ----------------- | ------ | ------- | ------- | ------- | -------- | -------- | -------- |
+| Kube-reserved CPU (millicores) | 60 | 100 | 140 | 180 | 260 | 420 | 740 |
+
+### Memory reservations
+
+In AKS, reserved memory consists of the sum of two values:
+
+**AKS 1.29 and later**
+
+* **`kubelet` daemon** has the *memory.available < 100 Mi* eviction rule by default. This rule ensures that a node has at least 100 Mi allocatable at all times. When a host is below that available memory threshold, the `kubelet` triggers the termination of one of the running pods and frees up memory on the host machine.
+* **A rate of memory reservations** set according to the lesser value of: *20 MB * Max Pods supported on the Node + 50 MB* or *25% of the total system memory resources*.
+
+    **Examples**:
+    * If the virtual machine (VM) provides 8 GB of memory and the node supports up to 30 pods, AKS reserves *20 MB * 30 Max Pods + 50 MB = 650 MB* for kube-reserved. `Allocatable space = 8 GB - 0.65 GB (kube-reserved) - 0.1 GB (eviction threshold) = 7.25 GB or 90.625% allocatable.`
+    * If the VM provides 4 GB of memory and the node supports up to 70 pods, AKS reserves *25% * 4 GB = 1000 MB* for kube-reserved, as this is less than *20 MB * 70 Max Pods + 50 MB = 1450 MB*.
+
+    For more information, see [Configure maximum pods per node in an AKS cluster][maximum-pods].
+
+**AKS versions prior to 1.29**
+
+* **`kubelet` daemon** has the *memory.available < 750 Mi* eviction rule by default. This rule ensures that a node has at least 750 Mi allocatable at all times. When a host is below that available memory threshold, the `kubelet` triggers the termination of one of the running pods and free up memory on the host machine.
+* **A regressive rate of memory reservations** for the kubelet daemon to properly function (*kube-reserved*).
+    * 25% of the first 4 GB of memory
+    * 20% of the next 4 GB of memory (up to 8 GB)
+    * 10% of the next 8 GB of memory (up to 16 GB)
+    * 6% of the next 112 GB of memory (up to 128 GB)
+    * 2% of any memory more than 128 GB
+
+> [!NOTE]
+> AKS reserves an extra 2 GB for system processes in Windows nodes that isn't part of the calculated memory.
+
+Memory and CPU allocation rules are designed to:
+
+* Keep agent nodes healthy, including some hosting system pods critical to cluster health.
+* Cause the node to report less allocatable memory and CPU than it would report if it weren't part of a Kubernetes cluster.
+
+For example, if a node offers 7 GB, it reports 34% of memory not allocatable including the 750 Mi hard eviction threshold.
+
+`0.75 + (0.25*4) + (0.20*3) = 0.75 GB + 1 GB + 0.6 GB = 2.35 GB / 7 GB = 33.57% reserved`
+
+In addition to reservations for Kubernetes itself, the underlying node OS also reserves an amount of CPU and memory resources to maintain OS functions.
+
+For associated best practices, see [Best practices for basic scheduler features in AKS][operator-best-practices-scheduler].
+
+## Next steps
+
+<!---LINKS--->
+[operator-best-practices-scheduler]: operator-best-practices-scheduler.md
+[maximum-pods]: concepts-network-ip-address-planning.md#maximum-pods-per-node