You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: AKS-Hybrid/availability-sets.md
+15-15Lines changed: 15 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,7 +3,7 @@ title: Availability sets in AKS enabled by Azure Arc
3
3
description: Learn how to enable availability sets in AKS enabled by Arc to improve the availability and distribution of your Kubernetes workloads.
4
4
ms.topic: how-to
5
5
author: sethmanheim
6
-
ms.date: 09/05/2024
6
+
ms.date: 09/06/2024
7
7
ms.author: sethm
8
8
ms.reviewer: rbaziwane
9
9
ms.lastreviewed: 08/15/2024
@@ -12,58 +12,58 @@ ms.lastreviewed: 08/15/2024
12
12
13
13
# Availability sets in AKS enabled by Azure Arc
14
14
15
-
*Availability sets* are logical groups of VMs that have weak anti-affinity relationships with each other, to ensure that they are spread evenly across the available fault domains in a physical cluster. A fault domain in this context is a physical host or a group of physical hosts. By using availability sets, we can improve the availability and distribution of your Kubernetes workloads and avoid scenarios where a single node failure can cause multiple VMs to go down or become unbalanced.
15
+
*Availability sets* are logical groups of VMs that have weak anti-affinity relationships with each other, to ensure that they are spread evenly across the available fault domains in a physical cluster. A fault domain in this context is a physical host or a group of physical hosts. By using availability sets, AKS Arc can improve the availability and distribution of your Kubernetes workloads. Availability sets can avoid scenarios in which a single node failure can cause multiple VMs to go down or become unbalanced.
16
16
17
17
## Overview
18
18
19
-
If you use AKS on Azure Stack HCI and Windows Server to run Kubernetes workloads on-premises, you might encounter some challenges with the current architecture. For example, you might notice that multiple virtual machines (VMs) within the same node pool can end up on the same physical host, which is not ideal for high availability. Or, you might see that VMs do not rebalance across physical hosts when a host recovers from an issue, resulting in uneven distribution of workloads. These issues can affect the performance and reliability of your applications, causing unnecessary disruption in your business operations.
19
+
If you use AKS on Azure Stack HCI and Windows Server to run Kubernetes workloads on-premises, you might encounter some challenges with the current architecture. For example, you might notice that multiple virtual machines (VMs) within the same node pool can exist on the same physical host, which is not ideal for high availability. Or, you might see that VMs do not rebalance across physical hosts when a host recovers from an issue, resulting in uneven distribution of workloads. These issues can affect the performance and reliability of your applications, causing unnecessary disruption in your business operations.
20
20
21
21
Availability sets offer several benefits for AKS on Azure Stack HCI and Windows Server users, such as:
22
22
23
-
-Improving the availability and resilience of your applications by avoiding scenarios in which multiple VMs within the same node pool or control plane go down or become unbalanced due to a single node failure.
24
-
-Optimizing the resource usage and performance of your cluster by ensuring that VMs are evenly distributed across the available nodes and not concentrated on a single node or a subset of nodes.
25
-
-Aligning with the best practices and expectations of your customers and partners who are looking for a reliable and consistent Kubernetes experience on-premises.
23
+
-Improves the availability and resilience of your applications by avoiding scenarios in which multiple VMs within the same node pool or control plane go down or become unbalanced due to a single node failure.
24
+
-Optimizes the resource usage and performance of your cluster by ensuring that VMs are evenly distributed across the available nodes and not concentrated on a single node or a subset of nodes.
25
+
-Aligns with the best practices and expectations of your customers and partners who are looking for a reliable and consistent on-premises Kubernetes experience.
26
26
27
27
## Enable availability sets
28
28
29
-
On AKS on Azure Stack HCI 23H2, the availability sets feature is enabled by default when you create a new node pool.
29
+
With AKS on Azure Stack HCI 23H2, the availability sets feature is enabled by default when you create a new node pool.
30
30
31
-
On AKS on Azure Stack HCI 22H2, the availability sets feature is disabled by default. To enable it, add the `-enableAvailabilitySet` parameter when you create a workload cluster.
31
+
With AKS on Azure Stack HCI 22H2, the availability sets feature is disabled by default. To enable it, add the `-enableAvailabilitySet` parameter when you create a workload cluster. For example:
32
32
33
33
```powershell
34
34
New-AksHciCluster -Name <name> -controlPlaneNodeCount 3 -osType Linux -kubernetesVersion $kubernetesVersion -enableAvailabilitySet
35
35
```
36
36
37
37
## How availability sets work in AKS enabled by Azure Arc
38
38
39
-
When you create a new AKS Arc cluster, we automatically create availability sets, one for the control plane VMs and another for each of the node pools in the cluster (each node pool has its own availability set). By defining this layout, we ensure that VMs of the same role (control plane or node pool) are never co-located on the same physical host and that they're distributed across the available nodes in a cluster.
39
+
When you create a new AKS Arc cluster, AKS Arc automatically creates availability sets, one for the control plane VMs and another one for each of the node pools in the cluster. Each node pool has its own availability set. With this layout, AKS Arc ensures that VMs of the same role (control plane or node pool) are never located on the same physical host and that they're distributed across the available nodes in a cluster.
40
40
41
41
Once the availability sets are created and the VMs assigned, the system automatically places them on the appropriate physical nodes. If a node fails, the system also automatically fails over the VMs to other nodes and rebalances them when the node recovers. This way, you can achieve high availability and optimal distribution of your Kubernetes workloads without manual intervention.
42
42
43
-
Consider an AKS on Azure Stack HCI 23H2 cluster with two physical host machines, **Host A** and **Host B**, three control plane VMs, and two worker node VMs, **Nodepool1VM1** and **Nodepool1VM2**. To ensure high availability of your Kubernetes applications, the node pool VMs must never share the same host, unless one of the hosts is temporarily unavailable for planned maintenance or capacity issue causing the VM (virtual machine) to be temporarily placed on an alternative host.
43
+
Consider an AKS on Azure Stack HCI 23H2 cluster with two physical host machines, **Host A** and **Host B**, three control plane VMs, and two worker node VMs, **Nodepool1VM1** and **Nodepool1VM2**. To ensure high availability of your Kubernetes applications, the node pool VMs must never share the same host, unless one of the hosts is temporarily unavailable for planned maintenance or capacity issue, which can cause the VM (virtual machine) to be temporarily placed on an alternative host.
44
44
45
45
In the following diagram, each color represents an anti-affinity group:
46
46
47
47
:::image type="content" source="media/availability-sets/anti-affinity-1.png" alt-text="Diagram showing hosts in anti-affinity group.":::
48
48
49
-
If host B goes down due to a reboot, **Control Plane VM2**, **Control Plane VM3**, and **Nodepool1VM2** fail over to **Host A** as shown in the following figure. Assuming your application is running pods in **NodePoolVM1**, this reboot has no impact on your application:
49
+
If **Host B** goes down due to a reboot, **Control Plane VM2**, **Control Plane VM3**, and **Nodepool1VM2** fail over to **Host A** as shown in the following figure. Assuming your application is running pods in **NodePoolVM1**, this reboot has no impact on your application:
50
50
51
51
:::image type="content" source="media/availability-sets/anti-affinity-2.png" alt-text="Diagram showing host B down.":::
52
52
53
-
In the old architecture, if host B came back online after a reboot, we could not guarantee that the VMs would move back from host A to host B (rebalancing), thus forcing the workloads to stay on the same host, and create a single point of failure, as shown in the following diagram:
53
+
In the old architecture, if **Host B** came back online after a reboot, there was no guarantee that the VMs would move back from Host A to Host B (rebalancing), thus forcing the workloads to stay on the same host, and create a single point of failure, as shown in the following diagram:
54
54
55
55
:::image type="content" source="media/availability-sets/anti-affinity-3.png" alt-text="Diagram showing no rebalancing.":::
56
56
57
-
Availability sets for AKS Arc help to rebalance VMs once a host recovers from temporary outage. In this example, **ControlPlaneVM2**, **ControlPlaneVM3**, and **Nodepool1VM2** automatically move to Host B as shown here:
57
+
Availability sets for AKS Arc can help to rebalance VMs once a host recovers from temporary outage. In this example, **ControlPlaneVM2**, **ControlPlaneVM3**, and **Nodepool1VM2** automatically move to **Host B**, as shown here:
58
58
59
59
:::image type="content" source="media/availability-sets/anti-affinity-1.png" alt-text="Diagram showing hosts in anti-affinity group.":::
60
60
61
61
> [!IMPORTANT]
62
-
> Availability sets in AKS Arc is a new feature that is still evolving and improving. We do not yet support manual configuration of the fault domains or availability sets. You can't change the fault domains of an availability set after it's created. VMs are assigned to an availability set at cluster creation, and can't be migrated to a different availability set.
62
+
> Availability sets in AKS Arc are a new feature that's still evolving and improving. We do not yet support manual configuration of the fault domains or availability sets. You can't change the fault domains of an availability set after it's created. VMs are assigned to an availability set at cluster creation, and can't be migrated to a different availability set.
63
63
64
64
## Add or delete machines
65
65
66
-
In a host deletion scenario, the host is no longer considered a part of the cluster. This typically occurs when you replace a machine due to hardware issues, or scale down the HCI cluster for other reasons. During a node outage, the node remains part of the HCI cluster but appears as **Down**.
66
+
In a host deletion scenario, the host is no longer considered a part of the cluster. This deletion typically occurs when you replace a machine due to hardware issues, or scale down the HCI cluster for other reasons. During a node outage, the node remains part of the HCI cluster but appears as **Down**.
67
67
68
68
If a physical machine (fault domain) is permanently deleted from the cluster, the availability set configuration isn't modified to reduce the number of fault domains. In this scenario, the availability set enters an unhealthy state. We recommend that you redeploy your workload clusters so that the availability set is updated with the proper number of fault domains.
0 commit comments