Skip to content

Commit 548539d

Browse files
Merge pull request #218120 from MGoedtel/bug36380
updated YAML code sample to fix formatting
2 parents 771f075 + 24a0749 commit 548539d

File tree

1 file changed

+26
-18
lines changed

1 file changed

+26
-18
lines changed

articles/aks/operator-best-practices-advanced-scheduler.md

Lines changed: 26 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,16 @@ titleSuffix: Azure Kubernetes Service
44
description: Learn the cluster operator best practices for using advanced scheduler features such as taints and tolerations, node selectors and affinity, or inter-pod affinity and anti-affinity in Azure Kubernetes Service (AKS)
55
services: container-service
66
ms.topic: conceptual
7-
ms.date: 03/09/2021
7+
ms.date: 11/11/2022
88

99
---
1010

1111
# Best practices for advanced scheduler features in Azure Kubernetes Service (AKS)
1212

1313
As you manage clusters in Azure Kubernetes Service (AKS), you often need to isolate teams and workloads. Advanced features provided by the Kubernetes scheduler let you control:
14+
1415
* Which pods can be scheduled on certain nodes.
15-
* How multi-pod applications can be appropriately distributed across the cluster.
16+
* How multi-pod applications can be appropriately distributed across the cluster.
1617

1718
This best practices article focuses on advanced Kubernetes scheduling features for cluster operators. In this article, you learn how to:
1819

@@ -23,22 +24,24 @@ This best practices article focuses on advanced Kubernetes scheduling features f
2324
2425
## Provide dedicated nodes using taints and tolerations
2526

26-
> **Best practice guidance:**
27+
> **Best practice guidance:**
2728
>
2829
> Limit access for resource-intensive applications, such as ingress controllers, to specific nodes. Keep node resources available for workloads that require them, and don't allow scheduling of other workloads on the nodes.
2930
30-
When you create your AKS cluster, you can deploy nodes with GPU support or a large number of powerful CPUs. You can use these nodes for large data processing workloads such as machine learning (ML) or artificial intelligence (AI).
31+
When you create your AKS cluster, you can deploy nodes with GPU support or a large number of powerful CPUs. You can use these nodes for large data processing workloads such as machine learning (ML) or artificial intelligence (AI).
3132

32-
Since this node resource hardware is typically expensive to deploy, limit the workloads that can be scheduled on these nodes. Instead, you'd dedicate some nodes in the cluster to run ingress services and prevent other workloads.
33+
Because this node resource hardware is typically expensive to deploy, limit the workloads that can be scheduled on these nodes. Instead, dedicate some nodes in the cluster to run ingress services and prevent other workloads.
3334

34-
This support for different nodes is provided by using multiple node pools. An AKS cluster provides one or more node pools.
35+
This support for different nodes is provided by using multiple node pools. An AKS cluster supports one or more node pools.
3536

3637
The Kubernetes scheduler uses taints and tolerations to restrict what workloads can run on nodes.
3738

3839
* Apply a **taint** to a node to indicate only specific pods can be scheduled on them.
3940
* Then apply a **toleration** to a pod, allowing them to *tolerate* a node's taint.
4041

41-
When you deploy a pod to an AKS cluster, Kubernetes only schedules pods on nodes whose taint aligns with the toleration. For example, assume you added a node pool in your AKS cluster for nodes with GPU support. You define name, such as *gpu*, then a value for scheduling. Setting this value to *NoSchedule* restricts the Kubernetes scheduler from scheduling pods with undefined toleration on the node.
42+
When you deploy a pod to an AKS cluster, Kubernetes only schedules pods on nodes whose taint aligns with the toleration. Taints and tolerations work together to ensure that pods are not scheduled onto inappropriate nodes. One or more taints are applied to a node, marking the the node so that it does not accept any pods that do not tolerate the taints.
43+
44+
For example, assume you added a node pool in your AKS cluster for nodes with GPU support. You define name, such as *gpu*, then a value for scheduling. Setting this value to *NoSchedule* restricts the Kubernetes scheduler from scheduling pods with undefined toleration on the node.
4245

4346
```azurecli-interactive
4447
az aks nodepool add \
@@ -85,9 +88,11 @@ For more information about how to use multiple node pools in AKS, see [Create an
8588
When you upgrade a node pool in AKS, taints and tolerations follow a set pattern as they're applied to new nodes:
8689

8790
#### Default clusters that use VM scale sets
91+
8892
You can [taint a node pool][taint-node-pool] from the AKS API to have newly scaled out nodes receive API specified node taints.
8993

9094
Let's assume:
95+
9196
1. You begin with a two-node cluster: *node1* and *node2*.
9297
1. You upgrade the node pool.
9398
1. Two additional nodes are created: *node3* and *node4*.
@@ -97,6 +102,7 @@ Let's assume:
97102
#### Clusters without VM scale set support
98103

99104
Again, let's assume:
105+
100106
1. You have a two-node cluster: *node1* and *node2*.
101107
1. You upgrade the node pool.
102108
1. An additional node is created: *node3*.
@@ -112,8 +118,8 @@ When you scale a node pool in AKS, taints and tolerations do not carry over by d
112118

113119
## Control pod scheduling using node selectors and affinity
114120

115-
> **Best practice guidance**
116-
>
121+
> **Best practice guidance**
122+
>
117123
> Control the scheduling of pods on nodes using node selectors, node affinity, or inter-pod affinity. These settings allow the Kubernetes scheduler to logically isolate workloads, such as by hardware in the node.
118124

119125
Taints and tolerations logically isolate resources with a hard cut-off. If the pod doesn't tolerate a node's taint, it isn't scheduled on the node.
@@ -153,7 +159,8 @@ spec:
153159
cpu: 4.0
154160
memory: 16Gi
155161
nodeSelector:
156-
hardware: highmem
162+
hardware:
163+
values: highmem
157164
```
158165

159166
When you use these scheduler options, work with your application developers and owners to allow them to correctly define their pod specifications.
@@ -162,7 +169,8 @@ For more information about using node selectors, see [Assigning Pods to Nodes][k
162169

163170
### Node affinity
164171

165-
A node selector is a basic solution for assigning pods to a given node. *Node affinity* provides more flexibility, allowing you to define what happens if the pod can't be matched with a node. You can:
172+
A node selector is a basic solution for assigning pods to a given node. *Node affinity* provides more flexibility, allowing you to define what happens if the pod can't be matched with a node. You can:
173+
166174
* *Require* that Kubernetes scheduler matches a pod with a labeled host. Or,
167175
* *Prefer* a match but allow the pod to be scheduled on a different host if no match is available.
168176

@@ -191,7 +199,8 @@ spec:
191199
- matchExpressions:
192200
- key: hardware
193201
operator: In
194-
values: highmem
202+
values:
203+
- highmem
195204
```
196205

197206
The *IgnoredDuringExecution* part of the setting indicates that the pod shouldn't be evicted from the node if the node labels change. The Kubernetes scheduler only uses the updated node labels for new pods being scheduled, not pods already scheduled on the nodes.
@@ -202,9 +211,10 @@ For more information, see [Affinity and anti-affinity][k8s-affinity].
202211

203212
One final approach for the Kubernetes scheduler to logically isolate workloads is using inter-pod affinity or anti-affinity. These settings define that pods either *shouldn't* or *should* be scheduled on a node that has an existing matching pod. By default, the Kubernetes scheduler tries to schedule multiple pods in a replica set across nodes. You can define more specific rules around this behavior.
204213

205-
For example, you have a web application that also uses an Azure Cache for Redis.
206-
1. You use pod anti-affinity rules to request that the Kubernetes scheduler distributes replicas across nodes.
207-
1. You use affinity rules to ensure each web app component is scheduled on the same host as a corresponding cache.
214+
For example, you have a web application that also uses an Azure Cache for Redis.
215+
216+
* You use pod anti-affinity rules to request that the Kubernetes scheduler distributes replicas across nodes.
217+
* You use affinity rules to ensure each web app component is scheduled on the same host as a corresponding cache.
208218

209219
The distribution of pods across nodes looks like the following example:
210220

@@ -213,7 +223,7 @@ The distribution of pods across nodes looks like the following example:
213223
| webapp-1 | webapp-2 | webapp-3 |
214224
| cache-1 | cache-2 | cache-3 |
215225

216-
Inter-pod affinity and anti-affinity provide a more complex deployment than node selectors or node affinity. With the deployment, you logically isolate resources and control how Kubernetes schedules pods on nodes.
226+
Inter-pod affinity and anti-affinity provide a more complex deployment than node selectors or node affinity. With the deployment, you logically isolate resources and control how Kubernetes schedules pods on nodes.
217227

218228
For a complete example of this web application with Azure Cache for Redis example, see [Co-locate pods on the same node][k8s-pod-affinity].
219229

@@ -226,14 +236,12 @@ This article focused on advanced Kubernetes scheduler features. For more informa
226236
* [Authentication and authorization][aks-best-practices-identity]
227237

228238
<!-- EXTERNAL LINKS -->
229-
[k8s-taints-tolerations]: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
230239
[k8s-node-selector]: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
231240
[k8s-affinity]: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
232241
[k8s-pod-affinity]: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#always-co-located-in-the-same-node
233242

234243
<!-- INTERNAL LINKS -->
235244
[aks-best-practices-scheduler]: operator-best-practices-scheduler.md
236-
[aks-best-practices-cluster-isolation]: operator-best-practices-cluster-isolation.md
237245
[aks-best-practices-identity]: operator-best-practices-identity.md
238246
[use-multiple-node-pools]: use-multiple-node-pools.md
239247
[taint-node-pool]: use-multiple-node-pools.md#specify-a-taint-label-or-tag-for-a-node-pool

0 commit comments

Comments
 (0)