Skip to content

Commit bb41161

Browse files
kelbrown20kelbrown20
authored
Merge pull request #63915 from kelbrown20/OSDOCS-7562-scheduling-workloads-multi-arch
OSDOCS#7562: Scheduling workloads on multi-arch clusters
2 parents 5be3956 + 44c2fc0 commit bb41161

File tree

3 files changed

+172
-1
lines changed

3 files changed

+172
-1
lines changed

‎modules/multi-architecture-scheduling-examples.adoc

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
// Module included in the following assembly
2+
//
3+
//post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="multi-architecture-scheduling-examples_{context}"]
7+
8+
= Sample multi-architecture node workload deployments
9+
10+
Before you schedule workloads on a cluster with compute nodes of different architectures, consider the following use cases:
11+
12+
Using node affinity to schedule workloads on a node:: You can allow a workload to be scheduled on only a set of nodes with architectures supported by its images, you can set the `spec.affinity.nodeAffinity` field in your pod's template specification.
13+
+
14+
.Example deployment with the `nodeAffinity` set to certain architectures
15+
[source,yaml]
16+
----
17+
apiVersion: apps/v1
18+
kind: Deployment
19+
metadata: # ...
20+
spec:
21+
# ...
22+
template:
23+
# ...
24+
spec:
25+
affinity:
26+
nodeAffinity:
27+
requiredDuringSchedulingIgnoredDuringExecution:
28+
nodeSelectorTerms:
29+
- matchExpressions:
30+
- key: kubernetes.io/arch
31+
operator: In
32+
values: <1>
33+
- amd64
34+
- arm64
35+
----
36+
<1> Specify the supported architectures. Valid values include `amd64`,`arm64`, or both values.
37+
38+
Tainting every node for a specific architecture:: You can taint a node to avoid workloads that are not compatible with its architecture to be scheduled on that node. In the case where your cluster is using a `MachineSet` object, you can add parameters to the `.spec.template.spec.taints` field to avoid workloads being scheduled on nodes with non-supported architectures.
39+
40+
* Before you can taint a node, you must scale down the `MachineSet` object or remove available machines. You can scale down the machine set by using one of following commands:
41+
+
42+
[source,terminal]
43+
----
44+
$ oc scale --replicas=0 machineset <machineset> -n openshift-machine-api
45+
----
46+
+
47+
Or:
48+
+
49+
[source,terminal]
50+
----
51+
$ oc edit machineset <machineset> -n openshift-machine-api
52+
----
53+
For more information on scaling machine sets, see "Modifying a compute machine set".
54+
55+
+
56+
--
57+
.Example `MachineSet` with a taint set
58+
[source,yaml]
59+
----
60+
apiVersion: machine.openshift.io/v1beta1
61+
kind: MachineSet
62+
metadata: # ...
63+
spec:
64+
# ...
65+
template:
66+
# ...
67+
spec:
68+
# ...
69+
taints:
70+
- effect: NoSchedule
71+
key: multi-arch.openshift.io/arch
72+
value: arm64
73+
----
74+
You can also set a taint on a specific node by running the following command:
75+
[source,terminal]
76+
----
77+
$ oc adm taint nodes <node-name> multi-arch.openshift.io/arch=arm64:NoSchedule
78+
----
79+
--
80+
81+
Creating a default toleration:: You can annotate a namespace so all of the workloads get the same default toleration by running the following command:
82+
+
83+
[source,terminal]
84+
----
85+
$ oc annotate namespace my-namespace \
86+
'scheduler.alpha.kubernetes.io/defaultTolerations'='[{"operator": "Exists", "effect": "NoSchedule", "key": "multi-arch.openshift.io/arch"}]'
87+
----
88+
89+
Tolerating architecture taints in workloads:: On a node with a defined taint, workloads will not be scheduled on that node. However, you can allow them to be scheduled by setting a toleration in the pod's specification.
90+
+
91+
.Example deployment with a toleration
92+
[source,yaml]
93+
----
94+
apiVersion: apps/v1
95+
kind: Deployment
96+
metadata: # ...
97+
spec:
98+
# ...
99+
template:
100+
# ...
101+
spec:
102+
tolerations:
103+
- key: "multi-arch.openshift.io/arch"
104+
value: "arm64"
105+
operator: "Equal"
106+
effect: "NoSchedule"
107+
----
108+
+
109+
This example deployment can also be allowed on nodes with the `multi-arch.openshift.io/arch=arm64` taint specified.
110+
111+
Using node affinity with taints and tolerations:: When a scheduler computes the set of nodes to schedule a pod, tolerations can broaden the set while node affinity restricts the set. If you set a taint to the nodes of a specific architecture, the following example toleration is required for scheduling pods.
112+
+
113+
.Example deployment with a node affinity and toleration set.
114+
[source,yaml]
115+
----
116+
apiVersion: apps/v1
117+
kind: Deployment
118+
metadata: # ...
119+
spec:
120+
# ...
121+
template:
122+
# ...
123+
spec:
124+
affinity:
125+
nodeAffinity:
126+
requiredDuringSchedulingIgnoredDuringExecution:
127+
nodeSelectorTerms:
128+
- matchExpressions:
129+
- key: kubernetes.io/arch
130+
operator: In
131+
values:
132+
- amd64
133+
- arm64
134+
tolerations:
135+
- key: "multi-arch.openshift.io/arch"
136+
value: "arm64"
137+
operator: "Equal"
138+
effect: "NoSchedule"
139+
----

‎modules/multi-architecture-scheduling-overview.adoc

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
// module included in the following assembly
2+
//
3+
//post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="multi-architecture-scheduling-overview_{context}"]
7+
= Scheduling workloads on clusters with multi-architecture compute machines
8+
9+
Before deploying a workload onto a cluster with compute nodes of different architectures, you must configure your compute node scheduling process so the pods in your cluster are correctly assigned.
10+
11+
You can schedule workloads onto multi-architecture nodes for your cluster in several ways. For example, you can use a node affinity or a node selector to select the node you want the pod to schedule onto. You can also use scheduling mechanisms, like taints and tolderations, when using node affinity or node selector to correctly schedule workloads.
12+
13+

‎post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,26 @@
22
:context: multi-architecture-compute-managing
33
[id="multi-architecture-compute-managing"]
44
= Managing your cluster with multi-architecture compute machines
5+
include::_attributes/common-attributes.adoc[]
56

6-
toc::[]
7+
toc::[]
8+
9+
== Scheduling workloads on clusters with multi-architecture compute machines
10+
11+
Deploying a workload on a cluster with compute nodes of different architectures requires attention and monitoring of your cluster. There might be further actions you need to take in order to successfully place pods in the nodes of your cluster.
12+
13+
For more detailed information on node affinity, scheduling, taints and tolerlations, see the following documentatinon:
14+
15+
* xref:../../nodes/scheduling/nodes-scheduler-taints-tolerations.adoc#nodes-scheduler-taints-tolerations[Controlling pod placement using node taints].
16+
17+
* xref:../../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity]
18+
19+
* xref:../../nodes/scheduling/nodes-scheduler-about.adoc#nodes-scheduler-about[Controlling pod placement using the scheduler]
20+
21+
include::modules/multi-architecture-scheduling-examples.adoc[leveloffset=+2]
22+
23+
.Additional resources
24+
25+
* xref:../../machine_management/modifying-machineset.adoc#machineset-modifying_modifying-machineset[Modifying a compute machine set]
726

827
include::modules/multi-architecture-import-imagestreams.adoc[leveloffset=+1]

0 commit comments

Comments
 (0)