Skip to content

Commit ae1d2bc

Browse files
michaelryanpeterMichael Burke
michaelryanpeter
authored and
Michael Burke
committed
OSDOCS-4111: [OLM][4.12] Document pod affinity overrides
1 parent 538207f commit ae1d2bc

File tree

5 files changed

+285
-1
lines changed

5 files changed

+285
-1
lines changed

‎modules/olm-overriding-operator-pod-affinity.adoc

Lines changed: 270 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,270 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/scheduling/nodes-scheduler-node-affinity.adoc
4+
// * nodes/scheduling/nodes-scheduler-pod-affinity.adoc
5+
// * operators/admin/olm-adding-operators-to-cluster.adoc
6+
7+
ifeval::["{context}" == "nodes-scheduler-pod-affinity"]
8+
:pod:
9+
endif::[]
10+
ifeval::["{context}" == "nodes-scheduler-node-affinity"]
11+
:node:
12+
endif::[]
13+
ifeval::["{context}" == "olm-adding-operators-to-a-cluster"]
14+
:olm:
15+
endif::[]
16+
17+
:_content-type: PROCEDURE
18+
[id="olm-overriding-operator-pod-affinity_{context}"]
19+
20+
ifdef::olm[]
21+
= Controlling where an Operator is installed
22+
endif::olm[]
23+
24+
ifdef::pod[]
25+
= Using pod affinity and anti-affinity to control where an Operator is installed
26+
endif::pod[]
27+
28+
ifdef::node[]
29+
= Using node affinity to control where an Operator is installed
30+
endif::node[]
31+
32+
By default, when you install an Operator, {product-title} installs the Operator pod to one of your worker nodes randomly. However, there might be situations where you want that pod scheduled on a specific node or set of nodes.
33+
34+
The following examples describe situations where you might want to schedule an Operator pod to a specific node or set of nodes:
35+
36+
* If an Operator requires a particular platform, such as `amd64` or `arm64`
37+
* If an Operator requires a particular operating system, such as Linux or Windows
38+
* If you want Operators that work together scheduled on the same host or on hosts located on the same rack
39+
* If you want Operators dispersed throughout the infrastructure to avoid downtime due to network or hardware issues
40+
41+
ifdef::olm[]
42+
You can control where an Operator pod is installed by adding node affinity, pod affinity, or pod anti-affinity constraints to the Operator's `Subscription` object. Node affinity is a set of rules used by the scheduler to determine where a pod can be placed. Pod affinity enables you to ensure that related pods are scheduled to the same node. Pod anti-affinity allows you to prevent a pod from being scheduled on a node.
43+
endif::olm[]
44+
45+
ifdef::pod[]
46+
You can control where an Operator pod is installed by adding a pod affinity or anti-affinity to the Operator's `Subscription` object.
47+
endif::pod[]
48+
49+
ifdef::node[]
50+
You can control where an Operator pod is installed by adding a node affinity constraints to the Operator's `Subscription` object.
51+
endif::node[]
52+
53+
ifdef::olm[]
54+
The following examples show how to use node affinity or pod anti-affinity to install an instance of the Custom Metrics Autoscaler Operator to a specific node in the cluster:
55+
endif::olm[]
56+
ifdef::node[]
57+
The following examples show how to use node affinity to install an instance of the Custom Metrics Autoscaler Operator to a specific node in the cluster:
58+
endif::node[]
59+
60+
ifndef::pod[]
61+
.Node affinity example that places the Operator pod on a specific node
62+
[source,terminal]
63+
----
64+
apiVersion: operators.coreos.com/v1alpha1
65+
kind: Subscription
66+
metadata:
67+
name: openshift-custom-metrics-autoscaler-operator
68+
namespace: openshift-keda
69+
spec:
70+
name: my-package
71+
source: my-operators
72+
sourceNamespace: operator-registries
73+
config:
74+
affinity:
75+
nodeAffinity: <1>
76+
requiredDuringSchedulingIgnoredDuringExecution:
77+
nodeSelectorTerms:
78+
- matchExpressions:
79+
- key: kubernetes.io/hostname
80+
operator: In
81+
values:
82+
- ip-10-0-163-94.us-west-2.compute.internal
83+
...
84+
----
85+
<1> A node affinity that requires the Operator's pod to be scheduled on a node named `ip-10-0-163-94.us-west-2.compute.internal`.
86+
87+
.Node affinity example that places the Operator pod on a node with a specific platform
88+
[source,terminal]
89+
----
90+
apiVersion: operators.coreos.com/v1alpha1
91+
kind: Subscription
92+
metadata:
93+
name: openshift-custom-metrics-autoscaler-operator
94+
namespace: openshift-keda
95+
spec:
96+
name: my-package
97+
source: my-operators
98+
sourceNamespace: operator-registries
99+
config:
100+
affinity:
101+
nodeAffinity: <1>
102+
requiredDuringSchedulingIgnoredDuringExecution:
103+
nodeSelectorTerms:
104+
- matchExpressions:
105+
- key: kubernetes.io/arch
106+
operator: In
107+
values:
108+
- arm64
109+
- key: kubernetes.io/os
110+
operator: In
111+
values:
112+
- linux
113+
----
114+
<1> A node affinity that requires the Operator's pod to be scheduled on a node with the `kubernetes.io/arch=arm64` and `kubernetes.io/os=linux` labels.
115+
endif::pod[]
116+
117+
ifdef::pod[]
118+
The following example shows how to use pod anti-affinity to prevent the installation the Custom Metrics Autoscaler Operator from any node that has pods with a specific label:
119+
endif::pod[]
120+
121+
ifndef::node[]
122+
.Pod affinity example that places the Operator pod on one or more specific nodes
123+
[source,terminal]
124+
----
125+
apiVersion: operators.coreos.com/v1alpha1
126+
kind: Subscription
127+
metadata:
128+
name: openshift-custom-metrics-autoscaler-operator
129+
namespace: openshift-keda
130+
spec:
131+
name: my-package
132+
source: my-operators
133+
sourceNamespace: operator-registries
134+
config:
135+
affinity:
136+
podAffinity:
137+
requiredDuringSchedulingIgnoredDuringExecution:
138+
- labelSelector:
139+
matchExpressions:
140+
- key: app
141+
operator: In
142+
values:
143+
- test
144+
topologyKey: kubernetes.io/hostname
145+
----
146+
<1> A pod affinity that places the Operator's pod on a node that has pods with the `app=test` label.
147+
148+
.Pod anti-affinity example that prevents the Operator pod from one or more specific nodes
149+
[source,terminal]
150+
----
151+
apiVersion: operators.coreos.com/v1alpha1
152+
kind: Subscription
153+
metadata:
154+
name: openshift-custom-metrics-autoscaler-operator
155+
namespace: openshift-keda
156+
spec:
157+
name: my-package
158+
source: my-operators
159+
sourceNamespace: operator-registries
160+
config:
161+
affinity:
162+
podAntiAffinity: <1>
163+
requiredDuringSchedulingIgnoredDuringExecution:
164+
- labelSelector:
165+
matchExpressions:
166+
- key: cpu
167+
operator: In
168+
values:
169+
- high
170+
topologyKey: kubernetes.io/hostname
171+
...
172+
----
173+
<1> A pod anti-affinity that prevents the Operator's pod from being scheduled on a node that has pods with the `cpu=high` label.
174+
endif::node[]
175+
176+
.Procedure
177+
178+
To control the placement of an Operator pod, complete the following steps:
179+
180+
. Install the Operator as usual.
181+
182+
. If needed, ensure that your nodes are labeled to properly respond to the affinity.
183+
184+
. Edit the Operator `Subscription` object to add an affinity:
185+
+
186+
ifndef::pod[]
187+
[source,terminal]
188+
----
189+
apiVersion: operators.coreos.com/v1alpha1
190+
kind: Subscription
191+
metadata:
192+
name: openshift-custom-metrics-autoscaler-operator
193+
namespace: openshift-keda
194+
spec:
195+
name: my-package
196+
source: my-operators
197+
sourceNamespace: operator-registries
198+
config:
199+
affinity: <1>
200+
nodeAffinity:
201+
requiredDuringSchedulingIgnoredDuringExecution:
202+
nodeSelectorTerms:
203+
- matchExpressions:
204+
- key: kubernetes.io/hostname
205+
operator: In
206+
values:
207+
- ip-10-0-185-229.ec2.internal
208+
...
209+
----
210+
ifdef::olm[]
211+
<1> Add a `nodeAffinity`, `podAffinity`, or `podAntiAffinity`. See the Additional resources section that follows for information about creating the affinity.
212+
endif::[]
213+
ifdef::node[]
214+
<1> Add a `nodeAffinity`.
215+
endif::[]
216+
endif::pod[]
217+
ifdef::pod[]
218+
[source,terminal]
219+
----
220+
apiVersion: operators.coreos.com/v1alpha1
221+
kind: Subscription
222+
metadata:
223+
name: openshift-custom-metrics-autoscaler-operator
224+
namespace: openshift-keda
225+
spec:
226+
name: my-package
227+
source: my-operators
228+
sourceNamespace: operator-registries
229+
config:
230+
affinity:
231+
podAntiAffinity: <1>
232+
requiredDuringSchedulingIgnoredDuringExecution:
233+
podAffinityTerm:
234+
labelSelector:
235+
matchExpressions:
236+
- key: kubernetes.io/hostname
237+
operator: In
238+
values:
239+
- ip-10-0-185-229.ec2.internal
240+
topologyKey: topology.kubernetes.io/zone
241+
...
242+
----
243+
<1> Add a `podAffinity` or `podAntiAffinity`.
244+
endif::pod[]
245+
246+
.Verification
247+
248+
* To ensure that the pod is deployed on the specific node, run the following command:
249+
+
250+
[source,terminal]
251+
----
252+
$ oc get pods -o wide
253+
----
254+
+
255+
.Example output
256+
+
257+
----
258+
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
259+
custom-metrics-autoscaler-operator-5dcc45d656-bhshg 1/1 Running 0 50s 10.131.0.20 ip-10-0-185-229.ec2.internal <none> <none>
260+
----
261+
262+
ifeval::["{context}" == "nodes-scheduler-pod-affinity"]
263+
:!pod:
264+
endif::[]
265+
ifeval::["{context}" == "nodes-scheduler-node-affinity"]
266+
:!node:
267+
endif::[]
268+
ifeval::["{context}" == "olm-adding-operators-to-a-cluster"]
269+
:!olm:
270+
endif::[]

‎modules/olm-pod-placement.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Controlling pod placement of Operator and Operand workloads has the following pr
1616
At this point, the project you created can be used to steer pods towards the specified nodes in the following scenarios:
1717

1818
For Operator pods::
19-
Administrators can create a `Subscription` object in the project. As a result, the Operator pods are placed on the specified nodes.
19+
Administrators can create a `Subscription` object in the project as described in the following section. As a result, the Operator pods are placed on the specified nodes.
2020

2121
For Operand pods::
2222
Using an installed Operator, users can create an application in the project, which places the custom resource (CR) owned by the Operator in the project. As a result, the Operand pods are placed on the specified nodes, unless the Operator is deploying cluster-wide objects or resources in other namespaces, in which case this customized pod placement does not apply.

‎nodes/scheduling/nodes-scheduler-node-affinity.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,8 @@ include::modules/nodes-scheduler-node-affinity-configuring-preferred.adoc[levelo
2626

2727
include::modules/nodes-scheduler-node-affinity-example.adoc[leveloffset=+1]
2828

29+
include::modules/olm-overriding-operator-pod-affinity.adoc[leveloffset=+1]
30+
2931
[id="nodes-scheduler-node-affinity-addtl-resources_{context}"]
3032
[role="_additional-resources"]
3133
== Additional resources

‎nodes/scheduling/nodes-scheduler-pod-affinity.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,3 +27,5 @@ include::modules/nodes-scheduler-pod-affinity-configuring.adoc[leveloffset=+1]
2727
include::modules/nodes-scheduler-pod-anti-affinity-configuring.adoc[leveloffset=+1]
2828

2929
include::modules/nodes-scheduler-pod-affinity-example.adoc[leveloffset=+1]
30+
31+
include::modules/olm-overriding-operator-pod-affinity.adoc[leveloffset=+1]

‎operators/admin/olm-adding-operators-to-cluster.adoc

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,13 @@ include::modules/olm-pod-placement.adoc[leveloffset=+1]
4848
* xref:../../nodes/scheduling/nodes-scheduler-node-selectors.adoc#nodes-scheduler-node-selectors-project_nodes-scheduler-node-selectors[Creating project-wide node selectors]
4949
* xref:../../nodes/scheduling/nodes-scheduler-taints-tolerations.adoc#nodes-scheduler-taints-tolerations-projects_nodes-scheduler-taints-tolerations[Creating a project with a node selector and toleration]
5050
endif::[]
51+
52+
include::modules/olm-overriding-operator-pod-affinity.adoc[leveloffset=+1]
53+
54+
[role="_additional-resources"]
55+
.Additional resources
56+
57+
* xref:../../nodes/scheduling/nodes-scheduler-pod-affinity.adoc#nodes-scheduler-pod-affinity-about_nodes-scheduler-pod-affinity[Understanding pod affinity]
58+
* xref:../../nodes/scheduling/nodes-scheduler-node-affinity.html#nodes-scheduler-node-affinity-about_nodes-scheduler-node-affinity[Understanding node affinity]
59+
* xref:../../nodes/nodes/nodes-nodes-working.adoc#nodes-nodes-working-updating_nodes-nodes-working[Understanding how to update labels on nodes].
60+

0 commit comments

Comments
 (0)