Skip to content

Commit 9b891af

Browse files
ousleypousleyp
committed
CNV-5646: Specifying nodes for virtualization components
1 parent d58d1ae commit 9b891af

11 files changed

+277
-2
lines changed

‎_topic_map.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2379,6 +2379,8 @@ Topics:
23792379
Topics:
23802380
- Name: Preparing your OpenShift cluster for OpenShift Virtualization
23812381
File: preparing-cluster-for-virt
2382+
- Name: Specifying nodes for OpenShift Virtualization components
2383+
File: virt-specifying-nodes-for-virtualization-components
23822384
- Name: Installing OpenShift Virtualization using the web console
23832385
File: installing-virt-web
23842386
- Name: Installing OpenShift Virtualization using the CLI

‎modules/virt-about-node-placement-virtualization-components.adoc‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * virt/install/virt-specifying-nodes-for-virtualization-components.adoc
4+
5+
[id="virt-about-node-placement-virtualization-components_{context}"]
6+
= About node placement for virtualization components
7+
8+
You might want to customize where {VirtProductName} deploys its components to ensure that:
9+
10+
* Virtual machines only deploy on nodes that are intended for virtualization workloads.
11+
* Operators only deploy on infrastructure nodes.
12+
* Certain nodes are unaffected by {VirtProductName}. For example, you have workloads unrelated to virtualization running on your cluster, and you want those workloads to be isolated from {VirtProductName}.
13+
14+
You can specify node placement rules for the {VirtProductName} Operators that Operator Lifecycle Manager (OLM) deploys by editing the OLM `Subscription` object.
15+
16+
To specify node placement rules for components that the {VirtProductName} Operators deploy, edit the HyperConverged Cluster custom resource (CR).
17+
18+
[id="about-node-placement-rules_{context}"]
19+
== About node placement rules
20+
21+
`nodeSelector`:: Allows pods to be scheduled on nodes that are labeled with the key-value pair or pairs that you specify in this field. The node must have labels that exactly match all listed pairs.
22+
`affinity`:: Enables you to use more expressive syntax to set rules that match nodes with pods. Affinity also allows for more nuance in how the rules are applied. For example, you can specify that a rule is a preference, rather than a hard requirement, so that pods are still scheduled if the rule is not satisfied.
23+
`tolerations`:: Allows pods to be scheduled on nodes that have matching taints. If a taint is applied to a node, that node only accepts pods that tolerate the taint.
24+
25+
[id="node-placement-olm-subscription-object_{context}"]
26+
== Node placement in the OLM Subscription object
27+
28+
To specify the nodes where OLM deploys the {VirtProductName} Operators, edit the `Subscription` object during {VirtProductName} installation. You can include node placement rules in the `spec.config` field, as shown in the following example:
29+
30+
[source,yaml,subs="attributes+"]
31+
----
32+
apiVersion: operators.coreos.com/v1alpha1
33+
kind: Subscription
34+
metadata:
35+
name: hco-operatorhub
36+
namespace: openshift-cnv
37+
spec:
38+
source: redhat-operators
39+
sourceNamespace: openshift-marketplace
40+
name: kubevirt-hyperconverged
41+
startingCSV: kubevirt-hyperconverged-operator.v{HCOVersion}
42+
channel: "stable"
43+
config: <1>
44+
----
45+
<1> The `config` field supports `nodeSelector` and `tolerations`, but it does not support `affinity`.
46+
47+
[id="node-placement-hco-cluster-cr_{context}"]
48+
== Node placement in the HyperConverged Cluster CR
49+
50+
To specify the nodes where {VirtProductName} deploys its components, you can include the `nodePlacement` object in the HyperConverged Cluster custom resource (CR) file that you create during {VirtProductName} installation. You can include `nodePlacement` under the `spec.infra` and `spec.workloads` fields, as shown in the following example:
51+
52+
[source,yaml]
53+
----
54+
apiVersion: hco.kubevirt.io/v1beta1
55+
kind: HyperConverged
56+
metadata:
57+
name: kubevirt-hyperconverged
58+
namespace: openshift-cnv
59+
spec:
60+
BareMetalPlatform: true
61+
infra:
62+
nodePlacement: <1>
63+
workloads:
64+
nodePlacement:
65+
----
66+
<1> The `nodePlacement` field supports `nodeSelector`, `affinity`, and `tolerations` fields.

‎modules/virt-example-node-placement-affinity-hyperconverged-cr.adoc‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * virt/install/virt-specifying-nodes-for-virtualization-components.adoc
4+
5+
[id="virt-example-node-placement-affinity-hyperconverged-cr_{context}"]
6+
= Example: Node placement with affinity in the HyperConverged Cluster CR
7+
8+
In this example, `affinity` is configured so that infrastructure resources are placed on nodes that are labeled with `example.io/example-infra-key = example-value` and workloads are placed on nodes labeled with `example.io/example-workloads-key = example-workloads-value`. Nodes that have more than eight CPUs are preferred for workloads, but if they are not available, pods are still scheduled.
9+
10+
[source,yaml]
11+
----
12+
apiVersion: hco.kubevirt.io/v1beta1
13+
kind: HyperConverged
14+
metadata:
15+
name: kubevirt-hyperconverged
16+
namespace: openshift-cnv
17+
spec:
18+
BareMetalPlatform: true
19+
infra:
20+
nodePlacement:
21+
affinity:
22+
nodeAffinity:
23+
requiredDuringSchedulingIgnoredDuringExecution:
24+
nodeSelectorTerms:
25+
- matchExpressions:
26+
- key: example.io/example-infra-key
27+
operator: In
28+
values:
29+
- example-infra-value
30+
workloads:
31+
nodePlacement:
32+
affinity:
33+
nodeAffinity:
34+
requiredDuringSchedulingIgnoredDuringExecution:
35+
nodeSelectorTerms:
36+
- matchExpressions:
37+
- key: example.io/example-workloads-key
38+
operator: In
39+
values:
40+
- example-workloads-value
41+
preferredDuringSchedulingIgnoredDuringExecution:
42+
- weight: 1
43+
preference:
44+
matchExpressions:
45+
- key: example.io/num-cpus
46+
operator: gt
47+
values:
48+
- 8
49+
----

‎modules/virt-example-node-placement-node-selector-hyperconverged-cr.adoc‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * virt/install/virt-specifying-nodes-for-virtualization-components.adoc
4+
5+
[id="virt-example-node-placement-node-selector-hyperconverged-cr_{context}"]
6+
= Example: Node placement with nodeSelector in the HyperConverged Cluster CR
7+
8+
In this example, `nodeSelector` is configured so that infrastructure resources are placed on nodes that are labeled with `example.io/example-infra-key = example-infra-value` and workloads are placed on nodes labeled with `example.io/example-workloads-key = example-workloads-value`.
9+
10+
[source,yaml]
11+
----
12+
apiVersion: hco.kubevirt.io/v1beta1
13+
kind: HyperConverged
14+
metadata:
15+
name: kubevirt-hyperconverged
16+
namespace: openshift-cnv
17+
spec:
18+
infra:
19+
nodePlacement:
20+
nodeSelector:
21+
example.io/example-infra-key: example-infra-value
22+
workloads:
23+
nodePlacement:
24+
nodeSelector:
25+
example.io/example-workloads-key: example-workloads-value
26+
----

‎modules/virt-example-node-placement-node-selector-olm-subscription.adoc‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * virt/install/virt-specifying-nodes-for-virtualization-components.adoc
4+
5+
[id="virt-example-node-placement-node-selector-olm-subscription_{context}"]
6+
= Example: Node placement with nodeSelector in the OLM Subscription object
7+
8+
In this example, `nodeSelector` is configured so that OLM places the {VirtProductName} Operators on nodes that are labeled with `example.io/example-infra-key = example-infra-value`.
9+
10+
[source,yaml,subs="attributes+"]
11+
----
12+
apiVersion: operators.coreos.com/v1alpha1
13+
kind: Subscription
14+
metadata:
15+
name: hco-operatorhub
16+
namespace: openshift-cnv
17+
spec:
18+
source: redhat-operators
19+
sourceNamespace: openshift-marketplace
20+
name: kubevirt-hyperconverged
21+
startingCSV: kubevirt-hyperconverged-operator.v{HCOVersion}
22+
channel: "stable"
23+
config:
24+
nodeSelector:
25+
example.io/example-infra-key: example-infra-value
26+
----

‎modules/virt-example-node-placement-tolerations-hyperconverged-cr.adoc‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * virt/install/virt-specifying-nodes-for-virtualization-components.adoc
4+
5+
[id="virt-example-node-placement-tolerations-hyperconverged-cr_{context}"]
6+
= Example: Node placement with tolerations in the HyperConverged Cluster CR
7+
8+
In this example, nodes that are reserved for {VirtProductName} components are labeled with the `key=virtualization:NoSchedule` taint. Only pods with the matching tolerations are scheduled to these nodes.
9+
10+
[source,yaml]
11+
----
12+
apiVersion: hco.kubevirt.io/v1beta1
13+
kind: HyperConverged
14+
metadata:
15+
name: kubevirt-hyperconverged
16+
namespace: openshift-cnv
17+
spec:
18+
workloads:
19+
nodePlacement:
20+
tolerations:
21+
- key: "key"
22+
operator: "Equal"
23+
value: "virtualization"
24+
effect: "NoSchedule"
25+
----

‎modules/virt-example-node-placement-tolerations-olm-subscription.adoc‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * virt/install/virt-specifying-nodes-for-virtualization-components.adoc
4+
5+
[id="virt-example-node-placement-tolerations-olm-subscription_{context}"]
6+
= Example: Node placement with tolerations in the OLM Subscription object
7+
8+
In this example, nodes that are reserved for OLM to deploy {VirtProductName} Operators are labeled with the `key=virtualization:NoSchedule` taint. Only pods with the matching tolerations are scheduled to these nodes.
9+
10+
[source,yaml,subs="attributes+"]
11+
----
12+
apiVersion: operators.coreos.com/v1alpha1
13+
kind: Subscription
14+
metadata:
15+
name: hco-operatorhub
16+
namespace: openshift-cnv
17+
spec:
18+
source: redhat-operators
19+
sourceNamespace: openshift-marketplace
20+
name: kubevirt-hyperconverged
21+
startingCSV: kubevirt-hyperconverged-operator.v{HCOVersion}
22+
channel: "stable"
23+
config:
24+
tolerations:
25+
- key: "key"
26+
operator: "Equal"
27+
value: "virtualization"
28+
effect: "NoSchedule"
29+
----

‎virt/install/installing-virt-cli.adoc‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@ Install {VirtProductName} to add virtualization functionality to your {product-t
88
cluster. You can subscribe to and deploy the {VirtProductName} Operators by
99
using the command line to apply manifests to your cluster.
1010

11+
[NOTE]
12+
====
13+
To specify the nodes where you want {VirtProductName} to install its components, xref:../../virt/install/virt-specifying-nodes-for-virtualization-components.adoc#virt-specifying-nodes-for-virtualization-components[configure node placement rules].
14+
====
15+
16+
[id="prerequisites_{context}"]
1117
== Prerequisites
1218
* Install {product-title} {product-version} on your cluster.
1319
* Install the OpenShift CLI (`oc`).

‎virt/install/installing-virt-web.adoc‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,12 @@ You can use the {product-title} {product-version}
1111
xref:../../web_console/web-console.adoc#web-console-overview_web-console[web console]
1212
to subscribe to and deploy the {VirtProductName} Operators.
1313

14+
[NOTE]
15+
====
16+
To specify the nodes where you want {VirtProductName} to install its components, xref:../../virt/install/virt-specifying-nodes-for-virtualization-components.adoc#virt-specifying-nodes-for-virtualization-components[configure node placement rules].
17+
====
18+
19+
[id="prerequisites_{context}"]
1420
== Prerequisites
1521
* Install {product-title} {product-version} on your cluster.
1622
* Log in as a user with `cluster-admin` permissions.

‎virt/install/virt-specifying-nodes-for-virtualization-components.adoc‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
[id="virt-specifying-nodes-for-virtualization-components"]
2+
= Specifying nodes for {VirtProductName} components
3+
include::modules/virt-document-attributes.adoc[]
4+
include::modules/common-attributes.adoc[]
5+
:context: virt-specifying-nodes-for-virtualization-components
6+
toc::[]
7+
8+
Specify the nodes where you want to deploy {VirtProductName} Operators, workloads, and controllers by configuring node placement rules during xref:../../virt/install/installing-virt-cli.adoc#installing-virt-cli[installation].
9+
10+
[NOTE]
11+
====
12+
You can configure node placement for some components after installing {VirtProductName}, but there must not be virtual machines present if you want to configure node placement for workloads.
13+
====
14+
15+
include::modules/virt-about-node-placement-virtualization-components.adoc[leveloffset=+1]
16+
17+
[id="additional-resources_{context}"]
18+
=== Additional resources
19+
20+
* xref:../../nodes/scheduling/nodes-scheduler-node-selectors.html#nodes-scheduler-node-selectors[Placing pods on specific nodes using node selectors]
21+
* xref:../../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity rules]
22+
* xref:../../nodes/scheduling/nodes-scheduler-taints-tolerations.adoc#nodes-scheduler-taints-tolerations[Controlling pod placement using node taints]
23+
24+
[id="example-manifests_{context}"]
25+
== Example manifests
26+
27+
The following example YAML snippets use `nodePlacement`, `affinity`, and `tolerations` objects to customize node placement for {VirtProductName} components.
28+
29+
[id="olm-subscription-examples_{context}"]
30+
=== Operator Lifecycle Manager Subscription examples
31+
32+
include::modules/virt-example-node-placement-tolerations-olm-subscription.adoc[leveloffset=+3]
33+
include::modules/virt-example-node-placement-node-selector-olm-subscription.adoc[leveloffset=+3]
34+
35+
[id="hyperconverged-cluster-cr-examples_{context}"]
36+
=== HyperConverged Cluster CR examples
37+
38+
include::modules/virt-example-node-placement-node-selector-hyperconverged-cr.adoc[leveloffset=+3]
39+
include::modules/virt-example-node-placement-affinity-hyperconverged-cr.adoc[leveloffset=+3]
40+
include::modules/virt-example-node-placement-tolerations-hyperconverged-cr.adoc[leveloffset=+3]

0 commit comments

Comments
 (0)