Merge pull request #31909 from ousleyp/cnv-7685

CNV-7685: node placement for VMs

Author: ousleyp

_topic_map.yml

@@ -2655,6 +2655,8 @@ Topics:
     Dir: advanced_vm_management
     Topics:
 #Advanced virtual machine configuration
+    - Name: Specifying nodes for virtual machines
+      File: virt-specifying-nodes-for-vms
     - Name: Automating management tasks
       File: virt-automating-management-tasks
     - Name: EFI mode for virtual machines

modules/virt-about-node-placement-vms.adoc

@@ -0,0 +1,29 @@
+// Module included in the following assemblies:
+//
+// * virt/virtual_machines/advanced_vm_management/virt-specifying-nodes-for-vms.adoc
+
+[id="virt-about-node-placement-vms_{context}"]
+= About node placement for virtual machines
+
+To ensure that virtual machines (VMs) run on appropriate nodes, you can configure node placement rules. You might want to do this if:
+
+* You have several VMs. To ensure fault tolerance, you want them to run on different nodes.
+* You have two chatty VMs. To avoid redundant inter-node routing, you want the VMs to run on the same node.
+* Your VMs require specific hardware features that are not present on all available nodes.
+* You have a pod that adds capabilities to a node, and you want to place a VM on that node so that it can use those capabilities.
+
+[NOTE]
+====
+Virtual machine placement relies on any existing node placement rules for workloads. If workloads are excluded from specific nodes on the component level, virtual machines cannot be placed on those nodes.
+====
+
+You can use the following rule types in the `spec` field of a `VirtualMachine` manifest:
+
+`nodeSelector`:: Allows virtual machines to be scheduled on nodes that are labeled with the key-value pair or pairs that you specify in this field. The node must have labels that exactly match all listed pairs.
+`affinity`:: Enables you to use more expressive syntax to set rules that match nodes with virtual machines. For example, you can specify that a rule is a preference, rather than a hard requirement, so that virtual machines are still scheduled if the rule is not satisfied. Pod affinity, pod anti-affinity, and node affinity are supported for virtual machine placement. Pod affinity works for virtual machines because the `VirtualMachine` workload type is based on the `Pod` object.
++
+[NOTE]
+====
+Affinity rules only apply during scheduling. {product-title} does not reschedule running workloads if the constraints are no longer met.
+====
+`tolerations`:: Allows virtual machines to be scheduled on nodes that have matching taints. If a taint is applied to a node, that node only accepts virtual machines that tolerate the taint.

modules/virt-example-vm-node-placement-node-affinity.adoc

@@ -0,0 +1,41 @@
+// Module included in the following assemblies:
+//
+// * virt/virtual_machines/advanced_vm_management/virt-specifying-nodes-for-vms.adoc
+
+[id="virt-example-vm-node-placement-node-affinity_{context}"]
+= Example: VM node placement with node affinity
+
+In this example, the VM must be scheduled on a node that has the label `example.io/example-key = example-value-1` or the label `example.io/example-key = example-value-2`. The constraint is met if only one of the labels is present on the node. If neither label is present, the VM is not scheduled.
+
+If possible, the scheduler avoids nodes that have the label `example-node-label-key = example-node-label-value`. However, if all candidate nodes have this label, the scheduler ignores this constraint.
+
+.Example VM manifest
+[source,yaml]
+----
+metadata:
+  name: example-vm-node-affinity
+apiVersion: kubevirt.io/v1alpha3
+kind: VirtualMachine
+spec:
+  affinity:
+    nodeAffinity:
+      requiredDuringSchedulingIgnoredDuringExecution: <1>
+        nodeSelectorTerms:
+        - matchExpressions:
+          - key: example.io/example-key
+            operator: In
+            values:
+            - example-value-1
+            - example-value-2
+      preferredDuringSchedulingIgnoredDuringExecution: <2>
+      - weight: 1
+        preference:
+          matchExpressions:
+          - key: example-node-label-key
+            operator: In
+            values:
+            - example-node-label-value
+...
+----
+<1> If you use the `requiredDuringSchedulingIgnoredDuringExecution` rule type, the VM is not scheduled if the constraint is not met.
+<2> If you use the `preferredDuringSchedulingIgnoredDuringExecution` rule type, the VM is still scheduled if the constraint is not met, as long as all required constraints are met.

modules/virt-example-vm-node-placement-node-selector.adoc

@@ -0,0 +1,27 @@
+// Module included in the following assemblies:
+//
+// * virt/virtual_machines/advanced_vm_management/virt-specifying-nodes-for-vms.adoc
+
+[id="virt-example-vm-node-placement-node-selector_{context}"]
+= Example: VM node placement with nodeSelector
+
+In this example, the virtual machine requires a node that has metadata containing both `example-key-1 = example-value-1` and `example-key-2 = example-value-2` labels.
+
+[WARNING]
+====
+If there are no nodes that fit this description, the virtual machine is not scheduled.
+====
+
+.Example VM manifest
+[source,yaml]
+----
+metadata:
+  name: example-vm-node-selector
+apiVersion: kubevirt.io/v1alpha3
+kind: VirtualMachine
+spec:
+  nodeSelector:
+    example-key-1: example-value-1
+    example-key-2: example-value-2
+...
+----

modules/virt-example-vm-node-placement-pod-affinity.adoc

@@ -0,0 +1,44 @@
+// Module included in the following assemblies:
+//
+// * virt/virtual_machines/advanced_vm_management/virt-specifying-nodes-for-vms.adoc
+
+[id="virt-example-vm-node-placement-pod-affinity_{context}"]
+= Example: VM node placement with pod affinity and pod anti-affinity
+
+In this example, the VM must be scheduled on a node that has a running pod with the label `example-key-1 = example-value-1`. If there is no such pod running on any node, the VM is not scheduled.
+
+If possible, the VM is not scheduled on a node that has any pod with the label `example-key-2 = example-value-2`. However, if all candidate nodes have a pod with this label, the scheduler ignores this constraint.
+
+.Example VM manifest
+[source,yaml]
+----
+metadata:
+  name: example-vm-pod-affinity
+apiVersion: kubevirt.io/v1alpha3
+kind: VirtualMachine
+spec:
+  affinity:
+    podAffinity:
+      requiredDuringSchedulingIgnoredDuringExecution: <1>
+      - labelSelector:
+          matchExpressions:
+          - key: example-key-1
+            operator: In
+            values:
+            - example-value-1
+        topologyKey: kubernetes.io/hostname
+    podAntiAffinity:
+      preferredDuringSchedulingIgnoredDuringExecution: <2>
+      - weight: 100
+        podAffinityTerm:
+          labelSelector:
+            matchExpressions:
+            - key: example-key-2
+              operator: In
+              values:
+              - example-value-2
+          topologyKey: kubernetes.io/hostname
+...
+----
+<1> If you use the `requiredDuringSchedulingIgnoredDuringExecution` rule type, the VM is not scheduled if the constraint is not met.
+<2> If you use the `preferredDuringSchedulingIgnoredDuringExecution` rule type, the VM is still scheduled if the constraint is not met, as long as all required constraints are met.

modules/virt-example-vm-node-placement-tolerations.adoc

@@ -0,0 +1,29 @@
+// Module included in the following assemblies:
+//
+// * virt/virtual_machines/advanced_vm_management/virt-specifying-nodes-for-vms.adoc
+
+[id="virt-example-vm-node-placement-tolerations_{context}"]
+= Example: VM node placement with tolerations
+
+In this example, nodes that are reserved for virtual machines are already labeled with the `key=virtualization:NoSchedule` taint. Because this virtual machine has matching `tolerations`, it can schedule onto the tainted nodes.
+
+[NOTE]
+====
+A virtual machine that tolerates a taint is not required to schedule onto a node with that taint.
+====
+
+.Example VM manifest
+[source,yaml]
+----
+metadata:
+  name: example-vm-tolerations
+apiVersion: kubevirt.io/v1alpha3
+kind: VirtualMachine
+spec:
+  tolerations:
+  - key: "key"
+    operator: "Equal"
+    value: "virtualization"
+    effect: "NoSchedule"
+...
+----

virt/install/virt-specifying-nodes-for-virtualization-components.adoc

@@ -20,6 +20,7 @@ include::modules/virt-about-node-placement-virtualization-components.adoc[levelo
 
 [id="node-placement-resources_{context}"]
 ==== Node placement rules
+* xref:../../virt/virtual_machines/advanced_vm_management/virt-specifying-nodes-for-vms.adoc#virt-specifying-nodes-for-vms[Specifying nodes for virtual machines]
 * xref:../../nodes/scheduling/nodes-scheduler-node-selectors.adoc#nodes-scheduler-node-selectors[Placing pods on specific nodes using node selectors]
 * xref:../../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity rules]
 * xref:../../nodes/scheduling/nodes-scheduler-taints-tolerations.adoc#nodes-scheduler-taints-tolerations[Controlling pod placement using node taints]

virt/virtual_machines/advanced_vm_management/virt-specifying-nodes-for-vms.adoc

@@ -0,0 +1,29 @@
+[id="virt-specifying-nodes-for-vms"]
+= Specifying nodes for virtual machines
+include::modules/virt-document-attributes.adoc[]
+include::modules/common-attributes.adoc[]
+:context: virt-specifying-nodes-for-vms
+
+toc::[]
+
+You can place virtual machines (VMs) on specific nodes by using node placement rules.
+
+include::modules/virt-about-node-placement-vms.adoc[leveloffset=+1]
+
+[id="node-placement-examples_{context}"]
+== Node placement examples
+
+The following example YAML file snippets use `nodePlacement`, `affinity`, and `tolerations` fields to customize node placement for virtual machines.
+
+include::modules/virt-example-vm-node-placement-node-selector.adoc[leveloffset=+2]
+include::modules/virt-example-vm-node-placement-pod-affinity.adoc[leveloffset=+2]
+include::modules/virt-example-vm-node-placement-node-affinity.adoc[leveloffset=+2]
+include::modules/virt-example-vm-node-placement-tolerations.adoc[leveloffset=+2]
+
+[id="additional-resources_{context}"]
+== Additional resources
+
+* xref:../../../virt/install/virt-specifying-nodes-for-virtualization-components.adoc#virt-specifying-nodes-for-virtualization-components[Specifying nodes for virtualization components]
+* xref:../../../nodes/scheduling/nodes-scheduler-node-selectors.adoc#nodes-scheduler-node-selectors[Placing pods on specific nodes using node selectors]
+* xref:../../../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity rules]
+* xref:../../../nodes/scheduling/nodes-scheduler-taints-tolerations.adoc#nodes-scheduler-taints-tolerations[Controlling pod placement using node taints]