Merge pull request #53376 from aireilly/telcodocs-1013

Author: adellape

_topic_maps/_topic_map.yml

@@ -2394,7 +2394,7 @@ Topics:
 - Name: Creating a performance profile
   File: cnf-create-performance-profiles
   Distros: openshift-origin,openshift-enterprise
-- Name: Workload partitioning on single-node OpenShift
+- Name: Workload partitioning in single-node OpenShift
   File: sno-du-enabling-workload-partitioning-on-single-node-openshift
   Distros: openshift-origin,openshift-enterprise
 - Name: Requesting CRI-O and Kubelet profiling data by using the Node Observability Operator

modules/sno-du-enabling-workload-partitioning.adoc

@@ -18,7 +18,7 @@ You can check that clusters are running the correct configuration. The following
 
 .Procedure
 
-. Check that the default Operator Hub sources are disabled. Run the following command:
+. Check that the default OperatorHub sources are disabled. Run the following command:
 +
 [source,terminal]
 ----
@@ -74,7 +74,7 @@ openshift-authentication-operator -- management
 +
 [IMPORTANT]
 ====
-Additional Operators must not be annotated for workload partitioning. In the output from the previous command, additional Operators should be listed without any value on the right-hand side of the `--` separator.
+Additional Operators must not be annotated for workload partitioning. In the output from the previous command, additional Operators should be listed without any value on the right side of the `--` separator.
 ====
 
 . Check that the `ClusterLogging` configuration is correct. Run the following commands:
@@ -564,7 +564,7 @@ $ oc get performanceprofile -o jsonpath="{ .items[0].spec.cpu.reserved }"
 .Example output
 [source,terminal]
 ----
-0-1,52-53
+0-3
 ----
 +
 [NOTE]

modules/ztp-checking-du-cluster-config.adoc

@@ -0,0 +1,25 @@
+// Module included in the following assemblies:
+//
+// * scalability_and_performance/ztp_far_edge/ztp-reference-cluster-configuration-for-vdu.adoc
+
+:_content-type: CONCEPT
+[id="ztp-workload-partitioning-sno_{context}"]
+= Workload partitioning in {sno} with GitOps ZTP
+
+Workload partitioning configures {product-title} services, cluster management workloads, and infrastructure pods to run on a reserved number of host CPUs.
+
+To configure workload partitioning with GitOps ZTP, you specify cluster management CPU resources with the `cpuset` field of the `SiteConfig` custom resource (CR) and the `reserved` field of the group `PolicyGenTemplate` CR.
+The GitOps ZTP pipeline uses these values to populate the required fields in the workload partitioning `MachineConfig` CR (`cpuset`) and the `PerformanceProfile` CR (`reserved`) that configure the {sno} cluster.
+
+[NOTE]
+====
+For maximum performance, ensure that the `reserved` and `isolated` CPU sets do not share CPU cores across NUMA zones.
+====
+
+* The workload partitioning `MachineConfig` CR pins the {product-title} infrastructure pods to a defined `cpuset` configuration.
+* The `PerformanceProfile` CR pins the systemd services to the reserved CPUs.
+
+[IMPORTANT]
+====
+The value for the `reserved` field specified in the `PerformanceProfile` CR must match the `cpuset` field in the workload partitioning `MachineConfig` CR.
+====

modules/ztp-enabling-workload-partitioning-sno.adoc

@@ -41,3 +41,49 @@ If Hyper-Threading is enabled, specify both threads for each core. The `CPUs` va
 }
 ----
 <1> The `cpuset` must match the `CPUs` value in `/etc/crio/crio.conf.d/01-workload-partitioning`.
+
+.Verification
+
+Check that the applications and cluster system CPU pinning is correct. Run the following commands:
+
+. Open a remote shell connection to the managed cluster:
++
+[source,terminal]
+----
+$ oc debug node/example-sno-1
+----
+
+. Check that the user applications CPU pinning is correct:
++
+[source,terminal]
+----
+sh-4.4# pgrep ovn | while read i; do taskset -cp $i; done
+----
++
+.Example output
+[source,terminal]
+----
+pid 8481's current affinity list: 0-3
+pid 8726's current affinity list: 0-3
+pid 9088's current affinity list: 0-3
+pid 9945's current affinity list: 0-3
+pid 10387's current affinity list: 0-3
+pid 12123's current affinity list: 0-3
+pid 13313's current affinity list: 0-3
+----
+
+. Check that the system applications CPU pinning is correct:
++
+[source,terminal]
+----
+sh-4.4# pgrep systemd | while read i; do taskset -cp $i; done
+----
++
+.Example output
+[source,terminal]
+----
+pid 1's current affinity list: 0-3
+pid 938's current affinity list: 0-3
+pid 962's current affinity list: 0-3
+pid 1197's current affinity list: 0-3
+----

modules/ztp-sno-du-enabling-workload-partitioning.adoc

@@ -1,6 +1,6 @@
 :_content-type: ASSEMBLY
 [id="sno-du-enabling-workload-partitioning-on-single-node-openshift"]
-= Workload partitioning on {sno}
+= Workload partitioning in {sno}
 include::_attributes/common-attributes.adoc[]
 :context: sno-du-enabling-workload-partitioning-on-single-node-openshift
 
@@ -23,4 +23,7 @@ When applying workload partitioning, use the Node Tuning Operator to implement t
 Workload partitioning introduces a new extended resource of `<workload-type>.workload.openshift.io/cores`
 for each defined CPU pool, or workload-type. Kubelet advertises these new resources and CPU requests by pods allocated to the pool are accounted for within the corresponding resource rather than the typical `cpu` resource. When workload partitioning is enabled, the `<workload-type>.workload.openshift.io/cores` resource allows access to the CPU capacity of the host, not just the default CPU pool.
 
-include::modules/sno-du-enabling-workload-partitioning.adoc[leveloffset=+1]
+[role="_additional-resources"]
+.Additional resources
+
+* For the recommended workload partitioning configuration for {sno} clusters, see xref:../scalability_and_performance/ztp_far_edge/ztp-reference-cluster-configuration-for-vdu.adoc#ztp-sno-du-enabling-workload-partitioning_sno-configure-for-vdu[Workload partitioning].

scalability_and_performance/sno-du-enabling-workload-partitioning-on-single-node-openshift.adoc

@@ -23,7 +23,7 @@ include::modules/ztp-generating-install-and-config-crs-manually.adoc[leveloffset
 [role="_additional-resources"]
 .Additional resources
 
-* xref:../../scalability_and_performance/ztp_far_edge/ztp-manual-install.adoc#ztp-manually-install-a-single-managed-cluster_ztp-manual-install[Installing a single managed cluster]
+* xref:../../scalability_and_performance/ztp_far_edge/ztp-reference-cluster-configuration-for-vdu.adoc#ztp-sno-du-enabling-workload-partitioning_sno-configure-for-vdu[Workload partitioning]
 
 * xref:../../installing/installing_bare_metal_ipi/ipi-install-installation-workflow.adoc#bmc-addressing_ipi-install-installation-workflow[BMC addressing]
 

scalability_and_performance/ztp_far_edge/ztp-manual-install.adoc

@@ -23,6 +23,13 @@ include::modules/ztp-du-host-firmware-requirements.adoc[leveloffset=+1]
 
 include::modules/ztp-managed-cluster-network-prereqs.adoc[leveloffset=+1]
 
+include::modules/ztp-enabling-workload-partitioning-sno.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+.Additional resources
+
+* For the recommended {sno} workload partitioning configuration, see xref:../../scalability_and_performance/ztp_far_edge/ztp-reference-cluster-configuration-for-vdu.adoc#ztp-sno-du-enabling-workload-partitioning_sno-configure-for-vdu[Workload partitioning].
+
 [id="ztp-sno-install-time-cluster-config"]
 == Recommended installation-time cluster configurations
 

scalability_and_performance/ztp_far_edge/ztp-reference-cluster-configuration-for-vdu.adoc

@@ -43,8 +43,8 @@ spec:
         bootMode: "UEFI" <10>
         rootDeviceHints:
           wwn: "0x11111000000asd123"
-        cpuset: "0-1,52-53"
-        nodeNetwork: <11>
+        cpuset: "0-1,52-53"  <11>
+        nodeNetwork: <12>
           interfaces:
             - name: eno1
               macAddress: "AA:BB:CC:DD:EE:11"
@@ -56,7 +56,7 @@ spec:
                 macAddress: "AA:BB:CC:DD:EE:11"
                 ipv4:
                   enabled: false
-                ipv6: <12>
+                ipv6: <13>
                   enabled: true
                   address:
                   - ip: 1111:2222:3333:4444::aaaa:1
@@ -84,5 +84,6 @@ spec:
 <8> Applies to all cluster types. Specifies the BMC address. ZTP supports iPXE and virtual media booting by using Redfish or IPMI protocols.
 <9> Create the `bmh-secret` CR that specifies the BMC credentials. Use the same namespace as the `SiteConfig` CR.
 <10> Use `UEFISecureBoot` to enable secure boot on the host.
-<11> Specifies the network settings for the node.
-<12> Configures the IPv6 address for the host. For {sno} clusters with static IP addresses, the node-specific API and Ingress IPs should be the same.
+<11> `cpuset` should match the value set in the cluster `PerformanceProfile` CR `.spec.cpu.reserved` field for workload partitioning.
+<12> Specifies the network settings for the node.
+<13> Configures the IPv6 address for the host. For {sno} clusters with static IP addresses, the node-specific API and Ingress IPs should be the same.

snippets/ztp-example-siteconfig.adoc

@@ -30,6 +30,11 @@ spec:
 <1> Ensure that the value for `name` matches that specified in the `spec.profile.data` field of `TunedPerformancePatch.yaml` and the `status.configuration.source.name` field of `validatorCRs/informDuValidator.yaml`.
 <2> Configures UEFI secure boot for the cluster host.
 <3> Set the isolated CPUs. Ensure all of the Hyper-Threading pairs match.
++
+[IMPORTANT]
+====
+The reserved and isolated CPU pools must not overlap and together must span all available cores. CPU cores that are not accounted for cause an undefined behaviour in the system.
+====
 <4> Set the reserved CPUs. When workload partitioning is enabled, system processes, kernel threads, and system container threads are restricted to these CPUs. All CPUs that are not isolated should be reserved.
 <5> Set the number of huge pages.
 <6> Set the huge page size.