Merge pull request #33927 from StephenJamesSmith/telcodocs-157

TELCODOCS-157: workload partitioning

Author: vikram-redhat

modules/cnf-deploying-the-du-infrastructure-profile.adoc

@@ -3,7 +3,7 @@
 //
 // *cnf-provisioning-and-deploying-a-distributed-unit.adoc
 
-[id="scalability_and_performance/cnf-deploying-the-du-infrastructure-profile_{context}"]
+[id="cnf-deploying-the-du-infrastructure-profile_{context}"]
 = Deploying the DU infrastructure profile
 
 [id="cnf-creating-the-performance-addon-operator-and-du-performance-profile_{context}"]

modules/cnf-du-configuring-a-performance-profile-to-support-workload-partitioning.adoc

@@ -0,0 +1,11 @@
+// Module included in the following assemblies:
+//
+// *scalability_and_performance/cnf-provisioning-and-installing-a-distributed-unit.adoc
+
+[id="cnf-du-configuring-a-performance-profile-to-support-workload-partitioning.adoc_{context}"]
+
+= Configuring a performance profile to support workload partitioning
+
+After you have configured workload partitioning, you need to ensure that the Performance Addon Operator has been installed and that you configured a performance profile.
+
+The reserved CPU IDs in the performance profile must match the workload partitioning CPU IDs.

modules/cnf-du-configuring-workload-partitioning.adoc

@@ -0,0 +1,20 @@
+// Module included in the following assemblies:
+//
+// *scalability_and_performance/cnf-provisioning-and-installing-a-distributed-unit.adoc
+
+[id="cnf-du-configuring-workload-partitioning_{context}"]
+
+= Configuring workload partitioning
+
+The following procedure outlines a high level, end to end workflow that installs a cluster with workload partitioning enabled and pods that
+are correctly scheduled to run on the management CPU partition.
+
+. Create a machine config manifest to configure CRI-O to partition management workloads. The cpuset that you specify must match the reserved cpuset that you specified in the performance-addon-operator profile.
+
+. Create another machine config manifest to write a configuration file for kubelet to enable the same workload partition. The file is only readable by the kubelet.
+
+. Run `openshift-install` to create the standard manifests, adds their extra manifests from steps 1 and 2, then creates the cluster.
+
+. For pods and namespaces that are correctly annotated, the CPU request values are zeroed out and converted to `<workload-type>.workload.openshift.io/cores`. This modified resource allows the pods to be constrained to the restricted CPUs.
+
+. The single node cluster starts with management components constrained to a subset of available CPUs.

modules/cnf-du-creating-a-machine-config-manifest-for-workload-partitioning.adoc

@@ -0,0 +1,81 @@
+// Module included in the following assemblies:
+//
+// *scalability_and_performance/cnf-provisioning-and-installing-a-distributed-unit.adoc
+
+[id="cnf-du-creating-a-machine-config-manifest-for-workload-partitioning_{context}"]
+
+= Creating a machine config manifest for workload partitioning
+
+Part of configuring workload partitioning requires you to provide a `MachineConfig` manifest during installation to configure CRI-O and kubelet for the workload types.
+
+The manifest, without the encoded file content, looks like this:
+
+[source,yaml]
+----
+apiVersion: machineconfiguration.openshift.io/v1
+kind: MachineConfig
+metadata:
+  labels:
+    machineconfiguration.openshift.io/role: master
+  name: 02-master-workload-partitioning
+spec:
+  config:
+    ignition:
+      version: 3.2.0
+    storage:
+      files:
+      - contents:
+          source: data:text/plain;charset=utf-8;base64,<01-workload-partitioning-content>
+        mode: 420
+        overwrite: true
+        path: /etc/crio/crio.conf.d/01-workload-partitioning
+        user:
+          name: root
+      - contents:
+          source: data:text/plain;charset=utf-8;base64,<openshift-workload-pinning content>
+        mode: 420
+        overwrite: true
+        path: /etc/kubernetes/openshift-workload-pinning
+        user:
+          name: root
+----
+
+The contents of `/etc/crio/crio.conf.d/01-workload-partitioning` should look like this.
+
+[source,yaml]
+----
+[crio.runtime.workloads.management]
+activation_annotation = "target.workload.openshift.io/management"
+annotation_prefix = "resources.workload.openshift.io"
+resources = { "cpushares" = 0, "cpuset" = "0-1,52-53" } <1>
+----
+<1> The `cpuset` value will vary based on the installation.
+
+If hyperthreading is enabled, specify both threads of each core. The `cpuset` must match the reserved CPU set specified in the performance profile.
+
+
+This content should be base64 encoded and provided in the `01-workload-partitioning-content` in the manifest above.
+
+The contents of `/etc/kubernetes/openshift-workload-pinning` should look like this:
+
+----
+{
+  "management": {
+    "cpuset": "0-1,52-53" <1>
+  }
+}
+----
+<1> The `cpuset` must match the value in `/etc/crio/crio.conf.d/01-workload-partitioning`.
+
+This content should be base64 encoded and provided in the `openshift-workload-pinning-content` in the preceding manifest.
+
+[NOTE]
+====
+The `cpuset` specified must match the reserved `cpuset` specified in the Performance Addon Operator profile.
+====
+
+[NOTE]
+====
+In this release, configuring machines for workload partitioning must be enabled during installation to work correctly.
+Once enabled, changes to the machine configs that enable the feature are not supported.
+====

modules/cnf-du-crio-configuration-for-workload-partitioning.adoc

@@ -0,0 +1,34 @@
+// Module included in the following assemblies:
+//
+// *scalability_and_performance/cnf-provisioning-and-deploying-a-distributed-unit.adoc
+
+[id="cnf-du-crio-configuration-for-workload-partitioning_{context}"]
+
+= CRI-O configuration for workload partitioning
+
+In support of workload partitioning, CRI-O supports new configuration settings. The configuration file is delivered to a host as part of a machine config.
+
+[source,terminal]
+----
+[crio.runtime.workloads.{workload-type}]
+  activation_annotation = "target.workload.openshift.io/<workload-type>" <1>
+  annotation_prefix = "resources.workload.openshift.io" <2>
+  resources = { "cpushares" = 0, "cpuset" = "0-1,52-53" } <3>
+----
+<1> Use the `activation_annotation` field to match pods that should be treated as having the workload type. The annotation key on the pod is compared for an exact match against the value specified in the configuration file. In this release, the only supported workload-type is `management`.
+
+<2> The `annotation_prefix` is the start of the annotation key that passes settings from the admission hook down to CRI-O.
+
+<3> The `resources` map associates annotation suffixes with default values. CRI-O defines a well-known set of resources and other values are not allowed. The `cpuset` value must match the kubelet configuration file and the reserved `cpuset` in the applied PerformanceProfile.
+
+In the management workload case, it is configured as follows:
+
+[source,terminal]
+----
+[crio.runtime.workloads.management]
+  activation_annotation = "target.workload.openshift.io/management"
+  annotation_prefix = "resources.workload.openshift.io"
+  resources = { "cpushares" = 0, "cpuset" = "0-1,52-53" }
+----
+
+Pods that have the `target.workload.openshift.io/management` annotation will have their `cpuset` configured to the value from the appropriate workload configuration. The CPU shares for each container in the pod are configured according to the `management.workload.openshift.io/cores` resource limit, which ensures the pod's CPU shares are enforced.