OCPBUGS-6268 updated the hardware enablement guide

Author: sheriff-rh

_topic_maps/_topic_map.yml

@@ -2559,7 +2559,7 @@ Topics:
 - Name: About specialized hardware and driver enablement
   File: about-hardware-enablement
 - Name: Driver Toolkit
-  File: psap-driver-toolkit
+  File: he-driver-toolkit
 - Name: Node Feature Discovery Operator
   File: psap-node-feature-discovery-operator
 ---

hardware_enablement/about-hardware-enablement.adoc

@@ -6,7 +6,6 @@ include::_attributes/common-attributes.adoc[]
 
 toc::[]
 
-Many applications require specialized hardware or software that depends on kernel modules or drivers. You can use driver containers to load out-of-tree kernel modules on {op-system-first} nodes. To deploy out-of-tree drivers during cluster installation, use the `kmods-via-containers` framework. To load drivers or kernel modules on an existing {product-title} cluster, {product-title} offers several tools:
+The Driver Toolkit (DTK) is a container image in the {product-title} payload which is meant to be used as a base image on which to build driver containers. The Driver Toolkit image contains the kernel packages commonly required as dependencies to build or install kernel modules as well as a few tools needed in driver containers. The version of these packages will match the kernel version running on the RHCOS nodes in the corresponding {product-title} release.
 
-* The Driver Toolkit is a container image that is a part of every {product-title} release. It contains the kernel packages and other common dependencies that are needed to build a driver or kernel module. The Driver Toolkit can be used as a base image for driver container image builds on {product-title}.
-* The Node Feature Discovery (NFD) Operator adds node labels for CPU capabilities, kernel version, PCIe device vendor IDs, and more.
+Driver containers are container images used for building and deploying out-of-tree kernel modules and drivers on container operating systems such as :op-system-first:. Kernel modules and drivers are software libraries running with a high level of privilege in the operating system kernel. They extend the kernel functionalities or provide the hardware-specific code required to control new devices. Examples include hardware devices like field-programmable gate arrays (FPGA) or graphics processing units (GPU), and software-defined storage solutions, which all require kernel modules on client machines. Driver containers are the first layer of the software stack used to enable these technologies on {product-title} deployments.

hardware_enablement/psap-driver-toolkit.adoc renamed to hardware_enablement/he-driver-toolkit.adoc

@@ -6,7 +6,7 @@ include::_attributes/common-attributes.adoc[]
 
 toc::[]
 
-Learn about the Driver Toolkit and how you can use it as a base image for driver containers for enabling special software and hardware devices on Kubernetes.
+Learn about the Driver Toolkit and how you can use it as a base image for driver containers for enabling special software and hardware devices on {product-title} deployments.
 
 :FeatureName: The Driver Toolkit
 

modules/psap-driver-toolkit-pulling.adoc

@@ -11,8 +11,8 @@ The `driver-toolkit` image is available from the link:https://registry.redhat.io
 [id="pulling-the-driver-toolkit-from-registry"]
 == Pulling the Driver Toolkit container image from registry.redhat.io
 
-Instructions for pulling the `driver-toolkit` image from `registry.redhat.io` with podman or in {product-title} can be found on the link:https://catalog.redhat.com/software/containers/openshift4/driver-toolkit-rhel8/604009d6122bd89307e00865?container-tabs=gti[Red Hat Ecosystem Catalog].
-The driver-toolkit image for the latest minor release will be tagged with the minor release version on registry.redhat.io for example `registry.redhat.io/openshift4/driver-toolkit-rhel8:v4.12`.
+Instructions for pulling the `driver-toolkit` image from `registry.redhat.io` with `podman` or in {product-title} can be found on the link:https://catalog.redhat.com/software/containers/openshift4/driver-toolkit-rhel8/604009d6122bd89307e00865?container-tabs=gti[Red Hat Ecosystem Catalog].
+The driver-toolkit image for the latest minor release are tagged with the minor release version on `registry.redhat.io`, for example: `registry.redhat.io/openshift4/driver-toolkit-rhel8:v{product-version}`.
 
 [id="pulling-the-driver-toolkit-from-payload"]
 == Finding the Driver Toolkit image URL in the payload
@@ -26,9 +26,13 @@ The driver-toolkit image for the latest minor release will be tagged with the mi
 
 . The image URL of the `driver-toolkit` corresponding to a certain release can be extracted from the release image using the `oc adm` command:
 +
-[source,terminal]
+[source,terminal,subs="attributes+"]
 ----
-$ oc adm release info 4.12.0 --image-for=driver-toolkit
+# For x86 image:
+$ oc adm release info quay.io/openshift-release-dev/ocp-release:{product-version}.z-x86_64 --image-for=driver-toolkit
+
+# For ARM image:
+$ oc adm release info quay.io/openshift-release-dev/ocp-release:{product-version}.z-aarch64 --image-for=driver-toolkit
 ----
 +
 .Example output
@@ -37,7 +41,7 @@ $ oc adm release info 4.12.0 --image-for=driver-toolkit
 quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0fd84aee79606178b6561ac71f8540f404d518ae5deff45f6d6ac8f02636c7f4
 ----
 
-. This image can be pulled using a valid pull secret, such as the pull secret required to install {product-title}.
+. This image can be obtained using a valid pull secret, such as the pull secret required to install {product-title}.
 
 [source,terminal]
 ----

modules/psap-driver-toolkit-using.adoc

@@ -6,11 +6,11 @@
 [id="using-the-driver-toolkit_{context}"]
 = Using the Driver Toolkit
 
-As an example, the Driver Toolkit can be used as the base image for building a very simple kernel module called simple-kmod.
+As an example, the Driver Toolkit can be used as the base image for building a very simple kernel module called `simple-kmod`.
 
 [NOTE]
 ====
-The Driver Toolkit contains the necessary dependencies, `openssl`, `mokutil`, and `keyutils`, needed to sign a kernel module. However, in this example, the simple-kmod kernel module is not signed and therefore cannot be loaded on systems with `Secure Boot` enabled.
+The Driver Toolkit includes the necessary dependencies, `openssl`, `mokutil`, and `keyutils`, needed to sign a kernel module. However, in this example, the `simple-kmod` kernel module is not signed and therefore cannot be loaded on systems with `Secure Boot` enabled.
 ====
 
 [id="create-simple-kmod-image_{context}"]
@@ -59,42 +59,40 @@ spec:
     - type: "ConfigChange"
     - type: "ImageChange"
   source:
-    git:
-      ref: "master"
-      uri: "https://github.com/openshift-psap/kvc-simple-kmod.git"
-    type: Git
     dockerfile: |
-      FROM DRIVER_TOOLKIT_IMAGE
+      ARG DTK
+      FROM ${DTK} as builder
+
+      ARG KVER
 
       WORKDIR /build/
-      
-      # Expecting kmod software version as an input to the build
-      ARG KMODVER
 
-      # Grab the software from upstream
       RUN git clone https://github.com/openshift-psap/simple-kmod.git
-      WORKDIR simple-kmod
-
-      # Build and install the module
-      RUN make all       KVER=$(rpm -q --qf "%{VERSION}-%{RELEASE}.%{ARCH}"  kernel-core) KMODVER=${KMODVER} \
-      && make install   KVER=$(rpm -q --qf "%{VERSION}-%{RELEASE}.%{ARCH}"  kernel-core) KMODVER=${KMODVER}
-
-      # Add the helper tools
-      WORKDIR /root/kvc-simple-kmod
-      ADD Makefile .
-      ADD simple-kmod-lib.sh .
-      ADD simple-kmod-wrapper.sh .
-      ADD simple-kmod.conf .
-      RUN mkdir -p /usr/lib/kvc/ \
-      && mkdir -p /etc/kvc/ \
-      && make install
-
-      RUN systemctl enable kmods-via-containers@simple-kmod
+
+      WORKDIR /build/simple-kmod
+
+      RUN make all install KVER=${KVER}
+
+      FROM registry.redhat.io/ubi8/ubi-minimal
+
+      ARG KVER
+
+      # Required for installing `modprobe`
+      RUN microdnf install kmod
+
+      COPY --from=builder /lib/modules/${KVER}/simple-kmod.ko /lib/modules/${KVER}/
+      COPY --from=builder /lib/modules/${KVER}/simple-procfs-kmod.ko /lib/modules/${KVER}/
+      RUN depmod ${KVER}
   strategy:
     dockerStrategy:
       buildArgs:
         - name: KMODVER
           value: DEMO
+          # $ oc adm release info quay.io/openshift-release-dev/ocp-release:<cluster version>-x86_64 --image-for=driver-toolkit
+        - name: DTK
+          value: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:34864ccd2f4b6e385705a730864c04a40908e57acede44457a783d739e377cae
+        - name: KVER
+          value: 4.18.0-372.26.1.el8_6.x86_64
   output:
     to:
       kind: ImageStreamTag
@@ -183,11 +181,14 @@ spec:
       - image: image-registry.openshift-image-registry.svc:5000/simple-kmod-demo/simple-kmod-driver-container:demo
         name: simple-kmod-driver-container
         imagePullPolicy: Always
-        command: ["/sbin/init"]
+        command: [sleep, infinity]
         lifecycle:
+          postStart:
+            exec:
+              command: ["modprobe", "-v", "-a" , "simple-kmod", "simple-procfs-kmod"]
           preStop:
             exec:
-              command: ["/bin/sh", "-c", "systemctl stop kmods-via-containers@simple-kmod"]
+              command: ["modprobe", "-r", "-a" , "simple-kmod", "simple-procfs-kmod"]
         securityContext:
           privileged: true
       nodeSelector:

modules/psap-driver-toolkit.adoc

@@ -8,9 +8,9 @@
 
 [discrete]
 == Background
-The Driver Toolkit is a container image in the {product-title} payload used as a base image on which you can build driver containers. The Driver Toolkit image contains the kernel packages commonly required as dependencies to build or install kernel modules, as well as a few tools needed in driver containers. The version of these packages will match the kernel version running on the {op-system-first} nodes in the corresponding {product-title} release. 
+The Driver Toolkit is a container image in the {product-title} payload used as a base image on which you can build driver containers. The Driver Toolkit image includes the kernel packages commonly required as dependencies to build or install kernel modules, as well as a few tools needed in driver containers. The version of these packages will match the kernel version running on the {op-system-first} nodes in the corresponding {product-title} release. 
 
-Driver containers are container images used for building and deploying out-of-tree kernel modules and drivers on container operating systems like {op-system}. Kernel modules and drivers are software libraries running with a high level of privilege in the operating system kernel. They extend the kernel functionalities or provide the hardware-specific code required to control new devices.  Examples include hardware devices like Field Programmable Gate Arrays (FPGA) or GPUs, and software-defined storage (SDS) solutions, such as Lustre parallel file systems, which require kernel modules on client machines. Driver containers are the first layer of the software stack used to enable these technologies on Kubernetes.
+Driver containers are container images used for building and deploying out-of-tree kernel modules and drivers on container operating systems like {op-system}. Kernel modules and drivers are software libraries running with a high level of privilege in the operating system kernel. They extend the kernel functionalities or provide the hardware-specific code required to control new devices. Examples include hardware devices like Field Programmable Gate Arrays (FPGA) or GPUs, and software-defined storage (SDS) solutions, such as Lustre parallel file systems, which require kernel modules on client machines. Driver containers are the first layer of the software stack used to enable these technologies on Kubernetes.
 
 The list of kernel packages in the Driver Toolkit includes the following and their dependencies:
 
@@ -27,7 +27,7 @@ In addition, the Driver Toolkit also includes the corresponding real-time kernel
 * `kernel-rt-modules`
 * `kernel-rt-modules-extra`
 
-The Driver Toolkit also has several tools which are commonly needed to build and install kernel modules, including:
+The Driver Toolkit also has several tools that are commonly needed to build and install kernel modules, including:
 
 * `elfutils-libelf-devel`
 * `kmod`
@@ -37,7 +37,6 @@ The Driver Toolkit also has several tools which are commonly needed to build and
 
 [discrete]
 == Purpose
-Prior to the Driver Toolkit's existence, you could install kernel packages in a pod or build config on {product-title} using link:https://www.openshift.com/blog/how-to-use-entitled-image-builds-to-build-drivercontainers-with-ubi-on-openshift[entitled builds] or by installing from the kernel RPMs in the hosts `machine-os-content`. The Driver Toolkit simplifies the process by removing the entitlement step, and avoids the privileged operation of accessing the machine-os-content in a pod. The Driver Toolkit can also be used by partners who have access to pre-released {product-title} versions to prebuild driver-containers for their hardware devices for future {product-title} releases.
-
-The Driver Toolkit is also used by the Special Resource Operator (SRO), which is currently available as a community Operator on OperatorHub. SRO supports out-of-tree and third-party kernel drivers and the support software for the underlying operating system. Users can create _recipes_ for SRO to build and deploy a driver container, as well as support software like a device plugin, or metrics. Recipes can include a build config to build a driver container based on the Driver Toolkit, or SRO can deploy a prebuilt driver container.
+Prior to the Driver Toolkit's existence, users would install kernel packages in a pod or build config on {product-title} using link:https://www.openshift.com/blog/how-to-use-entitled-image-builds-to-build-drivercontainers-with-ubi-on-openshift[entitled builds] or by installing from the kernel RPMs in the hosts `machine-os-content`. The Driver Toolkit simplifies the process by removing the entitlement step, and avoids the privileged operation of accessing the machine-os-content in a pod. The Driver Toolkit can also be used by partners who have access to pre-released {product-title} versions to prebuild driver-containers for their hardware devices for future {product-title} releases.
 
+The Driver Toolkit is also used by the Kernel Module Management (KMM), which is currently available as a community Operator on OperatorHub. KMM supports out-of-tree and third-party kernel drivers and the support software for the underlying operating system. Users can create modules for KMM to build and deploy a driver container, as well as support software like a device plugin, or metrics. Modules can include a build config to build a driver container-based on the Driver Toolkit, or KMM can deploy a prebuilt driver container.