Skip to content

Commit ec1ead8

Browse files
StephenJamesSmithStephenJamesSmith
committed
TELCODOCS-1278: First draft
1 parent fb56db2 commit ec1ead8

8 files changed

+166
-0
lines changed

‎hardware_enablement/kmm-kernel-module-management.adoc

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,12 +62,34 @@ include::modules/kmm-adding-the-keys-for-secureboot.adoc[leveloffset=+1]
6262
include::modules/kmm-checking-the-keys.adoc[leveloffset=+2]
6363
include::modules/kmm-signing-a-prebuilt-driver-container.adoc[leveloffset=+1]
6464
include::modules/kmm-building-and-signing-a-moduleloader-container-image.adoc[leveloffset=+1]
65+
[role="_additional-resources"]
6566
.Additional resources
6667
For information on creating a service account, see link:https://docs.openshift.com/container-platform/4.12/authentication/understanding-and-creating-service-accounts.html#service-accounts-managing_understanding-service-accounts[Creating service accounts].
6768
6869
// Added for TELCODOCS-1277
6970
include::modules/kmm-customizing-upgrades-for-kernel-modules.adoc[leveloffset=+1]
7071
72+
// Added for TELCODOCS-1278
73+
include::modules/kmm-day1-kernel-module-loading.adoc[leveloffset=+1]
74+
[role="_additional-resources"]
75+
.Additional resources
76+
* link:https://docs.openshift.com/container-platform/4.13/post_installation_configuration/machine-configuration-tasks.html#machine-config-operator_post-install-machine-configuration-tasks[Machine Config Operator]
77+
78+
include::modules/kmm-day1-supported-use-cases.adoc[leveloffset=+2]
79+
include::modules/kmm-day1-oot-kernel-module-loading-flow.adoc[leveloffset=+2]
80+
include::modules/kmm-day1-kernel-module-image.adoc[leveloffset=+2]
81+
[role="_additional-resources"]
82+
.Additional resources
83+
* link:https://docs.openshift.com/container-platform/4.13/hardware_enablement/psap-driver-toolkit.html[Driver Toolkit]
84+
85+
include::modules/kmm-day1-in-tree-module-replacement.adoc[leveloffset=+2]
86+
include::modules/kmm-day1-mco-yaml-creation.adoc[leveloffset=+2]
87+
include::modules/kmm-day1-machineconfigpool.adoc[leveloffset=+2]
88+
89+
[role="_additional-resources"]
90+
.Additional resources
91+
* link:https://www.redhat.com/en/blog/openshift-container-platform-4-how-does-machine-config-pool-work[About MachineConfigPool]
92+
7193
include::modules/kmm-debugging-and-troubleshooting.adoc[leveloffset=+1]
7294
7395
// Added for TELCODOCS-1067

‎modules/kmm-day1-in-tree-module-replacement.adoc

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * hardware_enablement/kmm-kernel-module-management.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="kmm-day1-in-tree-module-replacement_{context}"]
7+
= In-tree module replacement
8+
9+
The Day 1 functionality always tries to replace the in-tree kernel module with the OOT version. If the in-tree kernel module is not loaded, the flow is not affected; the service proceeds and loads the OOT kernel module.

‎modules/kmm-day1-kernel-module-image.adoc

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * hardware_enablement/kmm-kernel-module-management.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="kmm-day1-kernel-module-image_{context}"]
7+
= The kernel module image
8+
9+
The Day 1 functionality uses the same DTK based image leveraged by Day 2 KMM builds. The out-of-tree kernel module should be located under `/opt/lib/modules/${kernelVersion}`.

‎modules/kmm-day1-kernel-module-loading.adoc

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * hardware_enablement/kmm-kernel-module-management.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="kmm-day1-kernel-module-loading_{context}"]
7+
= Day 1 kernel module loading
8+
9+
Kernel Module Management (KMM) is typically a Day 2 Operator. Kernel modules are loaded only after the complete initialization of a Linux (RHCOS) server. However, in some scenarios the kernel module must be loaded at an earlier stage. Day 1 functionality allows you to use the Machine Config Operator (MCO) to load kernel modules during the Linux `systemd` initialization stage.

‎modules/kmm-day1-machineconfigpool.adoc

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * hardware_enablement/kmm-kernel-module-management.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="kmm-day1-machineconfigpool_{context}"]
7+
= The MachineConfigPool
8+
9+
The `MachineConfigPool` identifies a collection of nodes that are affected by the applied MCO.
10+
11+
[source,yaml]
12+
----
13+
kind: MachineConfigPool
14+
metadata:
15+
name: sfc
16+
spec:
17+
machineConfigSelector: <1>
18+
matchExpressions:
19+
- {key: machineconfiguration.openshift.io/role, operator: In, values: [worker, sfc]}
20+
nodeSelector: <2>
21+
matchLabels:
22+
node-role.kubernetes.io/sfc: ""
23+
paused: false
24+
maxUnavailable: 1
25+
----
26+
<1> Matches the labels in the MachineConfig.
27+
<2> Matches the labels on the node.
28+
29+
There are predefined `MachineConfigPools` in the OCP cluster:
30+
31+
* `worker`: Targets all worker nodes in the cluster
32+
33+
* `master`: Targets all master nodes in the cluster
34+
35+
Define the following `MachineConfig` to target the master `MachineConfigPool`:
36+
37+
[source,yaml]
38+
----
39+
metadata:
40+
labels:
41+
machineconfiguration.opensfhit.io/role: master
42+
----
43+
44+
45+
Define the following `MachineConfig` to target the worker `MachineConfigPool`:
46+
47+
[source,yaml]
48+
----
49+
metadata:
50+
labels:
51+
machineconfiguration.opensfhit.io/role: worker
52+
----

‎modules/kmm-day1-mco-yaml-creation.adoc

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * hardware_enablement/kmm-kernel-module-management.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="kmm-day1-mco-yaml-creation_{context}"]
7+
= MCO yaml creation
8+
9+
KMM provides an API to create an MCO YAML manifest for the Day 1 functionality:
10+
11+
[source,console]
12+
----
13+
ProduceMachineConfig(machineConfigName, machineConfigPoolRef, kernelModuleImage, kernelModuleName string) (string, error)
14+
----
15+
16+
The returned output is a string representation of the MCO YAML manifest to be applied. It is up to the customer to apply this YAML.
17+
18+
The parameters are:
19+
20+
`machineConfigName`:: The name of the MCO YAML manifest. This parameter is set as the `name` parameter of the metadata of the MCO YAML manifest.
21+
22+
`machineConfigPoolRef`:: The `MachineConfigPool` name used to identify the targeted nodes.
23+
24+
`kernelModuleImage`:: The name of the container image that includes the OOT kernel module.
25+
26+
`kernelModuleName`:: The name of the OOT kernel module. This parameter is used both to unload the in-tree kernel module (if loaded into the kernel) and to load the OOT kernel module.
27+
28+
The API is located under `pkg/mcproducer` package of the KMM source code. The KMM operator does not need to be running to use the Day 1 functionality. You only need to import the `pkg/mcproducer` package into their operator/utility code, call the API, and apply the produced MCO YAML to the cluster.

‎modules/kmm-day1-oot-kernel-module-loading-flow.adoc

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * hardware_enablement/kmm-kernel-module-management.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="kmm-day1-oot-kernel-module-loading-flow_{context}"]
7+
= OOT kernel module loading flow
8+
9+
The loading of the out-of-tree (OOT) kernel module leverages the Machine Config Operator (MCO). The flow sequence is as follows:
10+
11+
.Procedure
12+
13+
. Apply a `MachineConfig` resource to the existing running cluster. In order to identify the necessary nodes that need to be updated,
14+
you must create an appropriate `MachineConfigPool` resource.
15+
16+
. MCO applies the reboots node by node. On any rebooted node, two new `systemd` services are deployed: `pull` service and `load` service.
17+
18+
. The `load` service is configured to run prior to the `NetworkConfiguration` service. The service tries to pull a predefined kernel module image and then, using that image, to unload an in-tree module and load an OOT kernel module.
19+
20+
. The `pull` service is configured to run after NetworkManager service. The service checks if the preconfigured kernel module image is located on the node's filesystem. If it is, the service exists normally, and the server continues with the boot process. If not, it pulls the image onto the node and reboots the node afterwards.

‎modules/kmm-day1-supported-use-cases.adoc

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * hardware_enablement/kmm-kernel-module-management.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="kmm-day1-supported-use-cases_{context}"]
7+
= Day 1 supported use cases
8+
9+
The Day 1 functionality supports a limited number of use cases. The main use case is to allow loading out-of-tree (OOT) kernel modules prior to NetworkManager service initialization. It does not support loading kernel module at the `initramfs` stage.
10+
11+
The following are the conditions needed for Day 1 functionality:
12+
13+
* The kernel module is not loaded in the kernel.
14+
15+
* The in-tree kernel module is loaded into the kernel, but can be unloaded and replaced by the OOT kernel module. This means that the in-tree module is not referenced by any other kernel modules.
16+
17+
* In order for Day 1 functionlity to work, the node must have a functional network interface, that is, an in-tree kernel driver for that interface. The OOT kernel module can be a network driver that will replace the functional network driver.

0 commit comments

Comments
 (0)