Skip to content

Commit a1d203d

Browse files
mburke5678mburke5678
authored
Merge pull request #87730 from mburke5678/on-cluster-layering-4.18-upgrades
MCO-1316 On-Cluster Layering GA - 4.18 upgrades and integrations
2 parents b768569 + de57136 commit a1d203d

8 files changed

+524
-68
lines changed

‎machine_configuration/mco-coreos-layering.adoc

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -184,13 +184,30 @@ include::modules/coreos-layering-configuring-on.adoc[leveloffset=+1]
184184

185185
.Additional resources
186186
* xref:../nodes/clusters/nodes-cluster-enabling-features.adoc#nodes-cluster-enabling[Enabling features using feature gates]
187+
* xref:../updating/updating_a_cluster/update-using-custom-machine-config-pools.adoc#update-using-custom-machine-config-pools-pause_update-using-custom-machine-config-pools[Pausing the machine config pools]
188+
189+
include::modules/coreos-layering-configuring-on-modifying.adoc[leveloffset=+2]
190+
191+
.Additional resources
192+
* xref:../updating/updating_a_cluster/update-using-custom-machine-config-pools.adoc#update-using-custom-machine-config-pools-pause_update-using-custom-machine-config-pools[Pausing the machine config pools]
193+
194+
include::modules/coreos-layering-configuring-on-extensions.adoc[leveloffset=+2]
195+
196+
.Additional resources
197+
* xref:../machine_configuration/machine-configs-configure.html#rhcos-add-extensions_machine-configs-configure[Adding extensions to RHCOS]
198+
* xref:../updating/updating_a_cluster/update-using-custom-machine-config-pools.adoc#update-using-custom-machine-config-pools-pause_update-using-custom-machine-config-pools[Pausing the machine config pools]
199+
200+
// Not in 4.18; maybe 4.19
201+
// include::modules/coreos-layering-configuring-on-rebuild.adoc[leveloffset=+2]
202+
203+
include::modules/coreos-layering-configuring-on-revert.adoc[leveloffset=+2]
187204

188205
include::modules/coreos-layering-configuring.adoc[leveloffset=+1]
189206

190207
.Additional resources
191208
xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-updating_mco-coreos-layering[Updating with a {op-system} custom layered image]
192209

193-
include::modules/coreos-layering-removing.adoc[leveloffset=+1]
210+
include::modules/coreos-layering-removing.adoc[leveloffset=+2]
194211
include::modules/coreos-layering-updating.adoc[leveloffset=+1]
195212

196213
////

‎modules/coreos-layering-configuring-on-extensions.adoc

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * machine_configuration/coreos-layering.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="coreos-layering-configuring-on-extensions_{context}"]
7+
= Installing extensions into an on-cluster custom layered image
8+
9+
You can install {op-system-first} extensions into your on-cluster custom layered image by creating a machine config that lists the extensions that you want to install. The Machine Config Operator (MCO) installs the extensions onto the nodes associated with a specific machine config pool (MCP).
10+
11+
For a list of the currently supported extensions, see "Adding extensions to RHCOS."
12+
13+
After you make the change, the MCO reboots the nodes associated with the specified machine config pool.
14+
15+
[NOTE]
16+
====
17+
include::snippets/coreos-layering-configuring-on-pause.adoc[]
18+
====
19+
20+
.Prerequisites
21+
22+
* You have opted in to on-cluster layering by creating a `MachineOSConfig` object.
23+
24+
.Procedure
25+
26+
. Create a YAML file for the machine config similar to the following example:
27+
+
28+
[source,yaml]
29+
----
30+
apiVersion: machineconfiguration.openshift.io/v1 <1>
31+
kind: MachineConfig
32+
metadata:
33+
labels:
34+
machineconfiguration.openshift.io/role: layered <2>
35+
name: 80-worker-extensions
36+
spec:
37+
config:
38+
ignition:
39+
version: 3.2.0
40+
extensions: <3>
41+
- usbguard
42+
- kerberos
43+
----
44+
<1> Specifies the `machineconfiguration.openshift.io/v1` API that is required for `MachineConfig` CRs.
45+
<2> Specifies the machine config pool to apply the `MachineConfig` object to.
46+
<3> Lists the {op-system-first} extensions that you want to install.
47+
48+
. Create the MCP object:
49+
+
50+
[source,terminal]
51+
----
52+
$ oc create -f <file_name>.yaml
53+
----
54+
55+
.Verification
56+
57+
. You can watch the build progress by using the following command:
58+
+
59+
[source,terminal]
60+
----
61+
$ oc get machineosbuilds
62+
----
63+
+
64+
.Example output
65+
[source,terminal]
66+
----
67+
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED
68+
layered-f8ab2d03a2f87a2acd449177ceda805d False True False False False <1>
69+
----
70+
<1> The value `True` in the `BUILDING` column indicates that the `MachineOSBuild` object is building. When the `SUCCEEDED` column reports `TRUE`, the build is complete.
71+
72+
. You can watch as the new machine config is rolled out to the nodes by using the following command:
73+
+
74+
[source,terminal]
75+
----
76+
$ oc get machineconfigpools
77+
----
78+
+
79+
.Example output
80+
[source,terminal]
81+
----
82+
NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE
83+
layered rendered-layered-221507009cbcdec0eec8ab3ccd789d18 False True False 1 0 0 0 167m <1>
84+
master rendered-master-a0b404d061a6183cc36d302363422aba True False False 3 3 3 0 3h38m
85+
worker rendered-worker-221507009cbcdec0eec8ab3ccd789d18 True False False 2 2 2 0 3h38m
86+
----
87+
<1> The value `FALSE` in the `UPDATED` column indicates that the `MachineOSBuild` object is building. When the `UPDATED` column reports `FALSE`, the new custom layered image has rolled out to the nodes.
88+
89+
. When the associated machine config pool is updated, check that the extensions were installed:
90+
91+
.. Open an `oc debug` session to the node by running the following command:
92+
+
93+
[source,terminal]
94+
----
95+
$ oc debug node/<node_name>
96+
----
97+
98+
.. Set `/host` as the root directory within the debug shell by running the following command:
99+
+
100+
[source,terminal]
101+
----
102+
sh-5.1# chroot /host
103+
----
104+
105+
.. Use an appropriate command to verify that the extensions were installed. The following example shows that the usbguard extension was installed:
106+
+
107+
[source,terminal]
108+
----
109+
sh-5.1# rpm -qa |grep usbguard
110+
----
111+
+
112+
.Example output
113+
[source,terminal]
114+
----
115+
usbguard-selinux-1.0.0-15.el9.noarch
116+
usbguard-1.0.0-15.el9.x86_64
117+
----

‎modules/coreos-layering-configuring-on-modifying.adoc

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * machine_configuration/coreos-layering.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="coreos-layering-configuring-on-modifying_{context}"]
7+
= Modifying a custom layered image
8+
9+
You can modify an on-cluster custom layered image, as needed. This allows you to install additional packages, remove existing packages, change the pull or push repositories, update secrets, or other similar changes. You can edit the `MachineOSConfig` object, apply changes to the YAML file that created the `MachineOSConfig` object, or create a new YAML file for that purpose.
10+
11+
If you modify and apply the `MachineOSConfig` object YAML or create a new YAML file, the YAML overwrites any changes you made directly to the `MachineOSConfig` object itself.
12+
13+
include::snippets//coreos-layering-configuring-on-pause.adoc[]
14+
15+
.Prerequisites
16+
17+
* You have opted in to on-cluster layering by creating a `MachineOSConfig` object.
18+
19+
.Procedure
20+
21+
* Modify an object to update the associated custom layered image:
22+
23+
.. Edit the `MachineOSConfig` object to modify the custom layered image. The following example adds the `rngd` daemon to nodes that already have the tree package that was installed using a custom layered image.
24+
+
25+
[source,yaml]
26+
----
27+
apiVersion: machineconfiguration.openshift.io/v1alpha1
28+
kind: MachineOSConfig
29+
metadata:
30+
name: layered-alpha1
31+
spec:
32+
machineConfigPool:
33+
name: layered
34+
buildInputs:
35+
containerFile:
36+
- containerfileArch: noarch
37+
content: |- <1>
38+
FROM configs AS final
39+
40+
RUN rpm-ostree install rng-tools && \
41+
systemctl enable rngd && \
42+
rpm-ostree cleanup -m && \
43+
ostree container commit
44+
45+
RUN rpm-ostree install tree && \
46+
ostree container commit
47+
imageBuilder:
48+
imageBuilderType: PodImageBuilder
49+
baseImagePullSecret:
50+
name: global-pull-secret-copy <2>
51+
renderedImagePushspec: image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/os-images:latest <3>
52+
renderedImagePushSecret: <4>
53+
name: new-secret-name
54+
buildOutputs:
55+
currentImagePullSecret:
56+
name: new-secret-name <5>
57+
----
58+
<1> Optional: Modify the Containerfile, for example to add or remove packages.
59+
<2> Optional: Update the secret needed to pull the base operating system image from the registry.
60+
<3> Optional: Modify the image registry to push the newly-built custom layered image to.
61+
<4> Optional: Update the secret needed to push the newly-built custom layered image to the registry.
62+
<5> Optional: Update the secret needed to pull the newly-built custom layered image from the registry.
63+
+
64+
When you save the changes, the MCO drains, cordons, and reboots the nodes. After the reboot, the node uses the cluster base {op-system-first} image. If your changes modify a secret only, no new build is triggered and no reboot is performed.
65+
66+
.Verification
67+
68+
. Verify that the new `MachineOSBuild` object was created by using the following command:
69+
+
70+
[source,terminal]
71+
----
72+
$ oc get machineosbuild
73+
----
74+
+
75+
.Example output
76+
[source,terminal]
77+
----
78+
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED
79+
layered-a5457b883f5239cdcb71b57e1a30b6ef False False True False False
80+
layered-f91f0f5593dd337d89bf4d38c877590b False True False False False <1>
81+
----
82+
<1> The value `True` in the `BUILDING` column indicates that the `MachineOSBuild` object is building. When the `SUCCEEDED` column reports `True`, the build is complete.
83+
84+
. You can watch as the new machine config is rolled out to the nodes by using the following command:
85+
+
86+
[source,terminal]
87+
----
88+
$ oc get machineconfigpools
89+
----
90+
+
91+
.Example output
92+
[source,terminal]
93+
----
94+
NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE
95+
layered rendered-layered-221507009cbcdec0eec8ab3ccd789d18 False True False 1 0 0 0 167m <1>
96+
master rendered-master-a0b404d061a6183cc36d302363422aba True False False 3 3 3 0 3h38m
97+
worker rendered-worker-221507009cbcdec0eec8ab3ccd789d18 True False False 2 2 2 0 3h38m
98+
----
99+
<1> The value `FALSE` in the `UPDATED` column indicates that the `MachineOSBuild` object is building. When the `UPDATED` column reports `FALSE`, the new custom layered image has rolled out to the nodes.
100+
101+
. When the node is back in the `Ready` state, check that the changes were applied:
102+
103+
.. Open an `oc debug` session to the node by running the following command:
104+
+
105+
[source,terminal]
106+
----
107+
$ oc debug node/<node_name>
108+
----
109+
110+
.. Set `/host` as the root directory within the debug shell by running the following command:
111+
+
112+
[source,terminal]
113+
----
114+
sh-5.1# chroot /host
115+
----
116+
117+
.. Use an appropriate command to verify that change was applied. The following examples shows that the `rngd` daemon was installed:
118+
+
119+
[source,terminal]
120+
----
121+
sh-5.1# rpm -qa |grep rng-tools
122+
----
123+
+
124+
.Example output
125+
[source,terminal]
126+
----
127+
rng-tools-6.17-3.fc41.x86_64
128+
----
129+
+
130+
[source,terminal]
131+
----
132+
sh-5.1# rngd -v
133+
----
134+
+
135+
.Example output
136+
[source,terminal]
137+
----
138+
rngd 6.16
139+
----

‎modules/coreos-layering-configuring-on-revert.adoc

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * machine_configuration/coreos-layering.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="coreos-layering-configuring-on-revert_{context}"]
7+
= Reverting an on-cluster custom layered image
8+
9+
You can revert an on-cluster custom layered image from nodes by removing the label for the machine config pool (MCP) that you specified in the `MachineOSConfig` object. After you remove the label, the Machine Config Operator (MCO) reboots the nodes in that MCP with the cluster base {op-system-first} image, along with any previously-made machine config changes, overriding the custom layered image.
10+
11+
[IMPORTANT]
12+
====
13+
If the node where the custom layered image is deployed uses a custom machine config pool, before you remove the label, make sure the node is associated with a second MCP.
14+
====
15+
16+
You can reapply the custom layered image to the node by using the `oc label node/<node_name> 'node-role.kubernetes.io/<mcp_name>='` label.
17+
18+
.Prerequisites
19+
20+
* You have opted in to on-cluster layering by creating a `MachineOSConfig` object.
21+
22+
.Procedure
23+
24+
* Remove the label from the node by using the following command:
25+
+
26+
[source,terminal]
27+
----
28+
$ oc label node/<node_name> node-role.kubernetes.io/<mcp_name>-
29+
----
30+
+
31+
When you save the changes, the MCO drains, cordons, and reboots the nodes. After the reboot, the node uses the cluster base {op-system-first} image.
32+
33+
.Verification
34+
35+
You can verify that the custom layered image is removed by performing the following checks:
36+
37+
. Check that the worker machine config pool is updating with the previous machine config:
38+
+
39+
[source,terminal]
40+
----
41+
$ oc get mcp
42+
----
43+
+
44+
.Sample output
45+
[source,terminal]
46+
----
47+
NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE
48+
layered rendered-layered-bde4e4206442c0a48b1a1fb35ba56e85 True False False 0 0 0 0 4h46m
49+
master rendered-master-8332482204e0b76002f15ecad15b6c2d True False False 3 3 3 0 5h26m
50+
worker rendered-worker-bde4e4206442c0a48b1a1fb35ba56e85 False True False 3 2 2 0 5h26m <1>
51+
----
52+
<1> The value `FALSE` in the `UPDATED` column indicates that the `MachineOSBuild` object is building. When the `UPDATED` column reports `FALSE`, the base image has rolled out to the nodes.
53+
54+
. Check the nodes to see that scheduling on the nodes is disabled. This indicates that the change is being applied:
55+
+
56+
[source,terminal]
57+
----
58+
$ oc get nodes
59+
----
60+
+
61+
.Example output
62+
[source,terminal]
63+
----
64+
NAME STATUS ROLES AGE VERSION
65+
ip-10-0-148-79.us-west-1.compute.internal Ready worker 32m v1.31.3
66+
ip-10-0-155-125.us-west-1.compute.internal Ready,SchedulingDisabled worker 35m v1.31.3
67+
ip-10-0-170-47.us-west-1.compute.internal Ready control-plane,master 42m v1.31.3
68+
ip-10-0-174-77.us-west-1.compute.internal Ready control-plane,master 42m v1.31.3
69+
ip-10-0-211-49.us-west-1.compute.internal Ready control-plane,master 42m v1.31.3
70+
ip-10-0-218-151.us-west-1.compute.internal Ready worker 31m v1.31.3
71+
----
72+
73+
. When the node is back in the `Ready` state, check that the node is using the base image:
74+
75+
.. Open an `oc debug` session to the node. For example:
76+
+
77+
[source,terminal]
78+
----
79+
$ oc debug node/<node_name>
80+
----
81+
82+
.. Set `/host` as the root directory within the debug shell:
83+
+
84+
[source,terminal]
85+
----
86+
sh-5.1# chroot /host
87+
----
88+
89+
.. Run an `rpm-ostree status` command to view that the base image is in use:
90+
+
91+
[source,terminal]
92+
----
93+
sh-5.1# rpm-ostree status
94+
----
95+
+
96+
.Example output
97+
+
98+
[source,terminal]
99+
----
100+
State: idle
101+
Deployments:
102+
* ostree-unverified-image:containers-storage:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:76721c875a2b79688be46b1dca654c2c6619a6be28b29a2822cd86c3f9d8e3c1
103+
Digest: sha256:76721c875a2b79688be46b1dca654c2c6619a6be28b29a2822cd86c3f9d8e3c1
104+
Version: 418.94.202501300706-0 (2025-01-30T07:10:58Z)
105+
----

0 commit comments

Comments
 (0)