Skip to content

Commit 80b2221

Browse files
mburke5678mburke5678
authored
Merge pull request #75955 from mburke5678/mco-layering-on-cluster
Docs for OCPSTRAT-748 On Cluster Layering: Phase 2 (tech preview)
2 parents a85a880 + 5538eb1 commit 80b2221

File tree

3 files changed

+304
-24
lines changed

3 files changed

+304
-24
lines changed

‎modules/coreos-layering-configuring-on.adoc

Lines changed: 232 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,232 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * post-installation_configuration/coreos-layering.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="coreos-layering-configuring-on_{context}"]
7+
= Using on-cluster layering to apply a custom layered image
8+
9+
To apply a custom layered image to your cluster by using the on-cluster build process, make a `MachineOSConfig` custom resource that includes a Containerfile, a machine config pool reference, repository push and pull secrets, and other parameters as described in the prerequisites.
10+
11+
When you create the object, the Machine Config Operator (MCO) creates a `MachineOSBuild` object and a `machine-os-builder` pod. The build process also creates transient objects, such as config maps, which are cleaned up after the build is complete.
12+
13+
When the build is complete, the MCO pushes the new custom layered image to your repository for use when deploying new nodes. You can see the digested image pull spec for the new custom layered image in the `MachineOSBuild` object and `machine-os-builder` pod.
14+
15+
You should not need to interact with these new objects or the `machine-os-builder` pod. However, you can use all of these resources for troubleshooting, if necessary.
16+
17+
You need a separate `MachineOSConfig` CR for each machine config pool where you want to use a custom layered image.
18+
19+
:FeatureName: On-cluster image layering
20+
include::snippets/technology-preview.adoc[]
21+
22+
.Prerequisites
23+
24+
* You have enabled the `TechPreviewNoUpgrade` feature set by using the feature gates. For more information, see "Enabling features using feature gates".
25+
26+
* You have the pull secret in the `openshift-machine-config-operator` namespace that the MCO needs to pull the base operating system image.
27+
28+
* You have the push secret that the MCO needs to push the new custom layered image to your registry.
29+
30+
* You have a pull secret that your nodes need to pull the new custom layered image from your registry. This should be a different secret than the one used to push the image to the repository.
31+
32+
* You are familiar with how to configure a Containerfile. Instructions on how to create a Containerfile are beyond the scope of this documentation.
33+
34+
* Optional: You have a separate machine config pool for the nodes where you want to apply the custom layered image.
35+
36+
.Procedure
37+
38+
. Create a `MachineOSConfig` object:
39+
40+
.. Create a YAML file similar to the following:
41+
+
42+
[source,terminal]
43+
----
44+
apiVersion: machineconfiguration.openshift.io/v1alpha1
45+
kind: MachineOSConfig
46+
metadata:
47+
name: layered
48+
spec:
49+
machineConfigPool:
50+
name: <mcp_name> <1>
51+
buildInputs:
52+
containerFile: # <2>
53+
- containerfileArch: noarch
54+
content: |-
55+
FROM configs AS final
56+
RUN rpm-ostree install cowsay && \
57+
ostree container commit
58+
imageBuilder: # <3>
59+
imageBuilderType: PodImageBuilder
60+
baseImagePullSecret: # <4>
61+
name: global-pull-secret-copy
62+
renderedImagePushspec: image-registry.openshift-image-registry.svc:5000/openshift/os-image:latest # <5>
63+
renderedImagePushSecret: # <6>
64+
name: builder-dockercfg-7lzwl
65+
buildOutputs: # <7>
66+
currentImagePullSecret:
67+
name: builder-dockercfg-7lzwl
68+
----
69+
<1> Specifies the name of the machine config pool associated with the nodes where you want to deploy the custom layered image.
70+
<2> Specifies the Containerfile to configure the custom layered image.
71+
<3> Specifies the name of the image builder to use. This must be `PodImageBuilder`.
72+
<4> Specifies the name of the pull secret that the MCO needs to pull the base operating system image from the registry.
73+
<5> Specifies the image registry to push the newly-built custom layered image to. This can be any registry that your cluster has access to. This example uses the internal {product-title} registry.
74+
<6> Specifies the name of the push secret that the MCO needs to push the newly-built custom layered image to that registry.
75+
<7> Specifies the secret required by the image registry that the nodes need to pull the newly-built custom layered image. This should be a different secret than the one used to push the image to your repository.
76+
77+
.. Create the `MachineOSConfig` object:
78+
+
79+
[source,terminal]
80+
----
81+
$ oc create -f <file_name>.yaml
82+
----
83+
84+
. If necessary, when the `MachineOSBuild` object has been created and is in the `READY` state, modify the node spec for the nodes where you want to use the new custom layered image:
85+
86+
.. Check that the `MachineOSBuild` object is `READY`. When the `SUCCEEDED` value is `True`, the build is complete.
87+
+
88+
[source,terminal]
89+
----
90+
$ oc get machineosbuild
91+
----
92+
+
93+
.Example output showing that the `MachineOSBuild` object is ready
94+
[source,terminal]
95+
----
96+
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED
97+
layered-rendered-layered-ad5a3cad36303c363cf458ab0524e7c0-builder False False True False False
98+
----
99+
100+
.. Edit the nodes where you want to deploy the custom layered image by adding a label for the machine config pool you specified in the `MachineOSConfig` object:
101+
+
102+
[source,terminal]
103+
----
104+
$ oc label node <node_name> 'node-role.kubernetes.io/<mcp_name>='
105+
----
106+
+
107+
--
108+
where:
109+
110+
node-role.kubernetes.io/<mcp_name>=:: Specifies a node selector that identifies the nodes to deploy the custom layered image.
111+
--
112+
+
113+
When you save the changes, the MCO drains, cordons, and reboots the nodes. After the reboot, the node will be using the new custom layered image.
114+
115+
.Verification
116+
117+
. Verify that the new pods are running by using the following command:
118+
+
119+
[source,terminal]
120+
----
121+
$ oc get pods -n <machineosbuilds_namespace>
122+
----
123+
+
124+
[source,terminal]
125+
----
126+
NAME READY STATUS RESTARTS AGE
127+
build-rendered-layered-ad5a3cad36303c363cf458ab0524e7c0 2/2 Running 0 2m40s # <1>
128+
# ...
129+
machine-os-builder-6fb66cfb99-zcpvq 1/1 Running 0 2m42s # <2>
130+
----
131+
<1> This is the build pod where the custom layered image is building.
132+
<2> This pod can be used for troubleshooting.
133+
134+
. Verify that the `MachineOSConfig` object contains a reference to the new custom layered image:
135+
+
136+
[source,terminal]
137+
----
138+
$ oc describe MachineOSConfig <object_name>
139+
----
140+
+
141+
[source,yaml]
142+
----
143+
apiVersion: machineconfiguration.openshift.io/v1alpha1
144+
kind: MachineOSConfig
145+
metadata:
146+
name: layered
147+
spec:
148+
buildInputs:
149+
baseImagePullSecret:
150+
name: global-pull-secret-copy
151+
containerFile:
152+
- containerfileArch: noarch
153+
content: ""
154+
imageBuilder:
155+
imageBuilderType: PodImageBuilder
156+
renderedImagePushSecret:
157+
name: builder-dockercfg-ng82t-canonical
158+
renderedImagePushspec: image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/os-image:latest
159+
buildOutputs:
160+
currentImagePullSecret:
161+
name: global-pull-secret-copy
162+
machineConfigPool:
163+
name: layered
164+
status:
165+
currentImagePullspec: image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/os-image@sha256:f636fa5b504e92e6faa22ecd71a60b089dab72200f3d130c68dfec07148d11cd # <1>
166+
----
167+
<1> Digested image pull spec for the new custom layered image.
168+
169+
. Verify that the `MachineOSBuild` object contains a reference to the new custom layered image.
170+
+
171+
[source,terminal]
172+
----
173+
$ oc describe machineosbuild <object_name>
174+
----
175+
+
176+
[source,yaml]
177+
----
178+
apiVersion: machineconfiguration.openshift.io/v1alpha1
179+
kind: MachineOSBuild
180+
metadata:
181+
name: layered-rendered-layered-ad5a3cad36303c363cf458ab0524e7c0-builder
182+
spec:
183+
desiredConfig:
184+
name: rendered-layered-ad5a3cad36303c363cf458ab0524e7c0
185+
machineOSConfig:
186+
name: layered
187+
renderedImagePushspec: image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/os-image:latest
188+
# ...
189+
status:
190+
conditions:
191+
- lastTransitionTime: "2024-05-21T20:25:06Z"
192+
message: Build Ready
193+
reason: Ready
194+
status: "True"
195+
type: Succeeded
196+
finalImagePullspec: image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/os-image@sha256:f636fa5b504e92e6faa22ecd71a60b089dab72200f3d130c68dfec07148d11cd # <1>
197+
----
198+
<1> Digested image pull spec for the new custom layered image.
199+
200+
. Verify that the appropriate nodes are using the new custom layered image:
201+
202+
.. Start a debug session as root for a control plane node:
203+
+
204+
[source,terminal]
205+
----
206+
$ oc debug node/<node_name>
207+
----
208+
209+
.. Set `/host` as the root directory within the debug shell:
210+
+
211+
[source,terminal]
212+
----
213+
sh-4.4# chroot /host
214+
----
215+
216+
.. Run the `rpm-ostree status` command to view that the custom layered image is in use:
217+
+
218+
[source,terminal]
219+
----
220+
sh-5.1# rpm-ostree status
221+
----
222+
+
223+
.Example output
224+
[source,terminal]
225+
----
226+
# ...
227+
Deployments:
228+
* ostree-unverified-registry:quay.io/openshift-release-dev/os-image@sha256:f636fa5b504e92e6faa22ecd71a60b089dab72200f3d130c68dfec07148d11cd # <1>
229+
Digest: sha256:bcea2546295b2a55e0a9bf6dd4789433a9867e378661093b6fdee0031ed1e8a4
230+
Version: 416.94.202405141654-0 (2024-05-14T16:58:43Z)
231+
----
232+
<1> Digested image pull spec for the new custom layered image.

‎modules/coreos-layering-configuring.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
:_mod-docs-content-type: PROCEDURE
66
[id="coreos-layering-configuring_{context}"]
7-
= Applying a {op-system} custom layered image
7+
= Using out-of-cluster layering to apply a custom layered image
88

99
You can easily configure {op-system-first} image layering on the nodes in specific machine config pools. The Machine Config Operator (MCO) reboots those nodes with the new custom layered image, overriding the base {op-system-first} image.
1010

0 commit comments

Comments
 (0)