Skip to content

Commit c4960ad

Browse files
authored
Merge pull request #56250 from mburke5678/nodes-mirror-by-tags
OSDOCS#4981: Allow mirroring images by tags
2 parents 9923aed + ae773e1 commit c4960ad

File tree

8 files changed

+177
-46
lines changed

8 files changed

+177
-46
lines changed

modules/builds-image-source.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ source:
4545
+
4646
[NOTE]
4747
====
48-
If your cluster uses an `ImageContentSourcePolicy` object to configure repository mirroring, you can use only global pull secrets for mirrored registries. You cannot add a pull secret to a project.
48+
If your cluster uses an `ImageDigestMirrorSet` or `ImageTagMirrorSet` object to configure repository mirroring, you can use only global pull secrets for mirrored registries. You cannot add a pull secret to a project.
4949
====
5050

5151
.Images that require pull secrets
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * openshift_images/image-configuration.adoc
4+
// * post_installation_configuration/preparing-for-users.adoc
5+
// * updating/updating-restricted-network-cluster/restricted-network-update.adoc
6+
7+
:_content-type: PROCEDURE
8+
[id="images-configuration-registry-mirror-convert_{context}"]
9+
= Converting ImageContentSourcePolicy (ICSP) files for image registry repository mirroring
10+
11+
Using an `ImageContentSourcePolicy` (ICSP) object to configure repository mirroring is a deprecated feature. This functionality is still included in {product-title} and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments.
12+
13+
ICSP objects are being replaced by `ImageDigestMirrorSet` and `ImageTagMirrorSet` objects to configure repository mirroring. If you have existing YAML files that you used to create `ImageContentSourcePolicy` objects, you can use the `oc adm migrate icsp` command to convert those files to an `ImageDigestMirrorSet` YAML file. The command updates the API to the current version, changes the `kind` value to `ImageDigestMirrorSet`, and changes `spec.repositoryDigestMirrors` to `spec.imageDigestMirrors`. The rest of the file is not changed.
14+
15+
For more information about `ImageDigestMirrorSet` or `ImageTagMirrorSet` objects, see "Configuring image registry repository mirroring" in the previous section.
16+
17+
.Prerequisites
18+
19+
* Ensure that you have access to the cluster as a user with the `cluster-admin` role.
20+
21+
* Ensure that you have `ImageContentSourcePolicy` objects on your cluster.
22+
23+
.Procedure
24+
25+
. Use the following command to convert one or more `ImageContentSourcePolicy` YAML files to an `ImageDigestMirrorSet` YAML file:
26+
+
27+
[source,terminal]
28+
----
29+
$ oc adm migrate iscp <file_name>.yaml <file_name>.yaml <file_name>.yaml --dest-dir <path_to_the_directory>
30+
----
31+
+
32+
--
33+
where:
34+
35+
`<file_name>`:: Specifies the name of the source `ImageContentSourcePolicy` YAML. You can list multiple file names.
36+
`--dest-dir`:: Optional: Specifies a directory for the output `ImageDigestMirrorSet` YAML. If unset, the file is written to the current directory.
37+
--
38+
+
39+
For example, the following command converts the `icsp.yaml` and `icsp-2.yaml` file and saves the new YAML files to the `idms-files` directory.
40+
+
41+
[source,terminal]
42+
----
43+
$ oc adm migrate icsp icsp.yaml icsp-2.yaml --dest-dir idms-files
44+
----
45+
+
46+
.Example output
47+
[source,terminal]
48+
----
49+
wrote ImageDigestMirrorSet to idms-files/imagedigestmirrorset_ubi8repo.5911620242173376087.yaml
50+
wrote ImageDigestMirrorSet to idms-files/imagedigestmirrorset_ubi9repo.6456931852378115011.yaml
51+
----
52+
53+
. Create the CR object by running the following command:
54+
+
55+
[source,terminal]
56+
----
57+
$ oc create -f <path_to_the_directory>/<file-name>.yaml
58+
----
59+
+
60+
--
61+
where:
62+
63+
`<path_to_the_directory>`:: Specifies the path to the directory, if you used the `--dest-dir` flag.
64+
`<file_name>`:: Specifies the name of the `ImageDigestMirrorSet` YAML.
65+
--
66+

modules/images-configuration-registry-mirror.adoc

Lines changed: 99 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,12 @@
88
[id="images-configuration-registry-mirror_{context}"]
99
= Configuring image registry repository mirroring
1010

11-
Setting up container registry repository mirroring enables you to do the following:
11+
Setting up container registry repository mirroring enables you to perform the following tasks:
1212

1313
* Configure your {product-title} cluster to redirect requests to pull images from a repository on a source image registry and have it resolved by a repository on a mirrored image registry.
1414
* Identify multiple mirrored repositories for each target repository, to make sure that if one mirror is down, another can be used.
1515
16-
The attributes of repository mirroring in {product-title} include:
16+
Repository mirroring in {product-title} includes the following attributes:
1717

1818
* Image pulls are resilient to registry downtimes.
1919
* Clusters in disconnected environments can pull images from critical locations, such as quay.io, and have registries behind a company firewall provide the requested images.
@@ -29,9 +29,20 @@ By pulling container images needed by {product-title} and then bringing those im
2929
3030
* After {product-title} installation:
3131
+
32-
Even if you don't configure mirroring during {product-title} installation, you can do so later using the `ImageContentSourcePolicy` object.
32+
If you did not configure mirroring during {product-title} installation, you can do so post-installation by using one of the following custom resource (CR) objects:
33+
+
34+
--
35+
** `ImageDigestMirrorSet`. This CR allows you to pull images from a mirrored registry by using digest specifications.
36+
+
37+
** `ImageTagMirrorSet`. This CR allows you to pull images from a mirrored registry by using image tags.
38+
--
39+
+
40+
[IMPORTANT]
41+
====
42+
Using an `ImageContentSourcePolicy` (ICSP) object to configure repository mirroring is a deprecated feature. Deprecated functionality is still included in {product-title} and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments. If you have existing YAML files that you used to create `ImageContentSourcePolicy` objects, you can use the `oc adm migrate icsp` command to convert those files to an `ImageDigestMirrorSet` YAML file. For more information, see "Converting ImageContentSourcePolicy (ICSP) files for image registry repository mirroring" in the following section.
43+
====
3344
34-
The following procedure provides a post-installation mirror configuration, where you create an `ImageContentSourcePolicy` object that identifies:
45+
Both of these custom resource objects identify the following information:
3546
--
3647
* The source of the container image repository you want to mirror.
3748
* A separate entry for each mirror repository you want to offer the content
@@ -40,11 +51,26 @@ requested from the source repository.
4051

4152
[NOTE]
4253
====
43-
You can only configure global pull secrets for clusters that have an `ImageContentSourcePolicy` object. You cannot add a pull secret to a project.
54+
If your cluster uses an `ImageDigestMirrorSet` or `ImageTagMirrorSet` object to configure repository mirroring, you can use only global pull secrets for mirrored registries. You cannot add a pull secret to a project.
4455
====
4556

57+
The following procedure creates a post-installation mirror configuration, where you create an `ImageDigestMirrorSet` object.
58+
4659
.Prerequisites
47-
* Access to the cluster as a user with the `cluster-admin` role.
60+
* Ensure that you have access to the cluster as a user with the `cluster-admin` role.
61+
62+
* Ensure that there are no `ImageContentSourcePolicy` objects on your cluster. For example, you can use the following command:
63+
+
64+
[source, terminal]
65+
----
66+
$ oc get ImageContentSourcePolicy
67+
----
68+
+
69+
.Example output
70+
[source, terminal]
71+
----
72+
No resources found
73+
----
4874
4975
.Procedure
5076

@@ -66,55 +92,71 @@ In this example, you have a container image registry that is named `example.io`
6692

6793
. Log in to your {product-title} cluster.
6894

69-
. Create an `ImageContentSourcePolicy` file (for example, `registryrepomirror.yaml`), replacing the source and mirrors with your own registry and repository pairs and images:
95+
. Create an `ImageDigestMirrorSet` or `ImageTagMirrorSet` CR, as needed, replacing the source and mirrors with your own registry and repository pairs and images:
7096
+
7197
[source,yaml]
7298
----
73-
apiVersion: operator.openshift.io/v1alpha1
74-
kind: ImageContentSourcePolicy
99+
apiVersion: config.openshift.io/v1 <1>
100+
kind: ImageDigestMirrorSet <2>
75101
metadata:
76102
name: ubi8repo
77103
spec:
78-
repositoryDigestMirrors:
104+
imageDigestMirrors: <3>
79105
- mirrors:
80-
- example.io/example/ubi-minimal <1>
81-
- example.com/example/ubi-minimal <2>
82-
source: registry.access.redhat.com/ubi8/ubi-minimal <3>
106+
- example.io/example/ubi-minimal <4>
107+
- example.com/example/ubi-minimal <5>
108+
source: registry.access.redhat.com/ubi8/ubi-minimal <6>
109+
mirrorSourcePolicy: AllowContactingSource <7>
83110
- mirrors:
84111
- mirror.example.com/redhat
85-
source: registry.redhat.io/openshift4 <4>
112+
source: registry.redhat.io/openshift4 <8>
113+
mirrorSourcePolicy: AllowContactingSource
86114
- mirrors:
87115
- mirror.example.com
88-
source: registry.redhat.io <5>
116+
source: registry.redhat.io <9>
117+
mirrorSourcePolicy: AllowContactingSource
89118
- mirrors:
90119
- mirror.example.net/image
91-
source: registry.example.com/example/myimage <6>
120+
source: registry.example.com/example/myimage <10>
121+
mirrorSourcePolicy: AllowContactingSource
92122
- mirrors:
93123
- mirror.example.net
94-
source: registry.example.com/example <7>
124+
source: registry.example.com/example <11>
125+
mirrorSourcePolicy: AllowContactingSource
95126
- mirrors:
96127
- mirror.example.net/registry-example-com
97-
source: registry.example.com <8>
128+
source: registry.example.com <12>
129+
mirrorSourcePolicy: AllowContactingSource
98130
----
99-
<1> Indicates the name of the image registry and repository.
100-
<2> Indicates multiple mirror repositories for each target repository. If one mirror is down, the target repository can use another mirror.
101-
<3> Indicates the registry and repository containing the content that is mirrored.
102-
<4> You can configure a namespace inside a registry to use any image in that namespace. If you use a registry domain as a source, the `ImageContentSourcePolicy` resource is applied to all repositories from the registry.
103-
<5> If you configure the registry name, the `ImageContentSourcePolicy` resource is applied to all repositories from a source registry to a mirror registry.
104-
<6> Pulls the image `mirror.example.net/image@sha256:...`.
105-
<7> Pulls the image `myimage` in the source registry namespace from the mirror `mirror.example.net/myimage@sha256:...`.
106-
<8> Pulls the image `registry.example.com/example/myimage` from the mirror registry `mirror.example.net/registry-example-com/example/myimage@sha256:...`. The `ImageContentSourcePolicy` resource is applied to all repositories from a source registry to a mirror registry `mirror.example.net/registry-example-com`.
107-
108-
. Create the new `ImageContentSourcePolicy` object:
131+
<1> Indicates the API to use with this CR. This must be `config.openshift.io/v1`.
132+
<2> Indicates the kind of object according to the pull type:
133+
** `ImageDigestMirrorSet`: Pulls a digest reference image.
134+
** `ImageTagMirrorSet`: Pulls a tag reference image.
135+
<3> Indicates the type of image pull method, either:
136+
** `imageDigestMirrors`: Use for an `ImageDigestMirrorSet` CR.
137+
** `imageTagMirrors`: Use for an `ImageTagMirrorSet` CR.
138+
<4> Indicates the name of the mirrored image registry and repository.
139+
<5> Optional: Indicates a secondary mirror repository for each target repository. If one mirror is down, the target repository can use another mirror.
140+
<6> Indicates the registry and repository source, which is the repository that is referred to in image pull specifications.
141+
<7> Optional: Indicates the fallback policy if the image pull fails:
142+
** `AllowContactingSource`: Allows continued attempts to pull the image from the source repository. This is the default.
143+
** `NeverContactSource`: Prevents continued attempts to pull the image from the source repository.
144+
<8> Optional: Indicates a namespace inside a registry, which allows you to use any image in that namespace. If you use a registry domain as a source, the object is applied to all repositories from the registry.
145+
<9> Optional: Indicates a registry, which allows you to use any image in that registry. If you specify a registry name, the object is applied to all repositories from a source registry to a mirror registry.
146+
<10> Pulls the image `registry.example.com/example/myimage@sha256:...` from the mirror `mirror.example.net/image@sha256:..`.
147+
<11> Pulls the image `registry.example.com/example/image@sha256:...` in the source registry namespace from the mirror `mirror.example.net/image@sha256:...`.
148+
<12> Pulls the image `registry.example.com/myimage@sha256` from the mirror registry `example.net/registry-example-com/myimage@sha256:...`. The `ImageContentSourcePolicy` resource is applied to all repositories from a source registry to a mirror registry `mirror.example.net/registry-example-com`.
149+
150+
. Create the new object:
109151
+
110152
[source,terminal]
111153
----
112154
$ oc create -f registryrepomirror.yaml
113155
----
114156
+
115-
After the `ImageContentSourcePolicy` object is created, the new settings are deployed to each node and the cluster starts using the mirrored repository for requests to the source repository.
157+
After the object is created, the Machine Config Operator (MCO) cordons the nodes as the new settings are deployed to each node. The MCO restarts the nodes for an `ImageTagMirrorSet` object only. The MCO does not restart the nodes for `ImageDigestMirrorSet` objects. When the nodes are uncordoned, the cluster starts using the mirrored repository for requests to the source repository.
116158

117-
. To check that the mirrored configuration settings, are applied, do the following on one of the nodes.
159+
. To check that the mirrored configuration settings are applied, do the following on one of the nodes.
118160

119161
.. List your nodes:
120162
+
@@ -134,8 +176,6 @@ ip-10-0-147-35.ec2.internal Ready worker 7m v1.25.0
134176
ip-10-0-153-12.ec2.internal Ready worker 7m v1.25.0
135177
ip-10-0-154-10.ec2.internal Ready master 11m v1.25.0
136178
----
137-
+
138-
The `Imagecontentsourcepolicy` resource does not restart the nodes.
139179

140180
.. Start the debugging process to access the node:
141181
+
@@ -166,6 +206,8 @@ the changes were made:
166206
sh-4.2# cat /etc/containers/registries.conf
167207
----
168208
+
209+
The following output represents a `registries.conf` file where an `ImageDigestMirrorSet` object and an `ImageTagMirrorSet` object were applied. The final two entries are marked `digest-only` and `tag-only` respectively.
210+
+
169211
.Example output
170212
[source,terminal]
171213
----
@@ -174,57 +216,71 @@ short-name-mode = ""
174216

175217
[[registry]]
176218
prefix = ""
177-
location = "registry.access.redhat.com/ubi8/ubi-minimal"
178-
mirror-by-digest-only = true
219+
location = "registry.access.redhat.com/ubi9/ubi-minimal" <1>
179220

180221
[[registry.mirror]]
181-
location = "example.io/example/ubi-minimal"
222+
location = "example.io/example/ubi-minimal" <2>
223+
pull-from-mirror = "digest-only" <3>
182224

183225
[[registry.mirror]]
184226
location = "example.com/example/ubi-minimal"
227+
pull-from-mirror = "digest-only"
185228

186229
[[registry]]
187230
prefix = ""
188231
location = "registry.example.com"
189-
mirror-by-digest-only = true
190232

191233
[[registry.mirror]]
192234
location = "mirror.example.net/registry-example-com"
235+
pull-from-mirror = "digest-only"
193236

194237
[[registry]]
195238
prefix = ""
196239
location = "registry.example.com/example"
197-
mirror-by-digest-only = true
198240

199241
[[registry.mirror]]
200242
location = "mirror.example.net"
243+
pull-from-mirror = "digest-only"
201244

202245
[[registry]]
203246
prefix = ""
204247
location = "registry.example.com/example/myimage"
205-
mirror-by-digest-only = true
206248

207249
[[registry.mirror]]
208250
location = "mirror.example.net/image"
251+
pull-from-mirror = "digest-only"
209252

210253
[[registry]]
211254
prefix = ""
212255
location = "registry.redhat.io"
213-
mirror-by-digest-only = true
214256

215257
[[registry.mirror]]
216258
location = "mirror.example.com"
259+
pull-from-mirror = "digest-only"
217260

218261
[[registry]]
219262
prefix = ""
220263
location = "registry.redhat.io/openshift4"
221-
mirror-by-digest-only = true
222264

223265
[[registry.mirror]]
224266
location = "mirror.example.com/redhat"
267+
pull-from-mirror = "digest-only"
268+
[[registry]]
269+
prefix = ""
270+
location = "registry.access.redhat.com/ubi8/ubi-minimal"
271+
blocked = true <4>
272+
273+
[[registry.mirror]]
274+
location = "example.io/example/ubi-minimal-tag"
275+
pull-from-mirror = "tag-only" <5>
225276
----
277+
<1> Indicates the repository that is referred to in a pull spec.
278+
<2> Indicates the mirror for that repository.
279+
<3> Indicates that the image pull from the mirror is a digest reference image.
280+
<4> Indicates that the `NeverContactSource` parameter is set for this repository.
281+
<5> Indicates that the image pull from the mirror is a tag reference image.
226282
227-
.. Pull an image digest to the node from the source and check if it is resolved by the mirror. `ImageContentSourcePolicy` objects support image digests only, not image tags.
283+
.. Pull an image to the node from the source and check if it is resolved by the mirror.
228284
+
229285
[source,terminal]
230286
----
@@ -239,3 +295,5 @@ If the repository mirroring procedure does not work as described, use the follow
239295
* The main registry is only used if no other mirror works.
240296
* From the system context, the `Insecure` flags are used as fallback.
241297
* The format of the `/etc/containers/registries.conf` file has changed recently. It is now version 2 and in TOML format.
298+
* You cannot add the same repository to both an `ImageDigestMirrorSet` and an `ImageTagMirrorSet` object.
299+

modules/troubleshooting-disabling-autoreboot-mco.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ The following modifications do not trigger a node reboot:
1717
** Changes to the global pull secret or pull secret in the `openshift-config` namespace.
1818
** Automatic rotation of the `/etc/kubernetes/kubelet-ca.crt` certificate authority (CA) by the Kubernetes API Server Operator.
1919
20-
* When the MCO detects changes to the `/etc/containers/registries.conf` file, such as adding or editing an `ImageContentSourcePolicy` object, it drains the corresponding nodes, applies the changes, and uncordons the nodes. The node drain does not happen when there is a mirror configuration change in the `ImageContentSourcePolicy` (ICSP) object such as a new mirror added to an existing registry or a new registry added that has the setting `mirror-by-digest-only=true`.
20+
* When the MCO detects changes to the `/etc/containers/registries.conf` file, such as adding or editing an `ImageDigestMirrorSet` or `ImageTagMirrorSet` object, it drains the corresponding nodes, applies the changes, and uncordons the nodes. The node drain does not happen when there is a mirror configuration change in the object, such as adding a new mirror to an existing registry by using an `ImageDigestMirrorSet` object. The MCO does restart the nodes if you add a new mirror to an existing registry by uisng an`ImageTagMirrorSet` object.
2121
====
2222

2323
To avoid unwanted disruptions, you can modify the machine config pool (MCP) to prevent automatic rebooting after the Operator makes changes to the machine config.

0 commit comments

Comments
 (0)