Skip to content

Commit 5bea570

Browse files
authored
Merge pull request #30230 from adellape/olm_offline
Add OLM mirroring using offline media
2 parents 681772d + 85d28c9 commit 5bea570

File tree

5 files changed

+141
-48
lines changed

5 files changed

+141
-48
lines changed

modules/olm-creating-catalog-from-index.adoc

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,9 @@ ifndef::openshift-origin[]
1414
:index-image: redhat-operator-index
1515
:tag: v{product-version}
1616
endif::[]
17+
ifeval::["{context}" == "olm-restricted-networks"]
18+
:olm-restricted-networks:
19+
endif::[]
1720

1821
[id="olm-creating-catalog-from-index_{context}"]
1922
= Creating a catalog from an index image
@@ -27,8 +30,8 @@ You can create an Operator catalog from an index image and apply it to an {produ
2730
.Procedure
2831

2932
. Create a `CatalogSource` object that references your index image.
30-
ifeval::["{context}" == "olm-restricted-networks"]
31-
If you used the `oc adm catalog mirror` command to mirror your catalog to a target registry, you can use the generated `catalogSource.yaml` file as is or further modify the configuration.
33+
ifdef::olm-restricted-networks[]
34+
If you used the `oc adm catalog mirror` command to mirror your catalog to a target registry, you can use the generated `catalogSource.yaml` file as a starting point.
3235
endif::[]
3336

3437
.. Modify the following to your specifications and save it as a `catalogSource.yaml` file:
@@ -38,20 +41,28 @@ endif::[]
3841
apiVersion: operators.coreos.com/v1alpha1
3942
kind: CatalogSource
4043
metadata:
44+
ifdef::olm-restricted-networks[]
45+
name: my-operator-catalog <.>
46+
endif::[]
47+
ifndef::olm-restricted-networks[]
4148
name: my-operator-catalog
49+
endif::[]
4250
namespace: openshift-marketplace
4351
spec:
4452
sourceType: grpc
45-
image: <registry>:<port>/<namespace>/{index-image}:{tag} <1>
53+
image: <registry>:<port>/<namespace>/{index-image}:{tag} <.>
4654
displayName: My Operator Catalog
47-
publisher: <publisher_name> <2>
55+
publisher: <publisher_name> <.>
4856
updateStrategy:
49-
registryPoll: <3>
57+
registryPoll: <.>
5058
interval: 30m
5159
----
52-
<1> Specify your index image.
53-
<2> Specify your name or an organization name publishing the catalog.
54-
<3> Catalog sources can automatically check for new versions to keep up to date.
60+
ifdef::olm-restricted-networks[]
61+
<.> If you mirrored content to local files before uploading to a registry, remove any backslash (`/`) characters from the `metadata.name` field to avoid an "invalid resource name" error when you create the object.
62+
endif::[]
63+
<.> Specify your index image.
64+
<.> Specify your name or an organization name publishing the catalog.
65+
<.> Catalog sources can automatically check for new versions to keep up to date.
5566

5667
.. Use the file to create the `CatalogSource` object:
5768
+

modules/olm-mirroring-catalog.adoc

Lines changed: 107 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -23,14 +23,14 @@ You can mirror the Operator content of a Red Hat-provided catalog, or a custom c
2323

2424
The `oc adm catalog mirror` command also automatically mirrors the index image that is specified during the mirroring process, whether it be a Red Hat-provided index image or your own custom-built index image, to the target registry. You can then use the mirrored index image to create a catalog source that allows Operator Lifecycle Manager (OLM) to load the mirrored catalog onto your {product-title} cluster.
2525

26-
For the steps in this procedure, the target registry is an existing mirror registry that is accessible by both your cluster and a workstation with unrestricted network access.
27-
2826
.Prerequisites
2927

30-
* Workstation with unrestricted network access
31-
* `podman` version 1.9.3+
28+
* Workstation with unrestricted network access.
29+
* `podman` version 1.9.3 or later.
3230
* Access to mirror registry that supports
33-
link:https://docs.docker.com/registry/spec/manifest-v2-2/[Docker v2-2]
31+
link:https://docs.docker.com/registry/spec/manifest-v2-2/[Docker v2-2].
32+
* Decide which namespace on your mirror registry you will use to store the mirrored Operator content. For example, you might create an `olm-mirror` namespace.
33+
* If your mirror registry does not have Internet access, connect removable media to your workstation with unrestricted network access.
3434
* If you are working with private registries, set the `REG_CREDS` environment variable to the file path of your registry credentials for use in later steps. For example, for the `podman` CLI:
3535
+
3636
[source,terminal]
@@ -40,52 +40,44 @@ $ REG_CREDS=${XDG_RUNTIME_DIR}/containers/auth.json
4040

4141
.Procedure
4242

43-
. On your workstation with unrestricted network access, use the `podman login` command to authenticate with the target mirror registry:
43+
ifndef::openshift-origin[]
44+
. If you want to mirror a Red Hat-provided catalog, run the following command on your workstation with unrestricted network access to authenticate with `registry.redhat.io`:
4445
+
4546
[source,terminal]
4647
----
47-
$ podman login <mirror_registry>
48+
$ podman login registry.redhat.io
4849
----
50+
endif::[]
4951

50-
ifndef::openshift-origin[]
51-
. Authenticate with `registry.redhat.io`:
52+
. The `oc adm catalog mirror` command extracts the contents of an index image to generate the manifests required for mirroring. The default behavior of the command generates manifests, then automatically mirrors all of the image content from the index image, as well as the index image itself, to your mirror registry. Alternatively, if your mirror registry is on a completely disconnected, or _airgapped_, host, you can first mirror the content to removable media, move the media to the disconnected environment, then mirror the content from the media to the registry.
53+
54+
** *Option A: If your mirror registry is on the same network* as your workstation with unrestricted network access, take the following actions on your workstation:
55+
56+
.. If your mirror registry requires authentication, run the following command to log in to the registry:
5257
+
5358
[source,terminal]
5459
----
55-
$ podman login registry.redhat.io
60+
$ podman login <mirror_registry>
5661
----
57-
endif::[]
5862

59-
. The `oc adm catalog mirror` command extracts the contents of an index image to generate the manifests required for mirroring. You can choose either of the following:
60-
+
61-
--
62-
* Allow the default behavior of the command to automatically mirror all of the image content from the index image, as well as the index image itself, to your mirror registry after generating manifests.
63-
* Add the `--manifests-only` flag to only generate the manifests required for mirroring, but do not actually mirror the image content to the registry yet. This can be useful for reviewing what will be mirrored, and it allows you to make any changes to the mapping list if you only require a subset of packages. You can then use that file with the `oc image mirror` command to mirror the modified list of images in a later step.
64-
+
65-
[NOTE]
66-
====
67-
The `--manifests-only` flag is intended for advanced selective mirroring of content from the catalog. The `opm index prune` command, if you used it previously to prune the index image, is suitable for most use cases.
68-
====
69-
--
70-
+
71-
On your workstation with unrestricted network access, run the following command:
63+
.. Run the following command to mirror the content:
7264
+
7365
[source,terminal]
7466
----
7567
$ oc adm catalog mirror \
7668
<index_image> \//<1>
77-
<mirror_registry>:<port> \//<2>
69+
<mirror_registry>:<port>/<namespace> \//<2>
7870
[-a ${REG_CREDS}] \//<3>
7971
[--insecure] \//<4>
8072
[--index-filter-by-os='<platform>/<arch>'] \//<5>
8173
[--manifests-only] <6>
8274
----
8375
<1> Specify the index image for the catalog you want to mirror. For example, this might be a pruned index image that you created previously, or one of the source index images for the default catalogs, such as `{index-image-pullspec}`.
84-
<2> Specify the target registry to mirror the Operator content to.
76+
<2> Specify the target registry and namespace to mirror the Operator content to, where `<namespace>` is any existing namespace on the registry. For example, you might create an `olm-mirror` namespace to push all mirrored content to.
8577
<3> Optional: If required, specify the location of your registry credentials file.
8678
<4> Optional: If you do not want to configure trust for the target registry, add the `--insecure` flag.
8779
<5> Optional: Specify which platform and architecture of the index image to select when multiple variants are available. Images are passed as `'<platform>/<arch>[/<variant>]'`. This does not apply to images referenced by the index. Valid values are `linux/amd64`, `linux/ppc64le`, and `linux/s390x`.
88-
<6> Optional: Only generate the manifests required for mirroring and do not actually mirror the image content to a registry.
80+
<6> Optional: Generate only the manifests required for mirroring, and do not actually mirror the image content to a registry. This option can be useful for reviewing what will be mirrored, and it allows you to make any changes to the mapping list if you require only a subset of packages. You can then use the `mapping.txt` file with the `oc image mirror` command to mirror the modified list of images in a later step. This flag is intended for only advanced selective mirroring of content from the catalog; the `opm index prune` command, if you used it previously to prune the index image, is suitable for most catalog management use cases.
8981
+
9082
.Example output
9183
[source,terminal,subs="attributes+"]
@@ -94,29 +86,111 @@ src image has index label for database path: /database/index.db
9486
using database path mapping: /database/index.db:/tmp/153048078
9587
wrote database to /tmp/153048078 <1>
9688
...
97-
wrote mirroring manifests to manifests-{index-image}-1614211642
89+
wrote mirroring manifests to manifests-{index-image}-1614211642 <2>
9890
----
9991
<1> Directory for the temporary `index.db` database generated by the command.
92+
<2> Record the manifests directory name that is generated. This directory name is used in a later step.
93+
94+
** *Option B: If your mirror registry is on a disconnected host,* take the following actions.
95+
96+
.. Run the following command on your workstation with unrestricted network access to mirror the content to local files:
97+
+
98+
[source,terminal]
99+
----
100+
$ oc adm catalog mirror \
101+
<index_image> \//<1>
102+
file:///local/index \//<2>
103+
[-a ${REG_CREDS}] \
104+
[--insecure]
105+
----
106+
<1> Specify the index image for the catalog you want to mirror. For example, this might be a pruned index image that you created previously, or one of the source index images for the default catalogs, such as `{index-image-pullspec}`.
107+
<2> Mirrors content to local files in your current directory.
108+
+
109+
.Example output
110+
[source,terminal]
111+
----
112+
...
113+
info: Mirroring completed in 5.93s (5.915MB/s)
114+
wrote mirroring manifests to manifests-my-index-1614985528 <1>
115+
116+
To upload local images to a registry, run:
117+
118+
oc adm catalog mirror file://local/index/myrepo/my-index:v1 REGISTRY/REPOSITORY <2>
119+
----
120+
<1> Record the manifests directory name that is generated. This directory name is used in a later step.
121+
<2> Record the expanded `file://` path that based on your provided index image. This path is used in a later step.
122+
123+
.. Copy the `v2/` directory that is generated in your current directory to removable media.
124+
125+
.. Physically remove the media and attach it to a host in the disconnected environment that has access to the mirror registry.
126+
127+
.. If your mirror registry requires authentication, run the following command on your host in the disconnected environment to log in to the registry:
128+
+
129+
[source,terminal]
130+
----
131+
$ podman login <mirror_registry>
132+
----
133+
134+
.. Run the following command from the parent directory containing the `v2/` directory to upload the images from local files to the mirror registry:
100135
+
101-
After running the command, a `manifests-<index_image_name>-<random_number>/` directory is created in the current directory and generates the following files:
136+
[source,terminal]
137+
----
138+
$ oc adm catalog mirror \
139+
file://local/index/<repo>/<index_image>:<tag> \//<1>
140+
<mirror_registry>:<port>/<namespace> \//<2>
141+
[-a ${REG_CREDS}] \
142+
[--insecure]
143+
----
144+
<1> Specify the `file://` path from the previous command output.
145+
<2> Specify the target registry and namespace to mirror the Operator content to, where `<namespace>` is any existing namespace on the registry. For example, you might create an `olm-mirror` namespace to push all mirrored content to.
146+
147+
. After mirroring the content to your registry, inspect the manifests directory that is generated in your current directory.
148+
+
149+
[NOTE]
150+
====
151+
The manifests directory name is used in a later step.
152+
====
153+
+
154+
If you mirrored content to a registry on the same network in the previous step, the directory name takes the following form:
155+
+
156+
[source,text]
157+
----
158+
manifests-<index_image_name>-<random_number>
159+
----
160+
+
161+
If you mirrored content to a registry on a disconnected host in the previous step, the directory name takes the following form:
162+
+
163+
[source,text]
164+
----
165+
manifests-index/<namespace>/<index_image_name>-<random_number>
166+
----
167+
+
168+
The manifests directory contains the following files, some of which might require further modification:
102169
+
103170
--
104171
* The `catalogSource.yaml` file is a basic definition for a `CatalogSource` object that is pre-populated with your index image tag and other relevant metadata. This file can be used as is or modified to add the catalog source to your cluster.
172+
+
173+
[IMPORTANT]
174+
====
175+
If you mirrored the content to local files, you must modify your `catalogSource.yaml` file to remove any backslash (`/`) characters from the `metadata.name` field. Otherwise, when you attempt to create the object, it fails with an "invalid resource name" error.
176+
====
105177
* The `imageContentSourcePolicy.yaml` file defines an `ImageContentSourcePolicy` object that can configure nodes to translate between the image references stored in Operator manifests and the mirrored registry.
106178
* The `mapping.txt` file contains all of the source images and where to map them in the target registry. This file is compatible with the `oc image mirror` command and can be used to further customize the mirroring configuration.
107-
--
108179
+
109180
[IMPORTANT]
110181
====
111-
If you used the `--manifests-only` flag in this step and want to further trim the subset of packages to be mirrored, see the steps in the _Mirroring a Package Manifest Format catalog image_ procedure about modifying your `mapping.txt` file and using the file with the `oc image mirror` command. After following those further actions, you can continue this procedure.
182+
If you used the `--manifests-only` flag during the mirroring process and want to further trim the subset of packages to be mirrored, see the steps in the "Mirroring a Package Manifest Format catalog image" procedure about modifying your `mapping.txt` file and using the file with the `oc image mirror` command. After following those further actions, you can continue this procedure.
112183
====
184+
--
113185

114-
. Create the `ImageContentSourcePolicy` object:
186+
. On a host with access to the disconnected cluster, create the `ImageContentSourcePolicy` object by running the following command to specify the `imageContentSourcePolicy.yaml` file in your manifests directory:
115187
+
116188
[source,terminal,subs="attributes+"]
117189
----
118-
$ oc create -f ./manifests-{index-image}-<random_number>/imageContentSourcePolicy.yaml
190+
$ oc create -f <path/to/manifests/dir>/imageContentSourcePolicy.yaml
119191
----
192+
+
193+
where `<path/to/manifests/dir>` is the path to the manifests directory for your mirrored content.
120194

121195
You can now create a `CatalogSource` object to reference your mirrored index image and Operator content.
122196

modules/olm-pruning-index-image.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,12 +26,12 @@ endif::[]
2626
[id="olm-pruning-index-image_{context}"]
2727
= Pruning an index image
2828

29-
An index image, based on the Operator Bundle Format, is a containerized snapshot of an Operator catalog. You can prune an index of all but a specified list of packages, creating a copy of the source index containing only the Operators that you want.
29+
An index image, based on the Operator Bundle Format, is a containerized snapshot of an Operator catalog. You can prune an index of all but a specified list of packages, which creates a copy of the source index containing only the Operators that you want.
3030

3131
ifeval::["{context}" != "olm-managing-custom-catalogs"]
3232
When configuring Operator Lifecycle Manager (OLM) to use mirrored content on restricted network {product-title} clusters, use this pruning method if you want to only mirror a subset of Operators from the default catalogs.
3333

34-
For the steps in this procedure, the target registry is an existing mirror registry that is accessible by both your cluster and a workstation with unrestricted network access. This example also shows pruning the index image for the default `{catalog-name}` catalog, but the process is the same for all index images.
34+
For the steps in this procedure, the target registry is an existing mirror registry that is accessible by your workstation with unrestricted network access. This example also shows pruning the index image for the default `{catalog-name}` catalog, but the process is the same for any index image.
3535
endif::[]
3636

3737
.Prerequisites

modules/olm-restricted-networks-configuring-operatorhub.adoc

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
[id="olm-restricted-networks-operatorhub_{context}"]
1010
= Disabling the default OperatorHub sources
1111

12-
Operator catalogs that source content provided by Red Hat and community projects are configured for OperatorHub by default during an {product-title} installation. Before configuring OperatorHub to instead use local catalog sources in a restricted network environment, you must disable the default catalogs.
12+
Operator catalogs that source content provided by Red Hat and community projects are configured for OperatorHub by default during an {product-title} installation. Before configuring OperatorHub to instead use local catalog sources in a restricted network environment, you must disable the default catalogs as a cluster administrator.
1313

1414
.Procedure
1515

@@ -20,3 +20,8 @@ Operator catalogs that source content provided by Red Hat and community projects
2020
$ oc patch OperatorHub cluster --type json \
2121
-p '[{"op": "add", "path": "/spec/disableAllDefaultSources", "value": true}]'
2222
----
23+
24+
[TIP]
25+
====
26+
Alternatively, you can use the web console to manage catalog sources. From the *Administration* -> *Cluster Settings* -> *Global Configuration* -> *OperatorHub* page, click the *Sources* tab, where you can create, delete, disable, and enable individual sources.
27+
====

0 commit comments

Comments
 (0)