Skip to content

Commit b3e6d46

Browse files
ecordellecordell
committed
proposal(relatedImages): update related images proposal to use kustomize
1 parent b3a5a66 commit b3e6d46

File tree

1 file changed

+82
-145
lines changed

1 file changed

+82
-145
lines changed

‎Documentation/contributors/design-proposals/related-images.md

Lines changed: 82 additions & 145 deletions
Original file line numberDiff line numberDiff line change
@@ -12,181 +12,118 @@ Operators often need to make use of other container images to perform their func
1212

1313
## Proposal
1414

15-
Introduce a new field `relatedImages` to the `ClusterServiceVersion` spec.
15+
We will take advantage of the [images metadata](https://github.com/kubernetes-sigs/kustomize/blob/master/docs/fields.md#images)
16+
for kustomization files and include a `kustomization.yaml` file within the bundle.
1617

17-
### ClusterServiceVersion Spec Changes
18-
19-
A new section `relatedImages` is added to the ClusterServiceVersionSpec.
18+
This can be used to write down additional metadata, or as a way to
19+
change only the images in a bundle without touching the rest of the definition (i.e. as part of a CI/CD process).
2020

2121
```yaml
22-
kind: ClusterServiceVersion
23-
metadata:
24-
name: etcd-operator
25-
spec:
26-
relatedImages:
27-
- name: default
28-
image: quay.io/coreos/etcd@sha256:12345
29-
annotation: default
30-
- name: etcd-2.1.5
31-
image: quay.io/coreos/etcd@sha256:12345
32-
- name: etcd-3.1.1
33-
image: quay.io/coreos/etcd@sha256:12345
22+
images:
23+
- name: postgres
24+
newName: my-registry/my-postgres
25+
newTag: v1
26+
- name: nginx
27+
newTag: 1.8.0
28+
- name: my-demo-app
29+
newName: my-app
30+
- name: alpine
31+
digest: sha256:24a0c4b4a4c0eb97a1aabb8e29f18e917d05abfe1b7a7c07857230879ce7d3d3
3432
```
3533
36-
These will be made available as annotations on the operator deployments, so that they can be used via downward API if desired. This may be particularly useful for operators that are tightly coupled to another particular image.
34+
([image type](https://github.com/kubernetes-sigs/kustomize/blob/master/pkg/image/image.go))
3735
38-
```yaml
39-
kind: Deployment
40-
metadata:
41-
name: etcd-operator
42-
annotations:
43-
default: quay.io/coreos/etcd@sha256:12345
44-
olm.relatedImage.etcd-2.1.5: quay.io/coreos/etcd@sha256:12345
45-
olm.relatedImage.etcd-3.1.1: quay.io/coreos/etcd@sha256:12345
46-
spec:
47-
replicas: 1
48-
selector:
49-
matchLabels:
50-
name: etcd-operator
51-
template:
52-
metadata:
53-
name: etcd-operator
54-
labels:
55-
name: etcd-operator
56-
spec:
57-
serviceAccountName: etcd-operator
58-
containers:
59-
- name: etcd-operator
60-
command:
61-
- etcd-operator
62-
- --create-crd=false
63-
- --defaultImage=${DEFAULT}
64-
image: quay.io/coreos/etcd-operator@sha256:c0301e4686c3ed4206e370b42de5a3bd2229b9fb4906cf85f3f30650424abec2
65-
env:
66-
- name: NAMESPACE
67-
valueFrom:
68-
fieldRef:
69-
fieldPath: metadata.namespace
70-
- name: DEFAULT
71-
valueFrom:
72-
fieldRef:
73-
fieldPath: metadata.annotations['default']
74-
```
36+
The images in the `image` list will be used to determine the set of related images that are required for the operator.
7537

76-
### Implementation
38+
The images in this list will be considered related even if applying the config to the ClusterServiceVersion would not
39+
transform it (i.e. the `name` of an image does not need to exist in the CSV).
7740

78-
#### ClusterServiceVersion Spec and Status
41+
The kustomization will be applied to the CSV, so the `image` config may be used to overwrite the images in the deployment in the CSV.
7942

80-
Spec needs to be updated to include the fields described above, and the openapi validation should be updated as well.
43+
### Why kustomize?
8144

82-
Note that the `name` field for related images must satisfy the annotation name conventions.
45+
There are several other existing approaches to associating images with an application.
8346

84-
#### Install Strategy
47+
CNAB: Using CNAB's `bundle.json` would allow us to associate image metadata, but comes with a heavy spec. CNAB bundles cannot be applied directly to a kubernetes cluster, they require additional tooling.
8548

86-
Most of the change will take place in the install strategy; which knows how to take the deployment spec defined in a CSV and check if the cluster is up-to-date, and apply changes if needed.
49+
ImageStream: OpenShift ClusterOperators include an `image-references` file that contains an ImageStream object. This allows listing objects, but is not meaningful when applied to a cluster (despite being a real object), and can only be applied to OpenShift clusters.
8750

88-
- The install strategy will now need to know about related images.
89-
- `CheckInstalled` will check that the annotations on the operator deployments include the `relatedImages` annotations.
90-
- `Install` will also need to project the relatedImages as annotations on the deployment.
91-
92-
#### Implementation Stages
51+
By using Kustomize's metadata, we:
9352

94-
- [ ] API Changes
95-
- [ ] Annotation Projection on Deployments
53+
- Have a way to list out images needed by the operator
54+
- Have a way to override the images needed by the operator (without touching the base manifests)
55+
- Retain `kubectl` compatibility; operator bundles can be applied to a cluster with `kubectl apply -k -f bundle`
9656

97-
### User Documentation
57+
### Operator Registry Changes
9858

99-
#### Associating Related Images
59+
If a bundle includes an `kustomization.yaml` file, images in this file are extracted during the `load` operation of a bundle into an
60+
operator-registry database. With the following rules:
10061

101-
Operators often need to make use of other container images to perform their functions. For example, the etcd operator
102-
makes use of etcd container images to create etcd clusters as requested by the user.
62+
- Images are pulled from the ClusterServiceVersion `container` definitions as if kustomize has been run over it.
63+
- Images are pulled from the `kustomization.yaml` file regardless of whether they are "used" by the bundle.
10364

104-
To indicate that such images are used by the operator, a ClusterServiceVersion author can fill out the `relatedImages`
105-
field on the CSV spec.
65+
If a bundle does not include a `kustomization.yaml` file, images are extracted from the ClusterServiceVersion `container` definitions.
10666

107-
These fields are optional, but should be filled out whenever possible. Tooling can take advantage of this information
108-
to ensure that all required images are available in the cluster.
67+
The `Query` interface for an operator-registry database will have two new APIs:
10968

110-
```yaml
111-
kind: ClusterServiceVersion
112-
metadata:
113-
name: etcd-operator
114-
spec:
115-
relatedImages:
116-
- name: default
117-
image: quay.io/coreos/etcd@sha256:12345
118-
annotation: default
119-
- name: etcd-2.1.5
120-
image: quay.io/coreos/etcd@sha256:12345
121-
- name: etcd-3.1.1
122-
image: quay.io/coreos/etcd@sha256:12345
69+
```go
70+
type Query interface {
71+
// ...
72+
ListImages(ctx context.Context) ([]string, error)
73+
GetImagesForBundle(ctx context.Context, csvName string) ([]string, error)
74+
}
12375
```
12476

77+
`ListImages` will list all images in an operator-registry database.
12578

126-
#### Using related images via downwardAPI
79+
### Example
12780

128-
The related images can be consumed by the operator deployment. This may be useful if, for example, the operator
129-
and operand images are tightly coupled. The `annotation` field from the `relatedImages` is used as the name of the annotation.
81+
```sh
82+
$ tree bundle
83+
bundle
84+
├── csv.yaml
85+
└── kustomization.yaml
86+
```
13087

131-
These will be made available as annotations on the operator deployments, so that they can be used via downward API if desired. This may be particularly useful for operators that are tightly coupled to another particular image.
88+
**bundle/csv.yaml**
13289

13390
```yaml
134-
kind: Deployment
135-
metadata:
136-
name: etcd-operator
137-
annotations:
138-
default: quay.io/coreos/etcd@sha256:12345
139-
olm.relatedImage.etcd-2.1.5: quay.io/coreos/etcd@sha256:12345
140-
olm.relatedImage.etcd-3.1.1: quay.io/coreos/etcd@sha256:12345
141-
spec:
142-
replicas: 1
143-
selector:
144-
matchLabels:
145-
name: etcd-operator
146-
template:
147-
metadata:
148-
name: etcd-operator
149-
labels:
150-
name: etcd-operator
151-
spec:
152-
serviceAccountName: etcd-operator
153-
containers:
154-
- name: etcd-operator
155-
command:
156-
- etcd-operator
157-
- --create-crd=false
158-
- --defaultImage=${DEFAULT}
159-
image: quay.io/coreos/etcd-operator@sha256:c0301e4686c3ed4206e370b42de5a3bd2229b9fb4906cf85f3f30650424abec2
160-
env:
161-
- name: NAMESPACE
162-
valueFrom:
163-
fieldRef:
164-
fieldPath: metadata.namespace
165-
- name: DEFAULT
166-
valueFrom:
167-
fieldRef:
168-
fieldPath: metadata.annotations['default']
91+
containers:
92+
- command:
93+
- etcd-operator
94+
- --create-crd=false
95+
image: quay.io/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b
96+
name: etcd-operator
16997
```
17098

171-
### Future Work
99+
**bundle/kustomization.yaml**
100+
```yaml
101+
images:
102+
- name: quay.io/coreos/etcd-operator
103+
newTag: latest
104+
- name: quay.io/coreos/etcd
105+
newTag: 3.0.5
106+
- name: quay.io/coreos/etcd
107+
digest: sha256:24a0c4b4a4c0eb97a1aabb8e29f18e917d05abfe1b7a7c07857230879ce7d3d3
108+
resources:
109+
- csv.yaml
110+
```
172111

173-
#### Required Images
112+
The list of images will then be:
174113

175-
Any of the related images may be marked required. This would prevent the operator from installing if the required image is unavailable.
114+
```
115+
quay.io/coreos/etcd-operator:latest
116+
quay.io/coreos/etcd:3.0.5
117+
quay.io/coreos/etcd@sha256:24a0c4b4a4c0eb97a1aabb8e29f18e917d05abfe1b7a7c07857230879ce7d3d3
118+
```
176119

120+
Note that `quay.io/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b` is not
121+
included, since it would be replaced with `:latest` if the `kustomization.yaml` were applied.
122+
123+
### Future Work
124+
125+
#### Override Operand
126+
127+
Add a `relatedImages` field to the ClusterServiceVersion, and make use of kustomize's [transformer configs](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/transformerconfigs/images/README.md) to teach it about those fields.
128+
`relatedImages` can be projected into operator deployments via downward API, which will allow the kustomization file to override operand images in addition to opeator images.
177129

178-
```yaml
179-
kind: ClusterServiceVersion
180-
metadata:
181-
name: etcd-operator
182-
spec:
183-
relatedImages:
184-
- name: default
185-
image: quay.io/coreos/etcd@sha256:12345
186-
annotation: default
187-
required: true
188-
- name: etcd-2.1.5
189-
image: quay.io/coreos/etcd@sha256:12345
190-
- name: etcd-3.1.1
191-
image: quay.io/coreos/etcd@sha256:12345
192-
```

0 commit comments

Comments
 (0)