Skip to content

Commit b6d9c91

Browse files
AkrogAkrog
committed
Docs: External dependencies
This patch introduces a couple of guides on how to do dependencies in the cinder-operator to explain simple and circular references for both running the operator locally and building the operator.
1 parent 5d96003 commit b6d9c91

File tree

3 files changed

+345
-0
lines changed

3 files changed

+345
-0
lines changed

‎CONTRIBUTING.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,9 @@ section](README.md#getting-started):
7070

7171
- [Run operator locally](docs/dev/local.md)
7272
- [Run operator in OpenShift using a custom image](docs/dev/custom-image.md)
73+
- [Running locally with external dependencies](docs/dev/local-dependencies.md)
74+
- [Running in OpenShift with external
75+
dependencies](docs/dev/custom-image-dependencies.md)
7376

7477
### Pull Requests
7578

‎docs/dev/custom-image-dependencies.md‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Running the operator in OpenShift with external dependencies
2+
3+
**NOTE**: This article [makes some assumptions](assumptions.md), make sure they
4+
are correct or adapt the steps accordingly.
5+
6+
If you've already [run the operator in OpenShift](custom-image.md) before,
7+
you'll be familiar with the process of running custom cinder-operator code
8+
by building and deploying a custom image.
9+
10+
Let's see what we have to do to run our operator in OpenShift when we need to
11+
compile and build containers with external dependencies.
12+
13+
This short guide builds on the knowledge provided in [guide to running the
14+
operator in OpenShift](custom-image.md) to explain external dependencies usage.
15+
16+
The steps will be the same, the only difference is that some additional steps
17+
are necessary between the [Preparation](custom-image.md#preparation) and the
18+
[Image build](custom-image.md#image-build) steps to indicate the new
19+
references.
20+
21+
To avoid repeating things again, this text assumes familiarity with the
22+
concepts presented in the [running locally with external
23+
dependencies](local-dependencies.md) article.
24+
25+
All the cases described in running the operator locally with external
26+
dependencies where the dependency was expressed as github repositories will
27+
work fine when building the containers, so we don't need to do anything
28+
special.
29+
30+
The difference comes when using local paths to reference the external
31+
dependencies, because those are outside of the root directory of the
32+
`Dockerfile` and therefore won't be present in the building container.
33+
34+
The solution is to use bind mounts.
35+
36+
### Unmerged local simple reference
37+
38+
Assuming the same example as running things locally we would do:
39+
40+
```
41+
cd ~/cinder-operator
42+
mkdir -p tmp/lib-common
43+
sudo mount --bind `realpath ../lib-common` `realpath tmp/lib-common`
44+
45+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/common=tmp/lib-common/modules/common
46+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/database=tmp/lib-common/modules/database
47+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/storage=tmp/lib-common/modules/storage
48+
49+
go mod tidy
50+
```
51+
52+
Then we can just continue with [building the
53+
image](custom-image.md#image-build) like we would normally do.
54+
55+
### Circular references
56+
57+
In the circular references case when using local paths it's a bit more
58+
complicated than the local run case, because the bind mount created from the
59+
cinder-operator to the openstack-operator is not preserved when we bind mount
60+
the cinder-operator from the openstack-operator, so we need to link it again.
61+
62+
We'll assume we have the dependency between cinder-operator and
63+
openstack-operator and that our local repositories have the code we want to
64+
use.
65+
66+
```
67+
cd ~/cinder-operator
68+
mkdir -p tmp/openstack-operator
69+
sudo mount --bind `realpath ../openstack-operator` `realpath tmp/openstack-operator`
70+
71+
go mod edit -replace github.com/openstack-k8s-operators/openstack-operator/apis=tmp/openstack-operator
72+
go mod tidy
73+
74+
75+
cd ~/openstack-operator
76+
mkdir -p tmp/cinder-operator
77+
sudo mount --bind `realpath ../cinder-operator` `realpath tmp/cinder-operator`
78+
sudo mount --bind `realpath ./` `realpath tmp/cinder-operator/tmp/openstack-operator`
79+
80+
go mod edit -replace github.com/openstack-k8s-operators/cinder-operator/api=tmp/cinder-operator/api
81+
go mod tidy
82+
```
83+
84+
Now we can continue with the [Image build steps](custom-image.md#image-build).

‎docs/dev/local-dependencies.md‎

Lines changed: 258 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,258 @@
1+
# Running the operator locally with external dependencies
2+
3+
**NOTE**: This article [makes some assumptions](assumptions.md), make sure they
4+
are correct or adapt the steps accordingly.
5+
6+
If you've already [run the operator locally](local.md) before, you'll be
7+
familiar with the process of removing the cinder podified services and the
8+
cinder-operator pod from the OpenShift cluster, building the operator and
9+
running it locally.
10+
11+
It's very likely that during the development of the cinder-operator you'll find
12+
yourself needing to use a newer version of a dependency or even needing to use
13+
an unmerged code of a dependency.
14+
15+
This short guide builds on the knowledge provided in [guide to locally run the
16+
operator](local.md) to explain those external dependencies usages.
17+
18+
The steps will be the same, the only difference is that some additional steps
19+
are necessary between the [Preparation](local.md#preparation) and the [Build
20+
and Run](local.md#build-and-run) steps to indicate the new references.
21+
22+
There are slight differences between the possible cases, lib-common dependency
23+
update, circular reference with the openstack-operator, the external dependency
24+
code being merged or not, having the external dependency's code locally or not,
25+
etc., so to facilitate its understanding we'll provide separate explanations
26+
for most of them.
27+
28+
### Merged simple reference
29+
30+
If we want to compile the cinder-operator with newer lib-common code that has
31+
already merged in its repository but don't want to wait for dependabot to
32+
propose an update to the project's dependencies and for it to get merged we
33+
can just run the following commands from the `cinder-operator` repository
34+
before [Building and Running the operator](local.md#build-and-run):
35+
36+
```
37+
go get github.com/openstack-k8s-operators/lib-common/modules/common
38+
go get github.com/openstack-k8s-operators/lib-common/modules/database
39+
go get github.com/openstack-k8s-operators/lib-common/modules/storage
40+
```
41+
42+
As you can imagine, we don't need to always run the 3 commands, if we just need
43+
1 of the 3 modules we can just ran that specific command.
44+
45+
**NOTE**: Using the ``go get`` command instead of ``go mod edit`` ensures that
46+
other `go.mod` adjustments are made as needed to satisfy constraints imposed
47+
by other modules.
48+
49+
After running these commands we can see that the `go.mod` and `go.sum` have
50+
been updated:
51+
52+
```
53+
$ git diff go.*
54+
55+
diff --git a/go.mod b/go.mod
56+
index 599f056..8e7d423 100644
57+
--- a/go.mod
58+
+++ b/go.mod
59+
@@ -9,9 +9,9 @@ require (
60+
github.com/openshift/api v3.9.0+incompatible
61+
github.com/openstack-k8s-operators/cinder-operator/api v0.0.0-20221010180347-a9a8efadf3c3
62+
github.com/openstack-k8s-operators/keystone-operator/api v0.0.0-20220927090553-6b3218c776f7
63+
- github.com/openstack-k8s-operators/lib-common/modules/common v0.0.0-20221103175706-2c39582ce513
64+
- github.com/openstack-k8s-operators/lib-common/modules/database v0.0.0-20220923094431-9fca0c85a9dc
65+
- github.com/openstack-k8s-operators/lib-common/modules/storage v0.0.0-20220923094431-9fca0c85a9dc
66+
+ github.com/openstack-k8s-operators/lib-common/modules/common v0.0.0-20221117092428-c1190ea3bf3d
67+
+ github.com/openstack-k8s-operators/lib-common/modules/database v0.0.0-20221117092428-c1190ea3bf3d
68+
+ github.com/openstack-k8s-operators/lib-common/modules/storage v0.0.0-20221117092428-c1190ea3bf3d
69+
github.com/openstack-k8s-operators/mariadb-operator/api v0.0.0-20221014164348-0a612ae8b391
70+
github.com/openstack-k8s-operators/openstack-operator/apis v0.0.0-20221107090218-8d63dba1ec13
71+
k8s.io/api v0.25.4
72+
diff --git a/go.sum b/go.sum
73+
index c19f296..2130ed6 100644
74+
--- a/go.sum
75+
+++ b/go.sum
76+
@@ -323,12 +323,18 @@ github.com/openstack-k8s-operators/keystone-operator/api v0.0.0-20220927090553-6
77+
github.com/openstack-k8s-operators/keystone-operator/api v0.0.0-20220927090553-6b3218c776f7/go.mod h1:q/owiyXlI2W4uQR4TeHPeeN75AGDfyZgQdNHeKUYN68=
78+
github.com/openstack-k8s-operators/lib-common/modules/common v0.0.0-20221103175706-2c39582ce513 h1:PSXOLFTskoG9R/YR4Pg5AOJYS3CEnFbZ2yVdrk9xOE4=
79+
github.com/openstack-k8s-operators/lib-common/modules/common v0.0.0-20221103175706-2c39582ce513/go.mod h1:KWqK7l2ej+rIYngoNUrxE2YjKGlRAAgJXXM0uU2R6XY=
80+
+github.com/openstack-k8s-operators/lib-common/modules/common v0.0.0-20221117092428-c1190ea3bf3d h1:/1FTHxBQJo4xM0GmJCX5wPCYmyLWTw1uHQKCydGH3mY=
81+
+github.com/openstack-k8s-operators/lib-common/modules/common v0.0.0-20221117092428-c1190ea3bf3d/go.mod h1:KWqK7l2ej+rIYngoNUrxE2YjKGlRAAgJXXM0uU2R6XY=
82+
github.com/openstack-k8s-operators/lib-common/modules/database v0.0.0-20220923094431-9fca0c85a9dc h1:87lUVT3MLRI4Vg0nHpupwPKXtykGX3hZzPl5k6Kcyng=
83+
github.com/openstack-k8s-operators/lib-common/modules/database v0.0.0-20220923094431-9fca0c85a9dc/go.mod h1:umGUqQO4JtgefAaIwZjP+TxfxsLMEEeK/6VNzk8ooaI=
84+
+github.com/openstack-k8s-operators/lib-common/modules/database v0.0.0-20221117092428-c1190ea3bf3d h1:lO5WmV9RjVAIxbr1HPjvqVy6niatdEPG+FyEbL4FMpc=
85+
+github.com/openstack-k8s-operators/lib-common/modules/database v0.0.0-20221117092428-c1190ea3bf3d/go.mod h1:umGUqQO4JtgefAaIwZjP+TxfxsLMEEeK/6VNzk8ooaI=
86+
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.0.0-20220915080953-f73a201a1da6 h1:MVNEHyqD0ZdO9jiyUSKw5M2T9Lc4l4Wx1pdC2/BSJ5Y=
87+
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.0.0-20220915080953-f73a201a1da6/go.mod h1:YsqouRH8DoZAjFaxcIErspk59BcwXtVjPxK/yV17Wrc=
88+
github.com/openstack-k8s-operators/lib-common/modules/storage v0.0.0-20220923094431-9fca0c85a9dc h1:Dud2dr25VhaZF9Av28nqmCeBfNkGWDckZ5TaajEcGFc=
89+
github.com/openstack-k8s-operators/lib-common/modules/storage v0.0.0-20220923094431-9fca0c85a9dc/go.mod h1:fhM62I45VF/5WVpOP1h9OpTfFn+lF2XGrT5jUBKEHVc=
90+
+github.com/openstack-k8s-operators/lib-common/modules/storage v0.0.0-20221117092428-c1190ea3bf3d h1:b2dqfShyX4NO/NMe3rTK1xWRF91ITobjQEfO6ftO6yM=
91+
+github.com/openstack-k8s-operators/lib-common/modules/storage v0.0.0-20221117092428-c1190ea3bf3d/go.mod h1:fhM62I45VF/5WVpOP1h9OpTfFn+lF2XGrT5jUBKEHVc=
92+
github.com/openstack-k8s-operators/mariadb-operator/api v0.0.0-20221014164348-0a612ae8b391 h1:Bd1e4CG/0gQbRoSH1EJLS1tin9XUjPR2s1e+dpBHiUs=
93+
github.com/openstack-k8s-operators/mariadb-operator/api v0.0.0-20221014164348-0a612ae8b391/go.mod h1:HiEKXmDSJ6Gl+pN7kK5CX1sgOjrxybux4Ob5pdUim1M=
94+
github.com/openstack-k8s-operators/openstack-operator/apis v0.0.0-20221107090218-8d63dba1ec13 h1:GkYSRpfdDav5HipJ2oIYqpY8crLaDVRBmhqlUztDTV4=
95+
```
96+
97+
Now we can proceed as usual with the [Building and
98+
Running](local.md#build-and-run) steps.
99+
100+
Once we have confirmed that our code changes work as expected with the new
101+
dependency version we can submit our PR as we normally do but including the
102+
`go.mod` and `go.sum` changes.
103+
104+
If we want to use a specific tag instead of `HEAD` we can provide it in our
105+
calls:
106+
107+
```
108+
go get github.com/openstack-k8s-operators/lib-common/modules/[email protected]
109+
go get github.com/openstack-k8s-operators/lib-common/modules/[email protected]
110+
go get github.com/openstack-k8s-operators/lib-common/modules/[email protected]
111+
```
112+
113+
### Unmerged local simple reference
114+
115+
There will be times when we'll be working on lib-common code in our local
116+
repository while using it in the cinder-operator and we want to test this.
117+
118+
In this case we cannot use the `go get` because our lib-common module is not
119+
within the cinder-operator command so we'll use the `go mod edit` command
120+
instead:
121+
122+
```
123+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/common=`realpath ../lib-common/modules/common`
124+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/database=`realpath ../lib-common/modules/database`
125+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/storage=`realpath ../lib-common/modules/storage`
126+
```
127+
128+
Once we have run those 3 commands we need to add any missing modules necessary
129+
to build the current module's packages and dependencies, so we'll run
130+
131+
```
132+
go mod tidy
133+
```
134+
135+
Now we can proceed as usual with the [Building and
136+
Running](local.md#build-and-run) steps.
137+
138+
Once we have confirmed that our code changes work as expected with the new
139+
dependency version we will need to submit the lib-common PR first and then the
140+
one in cinder-operator with the new dependency, as explained in the previous
141+
section.
142+
143+
### Simple reference in PR
144+
145+
Somebody may have submitted a PR to lib-common and we want to start writing
146+
code in the cinder-operator before that PR gets merged.
147+
148+
Here we can either download the PR to our own local repository or just
149+
reference the one from GitHub.
150+
151+
As an example, let's look at the `extraVolumes` effort and the [lib-common
152+
PR#88](https://github.com/openstack-k8s-operators/lib-common/pull/88).
153+
154+
To reference it directly we'll go to GitHub's website and look for the source
155+
of the PR, which in this case it's [fmount's extra_volumes branch](
156+
https://github.com/fmount/lib-common/tree/extra_volumes).
157+
158+
So now we replace our lib-common with that one:
159+
160+
```
161+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/common=github.com/fmount/lib-common/modules/common@extra_volumes
162+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/database=github.com/fmount/lib-common/modules/database@extra_volumes
163+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/storage=github.com/fmount/lib-common/modules/storage@extra_volumes
164+
165+
go mod tidy
166+
```
167+
168+
If we have a local repository and we may want to make changes to the lib-common
169+
code ourselves (in case we find some issue) we can pull the PR and then
170+
reference it:
171+
172+
```
173+
cd ../lib-common
174+
175+
git fetch upstream pull/88/head:extra_volumes
176+
git checkout extra_volumes
177+
178+
cd ../cinder-operator
179+
180+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/common=`realpath ../lib-common/modules/common`
181+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/database=`realpath ../lib-common/modules/database`
182+
go mod edit -replace github.com/openstack-k8s-operators/lib-common/modules/storage=`realpath ../lib-common/modules/storage`
183+
184+
go mod tidy
185+
```
186+
187+
Now we can proceed as usual with the [Building and
188+
Running](local.md#build-and-run) steps.
189+
190+
### Circular references
191+
192+
There are times where we have circular dependencies where cinder-operator needs
193+
a new structure defined in the openstack-operator but at the same time the
194+
openstack-operator uses the cinder-operator structures to define the
195+
`OpenStackControlPlane`.
196+
197+
An example of this happening is the work on the `TrasportURL` that was
198+
introduced in [openstack-operator
199+
PR#37](https://github.com/openstack-k8s-operators/openstack-operator/pull/27)
200+
and used in the [cinder-operator
201+
PR#62](https://github.com/openstack-k8s-operators/cinder-operator/pull/62).
202+
203+
The solution of how to replace the dependencies is pretty much the same as the
204+
one we've seen in the previous section, the only difference is that we need to
205+
do it in both repositories.
206+
207+
We see in github that the cinder-operator PR source branch is
208+
`use_transport_url_crd` from `abays` and the openstack-operator PR source
209+
branch is `rabbitmq_transporturl` from `dprince`, so we replace their
210+
dependencies:
211+
212+
```
213+
cd ~/cinder-operator
214+
go mod edit -replace github.com/openstack-k8s-operators/openstack-operator/apis=github.com/dprince/openstack-operator@rabbitmq_transporturl
215+
go mod tidy
216+
217+
cd ~/openstack-operator
218+
go mod edit -replace github.com/openstack-k8s-operators/cinder-operator/api=github.com/abays/cinder-operator@use_transport_url_crd
219+
go mod tidy
220+
```
221+
222+
In this case since we have also modified the openstack-operator we'll have to
223+
stop it in our OpenShift cluster and run it locally before we can proceed with
224+
[building and running the cinder-operator](local.md#build-and-run).
225+
226+
```sh
227+
CSV=`oc get -l operators.coreos.com/openstack-operator.openstack= -o custom-columns=CSV:.metadata.name --no-headers csv`
228+
229+
oc edit csv $CSV
230+
```
231+
232+
This will drop us in our editor with the contents of CSV YAML manifest where
233+
we'll search for the first instance of `name:
234+
openstack-operator-controller-manager`, and we should see something like:
235+
236+
```
237+
- label:
238+
control-plane: controller-manager
239+
name: openstack-operator-controller-manager
240+
spec:
241+
replicas: 1
242+
selector:
243+
matchLabels:
244+
control-plane: controller-manager
245+
```
246+
247+
Where we see `replicas: 1` change it to `replicas: 0`, save and exit. This
248+
triggers the termination of the openstack-operator pod.
249+
250+
Now we just build and run our local openstack-operator:
251+
252+
```
253+
cd ~/openstack-operator
254+
make run
255+
```
256+
257+
Now we can continue, in another terminal, [running the cinder-operator
258+
locally](local.md#build-and-run).

0 commit comments

Comments
 (0)