Skip to content

Commit b6b64ab

Browse files
vidarnovidarno
committed
Add pages for Custom Resources - document Metal3cluster, Metal3Machine and Metal3MachineTemplate
Signed-off-by: Vidar Normann <1858786+vidarno@users.noreply.github.com>
1 parent 906e6ce commit b6b64ab

File tree

5 files changed

+333
-0
lines changed

5 files changed

+333
-0
lines changed

‎docs/user-guide/src/SUMMARY.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,10 @@
4141
- [Old Installation Process](ironic/ironic_installation.md)
4242
- [Cluster-api-provider-metal3](capm3/introduction.md)
4343
- [Install Metal³ provider](capm3/installation_guide.md)
44+
- [Custom Resources](capm3/custom_resources.md)
45+
- [Metal3Cluster](capm3/metal3cluster.md)
46+
- [Metal3Machine](capm3/metal3machine.md)
47+
- [Metal3MachineTemplate](capm3/metal3machinetemplate.md)
4448
- [Features](capm3/features.md)
4549
- [Remediation](capm3/remediaton.md)
4650
- [Node Reuse](capm3/node_reuse.md)

‎docs/user-guide/src/capm3/custom_resources.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Cluster-api-provider-metal3 Custom Resources
2+
- [Metal3Cluster](./metal3cluster.md)
3+
- [Metal3Machine](./metal3machine.md)
4+
- [Metal3MachineTemplate](./metal3machinetemplate.md)

‎docs/user-guide/src/capm3/metal3cluster.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Metal3Cluster
2+
3+
The metal3Cluster object contains information related to the deployment of the
4+
cluster on Baremetal. It currently has two specification fields :
5+
6+
- **controlPlaneEndpoint**: contains the target cluster API server address and
7+
port
8+
- **noCloudProvider(Deprecated use CloudProviderEnabled)**: (true/false) Whether
9+
the cluster will not be deployed with an external cloud provider. If set to
10+
true, CAPM3 will patch the target cluster node objects to add a providerID.
11+
This will allow the CAPI process to continue even if the cluster is deployed
12+
without cloud provider.
13+
- **CloudProviderEnabled**: (true/false) Whether the cluster will be deployed
14+
with an external cloud provider. If set to false, CAPM3 will patch the target
15+
cluster node objects to add a providerID. This will allow the CAPI process to
16+
continue even if the cluster is deployed without cloud provider.
17+
18+
Example metal3cluster :
19+
20+
```yaml
21+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
22+
kind: Metal3Cluster
23+
metadata:
24+
name: m3cluster
25+
namespace: metal3
26+
spec:
27+
controlPlaneEndpoint:
28+
host: 192.168.111.249
29+
port: 6443
30+
cloudProviderEnabled: false
31+
```

‎docs/user-guide/src/capm3/metal3machine.md‎

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
# Metal3Machine
2+
3+
4+
The Metal3Machine contains information related to the deployment of the
5+
BareMetalHost such as the image and the host selector. For each machine, there
6+
must be a Metal3Machine.
7+
8+
9+
The fields are:
10+
11+
12+
- **image** -- This includes two sub-fields, `url` and `checksum`, which include
13+
the URL to the image and the URL to a checksum for that image. These fields
14+
are required. The image will be used for provisioning of the `BareMetalHost`
15+
chosen by the `Machine` actuator.
16+
17+
18+
- **userData** -- This includes two sub-fields, `name` and `namespace`, which
19+
reference a `Secret` that contains base64 encoded user-data to be written to a
20+
config drive on the provisioned `BareMetalHost`. This field is optional and is
21+
automatically set by CAPM3 with the userData from the machine object. If you
22+
want to overwrite the userData, this should be done in the CAPI machine.
23+
24+
25+
- **dataTemplate** -- This includes a reference to a Metal3DataTemplate object
26+
containing the metadata and network data templates, and includes two fields,
27+
`name` and `namespace`.
28+
29+
30+
- **metaData** is a reference to a secret containing the metadata rendered from
31+
the Metal3DataTemplate metadata template object automatically. In case this
32+
would not be managed by the Metal3DataTemplate controller, if provided by the
33+
user for example, the ownerreference should be set properly to ensure that the
34+
secret belongs to the cluster ownerReference tree (see
35+
[doc](https://cluster-api.sigs.k8s.io/clusterctl/provider-contract.html#ownerreferences-chain)).
36+
37+
38+
- **networkData** is a reference to a secret containing the network data
39+
rendered from the Metal3DataTemplate metadata template object automatically.
40+
In case this would not be managed by the Metal3DataTemplate controller, if
41+
provided by the user for example, the ownerreference should be set properly to
42+
ensure that the secret belongs to the cluster ownerReference tree (see
43+
[doc](https://cluster-api.sigs.k8s.io/clusterctl/provider-contract.html#ownerreferences-chain)).
44+
The content of the secret should be a yaml equivalent of a json object that
45+
follows the format definition that can be found
46+
[here](https://docs.openstack.org/nova/latest/_downloads/9119ca7ac90aa2990e762c08baea3a36/network_data.json).
47+
48+
49+
- **hostSelector** -- Specify criteria for matching labels on `BareMetalHost`
50+
objects. This can be used to limit the set of available `BareMetalHost`
51+
objects chosen for this `Machine`.
52+
53+
54+
- **automatedCleaningMode** -- An interface to enable or disable Ironic
55+
automated cleaning during provisioning or deprovisioning of a host. When set
56+
to `disabled`, automated cleaning will be skipped, where `metadata` value
57+
enables it. It is recommended to tune the cleaning via metal3MachineTemplate
58+
rather than metal3Machine. When `spec.template.spec.automatedCleaningMode`
59+
field of metal3MachineTemplate is updated, metal3MachineTemplate controller
60+
will update all the metal3Machines (generated from the metal3MachineTemplate)
61+
and eventually BareMetalHosts with the same value.
62+
63+
64+
The `metaData` and `networkData` field in the `spec` section are for the user to
65+
give directly a secret to use as metaData or networkData. The `userData`,
66+
`metaData` and `networkData` fields in the `status` section are for the
67+
controller to store the reference to the secret that is actually being used,
68+
whether it is from one of the spec fields, or somehow generated. This is aimed
69+
at making a clear difference between the desired state from the user (whether it
70+
is with a DataTemplate reference, or direct `metaData` or `userData` secrets)
71+
and what the controller is actually using.
72+
73+
74+
The `dataTemplate` field consists of an object reference to a Metal3DataTemplate
75+
object containing the templates for the metadata and network data generation for
76+
this Metal3Machine. The `renderedData` field is a reference to the Metal3Data
77+
object created for this machine. If the dataTemplate field is set but either the
78+
`renderedData`, `metaData` or `networkData` fields in the status are unset, then
79+
the Metal3Machine controller will wait until it can find the Metal3Data object
80+
and the rendered secrets. It will then populate those fields.
81+
82+
83+
When CAPM3 controller will set the different fields in the BareMetalHost, it
84+
will reference the metadata secret and the network data secret in the
85+
BareMetalHost. If any of the `metaData` or `networkData` status fields are
86+
unset, that field will also remain unset on the BareMetalHost.
87+
88+
89+
When the Metal3Machine gets deleted, the CAPM3 controller will remove its
90+
ownerreference from the data template object. This will trigger the deletion of
91+
the generated Metal3Data object and the secrets generated for this machine.
92+
93+
94+
### hostSelector Examples
95+
96+
97+
The `hostSelector field has two possible optional sub-fields:
98+
99+
100+
- **matchLabels** -- Key/value pairs of labels that must match exactly.
101+
102+
103+
- **matchExpressions** -- A set of expressions that must evaluate to true for
104+
the labels on a `BareMetalHost`.
105+
106+
107+
Valid operators include:
108+
109+
110+
- **!** -- Key does not exist. Values ignored.
111+
- **=** -- Key equals specified value. There must only be one value specified.
112+
- **==** -- Key equals specified value. There must only be one value specified.
113+
- **in** -- Value is a member of a set of possible values
114+
- **!=** -- Key does not equal the specified value. There must only be one value
115+
specified.
116+
- **notin** -- Value not a member of the specified set of values.
117+
- **exists** -- Key exists. Values ignored.
118+
- **gt** -- Value is greater than the one specified. Value must be an integer.
119+
- **lt** -- Value is less than the one specified. Value must be an integer.
120+
121+
122+
Example 1: Only consider a `BareMetalHost` with label `key1` set to `value1`.
123+
124+
125+
```yaml
126+
spec:
127+
providerSpec:
128+
value:
129+
hostSelector:
130+
matchLabels:
131+
key1: value1
132+
```
133+
134+
135+
Example 2: Only consider `BareMetalHost` with both `key1` set to `value1` AND
136+
`key2` set to `value2`.
137+
138+
139+
```yaml
140+
spec:
141+
providerSpec:
142+
value:
143+
hostSelector:
144+
matchLabels:
145+
key1: value1
146+
key2: value2
147+
```
148+
149+
150+
Example 3: Only consider `BareMetalHost` with `key3` set to either `a`, `b`, or
151+
`c`.
152+
153+
154+
```yaml
155+
spec:
156+
providerSpec:
157+
value:
158+
hostSelector:
159+
matchExpressions:
160+
- key: key3
161+
operator: in
162+
values: [‘a’, ‘b’, ‘c’]
163+
```
164+
165+
166+
Example 3: Only consider `BareMetalHost` with `key1` set to `value1` AND `key2`
167+
set to `value2` AND `key3` set to either `a`, `b`, or `c`.
168+
169+
170+
```yaml
171+
spec:
172+
providerSpec:
173+
value:
174+
hostSelector:
175+
matchLabels:
176+
key1: value1
177+
key2: value2
178+
matchExpressions:
179+
- key: key3
180+
operator: in
181+
values: [‘a’, ‘b’, ‘c’]
182+
```
183+
184+
185+
### Metal3Machine example
186+
187+
188+
```yaml
189+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
190+
kind: Metal3Machine
191+
metadata:
192+
name: controlplane-0
193+
namespace: metal3
194+
spec:
195+
automatedCleaningMode: metadata
196+
image:
197+
checksum: http://172.22.0.1/images/UBUNTU_22.04_NODE_IMAGE_K8S_v1.29.0-raw.img.sha256sum
198+
checksumType: sha256
199+
format: raw
200+
url: http://172.22.0.1/images/UBUNTU_22.04_NODE_IMAGE_K8S_v1.29.0-raw.img
201+
hostSelector:
202+
matchLabels:
203+
key1: value1
204+
matchExpressions:
205+
key: key2
206+
operator: in
207+
values: { ‘abc’, ‘123’, ‘value2’ }
208+
dataTemplate:
209+
Name: controlplane-metadata
210+
metaData:
211+
Name: controlplane-0-metadata-0
212+
```

‎docs/user-guide/src/capm3/metal3machinetemplate.md‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Metal3MachineTemplate
2+
3+
4+
The Metal3MachineTemplate contains following two specification fields:
5+
6+
7+
- **nodeReuse**: (true/false) Whether the same pool of BareMetalHosts will be
8+
re-used during the upgrade/remediation operations. By default set to false, if
9+
set to true, CAPM3 Machine controller will pick the same pool of
10+
BareMetalHosts that were released while upgrading/remediation - for the next
11+
provisioning phase.
12+
- **template**: is a template containing the data needed to create a
13+
Metal3Machine.
14+
15+
16+
### Enabling nodeReuse feature
17+
18+
19+
This feature can be desirable and enabled in scenarios such as upgrade or node
20+
remediation. For example, the same pool of hosts need to be used after cluster
21+
upgrade and no data of secondary storage should be lost. To achieve that:
22+
23+
24+
1. `spec.nodeReuse` field of metal3MachineTemplate must be set to `True`. This
25+
tells that we want to reuse the same hosts after the upgrade, or to be exact
26+
same BareMetalHosts should be provisioned.
27+
28+
29+
1. `spec.template.spec.automatedCleaningMode` field of metal3MachineTemplate
30+
must be set to `disabled`. This tells that we want secondary/hosted storage
31+
data to persist even after upgrade.
32+
33+
34+
Above field changes need to be made before you start upgrading your cluster.
35+
36+
37+
#### Node Reuse flow
38+
39+
40+
When `spec.nodeReuse` field of metal3MachineTemplate is set to `True`, CAPM3
41+
Machine controller:
42+
43+
44+
- Sets `infrastructure.cluster.x-k8s.io/node-reuse` label to the
45+
corresponding CAPI object name (a `controlplane.cluster.x-k8s.io`
46+
object such as `KubeadmControlPlane` or a `MachineDeployment`) on the
47+
BareMetalHost during deprovisioning;
48+
- Selects the BareMetalHost that contains
49+
`infrastructure.cluster.x-k8s.io/node-reuse` label and matches exact
50+
same CAPI object name set in the previous step during next
51+
provisioning.
52+
53+
54+
Example Metal3MachineTemplate :
55+
56+
57+
```yaml
58+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
59+
kind: Metal3MachineTemplate
60+
metadata:
61+
name: m3mt-0
62+
namespace: metal3
63+
spec:
64+
nodeReuse: false
65+
template:
66+
spec:
67+
automatedCleaningMode: metadata
68+
image:
69+
checksum: http://172.22.0.1/images/UBUNTU_22.04_NODE_IMAGE_K8S_v1.29.0-raw.img.sha256sum
70+
checksumType: sha256
71+
format: raw
72+
url: http://172.22.0.1/images/UBUNTU_22.04_NODE_IMAGE_K8S_v1.29.0-raw.img
73+
hostSelector:
74+
matchLabels:
75+
key1: value1
76+
matchExpressions:
77+
key: key2
78+
operator: in
79+
values: { ‘abc’, ‘123’, ‘value2’ }
80+
dataTemplate:
81+
Name: m3mt-0-metadata
82+
```

0 commit comments

Comments
 (0)