Skip to content

Commit 51cb660

Browse files
rohennesrohennes
committed
Adding concepts and procedures for adding BMHs to UPI clusters
1 parent 4e60745 commit 51cb660

9 files changed

+452
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -302,6 +302,8 @@ Topics:
302302
File: installing-bare-metal-network-customizations
303303
- Name: Installing a user-provisioned bare metal cluster on a restricted network
304304
File: installing-restricted-networks-bare-metal
305+
- Name: Scaling a user-provisioned installation with the bare metal operator
306+
File: scaling-a-user-provisioned-cluster-with-the-bare-metal-operator
305307
- Name: Installing on-premise with Assisted Installer
306308
Dir: installing_on_prem_assisted
307309
Distros: openshift-enterprise

‎installing/installing_bare_metal/scaling-a-user-provisioned-cluster-with-the-bare-metal-operator.adoc

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
:_content-type: ASSEMBLY
2+
[id="scaling-a-user-provisioned-cluster-with-the-bare-metal-operator"]
3+
= Scaling a user-provisioned cluster with the Bare Metal Operator
4+
include::_attributes/common-attributes.adoc[]
5+
:context: scaling-a-user-provisioned-cluster-with-the-bare-metal-operator
6+
7+
toc::[]
8+
9+
After deploying a user-provisioned infrastructure cluster, you can use the Bare Metal Operator (BMO) and other metal3 components to scale bare-metal hosts in the cluster. This approach helps to you to scale a user-provisioned cluster in a more automated way.
10+
11+
include::modules/about-scaling-a-user-provisioned-installation-with-the-bare-metal-operator.adoc[leveloffset=+1]
12+
include::modules/upi-prerequisites-for-scaling-a-upi-cluster.adoc[leveloffset=+2]
13+
include::modules/upi-limitations-for-scaling-a-upi-cluster.adoc[leveloffset=+2]
14+
include::modules/configuring-a-provisioning-resource-to-scale-user-provisioned-clusters.adoc[leveloffset=+1]
15+
include::modules/upi-provisioning-new-hosts-in-a-upi-cluster.adoc[leveloffset=+1]
16+
include::modules/upi-managing-existing-hosts-in-a-upi-cluster.adoc[leveloffset=+1]
17+
include::modules/upi-removing-hosts-from-a-upi-cluster.adoc[leveloffset=+1]
18+
19+

‎modules/about-scaling-a-user-provisioned-installation-with-the-bare-metal-operator.adoc

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * installing/installing_bare_metal/scaling-a-user-provisioned-cluster-with-the-bare-metal-operator.adoc
4+
5+
:_content-type: CONCEPT
6+
7+
[id="about-scaling-a-user-provisioned-cluster-with-the-bare-metal-operator_{context}"]
8+
= About scaling a user-provisioned cluster with the Bare Metal Operator
9+
10+
You can scale user-provisioned infrastructure clusters by using the Bare Metal Operator (BMO) and other metal3 components. User-provisioned infrastructure installations do not feature the Machine API Operator. The Machine API Operator typically manages the lifecycle of bare-metal hosts in a cluster. However, it is possible to use the BMO and other metal3 components to scale nodes in user-provisioned clusters without requiring the Machine API Operator.

‎modules/configuring-a-provisioning-resource-to-scale-user-provisioned-clusters.adoc

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * installing/installing_bare_metal/scaling-a-user-provisioned-cluster-with-the-bare-metal-operator.adoc
4+
:_content-type: PROCEDURE
5+
6+
[id="configuring-a-provisioning-resource-to-scale-user-provisioned-clusters_{context}"]
7+
= Configuring a provisioning resource to scale user-provisioned clusters
8+
9+
Create a `Provisioning` custom resource (CR) to enable Metal platform components on a user-provisioned infrastructure cluster.
10+
11+
.Prerequisites
12+
13+
* You installed a user-provisioned infrastructure cluster on bare metal.
14+
15+
.Procedure
16+
17+
. Create a `Provisioning` CR.
18+
19+
.. Save the following YAML in the `provisioning.yaml` file:
20+
+
21+
[source,yaml]
22+
----
23+
apiVersion: metal3.io/v1alpha1
24+
kind: Provisioning
25+
metadata:
26+
name: provisioning-configuration
27+
spec:
28+
provisioningNetwork: "Disabled"
29+
watchAllNamespaces: false
30+
----
31+
+
32+
[NOTE]
33+
====
34+
{product-title} {product-version} does not support enabling a provisioning network when you scale a user-provisioned cluster by using the Bare Metal Operator.
35+
====
36+
37+
. Create the `Provisioning` CR by running the following command:
38+
+
39+
[source,terminal]
40+
----
41+
$ oc create -f provisioning.yaml
42+
----
43+
+
44+
.Example output
45+
[source,terminal]
46+
----
47+
provisioning.metal3.io/provisioning-configuration created
48+
----
49+
50+
.Verification
51+
52+
* Verify that the provisioning service is running by running the following command:
53+
+
54+
[source,terminal]
55+
----
56+
$ oc get pods -n openshift-machine-api
57+
----
58+
+
59+
.Example output
60+
[source,terminal]
61+
----
62+
NAME READY STATUS RESTARTS AGE
63+
cluster-autoscaler-operator-678c476f4c-jjdn5 2/2 Running 0 5d21h
64+
cluster-baremetal-operator-6866f7b976-gmvgh 2/2 Running 0 5d21h
65+
control-plane-machine-set-operator-7d8566696c-bh4jz 1/1 Running 0 5d21h
66+
ironic-proxy-64bdw 1/1 Running 0 5d21h
67+
ironic-proxy-rbggf 1/1 Running 0 5d21h
68+
ironic-proxy-vj54c 1/1 Running 0 5d21h
69+
machine-api-controllers-544d6849d5-tgj9l 7/7 Running 1 (5d21h ago) 5d21h
70+
machine-api-operator-5c4ff4b86d-6fjmq 2/2 Running 0 5d21h
71+
metal3-6d98f84cc8-zn2mx 5/5 Running 0 5d21h
72+
metal3-image-customization-59d745768d-bhrp7 1/1 Running 0 5d21h
73+
----
74+

‎modules/upi-limitations-for-scaling-a-upi-cluster.adoc

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * installing/installing_bare_metal/scaling-a-user-provisioned-cluster-with-the-bare-metal-operator.adoc
4+
:_content-type: CONCEPT
5+
6+
[id="upi-limitations-for-scaling-a-upi-cluster_{context}"]
7+
= Limitations for scaling a user-provisioned cluster
8+
9+
* You cannot use a provisioning network to scale user-provisioned infrastructure clusters by using the Bare Metal Operator (BMO).
10+
** Consequentially, you can only use bare-metal host drivers that support virtual media networking booting, for example `redfish-virtualmedia` and `idrac-virtualmedia`.
11+
* You cannot scale `MachineSet` objects in user-provisioned infrastructure clusters by using the BMO.

‎modules/upi-managing-existing-hosts-in-a-upi-cluster.adoc

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * installing/installing_bare_metal/scaling-a-user-provisioned-cluster-with-the-bare-metal-operator.adoc
4+
:_content-type: PROCEDURE
5+
[id="upi-managing-existing-hosts-in-a-upi-cluster_{context}"]
6+
= Optional: Managing existing hosts in a user-provisioned cluster by using the BMO
7+
8+
Optionally, you can use the Bare Metal Operator (BMO) to manage existing bare-metal controller hosts in a user-provisioned cluster by creating a `BareMetalHost` object for the existing host.
9+
It is not a requirement to manage existing user-provisioned hosts; however, you can enroll them as externally-provisioned hosts for inventory purposes.
10+
11+
[IMPORTANT]
12+
====
13+
To manage existing hosts by using the BMO, you must set the `spec.externallyProvisioned` specification in the `BareMetalHost` custom resource to `true` to prevent the BMO from re-provisioning the host.
14+
====
15+
16+
.Prerequisites
17+
18+
* You created a user-provisioned bare-metal cluster.
19+
* You have baseboard management controller (BMC) access to the hosts.
20+
* You deployed a provisioning service in the cluster by creating a `Provisioning` CR.
21+
22+
.Procedure
23+
24+
. Create the `Secret` CR and the `BareMetalHost` CR.
25+
26+
.. Save the following YAML in the `controller.yaml` file:
27+
+
28+
[source,yaml]
29+
----
30+
---
31+
apiVersion: v1
32+
kind: Secret
33+
metadata:
34+
name: controller1-bmc
35+
namespace: openshift-machine-api
36+
type: Opaque
37+
data:
38+
username: <base64_of_uid>
39+
password: <base64_of_pwd>
40+
---
41+
apiVersion: metal3.io/v1alpha1
42+
kind: BareMetalHost
43+
metadata:
44+
name: controller1
45+
namespace: openshift-machine-api
46+
spec:
47+
bmc:
48+
address: <protocol>://<bmc_url> <1>
49+
credentialsName: "controller1-bmc"
50+
bootMACAddress: <nic1_mac_address>
51+
customDeploy:
52+
method: install_coreos
53+
externallyProvisioned: true <2>
54+
online: true
55+
userData:
56+
name: controller-user-data-managed
57+
namespace: openshift-machine-api
58+
----
59+
<1> You can only use bare-metal host drivers that support virtual media networking booting, for example `redfish-virtualmedia` and `idrac-virtualmedia`.
60+
<2> You must set the value to true to prevent the BMO from re-provisioning the bare-metal controller host.
61+
62+
. Create the bare-metal host object by running the following command:
63+
+
64+
[source,terminal]
65+
----
66+
$ oc create -f controller.yaml
67+
----
68+
+
69+
.Example output
70+
[source,terminal]
71+
----
72+
secret/controller1-bmc created
73+
baremetalhost.metal3.io/controller1 created
74+
----
75+
76+
.Verification
77+
78+
* Verify that the BMO created the bare-metal host object by running the following command:
79+
+
80+
[source,terminal]
81+
----
82+
$ oc get bmh -A
83+
----
84+
+
85+
.Example output
86+
[source,terminal]
87+
----
88+
NAMESPACE NAME STATE CONSUMER ONLINE ERROR AGE
89+
openshift-machine-api controller1 externally provisioned true 13s
90+
----
91+

‎modules/upi-prerequisites-for-scaling-a-upi-cluster.adoc

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * installing/installing_bare_metal/scaling-a-user-provisioned-cluster-with-the-bare-metal-operator.adoc
4+
:_content-type: CONCEPT
5+
6+
[id="upi-prequisites-for-scaling-a-upi-cluster_{context}"]
7+
= Prerequisites for scaling a user-provisioned cluster
8+
9+
* You installed a user-provisioned infrastructure cluster on bare metal.
10+
* You have baseboard management controller (BMC) access to the hosts.

‎modules/upi-provisioning-new-hosts-in-a-upi-cluster.adoc

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * installing/installing_bare_metal/scaling-a-user-provisioned-cluster-with-the-bare-metal-operator.adoc
4+
:_content-type: PROCEDURE
5+
[id="upi-provisioning-new-hosts-in-a-upi-cluster_{context}"]
6+
= Provisioning new hosts in a user-provisioned cluster by using the BMO
7+
8+
You can use the Bare Metal Operator (BMO) to provision bare-metal hosts in a user-provisioned cluster by creating a `BareMetalHost` custom resource (CR).
9+
10+
[NOTE]
11+
====
12+
To provision bare-metal hosts to the cluster by using the BMO, you must set the `spec.externallyProvisioned` specification in the `BareMetalHost` custom resource to `false`.
13+
====
14+
15+
.Prerequisites
16+
17+
* You created a user-provisioned bare-metal cluster.
18+
* You have baseboard management controller (BMC) access to the hosts.
19+
* You deployed a provisioning service in the cluster by creating a `Provisioning` CR.
20+
21+
.Procedure
22+
23+
. Create the `Secret` CR and the `BareMetalHost` CR.
24+
25+
.. Save the following YAML in the `bmh.yaml` file:
26+
+
27+
[source,yaml]
28+
----
29+
---
30+
apiVersion: v1
31+
kind: Secret
32+
metadata:
33+
name: worker1-bmc
34+
namespace: openshift-machine-api
35+
type: Opaque
36+
data:
37+
username: <base64_of_uid>
38+
password: <base64_of_pwd>
39+
---
40+
apiVersion: metal3.io/v1alpha1
41+
kind: BareMetalHost
42+
metadata:
43+
name: worker1
44+
namespace: openshift-machine-api
45+
spec:
46+
bmc:
47+
address: <protocol>://<bmc_url> <1>
48+
credentialsName: "worker1-bmc"
49+
bootMACAddress: <nic1_mac_address>
50+
externallyProvisioned: false <2>
51+
customDeploy:
52+
method: install_coreos
53+
online: true
54+
userData:
55+
name: worker-user-data-managed
56+
namespace: openshift-machine-api
57+
----
58+
<1> You can only use bare-metal host drivers that support virtual media networking booting, for example `redfish-virtualmedia` and `idrac-virtualmedia`.
59+
<2> You must set the `spec.externallyProvisioned` specification in the `BareMetalHost` custom resource to `false`. The default value is `false`.
60+
61+
. Create the bare-metal host object by running the following command:
62+
+
63+
[source,terminal]
64+
----
65+
$ oc create -f bmh.yaml
66+
----
67+
+
68+
.Example output
69+
[source,terminal]
70+
----
71+
secret/worker1-bmc created
72+
baremetalhost.metal3.io/worker1 created
73+
----
74+
75+
. Approve all certificate signing requests (CSRs).
76+
77+
.. Verify that the provisioning state of the host is `provisioned` by running the following command:
78+
+
79+
[source,terminal]
80+
----
81+
$ oc get bmh -A
82+
----
83+
+
84+
.Example output
85+
[source,terminal]
86+
----
87+
NAMESPACE NAME STATE CONSUMER ONLINE ERROR AGE
88+
openshift-machine-api controller1 externally provisioned true 5m25s
89+
openshift-machine-api worker1 provisioned true 4m45s
90+
----
91+
92+
.. Get the list of pending CSRs by running the following command:
93+
+
94+
[source,terminal]
95+
----
96+
$ oc get csr
97+
----
98+
+
99+
.Example output
100+
[source,terminal]
101+
----
102+
NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION
103+
csr-gfm9f 33s kubernetes.io/kube-apiserver-client-kubelet system:serviceaccount:openshift-machine-config-o
104+
perator:node-bootstrapper <none> Pending
105+
----
106+
107+
.. Approve the CSR by running the following command:
108+
+
109+
[source,terminal]
110+
----
111+
$ oc adm certificate approve <csr_name>
112+
----
113+
+
114+
.Example output
115+
[source,terminal]
116+
----
117+
certificatesigningrequest.certificates.k8s.io/<csr_name> approved
118+
----
119+
120+
.Verification
121+
122+
* Verify that the node is ready by running the following command:
123+
+
124+
[source,terminal]
125+
----
126+
$ oc get nodes
127+
----
128+
+
129+
.Example output
130+
[source,terminal]
131+
----
132+
NAME STATUS ROLES AGE VERSION
133+
app1 Ready worker 47s v1.24.0+dc5a2fd
134+
controller1 Ready master,worker 2d22h v1.24.0+dc5a2fd
135+
----
136+

0 commit comments

Comments
 (0)