Merge pull request #42991 from ktothill/TELCODOCS-324

TELCODOCS-324: IPI Expanding Cluster Without Provisioner Node

Author: kalexand-rh

installing/installing_bare_metal_ipi/ipi-install-overview.adoc

@@ -6,15 +6,17 @@ include::_attributes/common-attributes.adoc[]
 
 Installer-provisioned installation provides support for installing {product-title} on bare metal nodes. This guide provides a methodology to achieving a successful installation.
 
-During installer-provisioned installation on bare metal, the installer on the bare metal node labeled as `provisioner` creates a bootstrap virtual machine (VM). The role of the bootstrap VM is to assist in the process of deploying an {product-title} cluster. The bootstrap VM connects to the `baremetal` network and to the `provisioning` network, if present, via the network bridges.
+During installer-provisioned installation on bare metal, the installation program on the bare metal node labeled as `provisioner` creates a bootstrap virtual machine (VM). The role of the bootstrap VM is to assist in the process of deploying an {product-title} cluster. The bootstrap VM connects to the `baremetal` network and to the `provisioning` network, if present, via the network bridges.
 
 image::210_OpenShift_Baremetal_IPI_Deployment_updates_0122_1.png[Deployment phase one]
 
-When the installation of {product-title} is complete and fully operational, the installer destroys the bootstrap VM automatically and moves the virtual IP addresses (VIPs) to the appropriate nodes accordingly. The API VIP moves to the control plane nodes and the Ingress VIP moves to the worker nodes.
+The provisioning node can be removed after the installation.
+
+When the installation of {product-title} is complete and fully operational, the installation program destroys the bootstrap VM automatically and moves the virtual IP addresses (VIPs) to the appropriate nodes accordingly. The API VIP moves to the control plane nodes and the Ingress VIP moves to the worker nodes.
 
 image::210_OpenShift_Baremetal_IPI_Deployment_updates_0122_2.png[Deployment phase two]
 
 [IMPORTANT]
 ====
 The `provisioning` network is optional, but it is required for PXE booting. If you deploy without a `provisioning` network, you must use a virtual media BMC addressing option such as `redfish-virtualmedia` or `idrac-virtualmedia`.
-====
+====

installing/installing_bare_metal_ipi/ipi-install-post-installation-configuration.adoc

@@ -12,4 +12,4 @@ include::modules/ipi-install-configuring-ntp-for-disconnected-clusters.adoc[leve
 
 include::modules/nw-enabling-a-provisioning-network-after-installation.adoc[leveloffset=+1]
 
-include::modules/nw-osp-configuring-external-load-balancer.adoc[leveloffset=+1]
+include::modules/nw-osp-configuring-external-load-balancer.adoc[leveloffset=+1]

installing/installing_bare_metal_ipi/ipi-install-prerequisites.adoc

@@ -8,8 +8,8 @@ toc::[]
 
 Installer-provisioned installation of {product-title} requires:
 
-ifdef::openshift-origin[. One provisioner node with {op-system-first} installed.]
-ifndef::openshift-origin[. One provisioner node with {op-system-base-full} 8.x installed.]
+ifdef::openshift-origin[. One provisioner node with {op-system-first} installed. The provisioning node can be removed after installation.]
+ifndef::openshift-origin[. One provisioner node with {op-system-base-full} 8.x installed. The provisioning node can be removed after installation.]
 . Three control plane nodes.
 . Baseboard Management Controller (BMC) access to each node.
 ifeval::[{product-version} > 4.5]

modules/ipi-install-preparing-the-bare-metal-node.adoc

@@ -22,33 +22,33 @@ Preparing the bare metal node requires executing the following procedure from th
 +
 [source,terminal]
 ----
-[kni@provisioner ~]$ curl -s https://mirror.openshift.com/pub/openshift-v4/clients/ocp/$VERSION/openshift-client-linux-$VERSION.tar.gz | tar zxvf - oc
+$ curl -s https://mirror.openshift.com/pub/openshift-v4/clients/ocp/$VERSION/openshift-client-linux-$VERSION.tar.gz | tar zxvf - oc
 ----
 +
 [source,terminal]
 ----
-[kni@provisioner ~]$ sudo cp oc /usr/local/bin
+$ sudo cp oc /usr/local/bin
 ----
 
-. Power off the bare metal node through the baseboard management controller and ensure it is off.
+. Power off the bare metal node by using the baseboard management controller, and ensure it is off.
 
-. Retrieve the user name and password of the bare metal node's baseboard management controller. Then, create `base64` strings from the user name and password. In the following example, the user name is `root` and the password is `password`.
+. Retrieve the user name and password of the bare metal node's baseboard management controller. Then, create `base64` strings from the user name and password:
 +
-[source,terminal]
+[source,terminal,subs="+quotes"]
 ----
-[kni@provisioner ~]$ echo -ne "root" | base64
+$ echo -ne "root" | base64
 ----
 +
 [source,terminal]
 ----
-[kni@provisioner ~]$ echo -ne "password" | base64
+$ echo -ne "password" | base64
 ----
 
 . Create a configuration file for the bare metal node.
 +
 [source,terminal]
 ----
-[kni@provisioner ~]$ vim bmh.yaml
+$ vim bmh.yaml
 ----
 +
 [source,yaml]
@@ -109,7 +109,7 @@ If the MAC address of an existing bare metal node matches the MAC address of a b
 +
 [source,terminal]
 ----
-[kni@provisioner ~]$ oc -n openshift-machine-api create -f bmh.yaml
+$ oc -n openshift-machine-api create -f bmh.yaml
 ----
 +
 .Example output
@@ -125,7 +125,7 @@ Where `<num>` will be the worker number.
 +
 [source,terminal]
 ----
-[kni@provisioner ~]$ oc -n openshift-machine-api get bmh openshift-worker-<num>
+$ oc -n openshift-machine-api get bmh openshift-worker-<num>
 ----
 +
 Where `<num>` is the worker node number.

modules/ipi-install-provisioning-the-bare-metal-node.adoc

@@ -117,4 +117,4 @@ $ ssh openshift-worker-<num>
 [source,terminal]
 ----
 [kni@openshift-worker-<num>]$ journalctl -fu kubelet
-----
+----

modules/ipi-install-troubleshooting-bootstrap-vm-cannot-boot.adoc

@@ -8,7 +8,7 @@
 During the deployment, it is possible for the bootstrap VM to fail to boot the cluster nodes, which prevents the VM from provisioning the nodes with the {op-system} image. This scenario can arise due to:
 
 * A problem with the `install-config.yaml` file.
-* Issues with out-of-band network access via the baremetal network.
+* Issues with out-of-band network access when using the baremetal network.
 
 To verify the issue, there are three containers related to `ironic`:
 
@@ -20,14 +20,14 @@ To verify the issue, there are three containers related to `ironic`:
 
 . Log in to the bootstrap VM:
 +
-[source,bash]
+[source,terminal]
 ----
 $ ssh [email protected]
 ----
 
 . To check the container logs, execute the following:
 +
-[source,bash]
+[source,terminal]
 ----
 [core@localhost ~]$ sudo podman logs -f <container-name>
 ----
@@ -41,7 +41,7 @@ The cluster nodes might be in the `ON` state when deployment started.
 Power off the {product-title} cluster nodes before you begin the
 installation over IPMI:
 
-[source,bash]
+[source,terminal]
 ----
 $ ipmitool -I lanplus -U root -P <password> -H <out-of-band-ip> power off
-----
+----

modules/ipi-install-troubleshooting-bootstrap-vm-inspecting-logs.adoc

@@ -66,4 +66,4 @@ If the bootstrap VM cannot access the URL to the images, use the `curl` command
 [source,terminal]
 ----
 [core@localhost ~]$ sudo podman logs <ironic-api>
-----
+----

modules/ipi-install-troubleshooting-bootstrap-vm.adoc

@@ -6,18 +6,18 @@
 
 = Bootstrap VM issues
 
-The {product-title} installer spawns a bootstrap node virtual machine, which handles provisioning the {product-title} cluster nodes.
+The {product-title} installation program spawns a bootstrap node virtual machine, which handles provisioning the {product-title} cluster nodes.
 
 .Procedure
 
-. About 10 to 15 minutes after triggering the installer, check to ensure the bootstrap VM is operational using the `virsh` command:
+. About 10 to 15 minutes after triggering the installation program, check to ensure the bootstrap VM is operational using the `virsh` command:
 +
-[source,bash]
+[source,terminal]
 ----
 $ sudo virsh list
 ----
 +
-[source,bash]
+[source,terminal]
 ----
  Id    Name                           State
  --------------------------------------------
@@ -33,12 +33,12 @@ If the bootstrap VM is not running after 10-15 minutes, troubleshoot why it is n
 
 . Verify `libvirtd` is running on the system:
 +
-[source,bash]
+[source,terminal]
 ----
 $ systemctl status libvirtd
 ----
 +
-[source,bash]
+[source,terminal]
 ----
 ● libvirtd.service - Virtualization daemon
    Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
@@ -56,16 +56,15 @@ If the bootstrap VM is operational, log in to it.
 
 . Use the `virsh console` command to find the IP address of the bootstrap VM:
 +
-[source,bash]
+[source,terminal]
 ----
 $ sudo virsh console example.com
 ----
 +
-[source,bash]
+[source,terminal]
 ----
 Connected to domain example.com
 Escape character is ^]
-
 Red Hat Enterprise Linux CoreOS 43.81.202001142154.0 (Ootpa) 4.3
 SSH host key: SHA256:BRWJktXZgQQRY5zjuAV0IKZ4WM7i4TiUyMVanqu9Pqg (ED25519)
 SSH host key: SHA256:7+iKGA7VtG5szmk2jB5gl/5EZ+SNcJ3a2g23o0lnIio (ECDSA)
@@ -88,7 +87,7 @@ When deploying an {product-title} cluster without the `provisioning` network, yo
 In the console output of the previous step, you can use the IPv6 IP address provided by `ens3` or the IPv4 IP provided by `ens4`.
 ====
 +
-[source,bash]
+[source,terminal]
 ----
 $ ssh [email protected]
 ----
@@ -97,11 +96,11 @@ If you are not successful logging in to the bootstrap VM, you have likely encoun
 
 * You cannot reach the `172.22.0.0/24` network. Verify network connectivity on the provisioner host specifically around the `provisioning` network bridge. This will not be the issue if you are not using the `provisioning` network.
 
-* You cannot reach the bootstrap VM via the public network. When attempting
+* You cannot reach the bootstrap VM through the public network. When attempting
 to SSH via `baremetal` network, verify connectivity on the
 `provisioner` host specifically around the `baremetal` network bridge.
 
 * You encountered `Permission denied (publickey,password,keyboard-interactive)`. When
 attempting to access the bootstrap VM, a `Permission denied` error
 might occur. Verify that the SSH key for the user attempting to log
-into the VM is set within the `install-config.yaml` file.
+into the VM is set within the `install-config.yaml` file.

modules/ipi-install-troubleshooting-cleaning-up-previous-installations.adoc

@@ -12,15 +12,15 @@ In the event of a previous failed deployment, remove the artifacts from the fail
 
 . Power off all bare metal nodes prior to installing the {product-title} cluster:
 +
-[source,bash]
+[source,terminal]
 ----
-$ ipmitool -I lanplus -U <user> -P <password> -H <management-server-ip> power off
+$ ipmitool -I lanplus -U _<user>_ -P _<password>_ -H _<management-server-ip>_ power off
 ----
 
 ifeval::[{product-version} >= 4.6]
 . Remove all old bootstrap resources if any are left over from a previous deployment attempt:
 +
-[source,bash]
+[source,terminal]
 ----
 for i in $(sudo virsh list | tail -n +3 | grep bootstrap | awk {'print $2'});
 do
@@ -37,7 +37,7 @@ endif::[]
 ifeval::[{product-version} < 4.6]
 . Remove all old bootstrap resources if any are left over from a previous deployment attempt:
 +
-[source,bash]
+[source,terminal]
 ----
 for i in $(sudo virsh list | tail -n +3 | grep bootstrap | awk {'print $2'});
 do
@@ -51,7 +51,7 @@ endif::[]
 
 . Remove the following from the `clusterconfigs` directory to prevent Terraform from failing:
 +
-[source,bash]
+[source,terminal]
 ----
 $ rm -rf ~/clusterconfigs/auth ~/clusterconfigs/terraform* ~/clusterconfigs/tls ~/clusterconfigs/metadata.json
-----
+----

modules/ipi-install-troubleshooting-reviewing-the-installation.adoc

@@ -12,12 +12,12 @@ After installation, ensure the installer deployed the nodes and pods successfull
 
 . When the {product-title} cluster nodes are installed appropriately, the following `Ready` state is seen within the `STATUS` column:
 +
-[source,bash]
+[source,terminal]
 ----
 $ oc get nodes
 ----
 +
-[source,bash]
+[source,terminal]
 ----
 NAME                   STATUS   ROLES           AGE  VERSION
 master-0.example.com   Ready    master,worker   4h   v1.23.0
@@ -28,7 +28,7 @@ master-2.example.com   Ready    master,worker   4h   v1.23.0
 . Confirm the installer deployed all pods successfully. The following command
 removes any pods that are still running or have completed as part of the output.
 +
-[source,bash]
+[source,terminal]
 ----
 $ oc get pods --all-namespaces | grep -iv running | grep -iv complete
-----
+----