Merge pull request #34121 from johnwilkins/network-requirements-feedback

IPI network requirements feedback

Author: mburke5678

installing/installing_bare_metal_ipi/ipi-install-prerequisites.adoc

@@ -13,15 +13,15 @@ ifndef::openshift-origin[. One provisioner node with {op-system-base-full} 8.x i
 . Baseboard Management Controller (BMC) access to each node.
 ifeval::[{product-version} > 4.5]
 . At least one network:
-.. One *required* routable network
-.. One *optional* network for provisioning nodes; and,
-.. One *optional* management network.
+.. One required routable network
+.. One optional network for provisioning nodes; and,
+.. One optional management network.
 endif::[]
 ifeval::[{product-version} < 4.6]
 . At least two networks:
-.. One *required* routable network
-.. One *required* network for provisioning nodes; and,
-.. One *optional* management network.
+.. One required routable network
+.. One required network for provisioning nodes; and,
+.. One optional management network.
 endif::[]
 
 Before starting an installer-provisioned installation of {product-title}, ensure the hardware environment meets the following requirements.

modules/ipi-install-network-requirements.adoc

@@ -5,33 +5,7 @@
 [id='network-requirements_{context}']
 = Network requirements
 
-Installer-provisioned installation of {product-title} involves several network requirements by default. First, installer-provisioned installation involves a non-routable `provisioning` network for provisioning the operating system on each bare metal node and a routable `baremetal` network. Since installer-provisioned installation deploys `ironic-dnsmasq`, the networks should have no other DHCP servers running on the same broadcast domain. Network administrators must reserve IP addresses for each node in the {product-title} cluster.
-
-ifeval::[{product-version} > 4.7]
-{product-title} 4.8 and later releases include functionality that uses cluster membership information to generate A/AAAA records. This resolves the node names to their IP addresses. Once the nodes are registered with the API, the cluster can disperse node information without using CoreDNS-mDNS. This eliminates the network traffic associated with multicast DNS.
-endif::[]
-
-.Network Time Protocol (NTP)
-
-ifeval::[{product-version} <= 4.7]
-Each {product-title} node in the cluster must have access to an NTP server. {product-title} nodes use NTP to synchronize their clocks. For example, cluster nodes use SSL certificates that require validation, which might fail if the date and time between the nodes are not in sync.
-
-[IMPORTANT]
-====
-Define a consistent clock date and time format in each cluster node's BIOS settings, or installation might fail.
-====
-endif::[]
-
-ifeval::[{product-version} > 4.7]
-Each {product-title} node in the cluster must have access to an NTP server. {product-title} nodes use NTP to synchronize their clocks. For example, cluster nodes use SSL certificates that require validation, which might fail if the date and time between the nodes are not in sync.
-
-[IMPORTANT]
-====
-Define a consistent clock date and time format in each cluster node's BIOS settings, or installation might fail.
-====
-
-In {product-title} 4.8 and later releases, you may reconfigure the control plane nodes to act as NTP servers on disconnected clusters, and reconfigure worker nodes to retrieve time from the control plane nodes.
-endif::[]
+Installer-provisioned installation of {product-title} involves several network requirements. First, installer-provisioned installation involves an optional non-routable `provisioning` network for provisioning the operating system on each bare metal node. Second, installer-provisioned installation involves a routable `baremetal` network.
 
 .Configuring NICs
 
@@ -73,6 +47,10 @@ For example:
 test-cluster.example.com
 ----
 
+ifeval::[{product-version}>4.7]
+{product-title} 4.8 and later releases include functionality that uses cluster membership information to generate A/AAAA records. This resolves the node names to their IP addresses. Once the nodes are registered with the API, the cluster can disperse node information without using CoreDNS-mDNS. This eliminates the network traffic associated with multicast DNS.
+endif::[]
+
 ifdef::upstream[]
 For assistance in configuring the DNS server, check xref:ipi-install-upstream-appendix[Appendix] section for:
 
@@ -81,6 +59,11 @@ For assistance in configuring the DNS server, check xref:ipi-install-upstream-ap
 
 endif::[]
 
+.Dynamic Host Configuration Protocol (DHCP) requirements
+
+By default, installer-provisioned installation deploys `ironic-dnsmasq` with DHCP enabled for the `provisioning` network. No other DHCP servers should be running on the `provisioning` network when the `provisioningNetwork` configuration setting is set to `managed`, which is the default value. If you have a DHCP server running on the `provisioning` network, you must set the `provisioningNetwork` configuration setting to `unmanaged` in the `install-config.yaml` file.
+
+Network administrators must reserve IP addresses for each node in the {product-title} cluster for the `baremetal` network on an external DHCP server.
 
 .Reserving IP addresses for nodes with the DHCP server
 
@@ -107,7 +90,7 @@ ifeval::[{product-version} > 4.6]
 [IMPORTANT]
 .Reserving IP addresses so they become static IP addresses
 ====
-Some administrators prefer to use static IP addresses so that each node's IP address remains constant in the absence of a DHCP server. To use static IP addresses in the {product-title} cluster, *reserve the IP addresses with an infinite lease*. During deployment, the installer will reconfigure the NICs from DHCP assigned addresses to static IP addresses. NICs with DHCP leases that are not infinite will remain configured to use DHCP.
+Some administrators prefer to use static IP addresses so that each node's IP address remains constant in the absence of a DHCP server. To use static IP addresses in the {product-title} cluster, reserve the IP addresses with an infinite lease. During deployment, the installer will reconfigure the NICs from DHCP assigned addresses to static IP addresses. NICs with DHCP leases that are not infinite will remain configured to use DHCP.
 ====
 endif::[]
 
@@ -145,6 +128,17 @@ For assistance in configuring the DHCP server, check xref:ipi-install-upstream-a
 - xref:creating-dhcp-reservations-using-dnsmasq-option2_{context}[Creating DHCP reservations with dnsmasq (Option 2)]
 endif::[]
 
+.Network Time Protocol (NTP)
+
+Each {product-title} node in the cluster must have access to an NTP server. {product-title} nodes use NTP to synchronize their clocks. For example, cluster nodes use SSL certificates that require validation, which might fail if the date and time between the nodes are not in sync.
+
+[IMPORTANT]
+====
+Define a consistent clock date and time format in each cluster node's BIOS settings, or installation might fail.
+====
+
+You may reconfigure the control plane nodes to act as NTP servers on disconnected clusters, and reconfigure worker nodes to retrieve time from the control plane nodes.
+
 ifeval::[{product-version} == 4.6]
 .Additional requirements with no provisioning network
 

modules/ipi-install-node-requirements.adoc

@@ -1,52 +1,51 @@
 // Module included in the following assemblies:
 //
 // * installing/installing_bare_metal_ipi/ipi-install-prerequisites.adoc
-
-[id='node-requirements_{context}']
-
+:product-version: 4.8
+[id="node-requirements_{context}"'"]
 = Node requirements
 
 Installer-provisioned installation involves a number of hardware node requirements:
 
-- *CPU architecture:* All nodes must use `x86_64` CPU architecture.
+* *CPU architecture:* All nodes must use `x86_64` CPU architecture.
 
-- *Similar nodes:* Red Hat recommends nodes have an identical configuration per role. That is, Red Hat recommends nodes be the same brand and model with the same CPU, memory, and storage configuration.
+* *Similar nodes:* Red Hat recommends nodes have an identical configuration per role. That is, Red Hat recommends nodes be the same brand and model with the same CPU, memory, and storage configuration.
 
 ifeval::[{product-version} < 4.5]
-- *Intelligent Platform Management Interface (IPMI):* Installer-provisioned installation requires IPMI enabled on each node.
+* *Intelligent Platform Management Interface (IPMI):* Installer-provisioned installation requires IPMI enabled on each node.
 endif::[]
 
 ifeval::[{product-version} > 4.4]
-- *Baseboard Management Controller:* The `provisioner` node must be able to access the baseboard management controller (BMC) of each {product-title} cluster node. You may use IPMI, Redfish, or a proprietary protocol.
+* *Baseboard Management Controller:* The `provisioner` node must be able to access the baseboard management controller (BMC) of each {product-title} cluster node. You may use IPMI, Redfish, or a proprietary protocol.
 endif::[]
 
 ifndef::openshift-origin[]
-- *Latest generation:* Nodes must be of the most recent generation. Installer-provisioned installation relies on BMC protocols, which must be compatible across nodes. Additionally, {op-system-base} 8 ships with the most recent drivers for RAID controllers. Ensure that the nodes are recent enough to support {op-system-base} 8 for the `provisioner` node and {op-system} 8 for the control plane and worker nodes.
+* *Latest generation:* Nodes must be of the most recent generation. Installer-provisioned installation relies on BMC protocols, which must be compatible across nodes. Additionally, {op-system-base} 8 ships with the most recent drivers for RAID controllers. Ensure that the nodes are recent enough to support {op-system-base} 8 for the `provisioner` node and {op-system} 8 for the control plane and worker nodes.
 endif::[]
 ifdef::openshift-origin[]
-- *Latest generation:* Nodes must be of the most recent generation. Installer-provisioned installation relies on BMC protocols, which must be compatible across nodes. Additionally, {op-system-first} ships with the most recent drivers for RAID controllers. Ensure that the nodes are recent enough to support {op-system} for the `provisioner` node and {op-system} for the control plane and worker nodes.
+* *Latest generation:* Nodes must be of the most recent generation. Installer-provisioned installation relies on BMC protocols, which must be compatible across nodes. Additionally, {op-system-first} ships with the most recent drivers for RAID controllers. Ensure that the nodes are recent enough to support {op-system} for the `provisioner` node and {op-system} for the control plane and worker nodes.
 endif::[]
 
-- *Registry node:* (Optional) If setting up a disconnected mirrored registry, it is recommended the registry reside in its own node.
+* *Registry node:* (Optional) If setting up a disconnected mirrored registry, it is recommended the registry reside in its own node.
 
-- *Provisioner node:* Installer-provisioned installation requires one `provisioner` node.
+* *Provisioner node:* Installer-provisioned installation requires one `provisioner` node.
 
-- *Control plane:* Installer-provisioned installation requires three control plane nodes for high availability.
+* *Control plane:* Installer-provisioned installation requires three control plane nodes for high availability.
 
-- *Worker nodes:* While not required, a typical production cluster has one or more worker nodes. Smaller clusters are more resource efficient for administrators and developers during development, production, and testing.
+* *Worker nodes:* While not required, a typical production cluster has one or more worker nodes. Smaller clusters are more resource efficient for administrators and developers during development, production, and testing.
 
-- *Network interfaces:* Each node must have at least one 10GB network interface for the routable `baremetal` network. Each node must have one 10GB network interface for a `provisioning` network *when using the `provisioning` network* for deployment. Using the `provisioning` network is the default configuration. Network interface names must follow the same naming convention across all nodes. For example, the first NIC name on a node, such as `eth0` or `eno1`, must be the same name on all of the other nodes. The same principle applies to the remaining NICs on each node.
+* *Network interfaces:* Each node must have at least one 10GB network interface for the routable `baremetal` network. Each node must have one 10GB network interface for a `provisioning` network when using the `provisioning` network for deployment. Using the `provisioning` network is the default configuration. Network interface names must follow the same naming convention across all nodes. For example, the first NIC name on a node, such as `eth0` or `eno1`, must be the same name on all of the other nodes. The same principle applies to the remaining NICs on each node.
 
 ifeval::[{product-version} > 4.3]
-- *Unified Extensible Firmware Interface (UEFI):* Installer-provisioned installation requires UEFI boot on all {product-title} nodes when using IPv6 addressing on the `provisioning` network. In addition, UEFI Device PXE Settings must be set to use the IPv6 protocol on the `provisioning` network NIC, but omitting the `provisioning` network removes this requirement.
+* *Unified Extensible Firmware Interface (UEFI):* Installer-provisioned installation requires UEFI boot on all {product-title} nodes when using IPv6 addressing on the `provisioning` network. In addition, UEFI Device PXE Settings must be set to use the IPv6 protocol on the `provisioning` network NIC, but omitting the `provisioning` network removes this requirement.
 endif::[]
 
 ifeval::[{product-version} == 4.7]
-- *Secure Boot:* Many production scenarios require nodes with Secure Boot enabled to verify the node only boots with trusted software, such as UEFI firmware drivers, EFI applications, and the operating system. To deploy an {product-title} cluster with Secure Boot, you must enable UEFI boot mode and Secure Boot on each control plane node and each worker node. Red Hat supports Secure Boot only when installer-provisioned installations use Red Fish Virtual Media. Red Hat does not support Secure Boot with self-generated keys.
+* *Secure Boot:* Many production scenarios require nodes with Secure Boot enabled to verify the node only boots with trusted software, such as UEFI firmware drivers, EFI applications, and the operating system. To deploy an {product-title} cluster with Secure Boot, you must enable UEFI boot mode and Secure Boot on each control plane node and each worker node. Red Hat supports Secure Boot only when installer-provisioned installations use Red Fish Virtual Media. Red Hat does not support Secure Boot with self-generated keys.
 endif::[]
 
 ifeval::[{product-version} > 4.7]
-- *Secure Boot:* Many production scenarios require nodes with Secure Boot enabled to verify the node only boots with trusted software, such as UEFI firmware drivers, EFI applications, and the operating system. You may deploy with Secure Boot manually or managed.
+* *Secure Boot:* Many production scenarios require nodes with Secure Boot enabled to verify the node only boots with trusted software, such as UEFI firmware drivers, EFI applications, and the operating system. You may deploy with Secure Boot manually or managed.
 +
 . *Manually:* To deploy an {product-title} cluster with Secure Boot manually, you must enable UEFI boot mode and Secure Boot on each control plane node and each worker node. Red Hat supports Secure Boot with manually enabled UEFI and Secure Boot only when installer-provisioned installations use Redfish virtual media. See "Configuring nodes for Secure Boot manually" in the "Configuring nodes" section for additional details.
 +

modules/ipi-install-preparing-the-provisioner-node-for-openshift-install.adoc

@@ -11,7 +11,7 @@ Perform the following steps to prepare the environment.
 
 . Log in to the provisioner node via `ssh`.
 
-. Create a non-root user (`kni`) and provide that user with `sudo` privileges.
+. Create a non-root user (`kni`) and provide that user with `sudo` privileges:
 +
 [source,terminal]
 ----
@@ -21,22 +21,22 @@ Perform the following steps to prepare the environment.
 # chmod 0440 /etc/sudoers.d/kni
 ----
 
-. Create an `ssh` key for the new user.
+. Create an `ssh` key for the new user:
 +
 [source,terminal]
 ----
 # su - kni -c "ssh-keygen -t ed25519 -f /home/kni/.ssh/id_rsa -N ''"
 ----
 
-. Log in as the new user on the provisioner node.
+. Log in as the new user on the provisioner node:
 +
 [source,terminal]
 ----
 # su - kni
 $
 ----
 
-. Use Red Hat Subscription Manager to register the provisioner node.
+. Use Red Hat Subscription Manager to register the provisioner node:
 +
 [source,terminal]
 ----
@@ -49,21 +49,21 @@ $ sudo subscription-manager repos --enable=rhel-8-for-x86_64-appstream-rpms --en
 For more information about Red Hat Subscription Manager, see link:https://access.redhat.com/documentation/en-us/red_hat_subscription_management/1/html-single/rhsm/index[Using and Configuring Red Hat Subscription Manager].
 ====
 
-. Install the following packages.
+. Install the following packages:
 +
 [source,terminal]
 ----
 $ sudo dnf install -y libvirt qemu-kvm mkisofs python3-devel jq ipmitool
 ----
 
-. Modify the user to add the `libvirt` group to the newly created user.
+. Modify the user to add the `libvirt` group to the newly created user:
 +
 [source,terminal]
 ----
 $ sudo usermod --append --groups libvirt <user>
 ----
 
-. Restart `firewalld` and enable the `http` service.
+. Restart `firewalld` and enable the `http` service:
 +
 [source,terminal]
 ----
@@ -72,14 +72,14 @@ $ sudo firewall-cmd --zone=public --add-service=http --permanent
 $ sudo firewall-cmd --reload
 ----
 
-. Start and enable the `libvirtd` service.
+. Start and enable the `libvirtd` service:
 +
 [source,terminal]
 ----
 $ sudo systemctl enable libvirtd --now
 ----
 
-. Create the `default` storage pool and start it.
+. Create the `default` storage pool and start it:
 +
 [source,terminal]
 ----
@@ -92,26 +92,48 @@ $ sudo virsh pool-autostart default
 +
 [NOTE]
 ====
-This step can also be run from the web console.
+You can also configure networking from the web console.
 ====
 +
+Export the `baremetal` network NIC name:
++
 [source,terminal]
 ----
 $ export PUB_CONN=<baremetal_nic_name>
-$ export PROV_CONN=<prov_nic_name>
+----
++
+Configure the `baremetal` network:
++
+[source,terminal]
+----
 $ sudo nohup bash -c "
-    nmcli con down \"$PROV_CONN\"
     nmcli con down \"$PUB_CONN\"
-    nmcli con delete \"$PROV_CONN\"
     nmcli con delete \"$PUB_CONN\"
     # RHEL 8.1 appends the word \"System\" in front of the connection, delete in case it exists
     nmcli con down \"System $PUB_CONN\"
     nmcli con delete \"System $PUB_CONN\"
-    nmcli connection add ifname provisioning type bridge con-name provisioning
-    nmcli con add type bridge-slave ifname \"$PROV_CONN\" master provisioning
     nmcli connection add ifname baremetal type bridge con-name baremetal
     nmcli con add type bridge-slave ifname \"$PUB_CONN\" master baremetal
     pkill dhclient;dhclient baremetal
+"
+----
++
+If you are deploying with a `provisioning` network, export the `provisioning` network NIC name:
++
+[source,terminal]
+----
+$ export PROV_CONN=<prov_nic_name>
+----
++
+If you are deploying with a `provisioning` network, configure the `provisioning` network:
++
+[source,terminal]
+----
+$ sudo nohup bash -c "
+    nmcli con down \"$PROV_CONN\"
+    nmcli con delete \"$PROV_CONN\"
+    nmcli connection add ifname provisioning type bridge con-name provisioning
+    nmcli con add type bridge-slave ifname \"$PROV_CONN\" master provisioning
     nmcli connection modify provisioning ipv6.addresses fd00:1101::1/64 ipv6.method manual
     nmcli con down provisioning
     nmcli con up provisioning
@@ -120,7 +142,7 @@ $ sudo nohup bash -c "
 +
 [NOTE]
 ====
-The `ssh` connection might disconnect after executing this step.
+The `ssh` connection might disconnect after executing these steps.
 
 The IPv6 address can be any address as long as it is not routable via the `baremetal` network.
 

modules/ipi-install-required-data-for-installation.adoc

@@ -11,6 +11,7 @@ Prior to the installation of the {product-title} cluster, gather the following i
 ** Examples
 *** Dell (iDRAC) IP
 *** HP (iLO) IP
+*** Fujitsu (iRMC) IP
 
 .When using the `provisioning` network