Merge pull request #41038 from johnwilkins/TELCODOCS-212

TELCODOCS-212: D/S Docs: SDN-2217/MPNETWORK-2/KNIDEPLOY-4589, IPI provisioning with static IP addressing for nodes and Virtual IPs

Author: lpettyjo

installing/installing_bare_metal_ipi/ipi-install-installation-workflow.adoc

@@ -27,6 +27,8 @@ include::modules/ipi-install-modifying-install-config-for-no-provisioning-networ
 
 include::modules/ipi-install-modifying-install-config-for-dual-stack-network.adoc[leveloffset=+2]
 
+include::modules/ipi-install-configuring-host-network-interfaces-in-the-install-config.yaml-file.adoc[leveloffset=+2]
+
 include::modules/ipi-install-configuring-managed-secure-boot-in-the-install-config-file.adoc[leveloffset=+2]
 
 include::modules/ipi-install-additional-install-config-parameters.adoc[leveloffset=+2]
@@ -51,7 +53,6 @@ ifeval::[{product-version} > 4.8]
 include::modules/ipi-install-configuring-bios-for-worker-node.adoc[leveloffset=+2]
 endif::[]
 
-
 include::modules/ipi-install-creating-a-disconnected-registry.adoc[leveloffset=+1]
 
 include::modules/ipi-install-deploying-routers-on-worker-nodes.adoc[leveloffset=+1]

modules/ipi-install-additional-install-config-parameters.adoc

@@ -206,4 +206,9 @@ The `hosts` parameter is a list of separate bare metal assets used to build the
 |
 | The MAC address of the NIC that the host uses for the `provisioning` network. Ironic retrieves the IP address using the `bootMACAddress` configuration setting. Then, it binds to the host.
 
+
+| `networkConfig`
+|
+| Set this optional parameter to configure the network interface of a host. See "(Optional) Configuring host network interfaces in the `install-config.yaml` file" for additional details.
+
 |===

modules/ipi-install-configuring-host-network-interfaces-in-the-install-config.yaml-file.adoc

@@ -0,0 +1,57 @@
+// This is included in the following assemblies:
+//
+// installing_bare_metal_ipi/ipi-install-installation-workflow.adoc
+
+[id="configuring-host-network-interfaces-in-the-install-config.yaml-file_{context}"]
+= (Optional) Configuring host network interfaces in the `install-config.yaml` file
+
+During installation, you can set the `networkConfig` configuration setting in the `install-config.yaml` file to configure host network interfaces using NMState. To use the `networkConfig` configuration setting, you must provide an NMState YAML configuration. See link:https://nmstate.io/examples.html#interfaces-ethernet[NMState] for additional examples of the NMState syntax.
+
+.Example
+[source,yaml]
+----
+  hosts:
+        - name: openshift-master-0
+          role: master
+          bmc:
+            address: redfish+http://<out-of-band-ip>/redfish/v1/Systems/
+            username: <user>
+            password: <password>
+            disableCertificateVerification: null
+          bootMACAddress: <NIC1_mac_address>
+          bootMode: UEFI
+          rootDeviceHints:
+            deviceName: "/dev/sda"
+          networkConfig: <1>
+            interfaces:
+            - name: <NIC1_name>
+              type: ethernet
+              state: up
+              ipv4:
+                address:
+                - ip: "<IP_address>"
+                  prefix-length: 24
+                enabled: true
+            dns-resolver:
+              config:
+                server:
+                - <DNS_IP_address>
+            routes:
+              config:
+              - destination: 0.0.0.0/0
+                next-hop-address: <IP_address>
+                next-hop-interface: <NIC1_name>
+----
+<1> Add NMState YAML syntax to configure host interfaces.
+
+[TIP]
+====
+Consider saving the `networkConfig` YAML syntax to a file and testing it using the NMState command line interface before including it in the `install-config.yaml` file, because the installer will not check the NMState YAML syntax. Execute `nmstatectl gc <yaml-config>` to test the syntax. Errors in the YAML syntax might result in a failure to apply the network configuration. Additionally, maintaining the validated YAML syntax is useful when applying changes using Kubernetes NMState after deployment or when expanding the cluster.
+====
+
+The most common use case for this functionality is to specify a static IP address on the `baremetal` network, but you can also configure other networks such as a storage network. This functionality will also support other NMState features such as VLAN, VXLAN, bridges, bonds, routes, MTU, and DNS resolver settings.
+
+[IMPORTANT]
+====
+Once deployed, you cannot modify the `networkConfig` configuration setting of `install-config.yaml` file to make changes to the host network interface. Use the Kubernetes NMState Operator to make changes to the host network interface after deployment.
+====

modules/ipi-install-network-requirements.adoc

@@ -104,7 +104,7 @@ For the `baremetal` network, a network administrator must reserve a number of IP
 [IMPORTANT]
 .Reserving IP addresses so they become static IP addresses
 ====
-Some administrators prefer to use static IP addresses so that each node's IP address remains constant in the absence of a DHCP server. To use static IP addresses in the {product-title} cluster, reserve the IP addresses with an infinite lease. During deployment, the installer will reconfigure the NICs from DHCP assigned addresses to static IP addresses. NICs with DHCP leases that are not infinite will remain configured to use DHCP.
+Some administrators prefer to use static IP addresses so that each node's IP address remains constant in the absence of a DHCP server. To configure static IP addresses with NMState, see "(Optional) Configuring host network interfaces in the `install-config.yaml` file" in the "Setting up the environment for an OpenShift installation" section.
 ====
 
 [IMPORTANT]

modules/ipi-install-preparing-the-bare-metal-node.adoc

@@ -1,55 +1,53 @@
 // This is included in the following assemblies:
 //
-// ipi-install-expanding-the-cluster.adoc
+// installing/installing_bare_metal_ipi/ipi-install-expanding-the-cluster.adoc
 
 :_content-type: PROCEDURE
 [id='preparing-the-bare-metal-node_{context}']
-
 = Preparing the bare metal node
 
 Expanding the cluster requires a DHCP server. Each node must have a DHCP reservation.
 
-ifeval::[{product-version}>4.6]
 [IMPORTANT]
 .Reserving IP addresses so they become static IP addresses
 ====
-Some administrators prefer to use static IP addresses so that each node's IP address remains constant in the absence of a DHCP server. To use static IP addresses in the {product-title} cluster, *reserve the IP addresses in the DHCP server with an infinite lease*. After the installer provisions the node successfully, the dispatcher script will check the node's network configuration. If the dispatcher script finds that the network configuration contains a DHCP infinite lease, it will recreate the connection as a static IP connection using the IP address from the DHCP infinite lease. NICs without DHCP infinite leases will remain unmodified.
+Some administrators prefer to use static IP addresses so that each node's IP address remains constant in the absence of a DHCP server. To configure static IP addresses with NMState, see "(Optional) Configuring host network interfaces in the `install-config.yaml` file" in the "Setting up the environment for an OpenShift installation" section for additional details.
 ====
-endif::[]
+
 
 Preparing the bare metal node requires executing the following procedure from the provisioner node.
 
 .Procedure
 
 . Get the `oc` binary, if needed. It should already exist on the provisioner node.
 +
-[source,bash]
+[source,terminal]
 ----
 [kni@provisioner ~]$ curl -s https://mirror.openshift.com/pub/openshift-v4/clients/ocp/$VERSION/openshift-client-linux-$VERSION.tar.gz | tar zxvf - oc
 ----
 +
-[source,bash]
+[source,terminal]
 ----
 [kni@provisioner ~]$ sudo cp oc /usr/local/bin
 ----
 
-. Power off the bare metal node via the baseboard management controller and ensure it is off.
+. Power off the bare metal node through the baseboard management controller and ensure it is off.
 
-. Retrieve the user name and password of the bare metal node's baseboard management controller. Then, create `base64` strings from the user name and password. In the following example, the user name is `root` and the password is `calvin`.
+. Retrieve the user name and password of the bare metal node's baseboard management controller. Then, create `base64` strings from the user name and password. In the following example, the user name is `root` and the password is `password`.
 +
-[source,bash]
+[source,terminal]
 ----
 [kni@provisioner ~]$ echo -ne "root" | base64
 ----
 +
-[source,bash]
+[source,terminal]
 ----
-[kni@provisioner ~]$ echo -ne "calvin" | base64
+[kni@provisioner ~]$ echo -ne "password" | base64
 ----
 
 . Create a configuration file for the bare metal node.
 +
-[source,bash]
+[source,terminal]
 ----
 [kni@provisioner ~]$ vim bmh.yaml
 ----
@@ -60,42 +58,63 @@ Preparing the bare metal node requires executing the following procedure from th
 apiVersion: v1
 kind: Secret
 metadata:
-  name: openshift-worker-<num>-bmc-secret
+  name: openshift-worker-<num>-bmc-secret <1>
 type: Opaque
 data:
-  username: <base64-of-uid>
-  password: <base64-of-pwd>
+  username: <base64-of-uid> <2>
+  password: <base64-of-pwd> <3>
 ---
 apiVersion: metal3.io/v1alpha1
 kind: BareMetalHost
 metadata:
-  name: openshift-worker-<num>
+  name: openshift-worker-<num> <1>
 spec:
   online: true
-  bootMACAddress: <NIC1-mac-address>
+  bootMACAddress: <NIC1-mac-address> <4>
   bmc:
-    address: <protocol>://<bmc-ip>
-    credentialsName: openshift-worker-<num>-bmc-secret
-----
-+
-Replace `<num>` for the worker number of the bare metal node in the two `name` fields and the `credentialsName` field. Replace `<base64-of-uid>` with the `base64` string of the user name. Replace `<base64-of-pwd>` with the `base64` string of the password. Replace `<NIC1-mac-address>` with the MAC address of the bare metal node's first NIC.
-+
-See the BMC addressing section for additional BMC configuration options. Replace `<protocol>` with the BMC protocol, such as IPMI, RedFish, or others.
-Replace `<bmc-ip>` with the IP address of the bare metal node's baseboard management controller.
+    address: <protocol>://<bmc-ip> <5>
+    credentialsName: openshift-worker-<num>-bmc-secret <1>
+  networkConfig: <6>
+    interfaces:
+    - name: <NIC1_name>
+      type: ethernet
+      state: up
+      ipv4:
+        address:
+        - ip: "<IP_address>"
+          prefix-length: 24
+        enabled: true
+    dns-resolver:
+      config:
+        server:
+        - <DNS_IP_address>
+    routes:
+      config:
+      - destination: 0.0.0.0/0
+        next-hop-address: <IP_address>
+        next-hop-interface: <NIC1_name>
+----
+<1> Replace `<num>` for the worker number of the bare metal node in the two `name` fields and the `credentialsName` field.
+<2> Replace `<base64-of-uid>` with the `base64` string of the user name.
+<3> Replace `<base64-of-pwd>` with the `base64` string of the password.
+<4> Replace `<NIC1-mac-address>` with the MAC address of the bare metal node's first NIC. See the BMC addressing section for additional BMC configuration options. Replace `<protocol>` with the BMC protocol, such as IPMI, RedFish, or others.
+<5> Replace `<bmc-ip>` with the IP address of the bare metal node's baseboard management controller.
+<6> Optional. You can set the `networkConfig` configuration option to configure host network interfaces. See "(Optional) Configuring host network interfaces in the `install-config.yaml` file" in the "Setting up the environment for an OpenShift installation" section for configuration details.
 +
 [NOTE]
 ====
-If the MAC address of an existing bare metal node matches the MAC address of a bare metal host that you are attempting to provision, then the Ironic installation will fail. If the host enrollment, inspection, cleaning, or other Ironic steps fail, the Bare Metal Operator retries the installation continuously. See xref:modules/ipi-install-diagnosing-duplicate-mac-address.adoc#ipi-install-diagnosing-duplicate-mac-address_{context}[Diagnosing a host duplicate MAC address] for more information.
+If the MAC address of an existing bare metal node matches the MAC address of a bare metal host that you are attempting to provision, then the Ironic installation will fail. If the host enrollment, inspection, cleaning, or other Ironic steps fail, the Bare Metal Operator retries the installation continuously. See "Diagnosing a host duplicate MAC address" for more information.
 ====
 
 . Create the bare metal node.
 +
-[source,bash]
+[source,terminal]
 ----
 [kni@provisioner ~]$ oc -n openshift-machine-api create -f bmh.yaml
 ----
 +
-[source,bash]
+.Example output
+[source,terminal]
 ----
 secret/openshift-worker-<num>-bmc-secret created
 baremetalhost.metal3.io/openshift-worker-<num> created
@@ -105,14 +124,15 @@ Where `<num>` will be the worker number.
 
 . Power up and inspect the bare metal node.
 +
-[source,bash]
+[source,terminal]
 ----
 [kni@provisioner ~]$ oc -n openshift-machine-api get bmh openshift-worker-<num>
 ----
 +
 Where `<num>` is the worker node number.
 +
-[source,bash]
+.Example output
+[source,terminal]
 ----
 NAME                 STATUS   PROVISIONING STATUS   CONSUMER   BMC                 HARDWARE PROFILE   ONLINE   ERROR
 openshift-worker-<num>   OK       ready                            ipmi://<out-of-band-ip>   unknown            true