add multipathing to installation procedure

SNiemann15 · SNiemann15 · commit 8649e04fcbb4 · 2021-02-05T18:06:18.000+01:00
diff --git a/modules/installation-complete-user-infra.adoc b/modules/installation-complete-user-infra.adoc
@@ -12,10 +12,20 @@ endif::[]
 ifeval::["{context}" == "installing-restricted-networks-bare-metal"]
 :restricted:
 endif::[]
+ifeval::["{context}" == "installing-ibm-z"]
+:ibm-z:
+endif::[]
 ifeval::["{context}" == "installing-restricted-networks-ibm-z"]
+:ibm-z:
+:restricted:
+endif::[]
+ifeval::["{context}" == "installing-ibm-power"]
+:ibm-power:
+endif::[]
+ifeval::["{context}" == "installing-restricted-networks-ibm-power"]
+:ibm-power:
 :restricted:
 endif::[]
-
 [id="installation-complete-user-infra_{context}"]
 = Completing installation on user-provisioned infrastructure
 
@@ -130,6 +140,73 @@ command.
 +
 If the pod logs display, the Kubernetes API server can communicate with the
 cluster machines.
+ifdef::ibm-z,ibm-power[]
+. For an installation with FCP, additional steps are required to enable multipathing. 
+.. To enable multipathing on master nodes, apply the following machine config file: 
++
+[source,yaml]
+--
+apiVersion: machineconfiguration.openshift.io/v1
+kind: MachineConfig
+metadata:
+  labels:
+    machineconfiguration.openshift.io/role: "master"
+  name: 99-master-kargs-mpath
+spec:
+  kernelArguments:
+    - 'rd.multipath=default'
+    - 'root=/dev/disk/by-label/dm-mpath-root'
+--
+
+.. To enable multipathing on worker nodes, apply the following machine config file: 
++
+[source,yaml]
+--
+apiVersion: machineconfiguration.openshift.io/v1
+kind: MachineConfig
+metadata:
+  labels:
+    machineconfiguration.openshift.io/role: "worker"
+  name: 99-master-kargs-mpath
+spec:
+  kernelArguments:
+    - 'rd.multipath=default'
+    - 'root=/dev/disk/by-label/dm-mpath-root'
+--
+endif::ibm-z,ibm-power[]
+ifdef::ibm-power[]
+.. To display a boot list and specify the possible boot devices if the system is booted in normal mode, enter the following command:
++
+[source,terminal]
+----
+$ bootlist -m normal -o
+sda
+----
+.. To update the boot list for normal mode and add alternate device names, enter the following command:
++
+[source,terminal]
+----
+$ bootlist -m normal -o /dev/sdc /dev/sdd /dev/sde
+sdc
+sdd
+sde
+----
++
+If the original boot disk path is down, the node reboots from the alternate device registered in the normal boot device list. 
+endif::ibm-power[]
+ifdef::ibm-z,ibm-power[]
+.. All the worker nodes are restarted. To monitor the process, enter the following command: 
++
+[source,terminal]
+----
+$ oc get nodes -w
+----
++
+[NOTE]
+====
+If you have additional machine types such as infrastructure nodes, repeat the process for these types.   
+====
+endif::ibm-z,ibm-power[]
 
 ifdef::restricted[]
 . Register your cluster on the link:https://cloud.redhat.com/openshift/register[Cluster registration] page.
@@ -144,6 +221,17 @@ endif::[]
 ifdef::openshift-origin[]
 :!restricted:
 endif::[]
+ifeval::["{context}" == "installing-ibm-z"]
+:!ibm-z:
+endif::[]
 ifeval::["{context}" == "installing-restricted-networks-ibm-z"]
+:!ibm-z:
 :!restricted:
 endif::[]
+ifeval::["{context}" == "installing-ibm-power"]
+:!ibm-power:
+endif::[]
+ifeval::["{context}" == "installing-restricted-networks-ibm-power"]
+:!ibm-power:
+:restricted:
+endif::[]
diff --git a/modules/installation-ibm-z-kvm-user-infra-machines-iso.adoc b/modules/installation-ibm-z-kvm-user-infra-machines-iso.adoc
@@ -16,7 +16,7 @@ Before you install a cluster on IBM Z infrastructure that you provision, you mus
 
 . Install the KVM/QEMU hypervisor on the {op-system-base} KVM host.
 
-. Obtain the {op-system-base} QEMU copy-on-write (QCOW2) disk image file from the link:https://access.redhat.com/downloads/content/290[Product Downloads] page on the Red Hat customer portal or the link:https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/4.2/latest/[{op-system} image mirror] page.
+. Obtain the {op-system-base} QEMU copy-on-write (QCOW2) disk image file from the link:https://access.redhat.com/downloads/content/290[Product Downloads] page on the Red Hat Customer Portal or from the link:https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/latest/[{op-system} image mirror] page.
 +
 [IMPORTANT]
 ====
diff --git a/modules/installation-ibm-z-user-infra-machines-iso.adoc b/modules/installation-ibm-z-user-infra-machines-iso.adoc
@@ -9,13 +9,13 @@ Before you install a cluster on IBM Z infrastructure that you provision, you mus
 
 .Prerequisites
 
-* An FTP server running on your provisioning machine that is accessible to the machines you create.
+* An HTTP or HTTPS server running on your provisioning machine that is accessible to the machines you create.
 
 .Procedure
 
 . Log in to Linux on your provisioning machine.
 
-. Obtain the {op-system-first} kernel, initramfs, and rootfs files from the link:https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/4.2/latest/[{op-system} image mirror].
+. Obtain the {op-system-first} kernel, initramfs, and rootfs files from the link:https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/latest/[{op-system} image mirror].
 +
 [IMPORTANT]
 ====
@@ -36,23 +36,25 @@ The rootfs image is the same for FCP and DASD.
 ====
 +
 . Create parameter files. The following parameters are specific for a particular virtual machine:
-** For `coreos.inst.install_dev=`, specify `dasda` for a DASD installation, or `sda` for FCP. Note that FCP requires `zfcp.allow_lun_scan=0`.
-** For `rd.dasd=`, specifys the DASD where {op-system} is to be installed.
-** `rd.zfcp=<adapter>,<wwpn>,<lun>` specifies the FCP disk to install {op-system} on.
 ** For `ip=`, specify the following seven entries:
 ... The IP address for the machine.
 ... An empty string.
 ... The gateway.
 ... The netmask.
 ... The machine host and domain name in the form `hostname.domainname`. Omit this value to let {op-system} decide.
 ... The network interface name. Omit this value to let {op-system} decide.
-... If you use static IP addresses, an empty string.
+... If you use static IP addresses, specify `none`.
 ** For `coreos.inst.ignition_url=`, specify the Ignition file for the machine role. Use `bootstrap.ign`, `master.ign`, or `worker.ign`. Only HTTP and HTTPS protocols are supported.
 ** For `coreos.live.rootfs_url=`, specify the matching rootfs artifact for the kernel and initramfs you are booting. Only HTTP and HTTPS protocols are supported.
-** All other parameters can stay as they are.
+
+** For installations on DASD-type disks, complete the following tasks:
+... For `coreos.inst.install_dev=`, specify `dasda`. 
+... Use `rd.dasd=` to specify the DASD where {op-system} is to be installed.
+... Leave all other parameters unchanged. 
 +
 Example parameter file, `bootstrap-0.parm`, for the bootstrap machine:
 +
+[source,terminal]
 ----
 rd.neednet=1 console=ttysclp0 coreos.inst.install_dev=dasda coreos.live.rootfs_url=http://
 cl1.provide.example.com:8080/assets/rhcos-live-rootfs.s390x.img
@@ -62,6 +64,40 @@ rd.znet=qeth,0.0.bdf0,0.0.bdf1,0.0.bdf2,layer2=1,portno=0 zfcp.allow_lun_scan=0
 !condev rd.dasd=0.0.3490
 ----
 
+
+** For installations on FCP-type disks, complete the following tasks:
+... Use `rd.zfcp=<adapter>,<wwpn>,<lun>` to specify the FCP disk where {op-system} is to be installed. For multipathing repeat this step for each additional path. 
+... For multipathing, set the following parameter: `rd.multipath=default`.
+... For multipathing, set the install device as: `coreos.inst.install_dev=/dev/mapper/mpatha`.
+... For single-path installation, set the install device as: `coreos.inst.install_dev=sda`.
++
+[NOTE]
+====
+If additional LUNs are configured with NPIV, FCP requires `zfcp.allow_lun_scan=0`.  
+====
+... Leave all other parameters unchanged.
++
+[NOTE]
+====
+Additional post-installation steps are required to fully enable multipathing. See “Completing installation on user-provisioned infrastructure” for more information.  
+====
++
+The following is an example parameter file `bootstrap-0.parm` for the bootstrap machine with multipathing:
++
+[source,terminal]
+----
+rd.neednet=1 rd.multipath=default console=ttysclp0 coreos.inst.install_dev=/dev/mapper/mpatha 
+coreos.live.rootfs_url=http://cl1.provide.example.com:8080/assets/rhcos-live-rootfs.s390x.img
+coreos.inst.ignition_url=http://cl1.provide.example.com:8080/ignition/bootstrap.ign
+ip=172.18.78.2::172.18.78.1:255.255.255.0:::none nameserver=172.18.78.1
+rd.znet=qeth,0.0.bdf0,0.0.bdf1,0.0.bdf2,layer2=1,portno=0 zfcp.allow_lun_scan=0 cio_ignore=all,
+!condev 
+rd.zfcp=0.0.1987,0x50050763070bc5e3,0x4008400B00000000
+rd.zfcp=0.0.19C7,0x50050763070bc5e3,0x4008400B00000000
+rd.zfcp=0.0.1987,0x50050763071bc5e3,0x4008400B00000000
+rd.zfcp=0.0.19C7,0x50050763071bc5e3,0x4008400B00000000
+----
+
 . Transfer the initramfs, kernel, parameter files, and {op-system} images to z/VM, for example with FTP. For details about how to transfer the files with FTP and boot from the virtual reader, see link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/installation_guide/sect-installing-zvm-s390[Installing under Z/VM].
 . Punch the files to the virtual reader of the z/VM guest virtual machine that is to become your bootstrap node.
 +
diff --git a/modules/installation-infrastructure-user-infra.adoc b/modules/installation-infrastructure-user-infra.adoc
@@ -25,7 +25,7 @@ page before you create the supporting infrastructure for your cluster.
 
 ifdef::ibm-z[]
 . Set up static IP addresses.
-. Set up an FTP server.
+. Set up an HTTP or HTTPS server to provide Ignition files to the cluster nodes.
 endif::ibm-z[]
 ifndef::ibm-z[]
 . Configure DHCP or set static IP addresses on each node.
diff --git a/modules/installation-network-user-infra.adoc b/modules/installation-network-user-infra.adoc
@@ -44,14 +44,14 @@ All the {op-system-first} machines require network in `initramfs` during boot
 to fetch Ignition config from the machine config server.
 
 ifdef::ibm-z[]
-During the initial boot, the machines require an FTP server in order to
+During the initial boot, the machines require an HTTP or HTTPS server to
 establish a network connection to download their Ignition config files.
 
 Ensure that the machines have persistent IP addresses and host names.
 endif::ibm-z[]
 ifndef::ibm-z[]
 During the initial boot, the machines require either a DHCP server
-or that static IP addresses be set on each host in the cluster in order to
+or that static IP addresses be set on each host in the cluster to
 establish a network connection, which allows them to download their Ignition config files.
 
 It is recommended to use the DHCP server to manage the machines for the cluster

Original file line number	Diff line number	Diff line change
`@@ -16,7 +16,7 @@ Before you install a cluster on IBM Z infrastructure that you provision, you mus`
`16`	`16`
`17`	`17`	`. Install the KVM/QEMU hypervisor on the {op-system-base} KVM host.`
`18`	`18`
`19`		`-. Obtain the {op-system-base} QEMU copy-on-write (QCOW2) disk image file from the link:https://access.redhat.com/downloads/content/290[Product Downloads] page on the Red Hat customer portal or the link:https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/4.2/latest/[{op-system} image mirror] page.`
	`19`	`+. Obtain the {op-system-base} QEMU copy-on-write (QCOW2) disk image file from the link:https://access.redhat.com/downloads/content/290[Product Downloads] page on the Red Hat Customer Portal or from the link:https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/latest/[{op-system} image mirror] page.`
`20`	`20`	`+`
`21`	`21`	`[IMPORTANT]`
`22`	`22`	`====`