Skip to content

Commit 61e5e54

Browse files
author
Bob
committed
BZ1973326: Update firstboot multipath doc
1 parent bad3784 commit 61e5e54

File tree

7 files changed

+190
-88
lines changed

7 files changed

+190
-88
lines changed

‎installing/installing_bare_metal/installing-bare-metal-network-customizations.adoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,8 @@ include::modules/installation-user-infra-machines-advanced.adoc[leveloffset=+2]
131131

132132
include::modules/installation-user-infra-machines-static-network.adoc[leveloffset=+3]
133133

134+
include::modules/rhcos-enabling-multipath.adoc[leveloffset=+2]
135+
134136
include::modules/architecture-rhcos-updating-bootloader.adoc[leveloffset=+2]
135137

136138
include::modules/installation-installing-bare-metal.adoc[leveloffset=+1]

‎installing/installing_bare_metal/installing-bare-metal.adoc‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ include::modules/cluster-entitlements.adoc[leveloffset=+1]
3232

3333
.Additional resources
3434

35-
* See xref:../../installing/installing_bare_metal/installing-restricted-networks-bare-metal.adoc#installing-restricted-networks-bare-metal[Installing a user-provisioned bare metal cluster on a restricted network] for more information about performing a restricted network installation on bare metal infrastructure that you provision.
35+
* See xref:../../installing/installing_bare_metal/installing-restricted-networks-bare-metal.adoc#installing-restricted-networks-bare-metal[Installing a user-provisioned bare metal cluster on a restricted network] for more information about performing a restricted network installation on bare metal infrastructure that you provision.
3636

3737
include::modules/installation-requirements-user-infra.adoc[leveloffset=+1]
3838

@@ -150,6 +150,12 @@ include::modules/installation-user-infra-machines-advanced.adoc[leveloffset=+2]
150150

151151
include::modules/installation-user-infra-machines-static-network.adoc[leveloffset=+3]
152152

153+
include::modules/rhcos-enabling-multipath.adoc[leveloffset=+2]
154+
155+
.Additional resources
156+
157+
* See xref:../../installing/installing_bare_metal/installing-bare-metal.adoc#creating-machines-bare-metal[Installing {op-system} and starting the {product-title} bootstrap process] for more information on using special `coreos.inst.*` arguments to direct the live installer.
158+
153159
include::modules/architecture-rhcos-updating-bootloader.adoc[leveloffset=+2]
154160

155161
include::modules/installation-installing-bare-metal.adoc[leveloffset=+1]

‎installing/installing_bare_metal/installing-restricted-networks-bare-metal.adoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,8 @@ include::modules/installation-user-infra-machines-advanced.adoc[leveloffset=+2]
152152

153153
include::modules/installation-user-infra-machines-static-network.adoc[leveloffset=+3]
154154

155+
include::modules/rhcos-enabling-multipath.adoc[leveloffset=+2]
156+
155157
include::modules/architecture-rhcos-updating-bootloader.adoc[leveloffset=+2]
156158

157159
include::modules/installation-installing-bare-metal.adoc[leveloffset=+1]

‎modules/installation-complete-user-infra.adoc‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -149,12 +149,17 @@ cluster machines.
149149
ifndef::ibm-power[]
150150
. For an installation with Fibre Channel Protocol (FCP), additional steps are required to enable multipathing. Do not enable multipathing during installation.
151151
+
152-
See "Enabling multipathing with kernel arguments on RHCOS" in the _Post-installation configuration_ documentation for more information.
152+
[NOTE]
153+
====
154+
When installing with multipath, it is strongly recommended to enable it at installation time, and not at a later time, which can cause problems.
155+
====
156+
+
157+
See "Enabling multipathing with kernel arguments on {op-system}" in the _Installing on bare metal_ documentation for more information.
153158
endif::ibm-power[]
154159
ifdef::ibm-power[]
155160
. Additional steps are required to enable multipathing. Do not enable multipathing during installation.
156161
+
157-
See "Enabling multipathing with kernel arguments on RHCOS" in the _Post-installation configuration_ documentation for more information.
162+
See the _Installing {op-system} and starting the {product-title} bootstrap process_ documentation for more information.
158163

159164
.. To display a boot list and specify the possible boot devices if the system is booted in normal mode, enter the following command:
160165
+

‎modules/rhcos-enabling-multipath-day-2.adoc‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * post_installation_configuration/machine-configuration-tasks.adoc
4+
5+
[id="rhcos-enabling-multipath-day-2_{context}"]
6+
= Enabling multipathing with kernel arguments on {op-system}
7+
8+
{op-system-first} supports multipathing on the primary disk, allowing stronger resilience to hardware failure to achieve higher host availability. Post-installation support is available by activating multipathing via the machine config.
9+
10+
[IMPORTANT]
11+
====
12+
Enabling multipathing during installation is supported and recommended for nodes provisioned in {product-title} 4.8 or higher. In setups where any I/O to non-optimized paths results in I/O system errors, you must enable multipathing at installation time. For more information about enabling multipathing during installation time, see "Enabling multipathing with kernel arguments on RHCOS" in the _Installing on bare metal_ documentation.
13+
====
14+
15+
[IMPORTANT]
16+
====
17+
On IBM Z and LinuxONE, you can enable multipathing only if you configured your cluster for it during installation. For more information, see "Installing {op-system} and starting the {product-title} bootstrap process" in _Installing a cluster with z/VM on IBM Z and LinuxONE_.
18+
====
19+
// Add xref once it's allowed.
20+
21+
.Prerequisites
22+
* You have a running {product-title} cluster that uses version 4.7 or later.
23+
* You are logged in to the cluster as a user with administrative privileges.
24+
25+
.Procedure
26+
27+
. To enable multipathing post-installation on control plane nodes:
28+
29+
* Create a machine config file, such as `99-master-kargs-mpath.yaml`, that instructs the cluster to add the `master` label and that identifies the multipath kernel argument, for example:
30+
+
31+
[source,yaml]
32+
----
33+
apiVersion: machineconfiguration.openshift.io/v1
34+
kind: MachineConfig
35+
metadata:
36+
labels:
37+
machineconfiguration.openshift.io/role: "master"
38+
name: 99-master-kargs-mpath
39+
spec:
40+
kernelArguments:
41+
- 'rd.multipath=default'
42+
- 'root=/dev/disk/by-label/dm-mpath-root'
43+
----
44+
45+
. To enable multipathing post-installation on worker nodes:
46+
47+
* Create a machine config file, such as `99-worker-kargs-mpath.yaml`, that instructs the cluster to add the `worker` label and that identifies the multipath kernel argument, for example:
48+
+
49+
[source,yaml]
50+
----
51+
apiVersion: machineconfiguration.openshift.io/v1
52+
kind: MachineConfig
53+
metadata:
54+
labels:
55+
machineconfiguration.openshift.io/role: "worker"
56+
name: 99-worker-kargs-mpath
57+
spec:
58+
kernelArguments:
59+
- 'rd.multipath=default'
60+
- 'root=/dev/disk/by-label/dm-mpath-root'
61+
----
62+
63+
. Create the new machine config by using either the master or worker YAML file you previously created:
64+
+
65+
[source,terminal]
66+
----
67+
$ oc create -f ./99-master-kargs-mpath.yaml
68+
----
69+
70+
. Check the machine configs to see that the new one was added:
71+
+
72+
[source,terminal]
73+
----
74+
$ oc get MachineConfig
75+
----
76+
+
77+
.Example output
78+
[source,terminal]
79+
----
80+
NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE
81+
00-master 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
82+
00-worker 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
83+
01-master-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
84+
01-master-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
85+
01-worker-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
86+
01-worker-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
87+
99-master-kargs-mpath 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 105s
88+
99-master-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
89+
99-master-ssh 3.2.0 40m
90+
99-worker-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
91+
99-worker-ssh 3.2.0 40m
92+
rendered-master-23e785de7587df95a4b517e0647e5ab7 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
93+
rendered-worker-5d596d9293ca3ea80c896a1191735bb1 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
94+
----
95+
96+
. Check the nodes:
97+
+
98+
[source,terminal]
99+
----
100+
$ oc get nodes
101+
----
102+
+
103+
.Example output
104+
[source,terminal]
105+
----
106+
NAME STATUS ROLES AGE VERSION
107+
ip-10-0-136-161.ec2.internal Ready worker 28m v1.20.0
108+
ip-10-0-136-243.ec2.internal Ready master 34m v1.20.0
109+
ip-10-0-141-105.ec2.internal Ready,SchedulingDisabled worker 28m v1.20.0
110+
ip-10-0-142-249.ec2.internal Ready master 34m v1.20.0
111+
ip-10-0-153-11.ec2.internal Ready worker 28m v1.20.0
112+
ip-10-0-153-150.ec2.internal Ready master 34m v1.20.0
113+
----
114+
+
115+
You can see that scheduling on each worker node is disabled as the change is being applied.
116+
117+
. Check that the kernel argument worked by going to one of the worker nodes and listing
118+
the kernel command line arguments (in `/proc/cmdline` on the host):
119+
+
120+
[source,terminal]
121+
----
122+
$ oc debug node/ip-10-0-141-105.ec2.internal
123+
----
124+
+
125+
.Example output
126+
[source,terminal]
127+
----
128+
Starting pod/ip-10-0-141-105ec2internal-debug ...
129+
To use host binaries, run `chroot /host`
130+
131+
sh-4.2# cat /host/proc/cmdline
132+
...
133+
rd.multipath=default root=/dev/disk/by-label/dm-mpath-root
134+
...
135+
136+
sh-4.2# exit
137+
----
138+
+
139+
You should see the added kernel arguments.

‎modules/rhcos-enabling-multipath.adoc‎

Lines changed: 26 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -1,122 +1,65 @@
11
// Module included in the following assemblies:
22
//
3-
// * post_installation_configuration/machine-configuration-tasks.adoc
3+
// * installing/installing_bare_metal/installing-bare-metal.adoc
4+
// * installing/installing_bare_metal/installing-bare-metal-network-customizations.adoc
5+
// * installing/installing_bare_metal/installing-restricted-networks-bare-metal.adoc
46

57
[id="rhcos-enabling-multipath_{context}"]
68
= Enabling multipathing with kernel arguments on {op-system}
79

810
{op-system} supports multipathing on the primary disk, allowing stronger resilience to hardware failure to achieve higher host availability.
911

10-
[IMPORTANT]
11-
====
12-
Multipathing is only supported when it is activated using the machine config as documented in the following procedure. It must be enabled after {op-system} installation.
13-
====
12+
You can enable multipathing at installation time for nodes that were provisioned in {product-title} 4.8 or later. While post-installation support is available by activating multipathing via the machine config, enabling multipathing during installation is recommended.
13+
14+
In setups where any I/O to non-optimized paths results in I/O system errors, you must enable multipathing at installation time.
1415

1516
[IMPORTANT]
1617
====
17-
On IBM Z and LinuxONE, you can enable multipathing only if you configured your cluster for it during installation. For more information, see "Creating {op-system-first} machines" in _Installing a cluster with z/VM on IBM Z and LinuxONE_.
18+
On IBM Z and LinuxONE, you can enable multipathing only if you configured your cluster for it during installation. For more information, see "Installing {op-system} and starting the {product-title} bootstrap process" in _Installing a cluster with z/VM on IBM Z and LinuxONE_.
1819
====
1920
// Add xref once it's allowed.
2021

22+
The following procedure enables multipath at installation time and appends kernel arguments to the `coreos-installer install` command so that the installed system itself will use multipath beginning from the first boot.
23+
2124
.Prerequisites
22-
* You have a running {product-title} cluster that uses version 4.7 or later.
25+
* You have a running {product-title} cluster that uses version 4.8 or later.
2326
* You are logged in to the cluster as a user with administrative privileges.
2427

2528
.Procedure
2629

27-
. To enable multipathing on control plane nodes (also known as the master nodes):
28-
29-
* Create a machine config file, such as `99-master-kargs-mpath.yaml`, that instructs the cluster to add the `master` label and that identifies the multipath kernel argument, for example:
30-
31-
+
32-
[source,yaml]
33-
----
34-
apiVersion: machineconfiguration.openshift.io/v1
35-
kind: MachineConfig
36-
metadata:
37-
labels:
38-
machineconfiguration.openshift.io/role: "master"
39-
name: 99-master-kargs-mpath
40-
spec:
41-
kernelArguments:
42-
- rd.multipath=default
43-
- root=/dev/disk/by-label/dm-mpath-root
44-
----
45-
46-
. To enable multipathing on worker nodes:
47-
48-
* Create a machine config file, such as `99-worker-kargs-mpath.yaml`, that instructs the cluster to add the `worker` label and that identifies the multipath kernel argument, for example:
49-
+
50-
[source,yaml]
51-
----
52-
apiVersion: machineconfiguration.openshift.io/v1
53-
kind: MachineConfig
54-
metadata:
55-
labels:
56-
machineconfiguration.openshift.io/role: "worker"
57-
name: 99-worker-kargs-mpath
58-
spec:
59-
kernelArguments:
60-
- rd.multipath=default
61-
- root=/dev/disk/by-label/dm-mpath-root
62-
----
63-
64-
. Create the new machine config by using either the master or worker YAML file you previously created:
30+
. To enable multipath and start the `multipathd` daemon, run the following command:
6531
+
6632
[source,terminal]
6733
----
68-
$ oc create -f ./99-master-kargs-mpath.yaml
34+
$ mpathconf --enable && systemctl start multipathd.service
6935
----
36+
** Optional: If booting the PXE or ISO, you can instead enable multipath by adding `rd.multipath=default` from the kernel command line.
7037

71-
. Check the machine configs to see that the new one was added:
38+
. Append the kernel arguments by invoking the `coreos-installer` program:
7239
+
73-
[source,terminal]
74-
----
75-
$ oc get MachineConfig
76-
----
40+
* If there is only one multipath device connected to the machine, it should be available at path `/dev/mapper/mpatha`. For example:
7741
+
78-
.Example output
7942
[source,terminal]
8043
----
81-
NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE
82-
00-master 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
83-
00-worker 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
84-
01-master-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
85-
01-master-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
86-
01-worker-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
87-
01-worker-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
88-
99-master-kargs-mpath 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 105s
89-
99-master-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
90-
99-master-ssh 3.2.0 40m
91-
99-worker-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
92-
99-worker-ssh 3.2.0 40m
93-
rendered-master-23e785de7587df95a4b517e0647e5ab7 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
94-
rendered-worker-5d596d9293ca3ea80c896a1191735bb1 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
44+
$ coreos-installer install /dev/mapper/mpatha \ <1>
45+
--append-karg rd.multipath=default \
46+
--append-karg root=/dev/disk/by-label/dm-mpath-root
9547
----
96-
97-
. Check the nodes:
48+
<1> Indicates the path of the single multipathed device.
9849
+
99-
[source,terminal]
100-
----
101-
$ oc get nodes
102-
----
50+
* If there are multiple multipath devices connected to the machine, or to be more explicit, instead of using `/dev/mapper/mpatha`, it is recommended to use the World Wide Name (WWN) symlink available in `/dev/disk/by-id`. For example:
10351
+
104-
.Example output
10552
[source,terminal]
10653
----
107-
NAME STATUS ROLES AGE VERSION
108-
ip-10-0-136-161.ec2.internal Ready worker 28m v1.20.0
109-
ip-10-0-136-243.ec2.internal Ready master 34m v1.20.0
110-
ip-10-0-141-105.ec2.internal Ready,SchedulingDisabled worker 28m v1.20.0
111-
ip-10-0-142-249.ec2.internal Ready master 34m v1.20.0
112-
ip-10-0-153-11.ec2.internal Ready worker 28m v1.20.0
113-
ip-10-0-153-150.ec2.internal Ready master 34m v1.20.0
54+
$ coreos-installer install /dev/disk/by-id/wwn-<wwn_ID> \ <1>
55+
--append-karg rd.multipath=default \
56+
--append-karg root=/dev/disk/by-label/dm-mpath-root
11457
----
58+
<1> Indicates the WWN ID of the target multipathed device. For example, `0xx194e957fcedb4841`.
11559
+
116-
You can see that scheduling on each worker node is disabled as the change is being applied.
60+
This symlink can also be used as the `coreos.inst.install_dev` kernel argument when using special `coreos.inst.*` arguments to direct the live installer. For more information, see "Installing {op-system} and starting the {product-title} bootstrap process".
11761

118-
. Check that the kernel argument worked by going to one of the worker nodes and listing
119-
the kernel command line arguments (in `/proc/cmdline` on the host):
62+
. Check that the kernel arguments worked by going to one of the worker nodes and listing the kernel command line arguments (in `/proc/cmdline` on the host):
12063
+
12164
[source,terminal]
12265
----

‎post_installation_configuration/machine-configuration-tasks.adoc‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,16 @@ include::modules/checking-mco-status.adoc[leveloffset=+2]
2424
You can use the tasks in this section to create `MachineConfig` objects that modify files, systemd unit files, and other operating system features running on {product-title} nodes. For more ideas on working with machine configs, see content related to link:https://access.redhat.com/solutions/5096731[adding] or
2525
link:https://access.redhat.com/solutions/4510281[updating] SSH authorized keys, link:https://access.redhat.com/verify-images-ocp4[verifying image signatures], link:https://access.redhat.com/solutions/4727321[enabling SCTP], and link:https://access.redhat.com/solutions/5170251[configuring iSCSI initiatornames] for {product-title}.
2626

27-
{product-title} supports link:https://coreos.github.io/ignition/configuration-v3_2/[Ignition specification version 3.2]. All new machine configs you create going forward should be based on Ignition specification version 3.2. If you are upgrading your {product-title} cluster, any existing Ignition specification version 2.x machine configs will be translated automatically to specification version 3.2.
27+
{product-title} supports link:https://coreos.github.io/ignition/configuration-v3_2/[Ignition specification version 3.2]. All new machine configs you create going forward should be based on Ignition specification version 3.2. If you are upgrading your {product-title} cluster, any existing Ignition specification version 2.x machine configs will be translated automatically to specification version 3.2.
2828

2929
include::modules/installation-special-config-chrony.adoc[leveloffset=+2]
3030
include::modules/nodes-nodes-kernel-arguments.adoc[leveloffset=+2]
31-
include::modules/rhcos-enabling-multipath.adoc[leveloffset=+2]
31+
include::modules/rhcos-enabling-multipath-day-2.adoc[leveloffset=+2]
32+
33+
.Additional resources
34+
35+
* See xref:../installing/installing_bare_metal/installing-bare-metal.adoc#rhcos-enabling-multipath_installing-bare-metal[Enabling multipathing with kernel arguments on RHCOS] for more information about enabling multipathing during installation time.
36+
3237
include::modules/nodes-nodes-rtkernel-arguments.adoc[leveloffset=+2]
3338
include::modules/machineconfig-modify-journald.adoc[leveloffset=+2]
3439
include::modules/rhcos-add-extensions.adoc[leveloffset=+2]

0 commit comments

Comments
 (0)