Skip to content

Commit 5f8bf47

Browse files
jeana-redhatjeana-redhat
authored
Merge pull request #53403 from SNiemann15/ibmz_post_install
[BZ1980770] Add new IBM Z post install assembly
2 parents 79d23ad + fdb3eaf commit 5f8bf47

File tree

5 files changed

+434
-0
lines changed

5 files changed

+434
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -528,6 +528,8 @@ Topics:
528528
File: connected-to-disconnected
529529
- Name: Cluster capabilities
530530
File: cluster-capabilities
531+
- Name: Configuring additional devices in an IBM Z or LinuxONE environment
532+
File: ibmz-post-install
531533
---
532534
Name: Updating clusters
533535
Dir: updating

‎modules/ibmz-configure-devices-manually.adoc

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * post-installation-configuration/ibmz-post-install.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="configure-additional-devices-manually_{context}"]
7+
= Configuring additional devices manually
8+
9+
Tasks in this section describe how to manually configure additional devices in an {ibmzProductName} or LinuxONE environment. This configuration method is persistent over node restarts but not {product-title} native and you need to redo the steps if you replace the node.
10+
11+
.Prerequisites
12+
13+
* You are logged in to the cluster as a user with administrative privileges.
14+
* The device must be available to the node.
15+
* In a z/VM environment, the device must be attached to the z/VM guest.
16+
17+
.Procedure
18+
19+
. Connect to the node via SSH:
20+
+
21+
[source,terminal]
22+
----
23+
$ ssh <user>@<node_ip_address>
24+
----
25+
+
26+
You can also start a debug session to the node:
27+
+
28+
[source,terminal]
29+
----
30+
$ oc debug node/<node_name>
31+
----
32+
33+
. Enable the devices with the `chzdev` command. For example:
34+
+
35+
[source,terminal]
36+
----
37+
$ sudo chzdev -e 0.0.8000
38+
sudo chzdev -e 1000-1002
39+
sude chzdev -e 4444
40+
sudo chzdev -e 0.0.8000:0x500507680d760026:0x00bc000000000000
41+
----

‎modules/ibmz-configure-devices-mco.adoc

Lines changed: 263 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,263 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * post-installation-configuration/ibmz-post-install.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="configure-additional-devices-using-mco_{context}"]
7+
= Configuring additional devices using the Machine Config Operator (MCO)
8+
9+
Tasks in this section describe how to use features of the Machine Config Operator to configure additional devices in an {ibmzProductName} or LinuxONE environment. Configuring devices with the MCO is persistent but only allows specific configurations for compute nodes. MCO does not allow control plane nodes to have different configurations.
10+
11+
.Prerequisites
12+
13+
* You are logged in to the cluster as a user with administrative privileges.
14+
* The device must be available to the z/VM guest.
15+
* The device is already attached.
16+
* The device is not included in the `cio_ignore` list, which can be set in the kernel parameters.
17+
* You have created a `MachineConfig` object file with the following YAML:
18+
+
19+
[source,yaml]
20+
----
21+
apiVersion: machineconfiguration.openshift.io/v1
22+
kind: MachineConfigPool
23+
metadata:
24+
name: worker0
25+
spec:
26+
machineConfigSelector:
27+
matchExpressions:
28+
- {key: machineconfiguration.openshift.io/role, operator: In, values: [worker,worker0]}
29+
nodeSelector:
30+
matchLabels:
31+
node-role.kubernetes.io/worker0: ""
32+
----
33+
34+
[id="configuring-fcp-host"]
35+
== Configuring a Fibre Channel Protocol (FCP) host
36+
37+
The following is an example of how to configure an FCP host adapter with N_Port Identifier Virtualization (NPIV) by adding a udev rule.
38+
39+
.Procedure
40+
41+
. Take the following sample udev rule `441-zfcp-host-0.0.8000.rules`:
42+
+
43+
[source,terminal]
44+
----
45+
ACTION=="add", SUBSYSTEM=="ccw", KERNEL=="0.0.8000", DRIVER=="zfcp", GOTO="cfg_zfcp_host_0.0.8000"
46+
ACTION=="add", SUBSYSTEM=="drivers", KERNEL=="zfcp", TEST=="[ccw/0.0.8000]", GOTO="cfg_zfcp_host_0.0.8000"
47+
GOTO="end_zfcp_host_0.0.8000"
48+
49+
LABEL="cfg_zfcp_host_0.0.8000"
50+
ATTR{[ccw/0.0.8000]online}="1"
51+
52+
LABEL="end_zfcp_host_0.0.8000"
53+
----
54+
55+
. Convert the rule to Base64 encoded, by running the following command:
56+
+
57+
[source,terminal]
58+
----
59+
$ base64 /path/to/file/
60+
----
61+
62+
. Copy the following MCO sample profile into a YAML file:
63+
+
64+
[source,yaml]
65+
----
66+
apiVersion: machineconfiguration.openshift.io/v1
67+
kind: MachineConfig
68+
metadata:
69+
labels:
70+
machineconfiguration.openshift.io/role: worker0 <1>
71+
name: 99-worker0-devices
72+
spec:
73+
config:
74+
ignition:
75+
version: 3.2.0
76+
storage:
77+
files:
78+
- contents:
79+
source: data:text/plain;base64,<encoded_base64_string> <2>
80+
filesystem: root
81+
mode: 420
82+
path: /etc/udev/rules.d/41-zfcp-host-0.0.8000.rules <3>
83+
----
84+
<1> The role you have defined in the machine config file.
85+
<2> The Base64 encoded string that you have generated in the previous step.
86+
<3> The path where the udev rule is located.
87+
88+
[id="configuring-fcp-lun"]
89+
== Configuring an FCP LUN
90+
The following is an example of how to configure an FCP LUN by adding a udev rule. You can add new FCP LUNs or add additional paths to LUNs that are already configured with multipathing.
91+
92+
.Procedure
93+
94+
. Take the following sample udev rule `41-zfcp-lun-0.0.8000:0x500507680d760026:0x00bc000000000000.rules`:
95+
+
96+
[source,terminal]
97+
----
98+
ACTION=="add", SUBSYSTEMS=="ccw", KERNELS=="0.0.8000", GOTO="start_zfcp_lun_0.0.8207"
99+
GOTO="end_zfcp_lun_0.0.8000"
100+
101+
LABEL="start_zfcp_lun_0.0.8000"
102+
SUBSYSTEM=="fc_remote_ports", ATTR{port_name}=="0x500507680d760026", GOTO="cfg_fc_0.0.8000_0x500507680d760026"
103+
GOTO="end_zfcp_lun_0.0.8000"
104+
105+
LABEL="cfg_fc_0.0.8000_0x500507680d760026"
106+
ATTR{[ccw/0.0.8000]0x500507680d760026/unit_add}="0x00bc000000000000"
107+
GOTO="end_zfcp_lun_0.0.8000"
108+
109+
LABEL="end_zfcp_lun_0.0.8000"
110+
----
111+
112+
. Convert the rule to Base64 encoded, by running the following command:
113+
+
114+
[source,terminal]
115+
----
116+
$ base64 /path/to/file/
117+
----
118+
119+
. Copy the following MCO sample profile into a YAML file:
120+
+
121+
[source,yaml]
122+
----
123+
apiVersion: machineconfiguration.openshift.io/v1
124+
kind: MachineConfig
125+
metadata:
126+
labels:
127+
machineconfiguration.openshift.io/role: worker0 <1>
128+
name: 99-worker0-devices
129+
spec:
130+
config:
131+
ignition:
132+
version: 3.2.0
133+
storage:
134+
files:
135+
- contents:
136+
source: data:text/plain;base64,<encoded_base64_string> <2>
137+
filesystem: root
138+
mode: 420
139+
path: /etc/udev/rules.d/41-zfcp-lun-0.0.8000:0x500507680d760026:0x00bc000000000000.rules <3>
140+
----
141+
<1> The role you have defined in the machine config file.
142+
<2> The Base64 encoded string that you have generated in the previous step.
143+
<3> The path where the udev rule is located.
144+
145+
[id="configuring-dasd"]
146+
== Configuring DASD
147+
148+
The following is an example of how to configure a DASD device by adding a udev rule.
149+
150+
.Procedure
151+
152+
. Take the following sample udev rule `41-dasd-eckd-0.0.4444.rules`:
153+
+
154+
[source,terminal]
155+
----
156+
ACTION=="add", SUBSYSTEM=="ccw", KERNEL=="0.0.4444", DRIVER=="dasd-eckd", GOTO="cfg_dasd_eckd_0.0.4444"
157+
ACTION=="add", SUBSYSTEM=="drivers", KERNEL=="dasd-eckd", TEST=="[ccw/0.0.4444]", GOTO="cfg_dasd_eckd_0.0.4444"
158+
GOTO="end_dasd_eckd_0.0.4444"
159+
160+
LABEL="cfg_dasd_eckd_0.0.4444"
161+
ATTR{[ccw/0.0.4444]online}="1"
162+
163+
LABEL="end_dasd_eckd_0.0.4444"
164+
----
165+
166+
. Convert the rule to Base64 encoded, by running the following command:
167+
+
168+
[source,terminal]
169+
----
170+
$ base64 /path/to/file/
171+
----
172+
173+
. Copy the following MCO sample profile into a YAML file:
174+
+
175+
[source,yaml]
176+
----
177+
apiVersion: machineconfiguration.openshift.io/v1
178+
kind: MachineConfig
179+
metadata:
180+
labels:
181+
machineconfiguration.openshift.io/role: worker0 <1>
182+
name: 99-worker0-devices
183+
spec:
184+
config:
185+
ignition:
186+
version: 3.2.0
187+
storage:
188+
files:
189+
- contents:
190+
source: data:text/plain;base64,<encoded_base64_string> <2>
191+
filesystem: root
192+
mode: 420
193+
path: /etc/udev/rules.d/41-dasd-eckd-0.0.4444.rules <3>
194+
----
195+
<1> The role you have defined in the machine config file.
196+
<2> The Base64 encoded string that you have generated in the previous step.
197+
<3> The path where the udev rule is located.
198+
199+
[id="configuring-qeth"]
200+
== Configuring qeth
201+
202+
The following is an example of how to configure a qeth device by adding a udev rule.
203+
204+
.Procedure
205+
206+
. Take the following sample udev rule `41-qeth-0.0.1000.rules`:
207+
+
208+
[source,terminal]
209+
----
210+
ACTION=="add", SUBSYSTEM=="drivers", KERNEL=="qeth", GOTO="group_qeth_0.0.1000"
211+
ACTION=="add", SUBSYSTEM=="ccw", KERNEL=="0.0.1000", DRIVER=="qeth", GOTO="group_qeth_0.0.1000"
212+
ACTION=="add", SUBSYSTEM=="ccw", KERNEL=="0.0.1001", DRIVER=="qeth", GOTO="group_qeth_0.0.1000"
213+
ACTION=="add", SUBSYSTEM=="ccw", KERNEL=="0.0.1002", DRIVER=="qeth", GOTO="group_qeth_0.0.1000"
214+
ACTION=="add", SUBSYSTEM=="ccwgroup", KERNEL=="0.0.1000", DRIVER=="qeth", GOTO="cfg_qeth_0.0.1000"
215+
GOTO="end_qeth_0.0.1000"
216+
217+
LABEL="group_qeth_0.0.1000"
218+
TEST=="[ccwgroup/0.0.1000]", GOTO="end_qeth_0.0.1000"
219+
TEST!="[ccw/0.0.1000]", GOTO="end_qeth_0.0.1000"
220+
TEST!="[ccw/0.0.1001]", GOTO="end_qeth_0.0.1000"
221+
TEST!="[ccw/0.0.1002]", GOTO="end_qeth_0.0.1000"
222+
ATTR{[drivers/ccwgroup:qeth]group}="0.0.1000,0.0.1001,0.0.1002"
223+
GOTO="end_qeth_0.0.1000"
224+
225+
LABEL="cfg_qeth_0.0.1000"
226+
ATTR{[ccwgroup/0.0.1000]online}="1"
227+
228+
LABEL="end_qeth_0.0.1000"
229+
----
230+
231+
. Convert the rule to Base64 encoded, by running the following command:
232+
+
233+
[source,terminal]
234+
----
235+
$ base64 /path/to/file/
236+
----
237+
238+
. Copy the following MCO sample profile into a YAML file:
239+
+
240+
[source,yaml]
241+
----
242+
apiVersion: machineconfiguration.openshift.io/v1
243+
kind: MachineConfig
244+
metadata:
245+
labels:
246+
machineconfiguration.openshift.io/role: worker0 <1>
247+
name: 99-worker0-devices
248+
spec:
249+
config:
250+
ignition:
251+
version: 3.2.0
252+
storage:
253+
files:
254+
- contents:
255+
source: data:text/plain;base64,<encoded_base64_string> <2>
256+
filesystem: root
257+
mode: 420
258+
path: /etc/udev/rules.d/41-dasd-eckd-0.0.4444.rules <3>
259+
----
260+
<1> The role you have defined in the machine config file.
261+
<2> The Base64 encoded string that you have generated in the previous step.
262+
<3> The path where the udev rule is located.
263+

‎modules/ibmz-enable-multipathing-fcp-luns.adoc

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * post-installation-configuration/ibmz-post-install.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="enabling-multipathing-fcp-luns_{context}"]
7+
= Enabling multipathing for FCP LUNs
8+
9+
Tasks in this section describe how to manually configure additional devices in an {ibmzProductName} or LinuxONE environment. This configuration method is persistent over node restarts but not {product-title} native and you need to redo the steps if you replace the node.
10+
11+
[IMPORTANT]
12+
====
13+
On {ibmzProductName} and LinuxONE, you can enable multipathing only if you configured your cluster for it during installation. For more information, see "Installing {op-system} and starting the {product-title} bootstrap process" in _Installing a cluster with z/VM on {ibmzProductName} and LinuxONE_.
14+
====
15+
16+
.Prerequisites
17+
18+
* You are logged in to the cluster as a user with administrative privileges.
19+
* You have configured multiple paths to a LUN with either method explained above.
20+
21+
.Procedure
22+
23+
. Connect to the node via SSH:
24+
+
25+
[source,terminal]
26+
----
27+
$ ssh <user>@<node_ip_address>
28+
----
29+
+
30+
You can also start a debug session to the node:
31+
+
32+
[source,terminal]
33+
----
34+
$ oc debug node/<node_name>
35+
----
36+
37+
. To enable multipathing, run the following command:
38+
+
39+
[source,terminal]
40+
----
41+
$ sudo /sbin/mpathconf --enable
42+
----
43+
44+
. To start the `multipathd` daemon, run the following command:
45+
+
46+
[source,terminal]
47+
----
48+
$ sudo multipath
49+
----
50+
51+
. Optional: To format your multipath device with fdisk, run the following command:
52+
+
53+
[source,terminal]
54+
----
55+
$ sudo fdisk /dev/mapper/mpatha
56+
----
57+
58+
.Verification
59+
60+
* To verify that the devices have been grouped, run the following command:
61+
+
62+
[source,terminal]
63+
----
64+
$ sudo multipath -II
65+
----
66+
+
67+
.Example output
68+
+
69+
[source,terminal]
70+
----
71+
mpatha (20017380030290197) dm-1 IBM,2810XIV
72+
size=512G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
73+
-+- policy='service-time 0' prio=50 status=enabled
74+
|- 1:0:0:6 sde 68:16 active ready running
75+
|- 1:0:1:6 sdf 69:24 active ready running
76+
|- 0:0:0:6 sdg 8:80 active ready running
77+
`- 0:0:1:6 sdh 66:48 active ready running
78+
----

0 commit comments

Comments
 (0)