You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/networking/clusternetwork.md
+88-46Lines changed: 88 additions & 46 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -97,49 +97,55 @@ The [witness node](../advanced/witness.md) is generally not involved in the cust
97
97
98
98
### Create a New Cluster Network
99
99
100
-
1. Go to **Networks > ClusterNetworks/Configs**, and then click **Create**.
100
+
:::tip
101
101
102
-
1. Specify a name for the network.
102
+
To simplify cluster maintenance, create one network configuration for each node or group of nodes. Without dedicated network configurations, certain maintenance tasks (for example, replacing old NICs with NICs in different slots) will require you to stop and/or migrate the affected virtual machines before updating the network configuration.
1. In the **Node Selector** tab, specify the name and choose one of the three methods to select nodes where the network configuration will apply. If you want to cover the unselected nodes, you can create another network configuration.
The method **Select all nodes**works only when all nodes use the exact same dedicated NICs for this specific custom cluster network. In other situations (for example, when the cluster has a [witness node](../advanced/witness.md)), you must select either of the remaining methods.
116
+
1. On the **Network Config:Create**screen, specify a name for the configuration.
117
117
118
-
:::
118
+
1. On the **Node Selector** tab, select the method for defining the scope of this specific network configuration.
119
119
120
-
1. Click the **Uplink** tab to add the NICs, and configure the bond options and link attributes. The bond mode defaults to `active-backup`.
120
+

121
121
122
-

123
-
124
-
:::note
122
+
:::note
125
123
126
-
- The NICs drop-down list shows all the common NICs on all the selected nodes. The drop-down list will change as you select different nodes.
127
-
- The text `enp7s3 (1/3 Down)` in the NICs drop-down list indicates that the enp7s3 NIC is down in one of the three selected nodes. In this case, you need to find the NIC, set it up, and refresh this page. After this, it should be selectable.
124
+
- The method *Select all nodes* works only when all nodes use the exact same dedicated NICs for this specific custom cluster network. In other situations (for example, when the cluster has a [witness node](../advanced/witness.md)), you must select either of the remaining methods.
125
+
- If you want the configuration to apply to nodes that are not covered by the selected method, you must create another network configuration.
128
126
129
-
:::
127
+
:::
130
128
131
-
:::note
129
+
1. On the **Uplink** tab, configure the following settings:
132
130
133
-
Starting with Harvester v1.1.2, Harvester supports updating network configs. Make sure to stop all affected VMs before updating network configs.
131
+
-**NICs**: The list contains NICs that are common to all selected nodes. NICs that cannot be selected are unavailable on one or more nodes and must be configured. Once troubleshooting is completed, refresh the screen and verify that the NICs can be selected.
132
+
-**Bond Options**: The default bonding mode is **active-backup**.
133
+
-**Attributes**: You must use the same MTU across all network configurations of a custom cluster network. If you do not specify an MTU, the default value **1500** is used.
134
134
135
-
To simplify cluster maintenance, create one network configuration for each node or group of nodes. Without dedicated network configurations, certain maintenance tasks (for example, replacing old NICs with NICs in different slots) will require you to stop and/or migrate the affected VMs before updating the network configuration.
135
+

136
136
137
-
:::
137
+
1. Click **Save**.
138
138
139
139
### Change a Network Configuration
140
140
141
141
Changes to existing network configurations may affect Harvester virtual machines and workloads, and external devices such as switches and routers. For more information, see [Network Topology](./deep-dive.md#network-topology).
142
142
143
+
:::info important
144
+
145
+
You must stop all affected virtual machines before changing a network configuration.
146
+
147
+
:::
148
+
143
149
The following sections outline the steps you must perform to change the MTU of a network configuration. The sample cluster network used in these sections has `cn-data` that was built with a MTU value `1500` and is intended to be changed with value `9000`.
144
150
145
151

@@ -187,43 +193,59 @@ If you must change the MTU, perform the following steps:
187
193
188
194
1. Test the new MTU on Harvester nodes using commands such as `ping`. You must send the messages to a Harvester node with the new MTU or a node with an external IP.
189
195
190
-
Example:
191
-
192
-
Suppose a CIDR `192.168.100.0/24` and gateway `192.168.100.1` is prepared for the cn-data network.
196
+
In the following example, the network is `cn-data`, the CIDR is `192.168.100.0/24`, and the gateway is `192.168.100.1`.
193
197
194
-
1. Set an IP 192.168.100.100 on bridge device
198
+
1. Set the IP `192.168.100.100` on the bridge device.
195
199
200
+
```
196
201
$ ip addr add dev cn-data-br 192.168.100.100/24
202
+
```
197
203
198
-
1. Add a route for destination IP like `8.8.8.8` via the gateway
204
+
1. Add a route for the destination IP (for example, `8.8.8.8`) via the gateway.
199
205
206
+
```
200
207
$ ip route add 8.8.8.8 via 192.168.100.1 dev cn-data-br
208
+
```
201
209
202
-
1. ping 8.8.8.8 from the new IP 192.168.100.100
210
+
1. Ping the destination IP from the new IP `192.168.100.100`.
203
211
212
+
```
204
213
$ ping 8.8.8.8 -I 192.168.100.100
205
214
PING 8.8.8.8 (8.8.8.8) from 192.168.100.100 : 56(84) bytes of data.
206
215
64 bytes from 8.8.8.8: icmp_seq=1 ttl=59 time=8.52 ms
207
216
64 bytes from 8.8.8.8: icmp_seq=2 ttl=59 time=8.90 ms
208
217
...
218
+
```
209
219
210
-
1. ping with different size to validate new MTU
220
+
1. Ping the destination IP with a different packet size to validate the new MTU.
211
221
222
+
```
212
223
$ ping 8.8.8.8 -s 8800 -I 192.168.100.100
213
224
214
225
PING 8.8.8.8 (8.8.8.8) from 192.168.100.100 : 8800(8828) bytes of data
215
226
216
227
The param `-s` specify the ping packet size, which can test if the new MTU really works
228
+
```
217
229
218
-
1. Remove the added test route
230
+
1. Remove the route that you used for testing.
219
231
232
+
```
220
233
$ ip route delete 8.8.8.8 via 192.168.100.1 dev cn-data-br
234
+
```
221
235
222
-
1. Remove the added test ip
236
+
1. Remove the IP that you used for testing.
223
237
238
+
```
224
239
$ ip addr delete 192.168.100.100/24 dev cn-data-br
240
+
```
241
+
242
+
1. Add back the network configurations that you removed.
243
+
244
+
:::info important
225
245
226
-
1. Add back the network configurations that you removed, change the MTU in each one, and verify that the new MTU was applied.
246
+
You must change the MTU in each one, and verify that the new MTU was applied.
247
+
248
+
:::
227
249
228
250
1. Edit the YAML of all virtual machine networks that are attached to the target cluster network.
229
251
@@ -268,7 +290,7 @@ If you must change the MTU, perform the following steps:
268
290
269
291
1. Start all virtual machines that are attached to the target cluster network.
270
292
271
-
The virtual machines should have inherited the new MTU. You can verify this in the guest operating system using the Linux `ip link` command and `ping 8.8.8.8 -s 8800` command.
293
+
The virtual machines should have inherited the new MTU. You can verify this in the guest operating system using the commands `ip link` and `ping 8.8.8.8 -s 8800`.
272
294
273
295
1. Verify that the virtual machine workloads are running normally.
274
296
@@ -295,7 +317,7 @@ The storage network is used by `driver.longhorn.io`, which is Harvester's defaul
295
317
296
318
If you must change the MTU, perform the following steps:
297
319
298
-
1. Stop all the VMs.
320
+
1. Stop all virtual machines.
299
321
300
322
1. Disable the Harvester [Storage Network](../advanced/storagenetwork.md#harvester-storage-network-setting).
301
323
@@ -313,7 +335,9 @@ If you must change the MTU, perform the following steps:
313
335
314
336
:::
315
337
316
-
1. Verify that the MTU was changed using the Linux `ip link` command. If the network configuration selects multiple Harvester nodes, run the command on each node.
338
+
1. Verify that the MTU was changed using the Linux `ip link` command.
339
+
340
+
If the network configuration selects multiple Harvester nodes, run the command on each node.
317
341
318
342
The output must show the new MTU of the related `*-br` device and the state `UP`. In the following example, the device is `cn-data-br` and the new MTU is `9000`.
319
343
@@ -333,47 +357,65 @@ If you must change the MTU, perform the following steps:
333
357
334
358
1. Test the new MTU on Harvester nodes using commands such as `ping`. You must send the messages to a Harvester node with the new MTU or to a node with an external IP.
335
359
336
-
Example:
360
+
In the following example, the network is `cn-data`, the CIDR is `192.168.100.0/24`, and the gateway is `192.168.100.1`.
337
361
338
-
Suppose a CIDR `192.168.100.0/24` and gateway `192.168.100.1` is prepared for the cn-data network.
339
-
340
-
1. Set an IP 192.168.100.100 on bridge device
362
+
1. Set the IP `192.168.100.100` on the bridge device.
341
363
364
+
```
342
365
$ ip addr add dev cn-data-br 192.168.100.100/24
366
+
```
343
367
344
-
1. Add a route for destination IP like `8.8.8.8` via the gateway
368
+
1. Add a route for the destination IP (for example, `8.8.8.8`) via the gateway.
345
369
370
+
```
346
371
$ ip route add 8.8.8.8 via 192.168.100.1 dev cn-data-br
372
+
```
347
373
348
-
1. ping 8.8.8.8 from the new IP 192.168.100.100
374
+
1. Ping the destination IP from the new IP `192.168.100.100`.
349
375
376
+
```
350
377
$ ping 8.8.8.8 -I 192.168.100.100
351
378
PING 8.8.8.8 (8.8.8.8) from 192.168.100.100 : 56(84) bytes of data.
352
379
64 bytes from 8.8.8.8: icmp_seq=1 ttl=59 time=8.52 ms
353
380
64 bytes from 8.8.8.8: icmp_seq=2 ttl=59 time=8.90 ms
354
381
...
382
+
```
355
383
356
-
1. ping with different size to validate new MTU
384
+
1. Ping the destination IP with a different packet size to validate the new MTU.
357
385
386
+
```
358
387
$ ping 8.8.8.8 -s 8800 -I 192.168.100.100
359
388
360
389
PING 8.8.8.8 (8.8.8.8) from 192.168.100.100 : 8800(8828) bytes of data
361
390
362
391
The param `-s` specify the ping packet size, which can test if the new MTU really works
392
+
```
363
393
364
-
1. Remove the added test route
394
+
1. Remove the route that you used for testing.
365
395
396
+
```
366
397
$ ip route delete 8.8.8.8 via 192.168.100.1 dev cn-data-br
398
+
```
367
399
368
-
1. Remove the added test ip
400
+
1. Remove the IP that you used for testing.
369
401
402
+
```
370
403
$ ip addr delete 192.168.100.100/24 dev cn-data-br
404
+
```
405
+
406
+
1. Add back the network configurations that you removed.
371
407
372
-
1. Add back the network configurations that you removed, change the MTU in each one, and verify that the new MTU was applied.
408
+
:::info important
373
409
374
-
1. Enable and configure the Harvester [storage network setting](../advanced/storagenetwork.md#harvester-storage-network-setting), ensuring that the [prerequisites](../advanced/storagenetwork.md#prerequisites) are met.
410
+
You must change the MTU in each one, and verify that the new MTU was applied.
375
411
376
-
Allow some time for the setting to be disabled, and then [verify that the change was applied](../advanced/storagenetwork.md#verify-configuration-is-completed).
412
+
:::
413
+
414
+
1. Enable and configure the Harvester [storage network setting](../advanced/storagenetwork.md#harvester-storage-network-setting).
415
+
416
+
Ensure that the [prerequisites](../advanced/storagenetwork.md#prerequisites) are met.
417
+
418
+
1. Allow some time for the setting to be enabled, and then [verify that the change was applied](../advanced/storagenetwork.md#verify-configuration-is-completed).
377
419
378
420
1. Edit the YAML of all virtual machine networks that are attached to the target cluster network.
0 commit comments