You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/networking/clusternetwork.md
+27-6Lines changed: 27 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -80,18 +80,16 @@ When a Harvester cluster is deployed, a cluster network named `mgmt` is automati
80
80
81
81
`mgmt` does not require a network configuration and is always enabled on all hosts. You cannot disable and delete `mgmt`.
82
82
83
-
_Change in behaviour as of v1.6.0_
84
-
85
-
The [primary VLAN ID](https://docs.harvesterhci.io/latest/install/harvester-configuration#installmanagement_interface) provided during installation is automatically added to the `mgmt-br` bridge and the `mgmt-bo` interface. You can [add secondary VLAN interfaces](#add-secondary-vlan-interfaces) after installation is completed.
86
-
87
-
::: note
83
+
:::note
88
84
89
-
In Harvester v1.5.0 and earlier versions, the entire VLAN ID range (2 to 4094) was assigned to the `mgmt` interfaces. This prevented hardware VLAN offloading from functioning correctly on certain network cards.
85
+
In Harvester v1.5.x and earlier versions, the entire VLAN ID range (2 to 4094) was assigned to the `mgmt` interfaces. However, this exceeded the upper limit of supported VLANs on certain network cards, so hardware VLAN offloading stopped working correctly.
90
86
91
87
For more information, see [issue #7650](https://github.com/harvester/harvester/issues/7650).
92
88
93
89
:::
94
90
91
+
As of v1.6.0,only the [primary VLAN ID](https://docs.harvesterhci.io/latest/install/harvester-configuration#installmanagement_interface) provided during installation is automatically added to the `mgmt-br` bridge and the `mgmt-bo` interface. You can [add secondary VLAN interfaces](#add-secondary-vlan-interfaces) after installation is completed.
92
+
95
93
During installation of the first cluster node, you can configure the MTU value for `mgmt` using the [`install.management_interface`](../install/harvester-configuration.md#installmanagement_interface) setting. The default value of the `mtu` field is `1500`, which is what `mgmt` typically uses. However, if you specify an MTU value other than `0` or `1500`, you must [add a corresponding annotation](#annotate-a-non-default-mtu-value-to-mgmt-after-installation) after the cluster is deployed.
96
94
97
95
:::caution
@@ -102,6 +100,29 @@ During installation of the first cluster node, you can configure the MTU value f
102
100
103
101
:::
104
102
103
+
### Add Secondary VLAN Interfaces
104
+
105
+
Add the following commands to the `/oem/90_custom.yaml` file and reboot the node.
106
+
107
+
-`/etc/wicked/scripts/setup_bond.sh` section
108
+
109
+
```
110
+
bridge vlan add vid <vlan-id> dev $INTERFACE
111
+
```
112
+
113
+
- `/etc/wicked/scripts/setup_bridge.sh` section
114
+
115
+
```
116
+
bridge vlan add vid <vlan-id> dev $INTERFACE self
117
+
bridge vlan add vid <vlan-id> dev mgmt-bo
118
+
```
119
+
120
+
:::info important
121
+
122
+
You must include a separate command for each distinct VLAN ID. Ensure that the `vlan-id` placeholder is replaced with the actual ID.
123
+
124
+
:::
125
+
105
126
#### Annotate a Non-Default MTU Value to `mgmt` After Installation
Copy file name to clipboardExpand all lines: docs/upgrade/v1-5-x-to-v1-6-0.md
+13-14Lines changed: 13 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -154,9 +154,9 @@ The workaround is to replace the above contents generated from gopkg.in/yaml.v3
154
154
155
155
Related issue: [#9033](https://github.com/harvester/harvester/issues/9033)
156
156
157
-
### 5. Network connectivity issue on secondary vlans for mgmt interface
157
+
### 5. Network connectivity lost on secondary VLAN interfaces on the `mgmt` cluster network
158
158
159
-
A change introduced in v1.6.0to fix a hardware VLAN offloading issue unintentionally created a new issue. All secondary VLAN interfaces previously connected to the `mgmt-br` bridge and `mgmt-bo` are removed from the management hosts after the cluster is upgraded to v1.6.x.
159
+
In v1.6.0, a feature change was introduced to only attach required VLAN interfaces to mgmt-br and mgmt-bo, instead of all secondary VLANs. This is intended behavior to reduce unnecessary VLAN provisioning.Due to this all secondary VLAN interfaces previously attached to the `mgmt-br` bridge and `mgmt-bo` are removed from the management hosts after the cluster is upgraded to v1.6.x.
160
160
161
161
:::warning
162
162
@@ -166,15 +166,16 @@ For more information, see [issue #7650](https://github.com/harvester/harvester/i
166
166
167
167
:::
168
168
169
-
Before upgrading to v1.6.x, perform the following steps:
169
+
After upgrading to v1.6.x, perform the following steps:
170
170
171
-
1.Identify secondary VLAN interfaces by running the following command on management hosts:
171
+
1.Verify VLANs attached to the `mgmt-br` and `mgmt-bo` by running the following command on management hosts:
172
172
173
173
```
174
174
bridge vlan show
175
175
```
176
+
The above outputs only the primary vlan part of `mgmt-br` and `mgmt-bo`
176
177
177
-
1. Manually add the VLAN interfaces to the `mgmt-br` bridge and the `mgmt-bo` interface by adding the following commands to the `/oem/90_custom.yaml` file:
178
+
1. Manually add the required secondary VLANs to the `mgmt-br` bridge and the `mgmt-bo` interface by adding the following commands to the `/oem/90_custom.yaml` file:
178
179
179
180
- `/etc/wicked/scripts/setup_bond.sh` section
180
181
@@ -191,21 +192,19 @@ Before upgrading to v1.6.x, perform the following steps:
191
192
192
193
:::info important
193
194
194
-
You must include a separate command for each distinct VLAN ID. Ensure that the <vlan-id> placeholder is replaced with the actual ID.
195
+
You must include a separate command for each distinct VLAN ID. Ensure that the `vlan-id` placeholder is replaced with the actual ID.
195
196
196
197
:::
197
198
198
199
1. Once the `/oem/90_custom.yaml` file is updated, reboot the management hosts.
199
200
200
-
1. Verify that the VLAN interfaces were added by running the following command on the hosts:
201
+
1. Verify that all the required VLANs were added by running the following command on the hosts:
201
202
202
203
```
203
204
bridge vlan show
204
205
```
205
206
206
-
#### Upgrade Scenario
207
-
208
-
Starting with v1.6.0, only the primary VLAN ID is assigned to the `mgmt-br` bridge and the `mgmt-bo` interface, rather than the entire VLAN ID range (2–4094). Consequently, communications that use secondary VLANs on the `mgmt` interface are affected.
207
+
#### Upgrade Scenario Example
209
208
210
209
In the following example, a v1.5.x cluster was initially installed with a [primary VLAN interface](../install/harvester-configuration.md#installmanagement_interface) (VLAN ID: `2021`). To add a secondary VLAN interface (VLAN ID: `2113`), the `/oem/99_vlan-ifcfg.yaml` file was created on the management hosts with the following contents:
211
210
@@ -228,7 +227,7 @@ stages:
228
227
DEFROUTE='no'
229
228
230
229
```
231
-
Users expect to create an additional vlan subinterface on mgmt interface `mgmt-br.2113` and assign IPv4 address and use it for L3 connectivity in addition to the primary `mgmt-br.2021` interface after upgrade to v1.6.x.
232
-
The additional vlan sub interface is created, but `mgmt-br` bridge and `mgmt-bo` will no longer be part of vlan `2113` as after reboot only the primary `vlan-id` will be assigned to the `mgmt-br` bridge and `mgmt-bo` using `/oem/90_custom.yaml`.Starting from v1.6.0, only primary vlan will be added to `mgmt-br` bridge and `mgmt-bo` instead of all 2-4094 vlans.Due to this communication using secondary vlans on mgmt interface is affected.
233
-
To mitigate the effects of this change, you must perform the workaround described in the previous section. This involves identifying secondary VLAN interfaces and then adding the necessary ones to the `mgmt-br` bridge and the `mgmt-bo` interface.
234
-
```
230
+
The typical expectation is that an additional VLAN sub-interface is created on the `mgmt` interface (`mgmt-br.2113`) and assigned an IPv4 address. In addition, this sub-interface and the primary interface (`mgmt-br.2021`) are both expected to be used for L3 connectivity after the cluster is upgraded to v1.6.x.
231
+
232
+
In actuality after upgrade to v1.6.0, however, the VLAN sub-interface is created but the secondary VLAN (VLAN ID: `2113`) is removed from the `mgmt-br` bridge and the `mgmt-bo` interface. After a reboot, only the primary VLAN ID is assigned to the `mgmt-br` bridge and the `mgmt-bo` interface (using the `/oem/90_custom.yaml` file).
233
+
To mitigate the effects of this change, you must perform the workaround described in the previous section. This involves identifying secondary VLAN interfaces and then adding the necessary ones to the `mgmt-br` bridge and the `mgmt-bo` interface.
Copy file name to clipboardExpand all lines: versioned_docs/version-v1.6/networking/clusternetwork.md
+29-12Lines changed: 29 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -80,21 +80,15 @@ When a Harvester cluster is deployed, a cluster network named `mgmt` is automati
80
80
81
81
`mgmt` does not require a network configuration and is always enabled on all hosts. You cannot disable and delete `mgmt`.
82
82
83
-
_Change in behaviour as of v1.6.0_
83
+
:::note
84
84
85
-
Starting from v1.6.0, only primary vlan configured during installation will be added to `mgmt-br` bridge and `mgmt-bo` instead of all 2-4094 vlans.Users can check this info using `bridge vlan show` command on the host.
86
-
Any additional vlans required on the `mgmt-br` bridge and `mgmt-bo` must be explicitly configured using the following, and repeat below for every secondary `vlan-id` or vlan range the user requires.
85
+
In Harvester v1.5.x and earlier versions, the entire VLAN ID range (2 to 4094) was assigned to the `mgmt` interfaces. However, this exceeded the upper limit of supported VLANs on certain network cards, so hardware VLAN offloading stopped working correctly.
87
86
88
-
Add the following under contents of /etc/wicked/scripts/setup_bond.sh in /oem/90_custom.yaml
89
-
```
90
-
bridge vlan add vid <vlan-id> dev $INTERFACE
91
-
```
87
+
For more information, see [issue #7650](https://github.com/harvester/harvester/issues/7650).
92
88
93
-
Add the following under contents of /etc/wicked/scripts/setup_bridge.sh in /oem/90_custom.yaml
94
-
```
95
-
bridge vlan add vid <vlan-id> dev $INTERFACE self
96
-
bridge vlan add vid <vlan-id> dev mgmt-bo
97
-
```
89
+
:::
90
+
91
+
As of v1.6.0,only the [primary VLAN ID](https://docs.harvesterhci.io/latest/install/harvester-configuration#installmanagement_interface) provided during installation is automatically added to the `mgmt-br` bridge and the `mgmt-bo` interface. You can [add secondary VLAN interfaces](#add-secondary-vlan-interfaces) after installation is completed.
98
92
99
93
During installation of the first cluster node, you can configure the MTU value for `mgmt` using the [`install.management_interface`](../install/harvester-configuration.md#installmanagement_interface) setting. The default value of the `mtu` field is `1500`, which is what `mgmt` typically uses. However, if you specify an MTU value other than `0` or `1500`, you must [add a corresponding annotation](#annotate-a-non-default-mtu-value-to-mgmt-after-installation) after the cluster is deployed.
100
94
@@ -106,6 +100,29 @@ During installation of the first cluster node, you can configure the MTU value f
106
100
107
101
:::
108
102
103
+
### Add Secondary VLAN Interfaces
104
+
105
+
Add the following commands to the `/oem/90_custom.yaml` file and reboot the node.
106
+
107
+
-`/etc/wicked/scripts/setup_bond.sh` section
108
+
109
+
```
110
+
bridge vlan add vid <vlan-id> dev $INTERFACE
111
+
```
112
+
113
+
- `/etc/wicked/scripts/setup_bridge.sh` section
114
+
115
+
```
116
+
bridge vlan add vid <vlan-id> dev $INTERFACE self
117
+
bridge vlan add vid <vlan-id> dev mgmt-bo
118
+
```
119
+
120
+
:::info important
121
+
122
+
You must include a separate command for each distinct VLAN ID. Ensure that the `vlan-id` placeholder is replaced with the actual ID.
123
+
124
+
:::
125
+
109
126
#### Annotate a Non-Default MTU Value to `mgmt` After Installation
Copy file name to clipboardExpand all lines: versioned_docs/version-v1.6/upgrade/v1-5-x-to-v1-6-0.md
+54-18Lines changed: 54 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -154,11 +154,59 @@ The workaround is to replace the above contents generated from gopkg.in/yaml.v3
154
154
155
155
Related issue: [#9033](https://github.com/harvester/harvester/issues/9033)
156
156
157
-
### 5. Network connectivity issue on secondary vlans for mgmt interface
157
+
### 5. Network connectivity lost on secondary VLAN interfaces on the `mgmt` cluster network
158
158
159
-
If users configure secondary vlan interfaces for mgmt interface in addition to the primary vlan configured during installation, then after upgrade from v1.5.x to v1.6.x, the additional vlans will not be part of the `mgmt-br` bridge and `mgmt-bo`due to which network connectivity on those secondary vlans are affected.
159
+
In v1.6.0, a feature change was introduced to only attach required VLAN interfaces to mgmt-br and mgmt-bo, instead of all secondary VLANs. This is intended behavior to reduce unnecessary VLAN provisioning.Due to this all secondary VLAN interfaces previously attached to the `mgmt-br` bridge and `mgmt-bo`are removed from the management hosts after the cluster is upgraded to v1.6.x.
160
160
161
-
For example, consider user having a mgmt interface configured with vlan `2021` during installation and have the following under `/oem/99_vlan-ifcfg.yaml`
161
+
:::warning
162
+
163
+
Workloads that rely on these interfaces will lose network connectivity.
164
+
165
+
For more information, see [issue #7650](https://github.com/harvester/harvester/issues/7650).
166
+
167
+
:::
168
+
169
+
After upgrading to v1.6.x, perform the following steps:
170
+
171
+
1. Verify VLANs attached to the `mgmt-br` and `mgmt-bo` by running the following command on management hosts:
172
+
173
+
```
174
+
bridge vlan show
175
+
```
176
+
The above outputs only the primary vlan part of `mgmt-br` and `mgmt-bo`
177
+
178
+
1. Manually add the required secondary VLANs to the `mgmt-br` bridge and the `mgmt-bo` interface by adding the following commands to the `/oem/90_custom.yaml` file:
179
+
180
+
- `/etc/wicked/scripts/setup_bond.sh` section
181
+
182
+
```
183
+
bridge vlan add vid <vlan-id> dev $INTERFACE
184
+
```
185
+
186
+
- `/etc/wicked/scripts/setup_bridge.sh` section
187
+
188
+
```
189
+
bridge vlan add vid <vlan-id> dev $INTERFACE self
190
+
bridge vlan add vid <vlan-id> dev mgmt-bo
191
+
```
192
+
193
+
:::info important
194
+
195
+
You must include a separate command for each distinct VLAN ID. Ensure that the `vlan-id` placeholder is replaced with the actual ID.
196
+
197
+
:::
198
+
199
+
1. Once the `/oem/90_custom.yaml` file is updated, reboot the management hosts.
200
+
201
+
1. Verify that all the required VLANs were added by running the following command on the hosts:
202
+
203
+
```
204
+
bridge vlan show
205
+
```
206
+
207
+
#### Upgrade Scenario Example
208
+
209
+
In the following example, a v1.5.x cluster was initially installed with a [primary VLAN interface](../install/harvester-configuration.md#installmanagement_interface) (VLAN ID: `2021`). To add a secondary VLAN interface (VLAN ID: `2113`), the `/oem/99_vlan-ifcfg.yaml` file was created on the management hosts with the following contents:
162
210
163
211
```
164
212
stages:
@@ -179,19 +227,7 @@ stages:
179
227
DEFROUTE='no'
180
228
181
229
```
182
-
Users expect to create an additional vlan sub interface on mgmt interface `mgmt-br.2113` and assign IPv4 address and use it for L3 connectivity in addition to the primary `mgmt-br.2021` interface after upgrade to v1.6.x.
183
-
The additional vlan sub interface is created, but `mgmt-br` bridge and `mgmt-bo` will no longer be part of vlan `2113` as after reboot only the primary `vlan-id` will be assigned to the `mgmt-br` bridge and `mgmt-bo` using `/oem/90_custom.yaml`.Starting from v1.6.0, only primary vlan will be added to `mgmt-br` bridge and `mgmt-bo` instead of all 2-4094 vlans.Due to this communication using secondary vlans on mgmt interface is affected.
184
-
Users can check this info using `bridge vlan show` command on the host.
230
+
The typical expectation is that an additional VLAN sub-interface is created on the `mgmt` interface (`mgmt-br.2113`) and assigned an IPv4 address. In addition, this sub-interface and the primary interface (`mgmt-br.2021`) are both expected to be used for L3 connectivity after the cluster is upgraded to v1.6.x.
185
231
186
-
Starting from v1.6.0,Users have to explicitly add the secondary vlans required to the `mgmt-br` bridge and `mgmt-bo` using the following and repeat below for every secondary `vlan-id` or vlan range the user requires.
187
-
188
-
Add the following under contents of /etc/wicked/scripts/setup_bond.sh in /oem/90_custom.yaml
189
-
```
190
-
bridge vlan add vid <vlan-id> dev $INTERFACE
191
-
```
192
-
193
-
Add the following under contents of /etc/wicked/scripts/setup_bridge.sh in /oem/90_custom.yaml
194
-
```
195
-
bridge vlan add vid <vlan-id> dev $INTERFACE self
196
-
bridge vlan add vid <vlan-id> dev mgmt-bo
197
-
```
232
+
In actuality after upgrade to v1.6.0, however, the VLAN sub-interface is created but the secondary VLAN (VLAN ID: `2113`) is removed from the `mgmt-br` bridge and the `mgmt-bo` interface. After a reboot, only the primary VLAN ID is assigned to the `mgmt-br` bridge and the `mgmt-bo` interface (using the `/oem/90_custom.yaml` file).
233
+
To mitigate the effects of this change, you must perform the workaround described in the previous section. This involves identifying secondary VLAN interfaces and then adding the necessary ones to the `mgmt-br` bridge and the `mgmt-bo` interface.
0 commit comments