Skip to content

Commit 373bf63

Browse files
Update docs/upgrade/v1-5-x-to-v1-6-0.md
Co-authored-by: Jillian Maroket <[email protected]> Signed-off-by: Renuka Devi Rajendran <[email protected]>
1 parent e129e5b commit 373bf63

File tree

4 files changed

+123
-50
lines changed

4 files changed

+123
-50
lines changed

docs/networking/clusternetwork.md

Lines changed: 27 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -80,18 +80,16 @@ When a Harvester cluster is deployed, a cluster network named `mgmt` is automati
8080

8181
`mgmt` does not require a network configuration and is always enabled on all hosts. You cannot disable and delete `mgmt`.
8282

83-
_Change in behaviour as of v1.6.0_
84-
85-
The [primary VLAN ID](https://docs.harvesterhci.io/latest/install/harvester-configuration#installmanagement_interface) provided during installation is automatically added to the `mgmt-br` bridge and the `mgmt-bo` interface. You can [add secondary VLAN interfaces](#add-secondary-vlan-interfaces) after installation is completed.
86-
87-
::: note
83+
:::note
8884

89-
In Harvester v1.5.0 and earlier versions, the entire VLAN ID range (2 to 4094) was assigned to the `mgmt` interfaces. This prevented hardware VLAN offloading from functioning correctly on certain network cards.
85+
In Harvester v1.5.x and earlier versions, the entire VLAN ID range (2 to 4094) was assigned to the `mgmt` interfaces. However, this exceeded the upper limit of supported VLANs on certain network cards, so hardware VLAN offloading stopped working correctly.
9086

9187
For more information, see [issue #7650](https://github.com/harvester/harvester/issues/7650).
9288

9389
:::
9490

91+
As of v1.6.0,only the [primary VLAN ID](https://docs.harvesterhci.io/latest/install/harvester-configuration#installmanagement_interface) provided during installation is automatically added to the `mgmt-br` bridge and the `mgmt-bo` interface. You can [add secondary VLAN interfaces](#add-secondary-vlan-interfaces) after installation is completed.
92+
9593
During installation of the first cluster node, you can configure the MTU value for `mgmt` using the [`install.management_interface`](../install/harvester-configuration.md#installmanagement_interface) setting. The default value of the `mtu` field is `1500`, which is what `mgmt` typically uses. However, if you specify an MTU value other than `0` or `1500`, you must [add a corresponding annotation](#annotate-a-non-default-mtu-value-to-mgmt-after-installation) after the cluster is deployed.
9694

9795
:::caution
@@ -102,6 +100,29 @@ During installation of the first cluster node, you can configure the MTU value f
102100

103101
:::
104102

103+
### Add Secondary VLAN Interfaces
104+
105+
Add the following commands to the `/oem/90_custom.yaml` file and reboot the node.
106+
107+
- `/etc/wicked/scripts/setup_bond.sh` section
108+
109+
```
110+
bridge vlan add vid <vlan-id> dev $INTERFACE
111+
```
112+
113+
- `/etc/wicked/scripts/setup_bridge.sh` section
114+
115+
```
116+
bridge vlan add vid <vlan-id> dev $INTERFACE self
117+
bridge vlan add vid <vlan-id> dev mgmt-bo
118+
```
119+
120+
:::info important
121+
122+
You must include a separate command for each distinct VLAN ID. Ensure that the `vlan-id` placeholder is replaced with the actual ID.
123+
124+
:::
125+
105126
#### Annotate a Non-Default MTU Value to `mgmt` After Installation
106127
107128
_Available as of v1.6.0_

docs/upgrade/v1-5-x-to-v1-6-0.md

Lines changed: 13 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -154,9 +154,9 @@ The workaround is to replace the above contents generated from gopkg.in/yaml.v3
154154

155155
Related issue: [#9033](https://github.com/harvester/harvester/issues/9033)
156156

157-
### 5. Network connectivity issue on secondary vlans for mgmt interface
157+
### 5. Network connectivity lost on secondary VLAN interfaces on the `mgmt` cluster network
158158

159-
A change introduced in v1.6.0 to fix a hardware VLAN offloading issue unintentionally created a new issue. All secondary VLAN interfaces previously connected to the `mgmt-br` bridge and `mgmt-bo` are removed from the management hosts after the cluster is upgraded to v1.6.x.
159+
In v1.6.0, a feature change was introduced to only attach required VLAN interfaces to mgmt-br and mgmt-bo, instead of all secondary VLANs. This is intended behavior to reduce unnecessary VLAN provisioning.Due to this all secondary VLAN interfaces previously attached to the `mgmt-br` bridge and `mgmt-bo` are removed from the management hosts after the cluster is upgraded to v1.6.x.
160160

161161
:::warning
162162

@@ -166,15 +166,16 @@ For more information, see [issue #7650](https://github.com/harvester/harvester/i
166166

167167
:::
168168

169-
Before upgrading to v1.6.x, perform the following steps:
169+
After upgrading to v1.6.x, perform the following steps:
170170

171-
1. Identify secondary VLAN interfaces by running the following command on management hosts:
171+
1. Verify VLANs attached to the `mgmt-br` and `mgmt-bo` by running the following command on management hosts:
172172

173173
```
174174
bridge vlan show
175175
```
176+
The above outputs only the primary vlan part of `mgmt-br` and `mgmt-bo`
176177
177-
1. Manually add the VLAN interfaces to the `mgmt-br` bridge and the `mgmt-bo` interface by adding the following commands to the `/oem/90_custom.yaml` file:
178+
1. Manually add the required secondary VLANs to the `mgmt-br` bridge and the `mgmt-bo` interface by adding the following commands to the `/oem/90_custom.yaml` file:
178179
179180
- `/etc/wicked/scripts/setup_bond.sh` section
180181
@@ -191,21 +192,19 @@ Before upgrading to v1.6.x, perform the following steps:
191192
192193
:::info important
193194
194-
You must include a separate command for each distinct VLAN ID. Ensure that the <vlan-id> placeholder is replaced with the actual ID.
195+
You must include a separate command for each distinct VLAN ID. Ensure that the `vlan-id` placeholder is replaced with the actual ID.
195196
196197
:::
197198
198199
1. Once the `/oem/90_custom.yaml` file is updated, reboot the management hosts.
199200
200-
1. Verify that the VLAN interfaces were added by running the following command on the hosts:
201+
1. Verify that all the required VLANs were added by running the following command on the hosts:
201202
202203
```
203204
bridge vlan show
204205
```
205206
206-
#### Upgrade Scenario
207-
208-
Starting with v1.6.0, only the primary VLAN ID is assigned to the `mgmt-br` bridge and the `mgmt-bo` interface, rather than the entire VLAN ID range (2–4094). Consequently, communications that use secondary VLANs on the `mgmt` interface are affected.
207+
#### Upgrade Scenario Example
209208
210209
In the following example, a v1.5.x cluster was initially installed with a [primary VLAN interface](../install/harvester-configuration.md#installmanagement_interface) (VLAN ID: `2021`). To add a secondary VLAN interface (VLAN ID: `2113`), the `/oem/99_vlan-ifcfg.yaml` file was created on the management hosts with the following contents:
211210
@@ -228,7 +227,7 @@ stages:
228227
DEFROUTE='no'
229228

230229
```
231-
Users expect to create an additional vlan sub interface on mgmt interface `mgmt-br.2113` and assign IPv4 address and use it for L3 connectivity in addition to the primary `mgmt-br.2021` interface after upgrade to v1.6.x.
232-
The additional vlan sub interface is created, but `mgmt-br` bridge and `mgmt-bo` will no longer be part of vlan `2113` as after reboot only the primary `vlan-id` will be assigned to the `mgmt-br` bridge and `mgmt-bo` using `/oem/90_custom.yaml`.Starting from v1.6.0, only primary vlan will be added to `mgmt-br` bridge and `mgmt-bo` instead of all 2-4094 vlans.Due to this communication using secondary vlans on mgmt interface is affected.
233-
To mitigate the effects of this change, you must perform the workaround described in the previous section. This involves identifying secondary VLAN interfaces and then adding the necessary ones to the `mgmt-br` bridge and the `mgmt-bo` interface.
234-
```
230+
The typical expectation is that an additional VLAN sub-interface is created on the `mgmt` interface (`mgmt-br.2113`) and assigned an IPv4 address. In addition, this sub-interface and the primary interface (`mgmt-br.2021`) are both expected to be used for L3 connectivity after the cluster is upgraded to v1.6.x.
231+
232+
In actuality after upgrade to v1.6.0, however, the VLAN sub-interface is created but the secondary VLAN (VLAN ID: `2113`) is removed from the `mgmt-br` bridge and the `mgmt-bo` interface. After a reboot, only the primary VLAN ID is assigned to the `mgmt-br` bridge and the `mgmt-bo` interface (using the `/oem/90_custom.yaml` file).
233+
To mitigate the effects of this change, you must perform the workaround described in the previous section. This involves identifying secondary VLAN interfaces and then adding the necessary ones to the `mgmt-br` bridge and the `mgmt-bo` interface.

versioned_docs/version-v1.6/networking/clusternetwork.md

Lines changed: 29 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -80,21 +80,15 @@ When a Harvester cluster is deployed, a cluster network named `mgmt` is automati
8080

8181
`mgmt` does not require a network configuration and is always enabled on all hosts. You cannot disable and delete `mgmt`.
8282

83-
_Change in behaviour as of v1.6.0_
83+
:::note
8484

85-
Starting from v1.6.0, only primary vlan configured during installation will be added to `mgmt-br` bridge and `mgmt-bo` instead of all 2-4094 vlans.Users can check this info using `bridge vlan show` command on the host.
86-
Any additional vlans required on the `mgmt-br` bridge and `mgmt-bo` must be explicitly configured using the following, and repeat below for every secondary `vlan-id` or vlan range the user requires.
85+
In Harvester v1.5.x and earlier versions, the entire VLAN ID range (2 to 4094) was assigned to the `mgmt` interfaces. However, this exceeded the upper limit of supported VLANs on certain network cards, so hardware VLAN offloading stopped working correctly.
8786

88-
Add the following under contents of /etc/wicked/scripts/setup_bond.sh in /oem/90_custom.yaml
89-
```
90-
bridge vlan add vid <vlan-id> dev $INTERFACE
91-
```
87+
For more information, see [issue #7650](https://github.com/harvester/harvester/issues/7650).
9288

93-
Add the following under contents of /etc/wicked/scripts/setup_bridge.sh in /oem/90_custom.yaml
94-
```
95-
bridge vlan add vid <vlan-id> dev $INTERFACE self
96-
bridge vlan add vid <vlan-id> dev mgmt-bo
97-
```
89+
:::
90+
91+
As of v1.6.0,only the [primary VLAN ID](https://docs.harvesterhci.io/latest/install/harvester-configuration#installmanagement_interface) provided during installation is automatically added to the `mgmt-br` bridge and the `mgmt-bo` interface. You can [add secondary VLAN interfaces](#add-secondary-vlan-interfaces) after installation is completed.
9892

9993
During installation of the first cluster node, you can configure the MTU value for `mgmt` using the [`install.management_interface`](../install/harvester-configuration.md#installmanagement_interface) setting. The default value of the `mtu` field is `1500`, which is what `mgmt` typically uses. However, if you specify an MTU value other than `0` or `1500`, you must [add a corresponding annotation](#annotate-a-non-default-mtu-value-to-mgmt-after-installation) after the cluster is deployed.
10094

@@ -106,6 +100,29 @@ During installation of the first cluster node, you can configure the MTU value f
106100

107101
:::
108102

103+
### Add Secondary VLAN Interfaces
104+
105+
Add the following commands to the `/oem/90_custom.yaml` file and reboot the node.
106+
107+
- `/etc/wicked/scripts/setup_bond.sh` section
108+
109+
```
110+
bridge vlan add vid <vlan-id> dev $INTERFACE
111+
```
112+
113+
- `/etc/wicked/scripts/setup_bridge.sh` section
114+
115+
```
116+
bridge vlan add vid <vlan-id> dev $INTERFACE self
117+
bridge vlan add vid <vlan-id> dev mgmt-bo
118+
```
119+
120+
:::info important
121+
122+
You must include a separate command for each distinct VLAN ID. Ensure that the `vlan-id` placeholder is replaced with the actual ID.
123+
124+
:::
125+
109126
#### Annotate a Non-Default MTU Value to `mgmt` After Installation
110127
111128
_Available as of v1.6.0_

versioned_docs/version-v1.6/upgrade/v1-5-x-to-v1-6-0.md

Lines changed: 54 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -154,11 +154,59 @@ The workaround is to replace the above contents generated from gopkg.in/yaml.v3
154154

155155
Related issue: [#9033](https://github.com/harvester/harvester/issues/9033)
156156

157-
### 5. Network connectivity issue on secondary vlans for mgmt interface
157+
### 5. Network connectivity lost on secondary VLAN interfaces on the `mgmt` cluster network
158158

159-
If users configure secondary vlan interfaces for mgmt interface in addition to the primary vlan configured during installation, then after upgrade from v1.5.x to v1.6.x, the additional vlans will not be part of the `mgmt-br` bridge and `mgmt-bo` due to which network connectivity on those secondary vlans are affected.
159+
In v1.6.0, a feature change was introduced to only attach required VLAN interfaces to mgmt-br and mgmt-bo, instead of all secondary VLANs. This is intended behavior to reduce unnecessary VLAN provisioning.Due to this all secondary VLAN interfaces previously attached to the `mgmt-br` bridge and `mgmt-bo` are removed from the management hosts after the cluster is upgraded to v1.6.x.
160160

161-
For example, consider user having a mgmt interface configured with vlan `2021` during installation and have the following under `/oem/99_vlan-ifcfg.yaml`
161+
:::warning
162+
163+
Workloads that rely on these interfaces will lose network connectivity.
164+
165+
For more information, see [issue #7650](https://github.com/harvester/harvester/issues/7650).
166+
167+
:::
168+
169+
After upgrading to v1.6.x, perform the following steps:
170+
171+
1. Verify VLANs attached to the `mgmt-br` and `mgmt-bo` by running the following command on management hosts:
172+
173+
```
174+
bridge vlan show
175+
```
176+
The above outputs only the primary vlan part of `mgmt-br` and `mgmt-bo`
177+
178+
1. Manually add the required secondary VLANs to the `mgmt-br` bridge and the `mgmt-bo` interface by adding the following commands to the `/oem/90_custom.yaml` file:
179+
180+
- `/etc/wicked/scripts/setup_bond.sh` section
181+
182+
```
183+
bridge vlan add vid <vlan-id> dev $INTERFACE
184+
```
185+
186+
- `/etc/wicked/scripts/setup_bridge.sh` section
187+
188+
```
189+
bridge vlan add vid <vlan-id> dev $INTERFACE self
190+
bridge vlan add vid <vlan-id> dev mgmt-bo
191+
```
192+
193+
:::info important
194+
195+
You must include a separate command for each distinct VLAN ID. Ensure that the `vlan-id` placeholder is replaced with the actual ID.
196+
197+
:::
198+
199+
1. Once the `/oem/90_custom.yaml` file is updated, reboot the management hosts.
200+
201+
1. Verify that all the required VLANs were added by running the following command on the hosts:
202+
203+
```
204+
bridge vlan show
205+
```
206+
207+
#### Upgrade Scenario Example
208+
209+
In the following example, a v1.5.x cluster was initially installed with a [primary VLAN interface](../install/harvester-configuration.md#installmanagement_interface) (VLAN ID: `2021`). To add a secondary VLAN interface (VLAN ID: `2113`), the `/oem/99_vlan-ifcfg.yaml` file was created on the management hosts with the following contents:
162210
163211
```
164212
stages:
@@ -179,19 +227,7 @@ stages:
179227
DEFROUTE='no'
180228

181229
```
182-
Users expect to create an additional vlan sub interface on mgmt interface `mgmt-br.2113` and assign IPv4 address and use it for L3 connectivity in addition to the primary `mgmt-br.2021` interface after upgrade to v1.6.x.
183-
The additional vlan sub interface is created, but `mgmt-br` bridge and `mgmt-bo` will no longer be part of vlan `2113` as after reboot only the primary `vlan-id` will be assigned to the `mgmt-br` bridge and `mgmt-bo` using `/oem/90_custom.yaml`.Starting from v1.6.0, only primary vlan will be added to `mgmt-br` bridge and `mgmt-bo` instead of all 2-4094 vlans.Due to this communication using secondary vlans on mgmt interface is affected.
184-
Users can check this info using `bridge vlan show` command on the host.
230+
The typical expectation is that an additional VLAN sub-interface is created on the `mgmt` interface (`mgmt-br.2113`) and assigned an IPv4 address. In addition, this sub-interface and the primary interface (`mgmt-br.2021`) are both expected to be used for L3 connectivity after the cluster is upgraded to v1.6.x.
185231
186-
Starting from v1.6.0,Users have to explicitly add the secondary vlans required to the `mgmt-br` bridge and `mgmt-bo` using the following and repeat below for every secondary `vlan-id` or vlan range the user requires.
187-
188-
Add the following under contents of /etc/wicked/scripts/setup_bond.sh in /oem/90_custom.yaml
189-
```
190-
bridge vlan add vid <vlan-id> dev $INTERFACE
191-
```
192-
193-
Add the following under contents of /etc/wicked/scripts/setup_bridge.sh in /oem/90_custom.yaml
194-
```
195-
bridge vlan add vid <vlan-id> dev $INTERFACE self
196-
bridge vlan add vid <vlan-id> dev mgmt-bo
197-
```
232+
In actuality after upgrade to v1.6.0, however, the VLAN sub-interface is created but the secondary VLAN (VLAN ID: `2113`) is removed from the `mgmt-br` bridge and the `mgmt-bo` interface. After a reboot, only the primary VLAN ID is assigned to the `mgmt-br` bridge and the `mgmt-bo` interface (using the `/oem/90_custom.yaml` file).
233+
To mitigate the effects of this change, you must perform the workaround described in the previous section. This involves identifying secondary VLAN interfaces and then adding the necessary ones to the `mgmt-br` bridge and the `mgmt-bo` interface.

0 commit comments

Comments
 (0)