Skip to content

Commit f673015

Browse files
kquinn1204kquinn1204
committed
documenting OCPBUGS-6674
1 parent ee4b3ff commit f673015

File tree

2 files changed

+65
-47
lines changed

2 files changed

+65
-47
lines changed

‎modules/nw-configure-sysctl-interface-sriov-network-bonded.adoc

Lines changed: 23 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
:_content-type: PROCEDURE
66
[id="configuring-sysctl-on-bonded-sriov-network_{context}"]
7-
= Configuring sysctl on a bonded SR-IOV network
7+
= Configuring sysctl on a bonded SR-IOV network
88

99
You can set interface specific `sysctl` settings on a bonded interface created from two SR-IOV interfaces. Do this by adding the tuning configuration to the optional `Plugins` parameter of the bond network attachment definition.
1010

@@ -22,16 +22,16 @@ To change specific interface-level network `sysctl` settings create the `SriovNe
2222
2323
.Procedure
2424

25-
. Create the `SriovNetwork` custom resource (CR) for the bonded interface as in the following example CR. Save the YAML as the file `sriov-network-attachment.yaml`.
25+
. Create the `SriovNetwork` custom resource (CR) for the bonded interface as in the following example CR. Save the YAML as the file `sriov-network-attachment.yaml`.
2626
+
2727
[source,yaml]
2828
----
29-
apiVersion: sriovnetwork.openshift.io/v1
30-
kind: SriovNetwork
31-
metadata:
29+
apiVersion: sriovnetwork.openshift.io/v1
30+
kind: SriovNetwork
31+
metadata:
3232
name: allvalidflags <1>
3333
namespace: openshift-sriov-network-operator <2>
34-
spec:
34+
spec:
3535
resourceName: policyallflags <3>
3636
networkNamespace: sysctl-tuning-test <4>
3737
capabilities: '{ "mac": true, "ips": true }' <5>
@@ -65,21 +65,20 @@ spec:
6565
"plugins":[
6666
{
6767
"type":"bond", <1>
68-
"ifname":"bond0", <2>
69-
"mode": "active-backup", <3>
70-
"failOverMac": 1, <4>
71-
"linksInContainer": true, <5>
68+
"mode": "active-backup", <2>
69+
"failOverMac": 1, <3>
70+
"linksInContainer": true, <4>
7271
"miimon": "100",
73-
"links": [ <6>
72+
"links": [ <5>
7473
{"name": "net1"},
7574
{"name": "net2"}
7675
],
77-
"ipam":{ <7>
76+
"ipam":{ <6>
7877
"type":"static"
7978
}
8079
},
8180
{
82-
"type":"tuning", <8>
81+
"type":"tuning", <7>
8382
"capabilities":{
8483
"mac":true
8584
},
@@ -99,19 +98,18 @@ spec:
9998
}'
10099
----
101100
<1> The type is `bond`.
102-
<2> The `ifname` attribute specifies the name of the bond interface.
103-
<3> The `mode` attribute specifies the bonding mode. The bonding modes supported are:
101+
<2> The `mode` attribute specifies the bonding mode. The bonding modes supported are:
104102

105-
* `balance-rr` - 0
103+
* `balance-rr` - 0
106104
* `active-backup` - 1
107105
* `balance-xor` - 2
108106
+
109107
For `balance-rr` or `balance-xor` modes, you must set the `trust` mode to `on` for the SR-IOV virtual function.
110-
<4> The `failover` attribute is mandatory for active-backup mode.
111-
<5> The `linksInContainer=true` flag informs the Bond CNI that the interfaces required are to be found inside the container. By default Bond CNI looks for these interfaces on the host which does not work for integration with SRIOV and Multus.
112-
<6> The `links` section defines which interfaces will be used to create the bond. By default, Multus names the attached interfaces as: "net", plus a consecutive number, starting with one.
113-
<7> A configuration object for the IPAM CNI plugin as a YAML block scalar. The plugin manages IP address assignment for the attachment definition. In this pod example IP addresses are configured manually, so in this case `ipam` is set to static.
114-
<8> Add additional capabilities to the device. For example, set the `type` field to `tuning`. Specify the interface-level network `sysctl` you want to set in the sysctl field. This example sets all interface-level network `sysctl` settings that can be set.
108+
<3> The `failover` attribute is mandatory for active-backup mode.
109+
<4> The `linksInContainer=true` flag informs the Bond CNI that the required interfaces are to be found inside the container. By default, Bond CNI looks for these interfaces on the host which does not work for integration with SRIOV and Multus.
110+
<5> The `links` section defines which interfaces will be used to create the bond. By default, Multus names the attached interfaces as: "net", plus a consecutive number, starting with one.
111+
<6> A configuration object for the IPAM CNI plugin as a YAML block scalar. The plugin manages IP address assignment for the attachment definition. In this pod example IP addresses are configured manually, so in this case,`ipam` is set to static.
112+
<7> Add additional capabilities to the device. For example, set the `type` field to `tuning`. Specify the interface-level network `sysctl` you want to set in the sysctl field. This example sets all interface-level network `sysctl` settings that can be set.
115113

116114
. Create the bond network attachment resource:
117115
+
@@ -133,8 +131,8 @@ $ oc get network-attachment-definitions -n <namespace> <1>
133131
.Example output
134132
[source,terminal]
135133
----
136-
NAME AGE
137-
bond-sysctl-network 22m
134+
NAME AGE
135+
bond-sysctl-network 22m
138136
allvalidflags 47m
139137
----
140138
+
@@ -188,7 +186,7 @@ spec:
188186
<2> Optional: The MAC address for the SR-IOV device that is allocated from the resource type defined in the SR-IOV network attachment definition CR. To use this feature, you also must specify `{ "mac": true }` in the SriovNetwork object.
189187
<3> Optional: IP addresses for the SR-IOV device that are allocated from the resource type defined in the SR-IOV network attachment definition CR. Both IPv4 and IPv6 addresses are supported. To use this feature, you also must specify `{ "ips": true }` in the `SriovNetwork` object.
190188

191-
. Apply the YAML:
189+
. Apply the YAML:
192190
+
193191
[source,terminal]
194192
----
@@ -214,7 +212,7 @@ tunepod 1/1 Running 0 47s
214212
+
215213
[source,terminal]
216214
----
217-
$ oc rsh -n sysctl-tuning-test tunepod
215+
$ oc rsh -n sysctl-tuning-test tunepod
218216
----
219217

220218
. Verify the values of the configured `sysctl` flag. Find the value `net.ipv6.neigh.IFNAME.base_reachable_time_ms` by running the following command::

‎modules/nw-sriov-cfg-bond-interface-with-virtual-functions.adoc

Lines changed: 42 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -35,13 +35,12 @@ apiVersion: "k8s.cni.cncf.io/v1"
3535
"type": "bond", <1>
3636
"cniVersion": "0.3.1",
3737
"name": "bond-net1",
38-
"ifname": "bond0", <2>
39-
"mode": "active-backup", <3>
40-
"failOverMac": 1, <4>
41-
"linksInContainer": true, <5>
38+
"mode": "active-backup", <2>
39+
"failOverMac": 1, <3>
40+
"linksInContainer": true, <4>
4241
"miimon": "100",
4342
"mtu": 1500,
44-
"links": [ <6>
43+
"links": [ <5>
4544
{"name": "net1"},
4645
{"name": "net2"}
4746
],
@@ -55,9 +54,8 @@ apiVersion: "k8s.cni.cncf.io/v1"
5554
}
5655
}'
5756
----
58-
<1> The type is `bond`.
59-
<2> The `ifname` attribute specifies the name of the bond interface.
60-
<3> The `mode` attribute specifies the bonding mode.
57+
<1> The cni-type is always set to `bond`.
58+
<2> The `mode` attribute specifies the bonding mode.
6159
+
6260
[NOTE]
6361
====
@@ -69,16 +67,16 @@ The bonding modes supported are:
6967
7068
For `balance-rr` or `balance-xor` modes, you must set the `trust` mode to `on` for the SR-IOV virtual function.
7169
====
72-
<4> The `failover` attribute is mandatory for active-backup mode.
73-
<5> The `linksInContainer=true` flag informs the Bond CNI that the interfaces required are to be found inside the container. By default Bond CNI looks for these interfaces on the host which does not work for integration with SRIOV and Multus.
74-
<6> The `links` section defines which interfaces will be used to create the bond. By default, Multus names the attached interfaces as: "net", plus a consecutive number, starting with one.
70+
<3> The `failover` attribute is mandatory for active-backup mode and must be set to 1.
71+
<4> The `linksInContainer=true` flag informs the Bond CNI that the required interfaces are to be found inside the container. By default, Bond CNI looks for these interfaces on the host which does not work for integration with SRIOV and Multus.
72+
<5> The `links` section defines which interfaces will be used to create the bond. By default, Multus names the attached interfaces as: "net", plus a consecutive number, starting with one.
7573

7674
[id="nw-sriov-cfg-creating-pod-using-interface_{context}"]
7775
== Creating a pod using a bond interface
7876

79-
You can now test the setup by creating a pod using a bond interface.
80-
81-
[source,yaml,subs="attributes+"]
77+
. Test the setup by creating a pod with a YAML file named for example `podbonding.yaml` with content similar to the following:
78+
+
79+
[source,yaml]
8280
----
8381
apiVersion: v1
8482
kind: Pod
@@ -90,12 +88,20 @@ apiVersion: v1
9088
spec:
9189
containers:
9290
- name: podexample
93-
image: quay.io/openshift/origin-network-interface-bond-cni:{product-version}.0
91+
image: quay.io/openshift/origin-network-interface-bond-cni:4.11.0
9492
command: ["/bin/bash", "-c", "sleep INF"]
9593
----
9694
<1> Note the network annotation: it contains two SR-IOV network attachments, and one bond network attachment. The bond attachment uses the two SR-IOV interfaces as bonded port interfaces.
9795

98-
You can inspect the pod interfaces with the following command:
96+
. Apply the yaml by running the following command:
97+
+
98+
[source,terminal]
99+
----
100+
$ oc apply -f podbonding.yaml
101+
----
102+
103+
. Inspect the pod interfaces with the following command:
104+
+
99105
[source,yaml]
100106
----
101107
$ oc rsh -n demo bondpod1
@@ -109,14 +115,28 @@ valid_lft forever preferred_lft forever
109115
link/ether 62:b1:b5:c8:fb:7a brd ff:ff:ff:ff:ff:ff
110116
inet 10.244.1.122/24 brd 10.244.1.255 scope global eth0
111117
valid_lft forever preferred_lft forever
112-
4: bond0: <BROADCAST,MULTICAST,UP,LOWER_UP400> mtu 1500 qdisc noqueue state UP qlen 1000
113-
link/ether 9e:23:69:42:fb:8a brd ff:ff:ff:ff:ff:ff
118+
4: net3: <BROADCAST,MULTICAST,UP,LOWER_UP400> mtu 1500 qdisc noqueue state UP qlen 1000
119+
link/ether 9e:23:69:42:fb:8a brd ff:ff:ff:ff:ff:ff <1>
114120
inet 10.56.217.66/24 scope global bond0
115121
valid_lft forever preferred_lft forever
116122
43: net1: <BROADCAST,MULTICAST,UP,LOWER_UP800> mtu 1500 qdisc mq master bond0 state UP qlen 1000
117-
link/ether 9e:23:69:42:fb:8a brd ff:ff:ff:ff:ff:ff <1>
118-
44: net2: <BROADCAST,MULTICAST,UP,LOWER_UP800> mtu 1500 qdisc mq master bond0 state UP qlen 1000
119123
link/ether 9e:23:69:42:fb:8a brd ff:ff:ff:ff:ff:ff <2>
124+
44: net2: <BROADCAST,MULTICAST,UP,LOWER_UP800> mtu 1500 qdisc mq master bond0 state UP qlen 1000
125+
link/ether 9e:23:69:42:fb:8a brd ff:ff:ff:ff:ff:ff <3>
126+
----
127+
<1> The bond interface is automatically named `net3`. To set a specific interface name add `@name` suffix to the pod’s `k8s.v1.cni.cncf.io/networks` annotation.
128+
<2> The `net1` interface is based on an SR-IOV virtual function.
129+
<3> The `net2` interface is based on an SR-IOV virtual function.
130+
+
131+
[NOTE]
132+
====
133+
If no interface names are configured in the pod annotation, interface names are assigned automatically as `net<n>`, with `<n>` starting at `1`.
134+
====
135+
136+
. Optional: If you want to set a specific interface name for example `bond0`, edit the `k8s.v1.cni.cncf.io/networks` annotation and set `bond0` as the interface name as follows:
137+
+
138+
[source,terminal]
120139
----
121-
<1> The `net1` interface is based on an SR-IOV virtual function.
122-
<2> The `net2` interface is based on an SR-IOV virtual function.
140+
annotations:
141+
k8s.v1.cni.cncf.io/networks: demo/sriovnet1, demo/sriovnet2, demo/bond-net1@bond0
142+
----

0 commit comments

Comments
 (0)