Skip to content

Commit b8bc5b2

Browse files
kquinn1204kquinn1204
committed
TELCODOCS-806-1 PR for setting interface sysctl with SR-IOV
1 parent 06cdc73 commit b8bc5b2

9 files changed

+648
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1101,6 +1101,8 @@ Topics:
11011101
File: configuring-sriov-ib-attach
11021102
- Name: Adding a pod to an SR-IOV network
11031103
File: add-pod
1104+
- Name: Tuning sysctl settings on an SR-IOV network
1105+
File: configuring-interface-sysctl-sriov-device
11041106
- Name: Using high performance multicast
11051107
File: using-sriov-multicast
11061108
- Name: Using DPDK and RDMA

‎modules/nw-configure-sysctl-interface-sriov-network-bonded.adoc

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
// Module included in the following assemblies:
2+
//
3+
//networking/hardware_networks/configuring-sriov-device.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="configuring-sysctl-on-bonded-sriov-network_{context}"]
7+
= Configuring sysctl on a bonded SR-IOV network
8+
9+
You can set interface specific `sysctl` settings on a bonded interface created from two SR-IOV interfaces. Do this by adding the tuning configuration to the optional `Plugins` parameter of the bond network attachment definition.
10+
11+
[NOTE]
12+
====
13+
Do not edit `NetworkAttachmentDefinition` custom resources that the SR-IOV Network Operator manages. Doing so might disrupt network traffic on your additional network.
14+
====
15+
16+
To change specific interface-level network `sysctl` settings create the `SriovNetwork` custom resource (CR) with the Container Network Interface (CNI) tuning plug-in by using the following procedure.
17+
18+
.Prerequisites
19+
20+
* Install the {product-title} CLI (oc).
21+
* Log in to the {product-title} cluster as a user with cluster-admin privileges.
22+
23+
.Procedure
24+
25+
. Create the `SriovNetwork` custom resource (CR) for the bonded interface as in the following example CR. Save the YAML as the file `sriov-network-attachment.yaml`.
26+
+
27+
[source,yaml]
28+
----
29+
apiVersion: sriovnetwork.openshift.io/v1
30+
kind: SriovNetwork
31+
metadata:
32+
name: allvalidflags <1>
33+
namespace: openshift-sriov-network-operator <2>
34+
spec:
35+
resourceName: policyallflags <3>
36+
networkNamespace: sysctl-tuning-test <4>
37+
capabilities: '{ "mac": true, "ips": true }' <5>
38+
----
39+
<1> A name for the object. The SR-IOV Network Operator creates a NetworkAttachmentDefinition object with same name.
40+
<2> The namespace where the SR-IOV Network Operator is installed.
41+
<3> The value for the `spec.resourceName` parameter from the `SriovNetworkNodePolicy` object that defines the SR-IOV hardware for this additional network.
42+
<4> The target namespace for the `SriovNetwork` object. Only pods in the target namespace can attach to the additional network.
43+
<5> Optional: The capabilities to configure for this additional network. You can specify `"{ "ips": true }"` to enable IP address support or `"{ "mac": true }"` to enable MAC address support.
44+
45+
. Create the `SriovNetwork` resource:
46+
+
47+
[source,terminal]
48+
----
49+
$ oc create -f sriov-network-attachment.yaml
50+
----
51+
52+
. Create a bond network attachment definition as in the following example CR. Save the YAML as the file `sriov-bond-network-interface.yaml`.
53+
+
54+
[source,yaml]
55+
----
56+
apiVersion: "k8s.cni.cncf.io/v1"
57+
kind: NetworkAttachmentDefinition
58+
metadata:
59+
name: bond-sysctl-network
60+
namespace: sysctl-tuning-test
61+
spec:
62+
config: '{
63+
"cniVersion":"0.4.0",
64+
"name":"bound-net",
65+
"plugins":[
66+
{
67+
"type":"bond", <1>
68+
"ifname":"bond0", <2>
69+
"mode": "active-backup", <3>
70+
"failOverMac": 1, <4>
71+
"linksInContainer": true, <5>
72+
"miimon": "100",
73+
"links": [ <6>
74+
{"name": "net1"},
75+
{"name": "net2"}
76+
],
77+
"ipam":{ <7>
78+
"type":"static"
79+
}
80+
},
81+
{
82+
"type":"tuning", <8>
83+
"capabilities":{
84+
"mac":true
85+
},
86+
"sysctl":{
87+
"net.ipv4.conf.IFNAME.accept_redirects": "0",
88+
"net.ipv4.conf.IFNAME.accept_source_route": "0",
89+
"net.ipv4.conf.IFNAME.disable_policy": "1",
90+
"net.ipv4.conf.IFNAME.secure_redirects": "0",
91+
"net.ipv4.conf.IFNAME.send_redirects": "0",
92+
"net.ipv6.conf.IFNAME.accept_redirects": "0",
93+
"net.ipv6.conf.IFNAME.accept_source_route": "1",
94+
"net.ipv6.neigh.IFNAME.base_reachable_time_ms": "20000",
95+
"net.ipv6.neigh.IFNAME.retrans_time_ms": "2000"
96+
}
97+
}
98+
]
99+
}'
100+
----
101+
<1> The type is `bond`.
102+
<2> The `ifname` attribute specifies the name of the bond interface.
103+
<3> The `mode` attribute specifies the bonding mode. The bonding modes supported are:
104+
105+
* `balance-rr` - 0
106+
* `active-backup` - 1
107+
* `balance-xor` - 2
108+
+
109+
For `balance-rr` or `balance-xor` modes, you must set the `trust` mode to `on` for the SR-IOV virtual function.
110+
<4> The `failover` attribute is mandatory for active-backup mode.
111+
<5> The `linksInContainer=true` flag informs the Bond CNI that the interfaces required are to be found inside the container. By default Bond CNI looks for these interfaces on the host which does not work for integration with SRIOV and Multus.
112+
<6> The `links` section defines which interfaces will be used to create the bond. By default, Multus names the attached interfaces as: "net", plus a consecutive number, starting with one.
113+
<7> A configuration object for the IPAM CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition. In this pod example IP addresses are configured manually, so in this case `ipam` is set to static.
114+
<8> Add additional capabilities to the device. For example, set the `type` field to `tuning`. Specify the interface-level network `sysctl` you want to set in the sysctl field. This example sets all interface-level network `sysctl` settings that can be set.
115+
116+
. Create the bond network attachment resource:
117+
+
118+
[source,terminal]
119+
----
120+
$ oc create -f sriov-bond-network-interface.yaml
121+
----
122+
123+
.Verifying that the `NetworkAttachmentDefinition` CR is successfully created
124+
125+
* Confirm that the SR-IOV Network Operator created the `NetworkAttachmentDefinition` CR by running the following command:
126+
+
127+
[source,terminal]
128+
----
129+
$ oc get network-attachment-definitions -n <namespace> <1>
130+
----
131+
<1> Replace `<namespace>` with the networkNamespace that you specified when configuring the network attachment, for example, `sysctl-tuning-test`.
132+
+
133+
.Example output
134+
[source,terminal]
135+
----
136+
NAME AGE
137+
bond-sysctl-network 22m
138+
allvalidflags 47m
139+
----
140+
+
141+
[NOTE]
142+
====
143+
There might be a delay before the SR-IOV Network Operator creates the CR.
144+
====
145+
146+
.Verifying that the additional SR-IOV network resource is successful
147+
148+
To verify that the tuning CNI is correctly configured and the additional SR-IOV network attachment is attached, do the following:
149+
150+
. Create a `Pod` CR. For example, save the following YAML as the file `examplepod.yaml`:
151+
+
152+
[source,yaml]
153+
----
154+
apiVersion: v1
155+
kind: Pod
156+
metadata:
157+
name: tunepod
158+
namespace: sysctl-tuning-test
159+
annotations:
160+
k8s.v1.cni.cncf.io/networks: |-
161+
[
162+
{"name": "allvalidflags"}, <1>
163+
{"name": "allvalidflags"},
164+
{
165+
"name": "bond-sysctl-network",
166+
"interface": "bond0",
167+
"mac": "0a:56:0a:83:04:0c", <2>
168+
"ips": ["10.100.100.200/24"] <3>
169+
}
170+
]
171+
spec:
172+
containers:
173+
- name: podexample
174+
image: centos
175+
command: ["/bin/bash", "-c", "sleep INF"]
176+
securityContext:
177+
runAsUser: 2000
178+
runAsGroup: 3000
179+
allowPrivilegeEscalation: false
180+
capabilities:
181+
drop: ["ALL"]
182+
securityContext:
183+
runAsNonRoot: true
184+
seccompProfile:
185+
type: RuntimeDefault
186+
----
187+
<1> The name of the SR-IOV network attachment definition CR.
188+
<2> Optional: The MAC address for the SR-IOV device that is allocated from the resource type defined in the SR-IOV network attachment definition CR. To use this feature, you also must specify `{ "mac": true }` in the SriovNetwork object.
189+
<3> Optional: IP addresses for the SR-IOV device that are allocated from the resource type defined in the SR-IOV network attachment definition CR. Both IPv4 and IPv6 addresses are supported. To use this feature, you also must specify `{ "ips": true }` in the `SriovNetwork` object.
190+
191+
. Apply the YAML:
192+
+
193+
[source,terminal]
194+
----
195+
$ oc apply -f examplepod.yaml
196+
----
197+
198+
. Verify that the pod is created by running the following command:
199+
+
200+
[source,terminal]
201+
----
202+
$ oc get pod -n sysctl-tuning-test
203+
----
204+
+
205+
.Example output
206+
+
207+
[source,terminal]
208+
----
209+
NAME READY STATUS RESTARTS AGE
210+
tunepod 1/1 Running 0 47s
211+
----
212+
213+
. Log in to the pod by running the following command:
214+
+
215+
[source,terminal]
216+
----
217+
$ oc rsh -n sysctl-tuning-test tunepod
218+
----
219+
220+
. Verify the values of the configured `sysctl` flag. Find the value `net.ipv6.neigh.IFNAME.base_reachable_time_ms` by running the following command::
221+
+
222+
[source,terminal]
223+
----
224+
$ sysctl net.ipv6.neigh.bond0.base_reachable_time_ms
225+
----
226+
+
227+
.Example output
228+
[source,terminal]
229+
----
230+
net.ipv6.neigh.bond0.base_reachable_time_ms = 20000
231+
----

0 commit comments

Comments
 (0)