Adds procedure to switch between NICs and DPU for BF2

Author: Steven Smith

_topic_maps/_topic_map.yml

@@ -1175,6 +1175,8 @@ Topics:
     File: using-pod-level-bonding
   - Name: Configuring hardware offloading
     File: configuring-hardware-offloading
+  - Name: Switching Bluefield-2 from NIC to DPU mode
+    File: switching-bf2-nic-dpu
   - Name: Uninstalling the SR-IOV Operator
     File: uninstalling-sriov-operator
 - Name: OVN-Kubernetes network plug-in

modules/proc-switching-bf2-nic.adoc

@@ -0,0 +1,95 @@
+// Module included in the following assemblies:
+//
+// * networking/switching-bf2-nic-dpu.adoc
+
+:_content-type: PROCEDURE
+[id="proc-switching-bf2-nic_{context}"]
+= Switching Bluefield-2 from DPU mode to NIC mode
+
+Use the following procedure to switch Bluefield-2 from data processing units (DPU) mode to network interface controller (NIC) mode.  
+
+[IMPORTANT]
+====
+Currently, only switching Bluefield-2 from DPU to NIC mode is supported. Switching from NIC mode to DPU mode is unsupported. 
+====
+
+.Prerequisites
+
+* You have installed the SR-IOV Network Operator. For more information, see "Installing SR-IOV Network Operator". 
+* You have updated Bluefield-2 to the latest firmware. For more information, see link:https://network.nvidia.com/support/firmware/bluefield2/[Firmware for NVIDIA BlueField-2]. 
+
+.Procedure
+
+. Add the following labels to each of your worker nodes by entering the following commands:
++
+[source,terminal]
+----
+$ oc label node <example_node_name_one> node-role.kubernetes.io/sriov=
+----
++
+[source,terminal]
+----
+$ oc label node <example_node_name_two> node-role.kubernetes.io/sriov=
+
+----
+
+. Create a machine config pool for the SR-IOV Operator, for example: 
++
+[source,yaml]
+----
+
+apiVersion: machineconfiguration.openshift.io/v1
+kind: MachineConfigPool
+metadata:
+  name: sriov
+spec:
+  machineConfigSelector:
+    matchExpressions:
+      - {key: machineconfiguration.openshift.io/role, operator: In, values: [worker,sriov]}
+  nodeSelector:
+    matchLabels:
+            node-role.kubernetes.io/sriov: ""
+----
+
+. Apply the following `machineconfig.yaml` file to the worker nodes:
++
+[source,yaml]
+----
+apiVersion: machineconfiguration.openshift.io/v1
+kind: MachineConfig
+metadata:
+  labels:
+    machineconfiguration.openshift.io/role: sriov
+  name: 99-bf2-dpu
+spec:
+  config:
+    ignition:
+      version: 3.2.0
+    storage:
+      files:
+      - contents:
+          source: data:text/plain;charset=utf-8;base64,ZmluZF9jb250YWluZXIoKSB7CiAgY3JpY3RsIHBzIC1vIGpzb24gfCBqcSAtciAnLmNvbnRhaW5lcnNbXSB8IHNlbGVjdCgubWV0YWRhdGEubmFtZT09InNyaW92LW5ldHdvcmstY29uZmlnLWRhZW1vbiIpIHwgLmlkJwp9CnVudGlsIG91dHB1dD0kKGZpbmRfY29udGFpbmVyKTsgW1sgLW4gIiRvdXRwdXQiIF1dOyBkbwogIGVjaG8gIndhaXRpbmcgZm9yIGNvbnRhaW5lciB0byBjb21lIHVwIgogIHNsZWVwIDE7CmRvbmUKISBzdWRvIGNyaWN0bCBleGVjICRvdXRwdXQgL2JpbmRhdGEvc2NyaXB0cy9iZjItc3dpdGNoLW1vZGUuc2ggIiRAIgo=
+        mode: 0755
+        overwrite: true
+        path: /etc/default/switch_in_sriov_config_daemon.sh
+    systemd:
+      units:
+        - name: dpu-switch.service
+          enabled: true
+          contents: |
+            [Unit]
+            Description=Switch BlueField2 card to NIC/DPU mode
+            RequiresMountsFor=%t/containers
+            Wants=network.target
+            After=network-online.target kubelet.service
+            [Service]
+            SuccessExitStatus=0 120
+            RemainAfterExit=True
+            ExecStart=/bin/bash -c '/etc/default/switch_in_sriov_config_daemon.sh nic || shutdown -r now' <1>
+            Type=oneshot
+            [Install]
+            WantedBy=multi-user.target
+----
+<1> Optional: The PCI address of a specific card can optionally be specified, for example `ExecStart=/bin/bash -c '/etc/default/switch_in_sriov_config_daemon.sh nic 0000:5e:00.0 || echo done'`. By default, the first device is selected. If there is more than one device, you must specify which PCI address to be used. The PCI address must be the same on all nodes that are switching Bluefield-2 from DPU mode to NIC mode. 
+
+. Wait for the worker nodes to restart. After restarting, the Bluefield-2 network device on the worker nodes is switched into NIC mode. 

networking/hardware_networks/switching-bf2-nic-dpu.adoc

@@ -0,0 +1,19 @@
+:_content-type: ASSEMBLY
+[id="switching-bf2-nic-dpu"]
+= Switching Bluefield-2 from DPU to NIC
+include::_attributes/common-attributes.adoc[]
+:context: switching-bf2-nic-dpu
+
+toc::[]
+
+You can switch the Bluefield-2 network device from data processing unit (DPU) mode to network interface controller (NIC) mode. 
+
+:FeatureName: Switching Bluefield-2 from data processing unit (DPU) mode to network interface controller (NIC) mode
+include::snippets/technology-preview.adoc[]
+
+include::modules/proc-switching-bf2-nic.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+.Additional resources 
+
+xref:../../networking/hardware_networks/installing-sriov-operator.adoc#installing-sr-iov-operator_installing-sriov-operator[Installing SR-IOV Network Operator]