DPDK does not work

[inss-lab]

Hello,

System Information:

• DU: Intel i7-14900 (see CPU file)
• RU: Benetel RAN550, 100 MHz, 4×2 MIMO
• NIC on DU: Intel E810 with GNSS
• Connection: DU NIC is connected directly to the RU without a network switch. The DU NIC is configured as the PTP grandmaster.

I am working with srsRAN and a Benetel RU to connect with COTS phones. Without DPDK, the phone can successfully connect to the gNB and access the Internet. However, the throughput is very low (about 1 Mbit/s) which I suspect is due to CPU processing limitations. This is why I wanted to try using DPDK.

I followed the instructions here: https://docs.srsran.com/projects/project/en/latest/tutorials/source/dpdk/source/index.html. The gNB starts successfully (see output below), but when using DPDK, the RU does not receive any data packets from the DU (monitored using kpi.sh). Phone cannot see the network. When I disable DPDK, kpi.sh on the RU kpi.sh shows normal RX and TX packet counts, and phone can access the network.

I have included detailed logs and configuration files below and in the attachments.

Any suggestions or guidance would be greatly appreciated.

--== srsRAN gNB (commit d90cd4e) ==--

EAL: Detected CPU lcores: 24
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 1 (Type 1)
EAL: Probe PCI driver: net_iavf (8086:1889) device: 0000:01:09.0 (socket -1)
iavf_execute_vf_cmd(): Cmd 26 not supported
iavf_set_hena(): Failed to execute command of OP_SET_RSS_HENA
iavf_default_rss_disable(): fail to disable default RSS,lack PF support
TELEMETRY: No legacy callbacks, legacy socket not created
iavf_dev_init_vlan(): Failed to update vlan offload
iavf_dev_configure(): configure VLAN failed: -95
iavf_set_rx_function(): request RXDID[1] in Queue[0] is legacy, set rx_pkt_burst as legacy for all queues
Initializing the Open Fronthaul Interface for sector#0: ul_compr=[BFP,9], dl_compr=[BFP,9], prach_compr=[BFP,9], prach_cp_enabled=true
Cell pci=1, bw=100 MHz, 4T2R, dl_arfcn=637212 (n78), dl_freq=3558.18 MHz, dl_ssb_arfcn=634464, ul_freq=3558.18 MHz

N2: Connection to AMF on 127.0.1.100:38412 completed
==== gNB started ===
Type to view help

/dpdk-23.11/usertools$ sudo ./dpdk-devbind.py -s

Network devices using DPDK-compatible driver
0000:01:09.0 'Ethernet Adaptive Virtual Function 1889' drv=vfio-pci unused=iavf

Network devices using kernel driver
0000:00:14.3 'Device 7af0' if=wlp0s20f3 drv=iwlwifi unused=vfio-pci Active
0000:00:1f.6 'Ethernet Connection (17) I219-LM 1a1c' if=enp0s31f6 drv=e1000e unused=vfio-pci
0000:01:00.0 'Ethernet Controller E810-C for SFP 1593' if=enp1s0f0 drv=ice unused=vfio-pci
0000:01:00.1 'Ethernet Controller E810-C for SFP 1593' if=enp1s0f1 drv=ice unused=vfio-pci Active
0000:01:00.2 'Ethernet Controller E810-C for SFP 1593' if=enp1s0f2 drv=ice unused=vfio-pci
0000:01:00.3 'Ethernet Controller E810-C for SFP 1593' if=enp1s0f3 drv=ice unused=vfio-pci

Linux inss-OptiPlex-Tower-Plus-7020 5.15.0-1033-realtime #36-Ubuntu SMP PREEMPT_RT Tue Feb 7 18:41:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

mount | grep huge
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=1024M)

Grub
GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 5.15.0-1033-realtime"
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=lsb_release -i -s 2> /dev/null || echo Debian
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

GRUB_CMDLINE_LINUX="default_hugepagesz=1GB hugepagesz=1G hugepages=20 skew_tick=1 intel_iommu=on intel_iommu=pt quiet splash vt.handoff=7"

cpuinfo_meminfo.txt

gnb.log

[ninjab3s]

Hi @inss-lab!

I see two potential issues in your setup. First of all the CPU you are using has Intel's big little architecture. We have seen in many deployments that these CPUs are not suited for realtime applications like our gNB. We strongly advise to use a different platform, best one that supports the regular AVX512 instructions. You can in theory run a RAN with this CPU but dont expect to run stable and I also dont think you wont get much throughput with it.

Second I think the NIC's firmware and drivers could be updated if you havent done so yet. Follow the steps below to upgrade the drivers and firmware:

Ice driver update:

1. Download the latest driver here: https://www.intel.com/content/www/us/en/download/19630/intel-network-adapter-driver-for-e810-series-devices-under-linux.html
2. unpack and navigate to the directory ice-2.2.9/src
3. In the src directory run the following commands:

make -j$(nproc)
sudo make -j$(nproc) install

iavf driver update:

1. Download the latest version here: https://www.intel.com/content/www/us/en/download/18159/intel-network-adapter-linux-virtual-function-driver-for-intel-ethernet-controller-700-and-e810-series.html
2. unpack and navigate to the directory iavf-4.13.16/src
3. In the src directory run the following commands:

make -j$(nproc)
sudo make -j$(nproc) install

Firmware upgrade:

1. Download the latest firmware here: https://www.intel.com/content/www/us/en/download/19626/non-volatile-memory-nvm-update-utility-for-intel-ethernet-network-adapters-e810-series-linux.html
2. Unpack and navigate to the directory E810/Linux_x64
3. In that directory run the following command:

sudo ./nvmupdate64e

The installer will confirm that the correct NIC is installed and ask you if you want to proceed with the upgrade. During the upgrade its recommended to make a backup of the current NVME image (you will be prompted with a question). Agree to all. Once finished please send me the output of the FW upgrade to confirm everything worked as expected.

Once you have done all these steps please reboot the server. To confirm everything worked out please send me the outputs of the following commands when you get a chance! Thanks!

ethtool -i enp1s0f0np0
modinfo iavf
modinfo ice

[inss-lab]

Hi @ninjab3s Thank you so much for the detailed instructions. The problem has been resolved by updating the drivers. We appreciate your help!