Skip to content

Commit 0f6afbb

Browse files
authored
Merge pull request #235701 from asudbring/linux-fixes-2
[Doc-a-thon] Linux doc-athon fixes for DPDK in a Azure Linux VM
2 parents b412ccd + 09608fc commit 0f6afbb

File tree

1 file changed

+56
-37
lines changed

1 file changed

+56
-37
lines changed

articles/virtual-network/setup-dpdk.md

Lines changed: 56 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,20 @@
11
---
22
title: DPDK in an Azure Linux VM
3+
titleSuffix: Azure Virtual Network
34
description: Learn the benefits of the Data Plane Development Kit (DPDK) and how to set up the DPDK on a Linux virtual machine.
45
services: virtual-network
56
author: asudbring
6-
manager: gedegrac
77
ms.service: virtual-network
88
ms.topic: how-to
9-
ms.workload: infrastructure-services
10-
ms.date: 05/12/2020
9+
ms.date: 04/24/2023
1110
ms.author: allensu
1211
---
1312

1413
# Set up DPDK in a Linux virtual machine
1514

1615
Data Plane Development Kit (DPDK) on Azure offers a faster user-space packet processing framework for performance-intensive applications. This framework bypasses the virtual machine’s kernel network stack.
1716

18-
In typical packet processing that uses the kernel network stack, the process is interrupt-driven. When the network interface receives incoming packets, there is a kernel interrupt to process the packet and a context switch from the kernel space to the user space. DPDK eliminates context switching and the interrupt-driven method in favor of a user-space implementation that uses poll mode drivers for fast packet processing.
17+
In typical packet processing that uses the kernel network stack, the process is interrupt-driven. When the network interface receives incoming packets, there's a kernel interrupt to process the packet and a context switch from the kernel space to the user space. DPDK eliminates context switching and the interrupt-driven method in favor of a user-space implementation that uses poll mode drivers for fast packet processing.
1918

2019
DPDK consists of sets of user-space libraries that provide access to lower-level resources. These resources can include hardware, logical cores, memory management, and poll mode drivers for network interface cards.
2120

@@ -25,7 +24,6 @@ DPDK can run on Azure virtual machines that are supporting multiple operating sy
2524

2625
**Higher packets per second (PPS)**: Bypassing the kernel and taking control of packets in the user space reduces the cycle count by eliminating context switches. It also improves the rate of packets that are processed per second in Azure Linux virtual machines.
2726

28-
2927
## Supported operating systems minimum versions
3028

3129
The following distributions from the Azure Marketplace are supported:
@@ -50,55 +48,47 @@ All Azure regions support DPDK.
5048

5149
## Prerequisites
5250

53-
Accelerated networking must be enabled on a Linux virtual machine. The virtual machine should have at least two network interfaces, with one interface for management. Enabling Accelerated networking on management interface is not recommended. Learn how to [create a Linux virtual machine with accelerated networking enabled](create-vm-accelerated-networking-cli.md).
51+
Accelerated networking must be enabled on a Linux virtual machine. The virtual machine should have at least two network interfaces, with one interface for management. Enabling Accelerated networking on management interface isn't recommended. Learn how to [create a Linux virtual machine with accelerated networking enabled](create-vm-accelerated-networking-cli.md).
5452

5553
On virtual machines that are using InfiniBand, ensure the appropriate `mlx4_ib` or `mlx5_ib` drivers are loaded, see [Enable InfiniBand](../virtual-machines/workloads/hpc/enable-infiniband.md).
5654

5755
## Install DPDK via system package (recommended)
5856

59-
### Ubuntu 18.04
57+
# [RHEL, CentOS](#tab/redhat)
6058

6159
```bash
62-
sudo add-apt-repository ppa:canonical-server/server-backports -y
63-
sudo apt-get update
64-
sudo apt-get install -y dpdk
60+
sudo yum install -y dpdk
6561
```
6662

67-
### Ubuntu 20.04 and newer
63+
# [openSUSE, SLES](#tab/suse)
6864

6965
```bash
70-
sudo apt-get install -y dpdk
66+
sudo zypper install -y dpdk
7167
```
7268

73-
### Debian 10 and newer
74-
75-
```bash
76-
sudo apt-get install -y dpdk
77-
```
78-
79-
## Install DPDK manually (not recommended)
80-
81-
### Install build dependencies
69+
# [Ubuntu, Debian](#tab/ubuntu)
8270

83-
#### Ubuntu 18.04
71+
### Ubuntu 18.04
8472

8573
```bash
8674
sudo add-apt-repository ppa:canonical-server/server-backports -y
8775
sudo apt-get update
88-
sudo apt-get install -y build-essential librdmacm-dev libnuma-dev libmnl-dev meson
76+
sudo apt-get install -y dpdk
8977
```
9078

91-
#### Ubuntu 20.04 and newer
79+
### Ubuntu 20.04/Debian 10 and newer
9280

9381
```bash
94-
sudo apt-get install -y build-essential librdmacm-dev libnuma-dev libmnl-dev meson
82+
sudo apt-get install -y dpdk
9583
```
9684

97-
#### Debian 10 and newer
85+
---
9886

99-
```bash
100-
sudo apt-get install -y build-essential librdmacm-dev libnuma-dev libmnl-dev meson
101-
```
87+
## Install DPDK manually (not recommended)
88+
89+
### Install build dependencies
90+
91+
# [RHEL, CentOS](#tab/redhat)
10292

10393
#### RHEL7.5/CentOS 7.5
10494

@@ -108,6 +98,8 @@ sudo dracut --add-drivers "mlx4_en mlx4_ib mlx5_ib" -f
10898
yum install -y gcc kernel-devel-`uname -r` numactl-devel.x86_64 librdmacm-devel libmnl-devel meson
10999
```
110100

101+
# [openSUSE, SLES](#tab/suse)
102+
111103
#### SLES 15 SP1
112104

113105
**Azure kernel**
@@ -128,11 +120,31 @@ zypper \
128120
--gpg-auto-import-keys install kernel-default-devel gcc make libnuma-devel numactl librdmacm1 rdma-core-devel meson
129121
```
130122

123+
# [Ubuntu, Debian](#tab/ubuntu)
124+
125+
#### Ubuntu 18.04
126+
127+
```bash
128+
sudo add-apt-repository ppa:canonical-server/server-backports -y
129+
sudo apt-get update
130+
sudo apt-get install -y build-essential librdmacm-dev libnuma-dev libmnl-dev meson
131+
```
132+
133+
#### Ubuntu 20.04/Debian 10 and newer
134+
135+
```bash
136+
sudo apt-get install -y build-essential librdmacm-dev libnuma-dev libmnl-dev meson
137+
```
138+
139+
---
131140
### Compile and install DPDK manually
132141

133142
1. [Download the latest DPDK](https://core.dpdk.org/download). Version 19.11 LTS or newer is required for Azure.
143+
134144
2. Build the default config with `meson builddir`.
145+
135146
3. Compile with `ninja -C builddir`.
147+
136148
4. Install with `DESTDIR=<output folder> ninja -C builddir install`.
137149

138150
## Configure the runtime environment
@@ -141,24 +153,27 @@ After restarting, run the following commands once:
141153

142154
1. Hugepages
143155

144-
* Configure hugepage by running the following command, once for each numa node:
156+
* Configure hugepage by running the following command, once for each numa node:
145157

146-
```bash
158+
```bash
147159
echo 1024 | sudo tee /sys/devices/system/node/node*/hugepages/hugepages-2048kB/nr_hugepages
148-
```
160+
```
149161

150-
* Create a directory for mounting with `mkdir /mnt/huge`.
151-
* Mount hugepages with `mount -t hugetlbfs nodev /mnt/huge`.
152-
* Check that hugepages are reserved with `grep Huge /proc/meminfo`.
162+
* Create a directory for mounting with `mkdir /mnt/huge`.
163+
164+
* Mount hugepages with `mount -t hugetlbfs nodev /mnt/huge`.
165+
166+
* Check that hugepages are reserved with `grep Huge /proc/meminfo`.
153167

154-
> [NOTE]
168+
> [!NOTE]
155169
> There is a way to modify the grub file so that hugepages are reserved on boot by following the [instructions](https://dpdk.org/doc/guides/linux_gsg/sys_reqs.html#use-of-hugepages-in-the-linux-environment) for the DPDK. The instructions are at the bottom of the page. When you're using an Azure Linux virtual machine, modify files under **/etc/config/grub.d** instead, to reserve hugepages across reboots.
156170
157171
2. MAC & IP addresses: Use `ifconfig –a` to view the MAC and IP address of the network interfaces. The *VF* network interface and *NETVSC* network interface have the same MAC address, but only the *NETVSC* network interface has an IP address. *VF* interfaces are running as subordinate interfaces of *NETVSC* interfaces.
158172
159173
3. PCI addresses
160174
161175
* Use `ethtool -i <vf interface name>` to find out which PCI address to use for *VF*.
176+
162177
* If *eth0* has accelerated networking enabled, make sure that testpmd doesn’t accidentally take over the *VF* pci device for *eth0*. If the DPDK application accidentally takes over the management network interface and causes you to lose your SSH connection, use the serial console to stop the DPDK application. You can also use the serial console to stop or start the virtual machine.
163178
164179
4. Load *ibuverbs* on each reboot with `modprobe -a ib_uverbs`. For SLES 15 only, also load *mlx4_ib* with `modprobe -a mlx4_ib`.
@@ -197,6 +212,7 @@ To run testpmd in root mode, use `sudo` before the *testpmd* command.
197212
If you're running testpmd with more than two NICs, the `--vdev` argument follows this pattern: `net_vdev_netvsc<id>,iface=<vf’s pairing eth>`.
198213
199214
3. After it's started, run `show port info all` to check port information. You should see one or two DPDK ports that are net_failsafe (not *net_mlx4*).
215+
200216
4. Use `start <port> /stop <port>` to start traffic.
201217

202218
The previous commands start *testpmd* in interactive mode, which is recommended for trying out testpmd commands.
@@ -238,6 +254,7 @@ The following commands periodically print the packets per second statistics:
238254
When you're running the previous commands on a virtual machine, change *IP_SRC_ADDR* and *IP_DST_ADDR* in `app/test-pmd/txonly.c` to match the actual IP address of the virtual machines before you compile. Otherwise, the packets are dropped before reaching the receiver.
239255
240256
### Advanced: Single sender/single forwarder
257+
241258
The following commands periodically print the packets per second statistics:
242259
243260
1. On the TX side, run the following command:
@@ -271,10 +288,12 @@ The following commands periodically print the packets per second statistics:
271288
--stats-period <display interval in seconds>
272289
```
273290
274-
When you're running the previous commands on a virtual machine, change *IP_SRC_ADDR* and *IP_DST_ADDR* in `app/test-pmd/txonly.c` to match the actual IP address of the virtual machines before you compile. Otherwise, the packets are dropped before reaching the forwarder. You won’t be able to have a third machine receive forwarded traffic, because the *testpmd* forwarder doesn’t modify the layer-3 addresses, unless you make some code changes.
291+
When you're running the previous commands on a virtual machine, change *IP_SRC_ADDR* and *IP_DST_ADDR* in `app/test-pmd/txonly.c` to match the actual IP address of the virtual machines before you compile. Otherwise, the packets are dropped before reaching the forwarder. You can't have a third machine receive forwarded traffic, because the *testpmd* forwarder doesn’t modify the layer-3 addresses, unless you make some code changes.
275292
276293
## References
277294
278295
* [EAL options](https://dpdk.org/doc/guides/testpmd_app_ug/run_app.html#eal-command-line-options)
296+
279297
* [Testpmd commands](https://dpdk.org/doc/guides/testpmd_app_ug/run_app.html#testpmd-command-line-options)
298+
280299
* [Packet dump commands](https://doc.dpdk.org/guides/tools/pdump.html#pdump-tool)

0 commit comments

Comments
 (0)