Skip to content

Commit 7fdd940

Browse files
committed
MANA DPDK: example revert, document example assumption
- swap device name back to bus-info - specify example assumption of 2 nics
1 parent 2072247 commit 7fdd940

File tree

1 file changed

+8
-7
lines changed

1 file changed

+8
-7
lines changed

articles/virtual-network/setup-dpdk-mana.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ The setup procedure for MANA DPDK is outlined in the [example code.](#example-te
2222
Legacy Azure Linux VMs rely on the mlx4 or mlx5 drivers and the accompanying hardware for accelerated networking. Azure DPDK users would select specific interfaces to include or exclude by passing bus addresses to the DPDK EAL. The setup procedure for MANA DPDK differs slightly, since the assumption of one bus address per Accelerated Networking interface no longer holds true. Rather than using a PCI bus address, the MANA PMD uses the MAC address to determine which interface it should bind to.
2323

2424
## MANA DPDK EAL Arguments
25-
The MANA PMD probes all devices and ports on the system when no `--vdev` argument is present; the `--vdev` argument is not mandatory. In testing environments it's often desirable to leave one (primary) interface available for servicing the SSH connection to the VM. To use DPDK with a subset of the available VFs, users should pass both the bus address of the MANA device and the MAC address of the interfaces in the `--vdev` argument. For more detail, example code is available to demonstrate [DPDK EAL initialization on MANA](#example-testpmd-setup-and-netvsc-test).
25+
The MANA PMD probes all devices and ports on the system when no `--vdev` argument is present; the `--vdev` argument is not mandatory. In testing environments, it's often desirable to leave one (primary) interface available for servicing the SSH connection to the VM. To use DPDK with a subset of the available VFs, users should pass both the bus address of the MANA device and the MAC address of the interfaces in the `--vdev` argument. For more detail, example code is available to demonstrate [DPDK EAL initialization on MANA](#example-testpmd-setup-and-netvsc-test).
2626

2727
For general information about the DPDK Environment Abstraction Layer (EAL):
2828
- [DPDK EAL Arguments for Linux](https://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#eal-in-a-linux-userland-execution-environment)
@@ -39,7 +39,7 @@ MANA DPDK requires the following set of drivers:
3939
1. [Libmana user-space drivers](https://github.com/linux-rdma/rdma-core/tree/master/providers/mana) (rdma-core v44 and later)
4040

4141
### Supported Marketplace Images
42-
A nonexhaustive list of images which have the requisite MANA patches for DPDK:
42+
A nonexhaustive list of images with backported patches for DPDK with MANA:
4343
- Red Hat Enterprise Linux 8.9
4444
- Red Hat Enterprise Linux 9.4
4545
- Canonical Ubuntu Server 20.04 (5.15.0-1045-azure)
@@ -92,6 +92,7 @@ Note the following example code for running DPDK with MANA. The direct-to-vf 'ne
9292

9393
>[!NOTE]
9494
>DPDK requires either 2MB or 1GB hugepages to be enabled
95+
>Example assumes an Azure VM with 2 accelerated networking NICs attached.
9596
9697
```bash
9798
# Enable 2MB hugepages.
@@ -130,7 +131,7 @@ echo $DEV_UUID > /sys/bus/vmbus/drivers/uio_hv_generic/bind
130131
dpdk-testpmd -l 1-3 --vdev="$BUS_INFO,mac=$MANA_MAC" -- --forward-mode=txonly --auto-start --txd=128 --rxd=128 --stats 2
131132

132133
# MANA multiple queue test (example assumes > 9 cores)
133-
dpdk-testpmd -l 1-9 --vdev="$BUS_INFO,mac=$MANA_MAC" -- --forward-mode=txonly --auto-start --nb-cores=8 --txd=128 --rxd=128 --txq=8 --rxq=8 --stats 2
134+
dpdk-testpmd -l 1-6 --vdev="$BUS_INFO,mac=$MANA_MAC" -- --forward-mode=txonly --auto-start --nb-cores=4 --txd=128 --rxd=128 --txq=8 --rxq=8 --stats 2
134135

135136
```
136137

@@ -155,12 +156,12 @@ EAL: Error - exiting with code: 1
155156
Cause: Cannot init EAL: Permission denied
156157
```
157158

158-
### Low throughput with use of --vdev="net_vdev_netvsc0,iface=eth1"
159+
### Low throughput with use of `--vdev="net_vdev_netvsc0,iface=eth1"`
159160

160161
Failover configuration of either the `net_failsafe` or `net_vdev_netvsc` poll-mode-drivers isn't recommended for high performance on Azure. The netvsc configuration with DPDK version 20.11 or higher may give better results. For optimal performance, ensure your Linux kernel, rdma-core, and DPDK packages meet the listed requirements for DPDK and MANA.
161162

162163
### Version mismatch for rdma-core
163-
Mismatches in rdma-core and the linux kernel can occur any time a user is building some combination of rdma-core, DPDK, and the linux kernel from source. This error can cause a number of issues, on MANA it will likely result in a failed probe of the MANA virtual function (VF).
164+
Mismatches in rdma-core and the linux kernel can occur any time a user is building some combination of rdma-core, DPDK, and the linux kernel from source. This type of version mismatch can cause a failed probe of the MANA virtual function (VF).
164165

165166
```log
166167
EAL: Probe PCI driver: net_mana (1414:ba) device: 7870:00:00.0 (socket 0)
@@ -205,7 +206,7 @@ enum rdma_driver_id {
205206
RDMA_DRIVER_QIB,
206207
RDMA_DRIVER_EFA,
207208
RDMA_DRIVER_SIW,
208-
RDMA_DRIVER_MANA, //<- Note MANA added as last member of enum
209+
RDMA_DRIVER_MANA, //<- MANA added as last member of enum after backporting
209210
};
210211

211212
// Example mismatched rdma-core ioctl verbs header
@@ -237,4 +238,4 @@ enum rdma_driver_id {
237238
};
238239
```
239240

240-
This mismatch will result in the MANA provider code failing to load. If you use `gdb` to trace the execution in this example you will find the provider for edrma is loaded instead. Either removing the erdma provider from the rdma-core source or forcing the ordering of the provider IDs will allow the MANA provider to load correctly.
241+
This mismatch results in the MANA provider code failing to load. Use `gdb` to trace the execution of `dpdk-testpmd` to confirm the edrma is loaded instead of the mana provider. The MANA driver_id must be consistent for both the kernel and rdma-core. The mana PMD will load correctly when those IDs match.

0 commit comments

Comments
 (0)