Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 32 additions & 27 deletions onramp/blueprints.rst
Original file line number Diff line number Diff line change
Expand Up @@ -422,30 +422,35 @@ UPF is used in both cases.

Enable SR-IOV and DPDK
~~~~~~~~~~~~~~~~~~~~~~~~~~
UPF performance can be improved by enabling SR-IOV and DPDK.

Pre-requisite:
* make sure virtualization and VT-d parameters are enabled in BIOS.
UPF performance can be improved by enabling SR-IOV and DPDK. This
blueprint supports both optimizations, where the former depends on the
server NIC(s) being SR-IOV capable. Before getting to the blueprint
itself, we note the following hardware-related prerequisites.

* make sure enough hugepage memory allocated, iommu enabled. These changes
can be made by updating
GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt default_hugepagesz=1G hugepagesz=1G hugepages=32 transparent_hugepage=never"
in /etc/default/grub
* Make sure virtualization and VT-d parameters are enabled in the BIOS.

Note: Number of hugepages = 2 X No of UPF Instances
* Make sure enough ``hugepage`` memory has been allocated and
``iommu`` is enabled. These changes can be made by updating
``/etc/default/grub``:

Once it is updated apply the changes by running below command,
.. code-block::

GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt default_hugepagesz=1G hugepagesz=1G hugepages=32 transparent_hugepage=never"

Note that the number of ``hugepages`` must be two times the number
of UPF Instances. Once the file is updated, apply the changes by running:

.. code-block::

$sudo update-grub
$sudo reboot
$ sudo update-grub
$ sudo reboot

You can verify the allocated hugepages using below command,
Verify the allocated ``hugepages`` using the following command:

.. code-block::

$cat /proc/meminfo | grep HugePages
$ cat /proc/meminfo | grep HugePages
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
FileHugePages: 0 kB
Expand All @@ -454,36 +459,36 @@ Pre-requisite:
HugePages_Rsvd: 0
HugePages_Surp: 0

* Create required VF devices(minimum 2 required per UPF) as follows,
(In this example the PF interface used is "ens801f0")
* Create the required VF devices, where a minimum of two is required
for each UPF. Using ``ens801f0`` as an example VF interface, this is
done as follows:

.. code-block::

echo 2 > /sys/class/net/ens801f0/device/sriov_numvfs
$ echo 2 > /sys/class/net/ens801f0/device/sriov_numvfs

Now retrieve the PCI address for the newly created VF devices using below command,
Retrieve the PCI address for the newly created VF devices using
the following command:

.. code-block::

ls -l /sys/class/net/ens801f0/device/virtfn*
$ ls -l /sys/class/net/ens801f0/device/virtfn*

* Clone the DPDK repo to use the binding tools,
* Clone the DPDK repo to use the binding tools:

.. code-block::

git clone https://github.com/DPDK/dpdk.git
cd dpdk
$ git clone https://github.com/DPDK/dpdk.git
$ cd dpdk

* Bind the VF devices to the vfio-pci driver as follows,
* Bind the VF devices to the ``vfio-pci`` driver as follows:

.. code-block::

./usertools/dpdk-devbind.py -b vfio-pci 0000:b1:01.0
./usertools/dpdk-devbind.py -b vfio-pci 0000:b1:01.1
$ ./usertools/dpdk-devbind.py -b vfio-pci 0000:b1:01.0
$ ./usertools/dpdk-devbind.py -b vfio-pci 0000:b1:01.1

This blueprint supports both optimizations, where the former depends on the
server NIC(s) being SR-IOV capable. The blueprint includes the
following:
Returning to the OnRamp blueprint, it includes the following:

* Global vars file ``vars/main-sriov.yml`` gives the overall blueprint
specification.
Expand Down
Loading