VPP Command_line_Arguments

Contents

• 1 Introduction
• 2 Command-line arguments
• 3 Configuration file
• 4 Configuration parameters
  • 4.1 unix { ... }
  • 4.2 tuntap { ... }
  • 4.3 api-trace { ... }
  • 4.4 api-segment { ... }
  • 4.5 cpu { ... }
  • 4.6 dpdk { ... }
  • 4.7 l2tp { ... }
  • 4.8 oam { ... }
  • 4.9 heapsize { ... }
  • 4.10 plugins { path ... }
  • 4.11 cj { ... }
  • 4.12 vhost-user { ... }
  • 4.13 l2learn { ... }
  • 4.14 socksvr { ... }
  • 4.15 statseg { ... }
  • 4.16 punt { ... }

Introduction

The VPP network stack comes with several configuration options that can be provided and applications built on the VPP stack have been known to require a dozen arguments, depending on requirements. This page describes commonly used configuration options. Please help maintain and improve this document!

The configuration options can be provided to VPP either as command-line arguments or in a configuration file.

Command-line arguments

Parameters are grouped by a section name. When providing more than one parameter to a section all parameters for that section must be wrapped in curly braces.

# /usr/bin/vpp unix { interactive cli-listen 127.0.0.1:5002 }
<startup diagnostic messages>
    _______    _        _   _____  ___ 
 __/ __/ _ \  (_)__    | | / / _ \/ _ \
 _/ _// // / / / _ \   | |/ / ___/ ___/
 /_/ /____(_)_/\___/   |___/_/  /_/    

vpp# <start-typing>

When providing only one such parameter the braces are optional. For example, the following command argument, unix interactive does not need braces:

# /usr/bin/vpp unix interactive

The command line can be presented as a single string or as several; anything given on the command line is concatenated with spaces into a single string before parsing.

VPP applications must be able to locate their own executable images. The simplest way to ensure this will work is to invoke a VPP application by giving its absolute path; for example: /usr/bin/vpp <options>. At startup, VPP applications parse through their own ELF-sections [primarily] to make lists of init, configuration, and exit handlers.

When developing with VPP, in gdb it's often sufficient to start an application like this:

# (gdb) run unix interactive

Configuration file

It is also possible to supply parameters in a startup configuration file the path of which is provided to the VPP application on its command line. The format of the configuration file is a simple text file with the same content as the command line but with the benefit of being able to use newlines to make the content easier to read.

The configuration file is commonly stored at /etc/vpp/startup.conf.

Here is an example of the configuration file:

unix {
  nodaemon
  log /var/log/vpp/vpp.log
  full-coredump
  cli-listen localhost:5002
}

api-trace {
  on
}

dpdk {
  dev 0000:03:00.0
}

VPP is then instructed to load this file with the -c option:

# /usr/bin/vpp -c /etc/vpp/startup.conf

Note: In VPP 18.04, the default log file location was moved from "/tmp/vpp.log" to "/var/log/vpp/vpp.log". The VPP code is indifferent to the file location. However, if SELinux is enabled, then the new location is required for the file to be properly labelled. Check your local startup.conf file for the log file location on your system.

Configuration parameters

This is not an exhaustive list of parameters available. Readers who find information is missing or inaccurate are encouraged to add to this wiki page!

You can find argument parsers in the source code by searching for instances of the VLIB_CONFIG_FUNCTION or VLIB_EARLY_CONFIG_FUNCTION macros. Their invocation (foo_config, "foo") will cause the function foo_config to receive all parameters given in a parameter block named "foo", for example: "foo { arg1 arg2 arg3 ... }".

unix { ... }

Parameter	Description
interactive	Attach CLI to stdin/out and provide a debugging command line interface. Implies nodaemon.
nodaemon	Do not fork / background the vpp process. Typical when invoking VPP applications from a process monitor. Set by default in the default startup.conf file.
exec <filename> startup-config <filename>	Read startup operational configuration from filename. The contents of the file will be performed as though entered at the CLI. The two keywords are aliases for the same function; if both are specified, only the last will have an effect.
log <filename>	Logs the startup configuration and all subsequent CLI commands in filename. Very useful in situations where folks don't remember or can't be bothered to include CLI commands in bug reports.
full-coredump	Ask the Linux kernel to dump all memory-mapped address regions, instead of just text+data+bss.
`cli-listen addr:port	`
cli-line-mode	Disable character-by-character I/O on stdin. Useful when combined with, for example, emacs M-x gud-gdb.
cli-prompt <string>	Configure the CLI prompt to be string.
cli-history-limit <nn>	Limit commmand history to nn lines. A value of 0 disables command history. Default value: 50
cli-no-banner	Disable the login banner on stdin and Telnet connections.
cli-no-pager	Disable the output pager.
cli-pager-buffer-limit <nn>	Limit pager buffer to nn lines of output. A value of 0 disables the pager. Default value: 100000

tuntap { ... }

The "tuntap" driver configures a point-to-point interface between the vpp engine and the local Linux kernel stack. This allows e.g. users to ssh to the host / VM / container via vpp "revenue" interfaces. It's marginally useful, and is currently disabled by default. To [dynamically] create TAP interfaces - the preferred scheme - see the "tap_connect" binary API

Parameter	Description
tuntap disable	Disable the tun/tap driver if desired
tuntap ethernet	Create a tap device (ethernet MAC) instead of a tun device (point-to-point tunnel)
tuntap have-normal-interface	Treat the host Linux stack as a routing peer instead of programming vpp interface L3 addresses onto the tun/tap devices. You'll need to manually configure the Linux network stack "vnet" interface, configure vlan subinterfaces if desired, etc.
tuntap name ishmael	Call the tun/tap device ishmael

Here's a typical multiple parameter invocation:

 tuntap { ethernet have-normal-interface name vpp1 }

api-trace { ... }

The ability to trace, dump, and replay control-plane API traces makes all the difference in the world when trying to understand what the control-plane has tried to ask the forwarding-plane to do.

Parameter	Description
on enable	Enable API trace capture from the beginning of time, and arrange for a post-mortem dump of the API trace if the application terminates abnormally. The two keywords are aliases for the same thing.
nitems <nnn>	Configure the circular trace buffer to contain the last entries. By default, the trace buffer captures the last 256K API messages received.
save-api-table <filename>	To save the api message table decode tables, configure a filename. Results in /tmp/<filename> Very handy for understanding api message changes between versions, identifying missing plugins, and so forth.

Typically, one simply enables the API message trace scheme:

api-trace { on }

api-segment { ... }

These values control various aspects of the binary API interface to VPP.

Parameter	Description
prefix <path>	Sets the prefix prepended to the name used for shared memory (SHM) segments. The default is empty, meaning shared memory segments are created directly in the SHM directory /dev/shm. It is worth noting that on many systems /dev/shm is a symbolic link to somewhere else in the file system; Ubuntu links it to /run/shm.
`uid <number	name>`
`gid <number	name>`

cpu { ... }

Command-line CPU configuration controls the creation of named thread types, and the cpu affinity thereof. As of this writing, the cpu configuration function supports these parameters:

Parameter	Description
workers <nn>	Create worker threads.
io <nn>	Create nn i/o threads.
main-thread-io	Handle i/o devices from thread 0, hand off traffic to worker threads. Requires "workers ".
skip-cores <nn>	Leave the low nn bits of the process affinity mask clear.

In some cases, it may be necessary to place thread instances manually:

Parameter	Description
main-core <n>	Assign main thread to specific core.
coremask-workers <hex-mask>	Place worker threads according to the bitmap hex-mask.
corelist-workers <list>	Same as coremask-workers but accepts a list of cores instead of a bitmap.
coremask-io <hex-mask>	Place I/O threads according to the bitmap hex-mask.
corelist-io <list>	Same as coremask-io but accepts a list of cores instead of a bitmap.

Note that the "main" thread always occupies the lowest core-id specified in the DPDK [process-level] coremask.

Here's a full-bore manual placement example:

/cisco/bin/vpe  unix interactive tuntap disable cpu { main-thread-io coremask-workers 18 coremask-stats 4 } dpdk {coremask 1e}

# taskset -a -p <vpe-pid>
pid 16251's current affinity mask: 2   # main thread
pid 16288's current affinity mask: ffffff # DPDK interrupt thread (not bound to a core)
pid 16289's current affinity mask: 4 # stats thread
pid 16290's current affinity mask: 8 # worker thread 0
pid 16291's current affinity mask: 10 # worker thread 1

dpdk { ... }

Command line DPDK configuration controls a number of parameters, including device whitelisting, the number of CPUs available for launching dpdk-eal-controlled threads, the number of I/O buffers, and the process affinity mask.

In addition, the DPDK configuration function attempts to support all of the DPDK EAL configuration parameters.

Popular options include:

Parameter	Description
coremask <hex>	Sets the process-level coremask. See [[previous section on cpu parameters
no-hugetlb	Don't use huge TLB pages. Potentially useful for running simulator images.
dev <pci-dev>	White-list [as in, attempt to drive] a specific PCI device. PCI-dev is a string of the form "DDDD:BB:SS.F" where: * DDDD = Domain * BB = Bus Number * SS = Slot number * F = Function This is the same format used in the linux sysfs tree (i.e. /sys/bus/pci/devices) for PCI device directory names. VLAN stripping for each PCI device on the white-list can be specified using [{vlan-strip-offload on
vdev <eal-command>	Provide a DPDK EAL command to specify bonded Ethernet interfaces, operating modes and PCI addresses of slave links. For example: vdev eth_bond0,mode=2,slave=0000:0f:00.0,slave=0000:11:00.0,xmit_policy=l34 vdev eth_bond1,mode=2,slave=0000:10:00.0,slave=0000:12:00.0,xmit_policy=l34 Only XOR balanced (mode 2) mode is supported.
num-mbufs <nn>	Number of I/O buffers. The default is 16384. You need to increase this if you have a lot of physical interfaces with multiple RSS queues.
kni <nn>	Number of KNI interfaces. Refer to the DPDK documentation
uio-driver <name>	Defaults to "igb_uio" [the Ubuntu default]. The names "uio_pci_generic" or "vfio-pci" are other possible values.

All of the DPDK EAL options should be available. See .../open-repo/vnet/vnet/devices/dpdk/dpdk.h, look at the set of "foreach_eal_XXX" macros.

l2tp { ... }

ip6-l2tpv3 configuration controls the method used to locate a specific ip6-l2tpv3 tunnel. The following settings are mutually exclusive

Parameter	Description
lookup-v6-src	Lookup tunnel by ip6 source address.
lookup-v6-dst	Lookup tunnel by ip6 destination address.
lookup-session-id	Lookup tunnel by l2tpv3 session identifier.

Refer to .../open-repo/vnet/vnet/devices/dpdk/{dpdk.h, init.c} for more details.

oam { ... }

Oam configuration controls the (ip4-icmp) interval, and number of misses allowed before reporting an oam target down to any registered listener.

Parameter	Description
interval <floating-point-seconds>	Interval between sending oam ip4 icmp messages.
misses-allowed <nn>	Number of misses before declaring an oam target down.

heapsize { ... }

Heapsize configuration controls the size of the main heap. The heap size is configured very early in the boot sequence, before loading plug-ins or doing much of anything else.

Parameter	Description
`heapsize M	G`

plugins { path ... }

Plugin path configuration controls the set of directories searched for vlib plugins. Supply a colon-separated list of (absolute) directory names:

Parameter	Description
plugins { path dir1:dir2:...:dirN }	Configure the search path for plugins.

cj { ... }

The cj thread-safe circular log buffer scheme is occasionally useful when chasing bugs. Calls to it should not be checked in. See .../vlib/vlib/unix/cj.c.

Parameter	Description
records <nn>	Configure the number of cj records in the circular buffer.
on	Turns on logging at the earliest possible moment.

vhost-user { ... }

Vhost-user configuration parameters control the vhost-user driver.

Parameter	Description
coalesce-frames <nn>	Subject to deadline-timer expiration - see next item - attempt to transmit at least nn packet frames.
coalesce-time <seconds>	Hold packets no longer than (floating-point) seconds before transmitting them.
dont-dump-memory	vhost-user shared-memory segments can add up to a large amount of memory, so it's handy to avoid adding them to corefiles when using a significant number of such interfaces.

l2learn { ... }

Configures the number of L2 (MAC) addresses in the L2 FIB at any one time. Defaults to 1M entries.

Parameter	Description
limit <nn>	Limits the size of the L2 FIB to nn concurrent entries.

socksvr { ... }

Configure the socket server for the binary API. (See .../vlibmemory/socket_api.c)

Parameter	Description
socket-name <filename>	Configure API socket filename
default	Default API filename: /run/vpp-api.sock

statseg { ... }

Configure the shared memory stats segment. (See .../vpp/stats/stat_segment.c)

Parameter	Default	Description
socket-name <filename>	/run/vpp/stats.sock	Configure the stats segment socket filename.
default	-	Sets the default value for socket-name parameter.
`per-node-counters <on	off>`	off
`size G	M	K

punt { ... }

Configure the punt socket for the binary API. (See .../vnet/ip/punt.c)

Parameter	Description
socket <filename>	Configure socket filename