User Plane Gateway (UPG) based on VPP

UPG implements a GTP-U user plane based on 3GPP TS 23.214 and 3GPP TS 29.244 Release 15. It is implemented as an out-of-tree plugin for FD.io VPP.

The possible uses for UPG are:

• User Plane Function (UPF) in 5G networks
• Packet Data Network Gateway User plane (PGW-U)
• Traffic Detection Function User plane (TDF-U)

Current State

UPG is used in production in conjunction with erGW as GGSN/PGW in multiple installation in several telecom operators (Tier 1 and smaller).

For the list of known issues, see KNOWN_ISSUES document.

Working features

• PFCP protocol
  • en/decoding of most IEs
  • heartbeat
  • node related messages
  • session related messages
• Uplink and Downlink Packet Detection Rules (PDR) and Forward Action Rules (FAR) -- (some parts)
• IPv4 -- inner and outer
• IPv6 -- inner and outer
• Usage Reporting Rules (URR)
• PFCP Session Reports
• Linked Usage Reports

No yet working

• Buffer Action Rules (BAR)
• QoS Enforcement Rule (QER)

Limitations

• FAR action with destination LI are not implemented
• Ethernet bearer support

Development setup

Design rationale for the development environment is this:

• provide an easily reproducible development and build environment usable both on CI and locally
• provide quick commands for common tasks
• simplify bisecting against upstream VPP
• discourage downstream VPP changes, as we should make effort to upstream them

Relevant parts of the source tree layout:

• hack/ contains helper scripts most of which are wrapped in make commands
• Makefile provides user interface for the environment
• upf/ contains the source code of the plugin¹
• upf/test/ contains the integration tests¹
• vpp/ contains VPP source code and is not committed
• vpp-patches/ contains patches to apply to the upstream VPP code (applied automatically via git am; generated by hand with git format-patch)
• vpp.spec contains the info on VPP repo, branch and commit to use

upg/ is symlinked to the VPP plugins directory, one can just go to vpp/ directory after make update-vpp and follow upstream VPP build instructions. But, as an alternative, UPG does provide some build helpers of its own.

There's a simple dockerized environment wrapped in 'make' commands. During the development cycle, VPP is cloned into vpp/ subdirectory from an upstream repository with some of the patches applied. This is done automatically but can also be done via make update-vpp which will delete all of the changes made to vpp/ directory. You can make temporary changes under vpp/ for debugging purposes.

The "build image" which is used for the devenv is tagged with a hash of Dockerfile.build as well as VPP's external dependencies.

The following make commands are supported:

• make image-debug builds a debug UPG Docker image named upg:debug
• make image-release builds a release UPG Docker image named upg:release
• make test-debug, make test-release build VPP and run UPG integration tests in debug and release mode respectively. The compilation results are retained under vpp/
• make retest-debug, make retest-release run UPG integration tests in debug and release mode respectively without building VPP. These can be run under make test-debug / make test-release to re-run the tests quickly if there were no UPG / VPP code changes
• make ensure-build-image checks if the build image exists or can be pulled and builds it otherwise
• make update-build-image-tag updates the build image tag in the Dockerfiles according to the hash calculated from Dockerfile.build and VPP external dependencies
• make install-hooks installs git hooks in the repo which prevent the user from making commits that contain ZZZZZ: substring. This is handy for debug print like clib_warning("ZZZZZ: i %d", i);
• make update-vpp re-clones VPP into vpp/ directory
• make buildenv runs an interactive shell inside the build environment with UPG and VPP sources mounted into the container
• make e2e-debug and make e2e-release build UPG in debug and release mode respectively and run E2E tests for it. For more information, see E2E test documentation
• make checkstyle performs style checks on the source code

If docker is used, one should set the following environment variable to enable wrapping the internally run commands in a docker container:

export UPG_BUILDENV=docker

It is also possible to use a k8s cluster to run the build container in a pod:

export UPG_BUILDENV=k8s
# optional: specify the node to run the build pod
export UPG_BUILDENV_NODE=somenode

In this case, the buildenv is run as statefulset inside the cluster. It can be removed using

hack/buildenv.sh clean

As an alternative, it’s possible to run commands such as

$ make -C vpp test TEST=test_upf V=2 EXTERN_TESTS=../../upf/test

using VPP’s usual Makefile. upf/ subdirectory is symlinked to vpp/src/plugins/upf (note though that there's an issue with VPP's test discovery mechanism, thus EXTERN_TEST=... variable is required to run the tests).

CI and releases

The CI for UPG-VPP is based on GitHub Actions. Currently, the CI only runs for pushes to branches in the repository itself. The jobs include:

• prepare: make sure build image is available for the commit
• build (debug + release): build the docker images and binaries / packages
• checkstyle: check for style errors in the code
• test: unit and e2e tests for release and debug builds
• conclude: intermediate job used for sync by the release workflow
• slack: internal notification job

The images built per-commit expire within 7 days.

When a tag is pushed, the release workflow is also run for it, re-tagging the images built as part of normal build process (preserving the old tags too). In case if the tag doesn't have test substring in it, it is also published as a release. The release notes list the PRs with the following tags:

• feature, enhancement: features
• fix: fixes
• test: tests

The releases for tags that contain pre substring are marked as pre-releases.

1: Historically, the project was named simply "UPF". There may be more UPF->UPG renames later