Hw. acceleration support

UltraGrid supports video encoding and decoding libavcodec HW acceleration with VA-API and vendor specific APIs (VDPAU, QuickSync, AMF, VideoToolbox, NVENC/CUVID). Currently the acceleration works best with VA-API on Intel and VDPAU on Nvidia proprietary drivers.

Table of Contents

• AMD support
  • Linux
    • VA-API
      • Encoding
      • Decoding
    • VDPAU
  • Windows
    • AMF
      • Tweaks
• QuickSync
  • Intel XE-Lp Alder Lake-P Graphics (Gen 12)
    • Encode
      • HEVC
    • Decode
      • HEVC
• Vulkan

AMD support

Linux

Acceleration support on AMD GPUs was tested whith RX 580 GPU on Ubuntu 18.04.02 with following results

VA-API

Encoding

H.264 encoding via VA-API, however the encoded stream produced on clean Ubuntu 18.04.2 install was broken and not decodable. After installing the amdgpu-pro-19.20-812932-ubuntu-18.04 driver and upgrading to linux kernel 5.0.0-17-generic it works correctly.

HEVC encoding via VA-API is broken, produces only artifacts

Decoding

H.264 get's decoded, however the decoded picture contains wrong colors (luma looks alright, looks like a wrong pixel format conversion is used somewhere)

VDPAU

VDPAU decoding for H.264 works, but only in copy mode. It looks like the GL-VDPAU interop doesn't properly map vdpau surfaces to GL textures.

Windows

AMF

Windows encode acceleration with AMF works but doesn't support intra-refresh which means that there will be required larger bandwidth to avoid packet loss due to excessive I-frames.

Tweaks

• enable intra refresh: https://ffmpeg.org/doxygen/trunk/amfenc__h264_8c_source.html (currently /26th Mar 21/, however, not in upstream)
• use quality constrain parameter, eg. :qp_i=36. This on one hand solves bitrate variance, on the other causes artifacts on the I-frames

QuickSync

Intel XE-Lp Alder Lake-P Graphics (Gen 12)

• CPU utilization seem to be rather even when using acceleration (some 2.5 core for one direction)

Encode

HEVC

• works using XV30 (10-bit YCbCr 4:4:4), yuyv422 or vuya (8-bit YCbCr 4:4:4). y210le (10-bit 4:2:2) can be used if intra refresh is disabled
• X2RGB10 (10-bit RGB) works but converts RGB to YCbCr internally
• bgra works when enabled HuC mode and low_power is enabled (VA-API entry point VAEntrypointEncSliceLP) – -c libavcodec:encoder=hevc_qsv:low_power=1

Decode

HEVC

• 10 and 12-bit YCbCr 4:4:4 works correctly
• RGB formats don't currently work - QSV decompresses them correctly as RGB but presents them (incorrectly) in a YCbCr container/pixelfmt xv36, xv30 or vuya (depending on bit-depth)
• UHD 4K resolution and YUV 4:4:4 (or equivalent RGB) causes "GPU Hang" errors, reported here and here; does occur with kernel 5.15 but not 6.0 regardless the in HuC mode
• not sure if there are not artifacts

The above color space issues can be tracked here.

Vulkan

UltraGrid supports Vulkan decode through ffmpeg. Currently only copy mode is supported (the hw. frames are transferred to cpu managed memory). To use this, UltraGrid needs to be build with a recent enough ffmpeg (release 6.2) and the system needs to have a recent version of the mesa package installed (23.1 for H.264 and 23.2 for HEVC).

The implementation has been observed to work on Intel and Nvidia GPUs. Currently, when running on Intel GPUs, the environment variable ANV_VIDEO_DECODE=1 needs to be set to enable Vulkan decode support in the driver.