Hw. acceleration support

UltraGrid supports video encoding and decoding libavcodec HW acceleration with VA-API and vendor specific APIs (VDPAU, QuickSync, AMF, VideoToolbox, NVENC/CUVID). Currently the acceleration works best with VA-API on Intel and VDPAU on Nvidia proprietary drivers.

Table of Contents

• AMD support
  • Linux
    • VA-API
      • Encoding
      • Decoding
    • VDPAU
  • Windows
    • AMF
      • Tweaks
• QuickSync
  • Intel XE-Lp Alder Lake-P Graphics (Gen 12)
    • Encode
      • HEVC
    • Decode
      • HEVC

AMD support

Linux

Acceleration support on AMD GPUs was tested whith RX 580 GPU on Ubuntu 18.04.02 with following results

VA-API

Encoding

H.264 encoding via VA-API, however the encoded stream produced on clean Ubuntu 18.04.2 install was broken and not decodable. After installing the amdgpu-pro-19.20-812932-ubuntu-18.04 driver and upgrading to linux kernel 5.0.0-17-generic it works correctly.

HEVC encoding via VA-API is broken, produces only artifacts

Decoding

H.264 get's decoded, however the decoded picture contains wrong colors (luma looks alright, looks like a wrong pixel format conversion is used somewhere)

VDPAU

VDPAU decoding for H.264 works, but only in copy mode. It looks like the GL-VDPAU interop doesn't properly map vdpau surfaces to GL textures.

Windows

AMF

Windows encode acceleration with AMF works but doesn't support intra-refresh which means that there will be required larger bandwidth to avoid packet loss due to excessive I-frames.

Tweaks

• enable intra refresh: https://ffmpeg.org/doxygen/trunk/amfenc__h264_8c_source.html (currently /26th Mar 21/, however, not in upstream)
• use quality constrain parameter, eg. :qp_i=36. This on one hand solves bitrate variance, on the other causes artifacts on the I-frames

QuickSync

Intel XE-Lp Alder Lake-P Graphics (Gen 12)

• CPU utilization seem to be rather even when using acceleration (some 2.5 core for one direction)

Encode

HEVC

• works using XV30 (10-bit YCbCr 4:4:4), yuyv422 or vuya (8-bit YCbCr 4:4:4). y210le (10-bit 4:2:2) can be used if intra refresh is disabled
• X2RGB10 (10-bit RGB) seems working but presumably it converts internally RGB to YCbCr while leaving RGB in metadata. It is blacklisted now but can be enabled by --param lavc-use-codec=x2rgb10le.
• bgra works when enabled HuC mode and low_power is enabled (VA-API entry point VAEntrypointEncSliceLP) – -c libavcodec:encoder=hevc_qsv:low_power=1

Decode

HEVC

• 10 and 12-bit YCbCr 4:4:4 works correctly
• RGB formats:
  • without HuC - don't currently work - QSV decompresses them correctly as RGB but presents them (incorrectly) in a YCbCr container/pixelfmt xv36, xv30 or vuya (depending on bit-depth)
  • with HuC - work correctly
• UHD 4K resolution and YUV 4:4:4 (or equivalent RGB) causes "GPU Hang" errors, reported here and here; does occur with kernel 5.15 but not 6.0 regardless the in HuC mode
• not sure if there are not artifacts

The above color space issues can be tracked here.