FFmpeg with vaapi hardware decoder on Allwinner H3

[yateya]

I am trying to make the hardware decoder work with ffmpeg on orangepipcplus (allwinner H3). Operating system in Ubuntu 16.04.3 from Armbian.

Following these Sunxi-cedrus steps I built bootlin cedrus kernel and I have /dev/video0, /dev/media0 and /dev/dri/card0 all in the built kernel.

Also cloned and built both libva-utils, libva and libva-v4l2-request. Test application v4l2-request-test works fine.

Downloaded and build FFmpeg-4.1.3 with this flags ./configure --enable-vaapi . Then ran ffmpeg with command ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/card0 -i /root/big_buck_bunny_480p_H264_AAC_25fps_1800K.MP4 -an output.mp4 -loglevel debug

ffmpeg version 4.1.3 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.10) 20160609
configuration: --enable-vaapi
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
Splitting the commandline.
Reading option '-hwaccel' ... matched as option 'hwaccel' (use HW accelerated decoding) with argument 'vaapi'.
Reading option '-vaapi_device' ... matched as option 'vaapi_device' (set VAAPI hardware device (DRM path or X11 display name)) with argument '/dev/dri/card0'.
Reading option '-i' ... matched as input url with argument '/root/big_buck_bunny_480p_H264_AAC_25fps_1800K.MP4'.
Reading option '-an' ... matched as option 'an' (disable audio) with argument '1'.
Reading option 'output.mp4' ... matched as output url.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option vaapi_device (set VAAPI hardware device (DRM path or X11 display name)) with argument /dev/dri/card0.
[AVHWDeviceContext @ 0x145b280] Opened VA display via DRM device /dev/dri/card0.
[AVHWDeviceContext @ 0x145b280] libva: VA-API version 1.5.0
[AVHWDeviceContext @ 0x145b280] libva: va_getDriverName() returns -1
[AVHWDeviceContext @ 0x145b280] libva: User requested driver 'v4l2_request'
[AVHWDeviceContext @ 0x145b280] libva: Trying to open /usr/local/lib/dri/v4l2_request_drv_video.so
[AVHWDeviceContext @ 0x145b280] libva: Found init function __vaDriverInit_1_5
[AVHWDeviceContext @ 0x145b280] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x145b280] Initialised VAAPI connection: version 1.5
[AVHWDeviceContext @ 0x145b280] Format 0x3231564e -> nv12.
[AVHWDeviceContext @ 0x145b280] VAAPI driver: v4l2-request.
[AVHWDeviceContext @ 0x145b280] Driver not found in known nonstandard list, using standard behaviour.
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url /root/big_buck_bunny_480p_H264_AAC_25fps_1800K.MP4.
Applying option hwaccel (use HW accelerated decoding) with argument vaapi.
Successfully parsed a group of options.
Opening an input file: /root/big_buck_bunny_480p_H264_AAC_25fps_1800K.MP4.
[NULL @ 0x14712f0] Opening '/root/big_buck_bunny_480p_H264_AAC_25fps_1800K.MP4' for reading
[file @ 0x1471940] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x14712f0] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x14712f0] ISO: File Type Major Brand: mp42
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x14712f0] Unknown dref type 0x206c7275 size 12
Last message repeated 3 times
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x14712f0] Before avformat_find_stream_info() pos: 17960803 bytes read:58567 seeks:1 nb_streams:4
[h264 @ 0x1472420] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x1472420] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x1472420] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x1472420] Format yuv420p chosen by get_format().
[h264 @ 0x1472420] Reinit context to 864x480, pix_fmt: yuv420p
[h264 @ 0x1472420] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
Last message repeated 5 times
[aac @ 0x1478ca0] skip whole frame, skip left: 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x14712f0] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x14712f0] After avformat_find_stream_info() pos: 293589 bytes read:354627 seeks:2 frames:14
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/root/big_buck_bunny_480p_H264_AAC_25fps_1800K.MP4':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: isommp423gp5
creation_time : 2011-08-04T02:50:53.000000Z
title : big_buck_bunny_480p_surround-fix.avi_002.AVI.MP4
artist : created with SUPER(C).v2011.bld.49
encoder : eRightSoft
comment : 10:51:04
Duration: 00:01:15.00, start: 0.000000, bitrate: 1915 kb/s
Stream #0:0(und), 13, 1/25: Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 854x480 (864x480) [SAR 1:1 DAR 427:240], 0/1, 1814 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc (default)
Metadata:
creation_time : 2011-08-05T02:50:25.000000Z
handler_name : VideoHandler
Stream #0:1(und), 1, 1/44100: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 95 kb/s (default)
Metadata:
creation_time : 2011-08-04T02:50:53.000000Z
handler_name : GPAC ISO Audio Handler
Stream #0:2(und), 0, 1/600: Data: none (mp4s / 0x7334706D), 0/1, 0 kb/s (default)
Metadata:
creation_time : 2011-08-04T02:50:57.000000Z
handler_name : GPAC MPEG-4 OD Handler
Stream #0:3(und), 0, 1/600: Data: none (mp4s / 0x7334706D), 0/1, 0 kb/s (default)
Metadata:
creation_time : 2011-08-04T02:50:57.000000Z
handler_name : GPAC MPEG-4 BIFS Handler
Successfully opened the file.
Parsing a group of options: output url output.mp4.
Applying option an (disable audio) with argument 1.
Successfully parsed a group of options.
Opening an output file: output.mp4.
File 'output.mp4' already exists. Overwrite ? [y/N] Y
[file @ 0x14769e0] Setting default whitelist 'file,crypto'
Successfully opened the file.
detected 4 logical cores
[h264 @ 0x148e2f0] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x148e2f0] nal_unit_type: 8(PPS), nal_ref_idc: 3
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 1 times
[h264 @ 0x148e2f0] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x148e2f0] Format vaapi_vld chosen by get_format().
[h264 @ 0x148e2f0] Format vaapi_vld requires hwaccel initialisation.
[h264 @ 0x148e2f0] Considering format 0x3231564e -> nv12.
[h264 @ 0x148e2f0] Picked nv12 (0x3231564e) as best match for yuv420p.
[AVHWFramesContext @ 0xb424b4d0] Created surface 0x4000000.
[AVHWFramesContext @ 0xb424b4d0] Direct mapping possible.
v4l2-request: Unable to set format for type 1: Device or resource busy
[AVHWFramesContext @ 0xb424b4d0] Failed to create surface: 1 (operation failed).
[h264 @ 0x148e2f0] Failed setup for format vaapi_vld: hwaccel initialisation returned error.
[h264 @ 0x148e2f0] Format vaapi_vld not usable, retrying get_format() without it.
[h264 @ 0x148e2f0] Format yuv420p chosen by get_format().
[h264 @ 0x148e2f0] Reinit context to 864x480, pix_fmt: yuv420p
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x15fe710] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x1569f00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x1550340] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 0x15dfb50] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[graph 0 input from stream 0:0 @ 0x145b110] Setting 'video_size' to value '854x480'
[graph 0 input from stream 0:0 @ 0x145b110] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x145b110] Setting 'time_base' to value '1/25'
[graph 0 input from stream 0:0 @ 0x145b110] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x145b110] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x145b110] Setting 'frame_rate' to value '25/1'
[graph 0 input from stream 0:0 @ 0x145b110] w:854 h:480 pixfmt:yuv420p tb:1/25 fr:25/1 sar:1/1 sws_param:flags=2
[format @ 0x15fda20] Setting 'pix_fmts' to value 'yuv420p'
[AVFilterGraph @ 0x14784b0] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[mpeg4 @ 0x1475a60] intra_quant_bias = 0 inter_quant_bias = -64
Output #0, mp4, to 'output.mp4':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: isommp423gp5
comment : 10:51:04
title : big_buck_bunny_480p_surround-fix.avi_002.AVI.MP4
artist : created with SUPER(C).v2011.bld.49
encoder : Lavf58.20.100
Stream #0:0(und), 0, 1/12800: Video: mpeg4, 1 reference frame (mp4v / 0x7634706D), yuv420p(left), 854x480 [SAR 1:1 DAR 427:240], 0/1, q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc (default)
Metadata:
creation_time : 2011-08-05T02:50:25.000000Z
handler_name : VideoHandler
encoder : Lavc58.35.100 mpeg4
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Clipping frame in rate conversion by 0.000008
[h264 @ 0x148e2f0] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 0x15fe710] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 0x1569f00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 0x1550340] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 0x15dfb50] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 0x148e2f0] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 0x15fe710] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
frame= 8 fps=0.0 q=7.8 Lsize= 107kB time=00:00:00.28 bitrate=3130.0kbits/s speed=0.998x
video:106kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.987936%
Input file #0 (/root/big_buck_bunny_480p_H264_AAC_25fps_1800K.MP4):
Input stream #0:0 (video): 12 packets read (267476 bytes); 9 frames decoded;
Input stream #0:1 (audio): 0 packets read (0 bytes);
Input stream #0:2 (data): 0 packets read (0 bytes);
Input stream #0:3 (data): 0 packets read (0 bytes);
Total: 12 packets (267476 bytes) demuxed
Output file #0 (output.mp4):
Output stream #0:0 (video): 8 frames encoded; 8 packets muxed (108509 bytes);
Total: 8 packets (108509 bytes) muxed
9 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x1476a30] Statistics: 2 seeks, 4 writeouts
[AVIOContext @ 0x1479a90] Statistics: 354627 bytes read, 2 seeks

The important part in the log

[h264 @ 0x1523ca0] Format vaapi_vld chosen by get_format().
[h264 @ 0x1523ca0] Format vaapi_vld requires hwaccel initialisation.
[h264 @ 0x1523ca0] Considering format 0x3231564e -> nv12.
[h264 @ 0x1523ca0] Picked nv12 (0x3231564e) as best match for yuv420p.
[AVHWFramesContext @ 0xb424b4d0] Created surface 0x4000000.
[AVHWFramesContext @ 0xb424b4d0] Direct mapping possible.
v4l2-request: Unable to set format for type 1: Device or resource busy
[AVHWFramesContext @ 0xb424b4d0] Failed to create surface: 1 (operation failed).
[h264 @ 0x1523ca0] Failed setup for format vaapi_vld: hwaccel initialisation returned error.
[h264 @ 0x1523ca0] Format vaapi_vld not usable, retrying get_format() without it.

[Dreyk007]

@yateya Please tell me which version (branch) of libva you used? I will not help you solve the problem, but it is important for me

[yateya]

@yateya Please tell me which version (branch) of libva you used? I will not help you solve the problem, but it is important for me

libva 2.4.1

[Dreyk007]

libva 2.4.1

thank you! In the end, did you manage to get it to work (ffmpeg or somethink like this +libva-v4l2-request)?

[nazar554]

Worked for me with libva 2.5.0, but only got output of first 5 seconds of the video (Arch Linux & Orange Pi Zero, mainline kernel linux-armv7)
libva-v4l2-request-git 2019.03.r7.g7f359be-1
log.txt

uname -a
Linux orangepi 5.3.1-1-ARCH #1 SMP PREEMPT Sat Sep 21 15:09:42 UTC 2019 armv7l GNU/Linux

vainfo                                                                                                                                                                   
error: can't connect to X server!
vainfo: VA-API version: 1.5 (libva 2.5.0)
vainfo: Driver version: v4l2-request
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264MultiviewHigh      : VAEntrypointVLD
      VAProfileH264StereoHigh         : VAEntrypointVLD