[Bug]: Low PSNR (~8 dB in best case/~25 dB in worst case below cuda) on NV12->RGB color conversion output

[dvrogozh]

Which component impacted?

Video Processing

Is it regression? Good in old configuration?

None

What happened?

On:

• Battle Image G21 (0xe20b)
• Ubuntu 25.04, kernel 6.14.0-28-generic
• Driver stack installed per https://dgpu-docs.intel.com/driver/client/overview.html instruction from Kobuk team PPA repository, versions:
  • intel-media-va-driver-non-free: 25.3.2-0ubuntu1~25.04~ppa2
• FFmpeg n6.1.2

Usage scenario:

• Video decoding (say, h264 with NV12 output) + color space conversion to RGB24. The reason behind RGB24 is that this is a format used by default in AI training, fine tuning and inference. Specifically, that's a format used in a https://github.com/pytorch/torchcodec.
• As Intel GPUs currently does not support RGB24, we perform conversion to one of supported RGB32 color format then just copy relevant data. The issue described below does not actually depend on that, but we give examples on RGB24 as this format allows to compare results with CUDA.

Results:

• Average PSNRs comparing to CPU processed reference:

Backend	Pipeline	Avg. PSNR
CUDA	torchcodec (ffmpeg-nvdec + self-written NPP color conversion)	50.872619
QSV	ffmpeg (dec + scale_qsv)	44.622594
VAAPI	torchcodec PR-558 (ffmpeg-vaapi dec + self-written VAAPI color conversion)	44.622594
VAAPI	ffmpeg-vaapi (dec + scale_vaapi) or torchcodec PR-832	24.701474

• Overall there are visible differences between Intel and CPU/CUDA results:

  • For the ffmpeg-vaapi scale_vaapi case there is visible color difference. Current assumption is that it's due to wrong color standard settings (see Analysis section below for details)
  • For the ffmpeg-qsv case there is visible differences in object positions (for the nasa clip, that's well seen comparing frames at indexes around 199)

• NOTE: For Intel GPUs torchcodec needs to be patched. There are 2 patch versions:

  • Enable Intel GPU support in torchcodec on Linux (xpu device) meta-pytorch/torchcodec#558 - that's ffmpeg-vaapi decoding + direct vaapi color conversion
  • Full (dec+filters) ffmpeg-vaapi pipeline for Intel GPU (xpu pytorch backend) meta-pytorch/torchcodec#832 - that's ffmpeg-vaapi for decoding + conversion (with ffmpeg-vaapi filters)

• For more details see "Detailed results" section below

Expectation:

1. Intel pipeline quality to be on par with CUDA (as of now best case scenarion is 8 dB behind)
2. ffmpeg-vaapi pipeline quality to be on par with ffmpeg-qsv (as of now 16 dB behind)

What's the usage scenario when you are seeing the problem?

Video Analytics

What impacted?

No response

Debug Information

Analysis:

• Self-written VAAPI color conversion sets VAProcPipelineParameterBuffer::surface_color_standard as VAProcColorStandardBT709, other values of VAProcPipelineParameterBuffer (except widht/heigh) were zeroed
• As experiment, setting VAProcPipelineParameterBuffer::output_color_standard to VAProcColorStandardBT601 does not change output stream
• I did not check how ffmpeg-qsv and libvpl sets these parameters, but ffmpeg-qsv quality fully matches self-written VAAPI color conversion pipeline
• ffmpeg-vaapi scale_vaapi filter sets VAProcColorStandardExplicit for both VAProcPipelineParameterBuffer::surface_color_standard and VAProcPipelineParameterBuffer::output_color_standard + respective input_color_properties and output_color_properties. These settings seem to be correct on ffmpeg side, but give significantly lower quality
• As an experiment, changing VAProcPipelineParameterBuffer::surface_color_standard or VAProcPipelineParameterBuffer::output_color_standard "fixes" scale_vaapi filter which matches quality level of ffmpeg-qsv or self-written VAAPI conversion

Detailed results:

• Below are sha1 checksums and PSNR values for a number of cases for above scenario: CPU processing, CUDA processing (on A10), ffmpeg VAAPI and ffmpeg QSV.
• Input stream can be found in torchcodec repo: https://github.com/pytorch/torchcodec/blob/main/test/resources/nasa_13013.mp4
• dump.py script is a small script which uses torchcodec to dump processed videos. It can be found in https://github.com/dvrogozh/notebook/blob/master/pytorch/run-torchcodec-on-intel-gpu.md
• Torchcodec results are given to show matching with ffmpeg cmdlines + to give CUDA reference
• CPU ffmpeg output is the reference for all PSNR calculations
• Cmdlines:

python3 dump.py -i test/resources/nasa_13013.mp4 -o nasa_13013_torchcodec_cpu.rgb -d cpu -s 0:390
python3 dump.py -i test/resources/nasa_13013.mp4 -o nasa_13013_torchcodec_cuda.rgb -d cuda:0 -s 0:390
python3 dump.py -i test/resources/nasa_13013.mp4 -o nasa_13013_torchcodec_cuda.rgb -d xpu:0 -s 0:390

ffmpeg -i test/resources/nasa_13013.mp4 -vf "scale=480:270:sws_flags=bilinear,format=rgb24" -y nasa_13013_ffmpeg_cpu.rgb
ffmpeg -i test/resources/nasa_13013.mp4 -vf "format=rgb24" -y nasa_13013_ffmpeg_cpu2.rgb
ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi -i test/resources/nasa_13013.mp4 -vf "scale_vaapi=480:270:format=rgba,hwdownload,format=rgba,format=rgb24" -y nasa_13013_ffmpeg_vaapi.rgb
ffmpeg -hwaccel qsv -qsv_device /dev/dri/renderD129 -c:v h264_qsv -i test/resources/nasa_13013.mp4 -vf "scale_qsv=480:270:format=rgb32,hwdownload,format=rgb32,format=rgb24" -y nasa_13013_ffmpeg_qsv.rgb

• NOTE: results marked as "patched ffmpeg-vaapi" correspond to the ffmpeg-vaapi modification to change input/output color standard at this place: https://github.com/FFmpeg/FFmpeg/blob/b1a4534186ca51b0457579fc05a5739eb2cc45cd/libavfilter/vaapi_vpp.c#L497
• sha1sums:

$ sha1sum *.rgb
7d307c4cfcf2680e413c943646894499aa641b2e  nasa_13013_ffmpeg_cpu.rgb                       # n6.1.2
7d307c4cfcf2680e413c943646894499aa641b2e  nasa_13013_ffmpeg_cpu2.rgb                      # n6.1.2
6dcf7083da51717e06902b81dcef11adaa7ae071  nasa_13013_torchcodec_cuda.rgb
718e35799ebfdd78d91914b474748cbb3095636b  nasa_13013_ffmpeg_qsv.rgb                       # n6.1.2
718e35799ebfdd78d91914b474748cbb3095636b  nasa_13013_ffmpeg_vaapi_in=BT709_out=BT709.rgb  # patched ffmpeg-vaapi
718e35799ebfdd78d91914b474748cbb3095636b  nasa_13013_ffmpeg_vaapi_out=BT601.rgb           # patched ffmpeg-vaapi
718e35799ebfdd78d91914b474748cbb3095636b  nasa_13013_ffmpeg_vaapi_out=BT709.rgb           # patched ffmpeg-vaapi
7d307c4cfcf2680e413c943646894499aa641b2e  nasa_13013_torchcodec_cpu.rgb
6dcf7083da51717e06902b81dcef11adaa7ae071  nasa_13013_torchcodec_cuda.rgb
2e036b96d4b10501f3b3b30b9d3e3313214fc6bc  nasa_13013_ffmpeg_vaapi.rgb                     # n6.1.2
2e036b96d4b10501f3b3b30b9d3e3313214fc6bc  nasa_13013_torchcodec_ffmpeg_vaapi_filters.rgb
718e35799ebfdd78d91914b474748cbb3095636b  nasa_13013_torchcodec_vaapi.rgb

• PNSR values:

ffmpeg -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_torchcodec_cuda.rgb -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_ffmpeg_cpu.rgb -filter_complex "psnr" -f null /dev/null
[Parsed_psnr_0 @ 0x561f70f6d2c0] PSNR r:53.017530 g:48.658710 b:52.270208 average:50.872619 min:49.593809 max:51.807851

ffmpeg -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_torchcodec_ffmpeg_vaapi_filters.rgb -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_ffmpeg_cpu.rgb -filter_complex "psnr" -f null /dev/null
[Parsed_psnr_0 @ 0x5588df31d180] PSNR r:24.489627 g:24.480980 b:25.169051 average:24.701474 min:24.309702 max:26.582759

ffmpeg -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_ffmpeg_vaapi.rgb -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_ffmpeg_cpu.rgb -filter_complex "psnr" -f null /dev/null
[Parsed_psnr_0 @ 0x64513338d8c0] PSNR r:24.489627 g:24.480980 b:25.169051 average:24.701474 min:24.309702 max:26.582759

ffmpeg -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_torchcodec_vaapi.rgb -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_ffmpeg_cpu.rgb -filter_complex "psnr" -f null /dev/null
[Parsed_psnr_0 @ 0x56d294e56940] PSNR r:40.279087 g:44.622594 b:41.263727 average:41.695772 min:39.498336 max:43.352288

ffmpeg -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_ffmpeg_qsv.rgb -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_ffmpeg_cpu.rgb -filter_complex "psnr" -f null /dev/null
[Parsed_psnr_0 @ 0x61a5d080b940] PSNR r:40.279087 g:44.622594 b:41.263727 average:41.695772 min:39.498336 max:43.352288

Do you want to contribute a patch to fix the issue?

None

[jiafengy1]

For output centering, shift will be done during AVS scaling, it may cause PSNR drop compared to CPU/CUDA path but no impact on quality.

[dvrogozh]

Regarding the difference between ffmpeg-vaapi and ffmpeg-qsv:

Backend	Pipeline	Avg. PSNR
QSV	ffmpeg (dec + scale_qsv)	44.622594
VAAPI	ffmpeg-vaapi (dec + scale_vaapi) or torchcodec PR-832	24.701474

There is a difference how QSV and ffmpeg-vaapi program VAProcPipelineParameterBuffer (see log snapshots below for details).

• QSV sets surface_color_standard=output_color_standard=2 which is VAProcColorStandardBT709
• ffmpeg-vaapi sets surface_color_standard=output_color_standard=13 which is VAProcColorStandardExplicit

Further, there is actually a difference how media driver handles these settings:

• QSV:
  

  media-driver/media_softlet/linux/common/vp/ddi/ddi_vp_functions.cpp

  Lines 2079 to 2106 in de05a3b

  	if (IS_RGB_FORMAT(vpHalSurf->Format) || (vpHalSurf->Format == Format_P8))
  	{
  	switch (colorStandard)
  	{
  	case VAProcColorStandardBT2020:
  	#if (VA_MAJOR_VERSION < 1)
  	if (flag & VA_SOURCE_RANGE_FULL)
  	#else
  	if (colorRange == VA_SOURCE_RANGE_FULL)
  	#endif
  	{
  	vpHalSurf->ColorSpace = CSpace_BT2020_RGB;
  	}
  	else
  	{
  	vpHalSurf->ColorSpace = CSpace_BT2020_stRGB;
  	}
  	break;
  	case VAProcColorStandardSTRGB:
  	vpHalSurf->ColorSpace = CSpace_stRGB;
  	break;
  	case VAProcColorStandardExplicit:
  	VpSetColorStandardExplictly(vpHalSurf, colorStandard, colorProperties);
  	break;
  	case VAProcColorStandardSRGB:
  	default:
  	vpHalSurf->ColorSpace = CSpace_sRGB;
  	break;

• ffmpeg-VAAPI:
  

  media-driver/media_softlet/linux/common/vp/ddi/ddi_vp_functions.cpp

  Lines 1703 to 1713 in de05a3b

  	if (IS_RGB_FORMAT(vpHalSurf->Format))
  	{
  	switch (colorProperties.colour_primaries)
  	{
  	case COLOUR_PRIMARY_BT2020:
  	vpHalSurf->ColorSpace = (colorProperties.color_range & VA_SOURCE_RANGE_REDUCED) ? CSpace_BT2020_stRGB : CSpace_BT2020_RGB;
  	break;
  	case COLOUR_PRIMARY_BT709:
  	case COLOUR_PRIMARY_BT601:
  	vpHalSurf->ColorSpace = (colorProperties.color_range & VA_SOURCE_RANGE_REDUCED) ? CSpace_stRGB : CSpace_sRGB;
  	break;

Note that color space selection for the output surface is different:

• For ffmpeg-qsv VAProcColorStandardBT709 path the color space is set as CSpace_sRGB unconditionally towards color range (default case hit)
• For ffmpeg-vaapi VAProcColorStandardExplicit driver uses CSpace_stRGB conditionally towards color range (limited)

Why driver has different settings for these 2 paths?

Detailed logs:

• QSV:

[43713.953283][ctx 0xd0000000]	--VAProcPipelineParameterBuffer
[43713.953284][ctx 0xd0000000]	  surface = 0x00000025
[43713.953284][ctx 0xd0000000]	  surface_region
[43713.953284][ctx 0xd0000000]	    x = 0
[43713.953284][ctx 0xd0000000]	    y = 0
[43713.953284][ctx 0xd0000000]	    width = 480
[43713.953285][ctx 0xd0000000]	    height = 270
[43713.953285][ctx 0xd0000000]	  surface_color_standard = 2
[43713.953285][ctx 0xd0000000]	  output_region
[43713.953285][ctx 0xd0000000]	    x = 0
[43713.953285][ctx 0xd0000000]	    y = 0
[43713.953286][ctx 0xd0000000]	    width = 480
[43713.953286][ctx 0xd0000000]	    height = 270
[43713.953286][ctx 0xd0000000]	  output_background_color = 0xff000000
[43713.953286][ctx 0xd0000000]	  output_color_standard = 2
[43713.953286][ctx 0xd0000000]	  pipeline_flags = 0x00000000
[43713.953287][ctx 0xd0000000]	  filter_flags = 0x00000200
[43713.953287][ctx 0xd0000000]	  num_filters = 0
[43713.953287][ctx 0xd0000000]	  filters = 0x650398aa4068
[43713.953288][ctx 0xd0000000]	  num_forward_references = 0x00000000
[43713.953288][ctx 0xd0000000]	  num_backward_references = 0x00000000
[43713.953288][ctx 0xd0000000]	  rotation_state = 0x00000000
[43713.953288][ctx 0xd0000000]	  blend_state = (NULL)
[43713.953289][ctx 0xd0000000]	  mirror_state = 0x00000000
[43713.953289][ctx 0xd0000000]	  num_additional_outputs = 0
[43713.953289][ctx 0xd0000000]	  input_surface_flag = 0x00000000
[43713.953289][ctx 0xd0000000]	  output_surface_flag = 0x00000000
[43713.953289][ctx 0xd0000000]	  input_color_properties
[43713.953289][ctx 0xd0000000]	    chroma_sample_location = 0x00
[43713.953290][ctx 0xd0000000]	    color_range = 1
[43713.953290][ctx 0xd0000000]	    colour_primaries = 1
[43713.953290][ctx 0xd0000000]	    transfer_characteristics = 1
[43713.953290][ctx 0xd0000000]	    matrix_coefficients = 0
[43713.953290][ctx 0xd0000000]	  output_color_properties
[43713.953291][ctx 0xd0000000]	    chroma_sample_location = 0x00
[43713.953291][ctx 0xd0000000]	    color_range = 1
[43713.953291][ctx 0xd0000000]	    colour_primaries = 1
[43713.953291][ctx 0xd0000000]	    transfer_characteristics = 1
[43713.953291][ctx 0xd0000000]	    matrix_coefficients = 0
[43713.953292][ctx 0xd0000000]	  processing_mode = 0
[43713.953292][ctx 0xd0000000]	  output_hdr_metadata = (NULL)

• ffmpeg-vaapi:

[43799.004518][ctx 0xd0000000]	--VAProcPipelineParameterBuffer
[43799.004526][ctx 0xd0000000]	  surface = 0x00000023
[43799.004527][ctx 0xd0000000]	  surface_region
[43799.004528][ctx 0xd0000000]	    x = 0
[43799.004529][ctx 0xd0000000]	    y = 0
[43799.004529][ctx 0xd0000000]	    width = 480
[43799.004530][ctx 0xd0000000]	    height = 270
[43799.004531][ctx 0xd0000000]	  surface_color_standard = 13
[43799.004532][ctx 0xd0000000]	  output_region = (NULL)
[43799.004532][ctx 0xd0000000]	  output_background_color = 0xff000000
[43799.004533][ctx 0xd0000000]	  output_color_standard = 13
[43799.004534][ctx 0xd0000000]	  pipeline_flags = 0x00000000
[43799.004535][ctx 0xd0000000]	  filter_flags = 0x00000200
[43799.004536][ctx 0xd0000000]	  num_filters = 0
[43799.004537][ctx 0xd0000000]	  filters = (nil)
[43799.004538][ctx 0xd0000000]	  num_forward_references = 0x00000000
[43799.004539][ctx 0xd0000000]	  num_backward_references = 0x00000000
[43799.004540][ctx 0xd0000000]	  rotation_state = 0x00000000
[43799.004540][ctx 0xd0000000]	  blend_state = (NULL)
[43799.004541][ctx 0xd0000000]	  mirror_state = 0x00000000
[43799.004542][ctx 0xd0000000]	  num_additional_outputs = 0
[43799.004542][ctx 0xd0000000]	  input_surface_flag = 0x00000000
[43799.004543][ctx 0xd0000000]	  output_surface_flag = 0x00000000
[43799.004544][ctx 0xd0000000]	  input_color_properties
[43799.004544][ctx 0xd0000000]	    chroma_sample_location = 0x06
[43799.004545][ctx 0xd0000000]	    color_range = 1
[43799.004546][ctx 0xd0000000]	    colour_primaries = 1
[43799.004547][ctx 0xd0000000]	    transfer_characteristics = 1
[43799.004547][ctx 0xd0000000]	    matrix_coefficients = 1
[43799.004548][ctx 0xd0000000]	  output_color_properties
[43799.004549][ctx 0xd0000000]	    chroma_sample_location = 0x06
[43799.004549][ctx 0xd0000000]	    color_range = 1
[43799.004550][ctx 0xd0000000]	    colour_primaries = 1
[43799.004551][ctx 0xd0000000]	    transfer_characteristics = 1
[43799.004551][ctx 0xd0000000]	    matrix_coefficients = 0
[43799.004552][ctx 0xd0000000]	  processing_mode = 0
[43799.004553][ctx 0xd0000000]	  output_hdr_metadata = (NULL)

[dvrogozh]

It occurs that the ffmpeg CPU format filter which we use to convert to RGB24 outputs full color range (pc in ffmpeg log output, and tv stands for limited). See snapshots of the ffmpeg output below:

$ ffmpeg -i test/resources/nasa_13013.mp4 -vf "format=rgb24" -y nasa_13013_ffmpeg_cpu2.rgb
...
  Stream #0:3[0x4](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 480x270 [SAR 1:1 DAR 16:9], 128 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
...
  Stream #0:0(eng): Video: rawvideo (RGB[24] / 0x18424752), rgb24(pc, gbr/bt709/bt709, progressive), 480x270 [SAR 1:1 DAR 16:9], q=2-31, 93218 kb/s, 29.97 fps, 29.97 tbn (default)
...

This selection seems to happen here in ffmpeg code: https://github.com/FFmpeg/FFmpeg/blob/2451e06f19fe148456e4994b6f1259683cb04ef3/libavfilter/avfiltergraph.c#L837

To match ffmpeg-vaapi with CPU we can use out_range=pc on the ffmpeg-vaapi cmdline. This will match the output with ffmpeg-qsv and improve PSNR:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi -i test/resources/nasa_13013.mp4 -vf "scale_vaapi=format=rgba:out_range=pc,hwdownload,format=rgba,format=rgb24" -y nasa_13013_ffmpeg_vaapi.rgb

ffmpeg -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_ffmpeg_vaapi.rgb -f rawvideo -pix_fmt rgb24 -s:v 480x270 -i nasa_13013_ffmpeg_cpu.rgb -filter_complex "psnr" -f null /dev/null
[Parsed_psnr_0 @ 0x75b988002ec0] PSNR r:40.279087 g:44.622594 b:41.263727 average:41.695772 min:39.498336 max:43.352288

[dvrogozh]

scale_vaapi filter has out_chroma_location= option. However, applying it to ffmpeg-vaapi pipeline described above does not make any difference to the output. All output streams are sha1sum identical. The setting gets applied to the output color properties of VAProcPipelineParameterBuffer which is RGBA. I guess driver does not recognize such settings for non-YUV output format?

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi -i $HOME/git/torchcodec/test/resources/nasa_13013.mp4 \
  -vf "scale_vaapi=format=rgba:out_range=pc:out_chroma_location=bottomcenter,hwdownload,format=rgba,format=rgb24" \
  -y nasa_13013_ffmpeg_vaapi_bottomcenter.rgb

This made me think to add additional scale_vaapi to first convert to NV12 changing chroma location and then convert to RGBA (note that decoded stream is marked by FFmpeg as 0x06 which is centerleft). This worked better in a sense that driver did produce different outputs.

LOC="topleft centerleft bottomleft topcenter centercenter bottomcenter"
for i in $LOC;
  ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi -i $HOME/git/torchcodec/test/resources/nasa_13013.mp4 \
    -vf "scale_vaapi=out_chroma_location=$i,scale_vaapi=format=rgba:out_range=pc,hwdownload,format=rgba,format=rgb24" \
    -y nasa_13013_ffmpeg_vaapi_$i.rgb
done

However, I noticed 2 things:

1. None of the results match the case of the initial single scale_vaapi filter. How can that be? Effectively we should have seen the match for the centerleft case which should not have modified input at all as it already had chroma sitting matching requested output. It should have just copied. But output is different. Why?
2. None of the results reach PSNR 50. Some of the outputs however has visually smaller objects pixel shifts comparing to CPU reference.

Chroma location	sha1sum	PSNR
single scale_vaapi (centerleft)	718e35799ebfdd78d91914b474748cbb3095636b	41.695772
bottomcenter	d43f0f153ad6118f3a73f3b3c55144f8cbd80966	41.755833
bottomleft	b7610f44a5abc78f198ee006c1f19e3747215922	40.895935
centercenter	b3d50c64a6388df00c3a98476055530a0772a4de	42.647297
centerleft	b3d50c64a6388df00c3a98476055530a0772a4de	42.647297
topcenter	3e91f4276dca1a047131ea40764d68b5f95a7b2e	42.023334
topleft	ffe4a639fee71b0d8648b000d8d91167ee5286d9	41.278626

Single scale_vaapi (centerleft input) pipeline:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi -i $HOME/git/torchcodec/test/resources/nasa_13013.mp4 \
  -vf "scale_vaapi=format=rgba:out_range=pc,hwdownload,format=rgba,format=rgb24" \
  -y nasa_13013_ffmpeg_vaapi.rgb

[peiyigu-intel]

scale_vaapi filter has out_chroma_location= option. However, applying it to ffmpeg-vaapi pipeline described above does not make any difference to the output. All output streams are sha1sum identical. The setting gets applied to the output color properties of VAProcPipelineParameterBuffer which is RGBA. I guess driver does not recognize such settings for non-YUV output format?

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi -i $HOME/git/torchcodec/test/resources/nasa_13013.mp4 \
  -vf "scale_vaapi=format=rgba:out_range=pc:out_chroma_location=bottomcenter,hwdownload,format=rgba,format=rgb24" \
  -y nasa_13013_ffmpeg_vaapi_bottomcenter.rgb

This made me think to add additional scale_vaapi to first convert to NV12 changing chroma location and then convert to RGBA (note that decoded stream is marked by FFmpeg as 0x06 which is centerleft). This worked better in a sense that driver did produce different outputs.

LOC="topleft centerleft bottomleft topcenter centercenter bottomcenter"
for i in $LOC;
  ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi -i $HOME/git/torchcodec/test/resources/nasa_13013.mp4 \
    -vf "scale_vaapi=out_chroma_location=$i,scale_vaapi=format=rgba:out_range=pc,hwdownload,format=rgba,format=rgb24" \
    -y nasa_13013_ffmpeg_vaapi_$i.rgb
done

However, I noticed 2 things:

1. None of the results match the case of the initial single scale_vaapi filter. How can that be? Effectively we should have seen the match for the centerleft case which should not have modified input at all as it already had chroma sitting matching requested output. It should have just copied. But output is different. Why?
2. None of the results reach PSNR 50. Some of the outputs however has visually smaller objects pixel shifts comparing to CPU reference.

Chroma location sha1sum PSNR
single scale_vaapi (centerleft) 718e35799ebfdd78d91914b474748cbb3095636b 41.695772
bottomcenter d43f0f153ad6118f3a73f3b3c55144f8cbd80966 41.755833
bottomleft b7610f44a5abc78f198ee006c1f19e3747215922 40.895935
centercenter b3d50c64a6388df00c3a98476055530a0772a4de 42.647297
centerleft b3d50c64a6388df00c3a98476055530a0772a4de 42.647297
topcenter 3e91f4276dca1a047131ea40764d68b5f95a7b2e 42.023334
topleft ffe4a639fee71b0d8648b000d8d91167ee5286d9 41.278626
Single scale_vaapi (centerleft input) pipeline:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi -i $HOME/git/torchcodec/test/resources/nasa_13013.mp4 \
  -vf "scale_vaapi=format=rgba:out_range=pc,hwdownload,format=rgba,format=rgb24" \
  -y nasa_13013_ffmpeg_vaapi.rgb

Hi @dvrogozh
Your workload involves converting from NV12 (420) to ARGB (444). In this case, for the output, ARGB does not require chroma siting since it is in 444 format. However, NV12 requires chroma siting because it needs to perform chroma upsampling. This process requires knowledge of the original chroma siting position. The driver distinguishes between the chroma siting settings for input and output respectively. Therefore, you would need to configure the chroma siting for the input rather than the output.

[dvrogozh]

Therefore, you would need to configure the chroma siting for the input rather than the output.

@peiyigu-intel, the chroma sitting is actually configured by ffmpeg-vaapi automatically per input stream from the decoder. In my case it programs 0x6 = VA_CHROMA_SITING_VERTICAL_CENTER | VA_CHROMA_SITING_HORIZONTAL_LEFT (centerleft). Manually overriding this setting to check if ffmpeg made a mistake does not actually give a better PSNR result - they all fluctuate around ~41dB:

Chroma location	sha1sum	PSNR
bottomcenter	d43f0f153ad6118f3a73f3b3c55144f8cbd80966	41.755833
bottomleft	b7610f44a5abc78f198ee006c1f19e3747215922	40.895935
centercenter	718e35799ebfdd78d91914b474748cbb3095636b	41.695772
centerleft	718e35799ebfdd78d91914b474748cbb3095636b	41.695772
topcenter	3e91f4276dca1a047131ea40764d68b5f95a7b2e	42.023334
topleft	ffe4a639fee71b0d8648b000d8d91167ee5286d9	41.278626

[dvrogozh]

I had a chance to run ffmpeg-vaapi on A770 which supports 2 modes of execution - one on SFC, another on EUs. Here are results:

work mode	GPU	sha1sum	PSNR
mode=default	A770	718e35799ebfdd78d91914b474748cbb3095636b	41.695772
mode=fast	A770	b9422cbeac251b63dc14821a15ae373dfd9609a5	50.323587
mode=hq	A770	718e35799ebfdd78d91914b474748cbb3095636b	41.695772
mode=nl_anamorphic	A770	718e35799ebfdd78d91914b474748cbb3095636b	41.695772
mode=default	B580	718e35799ebfdd78d91914b474748cbb3095636b	41.695772
mode=hq	B580	718e35799ebfdd78d91914b474748cbb3095636b	41.695772
mode=fast	B580	718e35799ebfdd78d91914b474748cbb3095636b	41.695772
mode=nl_anamorphic	B580	718e35799ebfdd78d91914b474748cbb3095636b	41.695772
cuda	A10	6dcf7083da51717e06902b81dcef11adaa7ae071	50.872619

As you can see, in the fast mode on A770 we are able to achieve similar PSNR value as CUDA. On B580 all modes behave the same. Questions:

1. Is "fast" mode on A770 running on EUs via the kernel and "hq" on SFC?
2. Is that a bug in the media driver that "fast" mode on B580 changed behavior or that's a planned change?

[peiyigu-intel]

1 We have confirmed the behavior of ffmpeg's fast mode. The true meaning of fast mode does not necessarily indicate using the EU path; it corresponds to VA_FILTER_SCALING_FAST = VPHAL_SCALING_PREFER_SFC_FOR_VEBOX. On legacy platforms like the A770 generation, when SFC_FOR_VEBOX is requested but no vebox features are available, it will utilize the EU path. This is the behavioral design for this generation of platforms. Starting with the B platform, the overall design philosophy requires utilizing VESFC whenever possible. Therefore, even without VEBOX features, it will route to SFC rather than EU. This represents the distinction between B and A platforms and is by design. For VAAPI, if one prefers to utilize the EU path as much as possible, the corresponding API is VA_PROC_PIPELINE_FAST. We understand that ffmpeg does not have this configuration option.

2 We have examined the results of the three paths - EU path, VE path, and SFC path - on both B and A platforms. The VE and SFC results are aligned. We manually extracted the input NV12 values one by one and performed verification using C code. The VE SFC results are correct. However, on both A and B platforms, the EU path differs due to computational precision issues, as it uses limited integer calculations with different rounding approaches, resulting in varying EU path outcomes. On newer platforms that have not yet undergone PV, we have comprehensively updated the EU path, implementing higher-precision float calculations. We have tested the new EU path, and its results are nearly aligned with the VE SFC results from A/B platforms. So, to summarize, it turns out that the old EU path is actually further from our reference.