How to load an image2 format from FFMPEG?

[intermension]

I'm creating thumbnails from video using FFMPEG pipes. When piped the image is returned an image2 format which i believe to be a YUV file of some kind. In .NET 47 I can do the following to load the image for subsequent processing:

using (var img = Image.FromStream(thumbStream))

and the conversion is automatically handled but in ImageSharp it fails:

using (Image image = Image.Load(thumbStream))
                    
Image cannot be loaded. Available decoders:
 - GIF : GifDecoder
 - TGA : TgaDecoder
 - PNG : PngDecoder
 - BMP : BmpDecoder
 - JPEG : JpegDecoder

Does anyone know if there is something simple i can do to resolve this?

[antonfirsov]

ImageSharp only has builtin support for the formats listed in the error message.

Maybe you can use FFMediaToolkit to load the stream and get the frame as ImageData as described here:
https://github.com/radek-k/FFMediaToolkit#code-samples

Then convert it to an ImageSharp image like this:
https://github.com/radek-k/FFMediaToolkit#usage-details

[JimBobSquarePants]

That's cool they have a dedicated example for us!

[intermension]

That's helpful. Thanks.

[JimBobSquarePants]

I’m curious... were you originally using System.Drawing? If so, I’m wondering if it could simply be a stream position issue?

[intermension]

Yes I was. I thought the same but setting position back to 0 did not resolve the issue.

[JimBobSquarePants]

Thanks... I wonder what the image format of the stream contents is?

[intermension]

It is actually a JFIF:

Header (from winhex)
FFD8FFE000104A46494600010200000100010000FFFE00104C61766335382E33352E31303000FFDB0043

So I'm not sure what ImageSharp uses to determine compatibility and if i save the stream to disk then image editors can use it.

[JimBobSquarePants]

Ok.... That's weird... We should be able to detect it.

ImageSharp/src/ImageSharp/Formats/Jpeg/JpegImageFormatDetector.cs

Lines 58 to 60 in eab04e4

	private bool IsJpeg(ReadOnlySpan<byte> header) =>
	header[0] == 0xFF && // 255
	header[1] == 0xD8; // 216

[antonfirsov]

Out of curiosity: how is JFIF/JPEG related to FFMPEG?