video: ov5640 support for vga , qvga and qqvga not there

[adityaganesh]

Describe the bug

The current driver for 0v5640 does not support vga , qvga or qqvga resolutions

Impact
The ov5640 camera supports the above mentioned resolutions but the driver does not support so using this driver with microcontrollers which have ram lesser than 614.4kB in its current state is not possible

Environment (please complete the following information):

• Windows 11
• Zephyr SDK release version 4.0

[adityaganesh]

Looks like the issue is resolved in main branch but not committed to the release branch of 4.0

[KurtE]

Quick note: Today I started experimenting to see if I could get this camera to work with the ArduinoCore-zephyr project, in my case
I was trying to configure it for a Portenta H7...

Note: The main camera code in camera.cpp, is looking at the table of Width, Height, Format for a match. And the current stuff up
in main still only has:

static const struct video_format_cap csi2_fmts[] = {
	OV5640_VIDEO_FORMAT_CAP(1280, 720, VIDEO_PIX_FMT_RGB565),
	OV5640_VIDEO_FORMAT_CAP(1280, 720, VIDEO_PIX_FMT_YUYV),
	OV5640_VIDEO_FORMAT_CAP(640, 480, VIDEO_PIX_FMT_RGB565),
	OV5640_VIDEO_FORMAT_CAP(640, 480, VIDEO_PIX_FMT_YUYV),
	{0}};

Whereas the OV2640 has a much more complete table:

static const struct video_format_cap fmts[] = {
	OV2640_VIDEO_FORMAT_CAP(160, 120, VIDEO_PIX_FMT_RGB565),   /* QQVGA */
	OV2640_VIDEO_FORMAT_CAP(176, 144, VIDEO_PIX_FMT_RGB565),   /* QCIF  */
	OV2640_VIDEO_FORMAT_CAP(240, 160, VIDEO_PIX_FMT_RGB565),   /* HQVGA */
	OV2640_VIDEO_FORMAT_CAP(240, 240, VIDEO_PIX_FMT_RGB565),   /* 240x240 */
	OV2640_VIDEO_FORMAT_CAP(320, 240, VIDEO_PIX_FMT_RGB565),   /* QVGA  */
	OV2640_VIDEO_FORMAT_CAP(352, 288, VIDEO_PIX_FMT_RGB565),   /* CIF   */
	OV2640_VIDEO_FORMAT_CAP(640, 480, VIDEO_PIX_FMT_RGB565),   /* VGA   */
	OV2640_VIDEO_FORMAT_CAP(800, 600, VIDEO_PIX_FMT_RGB565),   /* SVGA  */
	OV2640_VIDEO_FORMAT_CAP(1024, 768, VIDEO_PIX_FMT_RGB565),  /* XVGA  */
	OV2640_VIDEO_FORMAT_CAP(1280, 1024, VIDEO_PIX_FMT_RGB565), /* SXGA  */
	OV2640_VIDEO_FORMAT_CAP(1600, 1200, VIDEO_PIX_FMT_RGB565), /* UXGA  */
	OV2640_VIDEO_FORMAT_CAP(160, 120, VIDEO_PIX_FMT_JPEG),     /* QQVGA */
	OV2640_VIDEO_FORMAT_CAP(176, 144, VIDEO_PIX_FMT_JPEG),     /* QCIF  */
	OV2640_VIDEO_FORMAT_CAP(240, 160, VIDEO_PIX_FMT_JPEG),     /* HQVGA */
	OV2640_VIDEO_FORMAT_CAP(320, 240, VIDEO_PIX_FMT_JPEG),     /* QVGA  */
	OV2640_VIDEO_FORMAT_CAP(352, 288, VIDEO_PIX_FMT_JPEG),     /* CIF   */
	OV2640_VIDEO_FORMAT_CAP(640, 480, VIDEO_PIX_FMT_JPEG),     /* VGA   */
	OV2640_VIDEO_FORMAT_CAP(800, 600, VIDEO_PIX_FMT_JPEG),     /* SVGA  */
	OV2640_VIDEO_FORMAT_CAP(1024, 768, VIDEO_PIX_FMT_JPEG),    /* XVGA  */
	OV2640_VIDEO_FORMAT_CAP(1280, 1024, VIDEO_PIX_FMT_JPEG),   /* SXGA  */
	OV2640_VIDEO_FORMAT_CAP(1600, 1200, VIDEO_PIX_FMT_JPEG),   /* UXGA  */
	{ 0 }
};

Will try more with 2640, but wish the OV5640 was updated.

Thanks.

[josuah]

I think I missed this message earlier!

The OV5640 has a built-in GPU that pemits to emmit JPEG data directly, with an integrated auto-focus, auto-exposure, auto-white-balance, which makes it a popular image sensor for makers and firmware developers, who can skip all the effort of doing all this image post processing on a low-power platform.

This however makes the OV5640 very complex (and in turn increases noise on the output image), and makes it the biggest image sensor driver of all Linux, so it is normal that it appears particularly complex.

~/Code/Linux/drivers/media/i2c master.$ wc -l *.c | sort -nr | head
   4058 ds90ub960.c <- not an image sensor, FDP-Link
   4031 ov5640.c <- this one
   3697 adv7604.c
   3690 ov64a40.c <- 64 MP image sensor
    [...]

Still, the basics are:

1. Undocumented registers permits to select a "mode of operation" that determines the base resolution and framerate
2. On top of this, extra configuration permits to control further the extra resolution.

The Zephyr drivers seems to combine them in just one big block without annotation:

zephyr/drivers/video/ov5640.c

Lines 469 to 474 in ed45960

	static const struct ov5640_reg csi2_low_res_params[] = {
	{0x3800, 0x00}, {0x3801, 0x00}, {0x3802, 0x00}, {0x3803, 0x04}, {0x3804, 0x0a},
	{0x3805, 0x3f}, {0x3806, 0x07}, {0x3807, 0x9b}, {0x3808, 0x02}, {0x3809, 0x80},
	{0x380a, 0x01}, {0x380b, 0xe0}, {0x380c, 0x07}, {0x380d, 0x68}, {0x380e, 0x03},
	{0x380f, 0xd8}, {0x3810, 0x00}, {0x3811, 0x10}, {0x3812, 0x00}, {0x3813, 0x06},
	{0x3814, 0x31}, {0x3815, 0x31}, {0x3824, 0x02}, {0x460c, 0x22}};

One way would be to directly add the extra registers for the missing resolution by looking at the now-public datasheet.

One way would be to first document the registers (with #define REG_THIS 0x2345) to help writing a new one.

[josuah]

For Portenta and very small resolutions, there is also the HM01B0 image sensor, 320x320 array, available on a shield.

It is not supported yet, but offers an alternative.
I am interested in porting it to Zephyr, but cannot give any time frame yet.

[ngphibang]

In fact, the ov5640 supports 2 modes:

• DVP (parallel) mode : in this mode, QVGA (320x240) and QQVGA (160x120) are recently supported (from Zephyr 4.1).
• CSI2 mode : It is in our plan to support QVGA and QQVGA in the near future

[KurtE]

For Portenta and very small resolutions, there is also the HM01B0 image sensor, 320x320 array, available on a shield.

I believe their current V2 shield now has the HM0360 on it. I believe both of these are gray scale cameras. For
lower resolution color cameras there are also OV767x cameras. Currently I believe that Zephyr only has support in it
for the OV7670 and not the OV7675, which Arduino currently sells for the Portenta and GIGA.

There is support in the Arduino library arducam_dvp, for both of these Monochrome cameras as well as OV7670 and OV7675 on the MBED releases for these boards. Hopefully should not be too hard to hopefully migrate this support into this code base.

In fact, the ov5640 supports 2 modes:

• DVP (parallel) mode : in this mode, QVGA (320x240) and QQVGA (160x120) are recently supported (from Zephyr 4.1).
• CSI2 mode : It is in our plan to support QVGA and QQVGA in the near future
  Will try out some of these modes soon, once the Arduino beta setup for zephyr is setup for Zephyr 4.1 (or beyond).

Side notes about the OV5640/OV2640 for setups, wishing to use smaller memory. There are a couple of other capabilities of these
cameras, that are useful. The ability to capture images using JPEG. The other is the ability to define a window of the image to be returned to the processor. One might be able to pan and zoom through the image

A few of us have played around with some of these capabilities on Teensy 4.x board (mainly T4.1 and MicroMod or custom).
The code is up at: https://github.com/mjs513/ArduCAM_t4

[ngphibang]

QVGA and QQVGA in CSI2 mode is now done in #87911.

The ability to capture images using JPEG.

I asked @CharlesDias last time if he can add support for JPEG as I don't have HW which can receive JPEG.

A few of us have played around with some of these capabilities on Teensy 4.x board (mainly T4.1 and MicroMod or custom).
The code is up at: https://github.com/mjs513/ArduCAM_t4

It would be nice if you can upstream these code to support these features