video: fix missing crop/compose handling in capture_to_lvgl sample & add missing conf files for stm32n657 fsbl variant

[avolmat-st]

PR #91878 added mechanism to perform crop and compose on a frame. Related API usage was added within the capture sample but not within the capture_to_lvgl leading to having this sample no more working on the stm32n6570_dk board since compose is mandatory on it in order to be able to display the large frame from sensor to the 800x480 display.

Current PR add the same handling done in capture sample app within the capture_to_lvgl app in order to have the sample app working on the stm32n6.

Moreover PR #91941 introduced a new variable (FSBL) to run application on the stm32n6. Since st_b_cams_imx_mb1854 shield and video sample app have board specific conf / overlays, add the conf for the FSBL variant as well.

[josuah]

This was introduced in the capture sample, but not capture_to_lvgl sample that is true.

Could it be tested on native_sim/native64 to confirm that the sample still works well (as it is easy to test with that platform)?

Thank you for the complement.

[avolmat-st]

This was introduced in the capture sample, but not capture_to_lvgl sample that is true.

Could it be tested on native_sim/native64 to confirm that the sample still works well (as it is easy to test with that platform)?

Thank you for the complement.

Yes it is still working:

west build -p -b native_sim/native/64 --snippet video-sw-generator samples/drivers/video/capture_to_lvgl/

[2/2] Running utility command for native_runner_executable
-- west flash: using runner native
*** Booting Zephyr OS build v4.2.0-rc1-4-gb37a2d7a34e7 ***
[00:00:00.000,000] main: - Device name: video-sw-generator
[00:00:00.000,000] main: - Capabilities:
[00:00:00.000,000] main: RGB3 width [64; 1920; 1] height [64; 1080; 1]
[00:00:00.000,000] main: YUYV width [64; 1920; 1] height [64; 1080; 1]
[00:00:00.000,000] main: RGBP width [64; 1920; 1] height [64; 1080; 1]
[00:00:00.000,000] main: BX24 width [64; 1920; 1] height [64; 1080; 1]
[00:00:00.000,000] main: RGGB width [64; 1920; 1] height [64; 1080; 1]
[00:00:00.000,000] main: GRBG width [64; 1920; 1] height [64; 1080; 1]
[00:00:00.000,000] main: BA81 width [64; 1920; 1] height [64; 1080; 1]
[00:00:00.000,000] main: GBRG width [64; 1920; 1] height [64; 1080; 1]
[00:00:00.000,000] main: - Format: RGBP 320x240 640
[00:00:00.000,000] main: - Capture started
^C
Stopped at 40.840s

and I got the colorbar displayed on the SDL display

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
76.7% Duplication on New Code

See analysis details on SonarQube Cloud

[josuah]

LGTM! Thank you for testing!

[KurtE]

@josuah @avolmat-st - Sorry to jump in here, after the fact, but was wondering about the changes to the two example
sketches.

That is they now both have code in them like:

	/* Set the crop setting if necessary */
#if CONFIG_VIDEO_SOURCE_CROP_WIDTH && CONFIG_VIDEO_SOURCE_CROP_HEIGHT
	sel.target = VIDEO_SEL_TGT_CROP;
	sel.rect.left = CONFIG_VIDEO_SOURCE_CROP_LEFT;
	sel.rect.top = CONFIG_VIDEO_SOURCE_CROP_TOP;
	sel.rect.width = CONFIG_VIDEO_SOURCE_CROP_WIDTH;
	sel.rect.height = CONFIG_VIDEO_SOURCE_CROP_HEIGHT;
	if (video_set_selection(video_dev, &sel)) {
		LOG_ERR("Unable to set selection crop");
		return 0;
	}
	LOG_INF("Selection crop set to (%u,%u)/%ux%u",
		sel.rect.left, sel.rect.top, sel.rect.width, sel.rect.height);
#endif

#if CONFIG_VIDEO_FRAME_HEIGHT || CONFIG_VIDEO_FRAME_WIDTH
#if CONFIG_VIDEO_FRAME_HEIGHT
	fmt.height = CONFIG_VIDEO_FRAME_HEIGHT;
#endif

#if CONFIG_VIDEO_FRAME_WIDTH
	fmt.width = CONFIG_VIDEO_FRAME_WIDTH;
#endif

	/*
	 * Check (if possible) if targeted size is same as crop
	 * and if compose is necessary
	 */
	sel.target = VIDEO_SEL_TGT_CROP;
	err = video_get_selection(video_dev, &sel);
	if (err < 0 && err != -ENOSYS) {
		LOG_ERR("Unable to get selection crop");
		return 0;
	}

And I potentially wish to run it on a board that does not using: video_stm32_dcmipp.c
like, maybe the arduino_nicla_vision, that uses the video_stm32_dcmi.c
which unless it was just added, I don't believe it has support for video_get_selection,
which I believe will cause both samples to fail.

I noticed this as I am experimenting adding it to video_stm32_dcmi.c and have it try to
forward this as well as the set to the camera. And trying to update gc2145.c to handle it.
As was I believe suggested in my PR: #91975, to for example
allow me to setup the camera, to the same resolution of my display. For example to an ST7796 at 480x320.

Still figuring out the order of calls that is necessary for this, as well as things like, how to reflect this
in the creation of buffers, etc.

Thanks
Kurt

[josuah]

like, maybe the arduino_nicla_vision, that uses the video_stm32_dcmi.c
which unless it was just added, I don't believe it has support for video_get_selection,
which I believe will cause both samples to fail.

I am trying west build -b arduino_nicla_vision/stm32h747xx/m7 and notice a lot of broken builds... but 2aa7cb5 (which introduced it) works well.

Many other commits after that did not, including many before introduction of video_set_selection().
It looks like you found another thing we need to fix! :D

The UVC sample seems to work, however.

Thank you for all the testing!

[josuah]

@KurtE I will pursue the Nicla Vision-specific issue there: #92712

This PR is regarding the capture_to_lvgl sample, the PR about the crop/compose ('video_set_selection()') is #91878

[avolmat-st]

@josuah @avolmat-st - Sorry to jump in here, after the fact, but was wondering about the changes to the two example sketches.

That is they now both have code in them like:

	/* Set the crop setting if necessary */
#if CONFIG_VIDEO_SOURCE_CROP_WIDTH && CONFIG_VIDEO_SOURCE_CROP_HEIGHT
	sel.target = VIDEO_SEL_TGT_CROP;
	sel.rect.left = CONFIG_VIDEO_SOURCE_CROP_LEFT;
	sel.rect.top = CONFIG_VIDEO_SOURCE_CROP_TOP;
	sel.rect.width = CONFIG_VIDEO_SOURCE_CROP_WIDTH;
	sel.rect.height = CONFIG_VIDEO_SOURCE_CROP_HEIGHT;
	if (video_set_selection(video_dev, &sel)) {
		LOG_ERR("Unable to set selection crop");
		return 0;
	}
	LOG_INF("Selection crop set to (%u,%u)/%ux%u",
		sel.rect.left, sel.rect.top, sel.rect.width, sel.rect.height);
#endif

#if CONFIG_VIDEO_FRAME_HEIGHT || CONFIG_VIDEO_FRAME_WIDTH
#if CONFIG_VIDEO_FRAME_HEIGHT
	fmt.height = CONFIG_VIDEO_FRAME_HEIGHT;
#endif

#if CONFIG_VIDEO_FRAME_WIDTH
	fmt.width = CONFIG_VIDEO_FRAME_WIDTH;
#endif

	/*
	 * Check (if possible) if targeted size is same as crop
	 * and if compose is necessary
	 */
	sel.target = VIDEO_SEL_TGT_CROP;
	err = video_get_selection(video_dev, &sel);
	if (err < 0 && err != -ENOSYS) {
		LOG_ERR("Unable to get selection crop");
		return 0;
	}

And I potentially wish to run it on a board that does not using: video_stm32_dcmipp.c like, maybe the arduino_nicla_vision, that uses the video_stm32_dcmi.c which unless it was just added, I don't believe it has support for video_get_selection, which I believe will cause both samples to fail.

Hum, no not having set_selection API availability shouldn't block the sample to run since -ENOSYS (which is what is returned if the API is not supported) is not going to block the sample. Actually upon request of @josuah I tested it with the nativesim and the video-sw-generator snippet and it is well working (even if set_selection is not supported in those case).

I noticed this as I am experimenting adding it to video_stm32_dcmi.c and have it try to forward this as well as the set to the camera. And trying to update gc2145.c to handle it. As was I believe suggested in my PR: #91975, to for example allow me to setup the camera, to the same resolution of my display. For example to an ST7796 at 480x320.

Still figuring out the order of calls that is necessary for this, as well as things like, how to reflect this in the creation of buffers, etc.

Thanks Kurt

[avolmat-st]

like, maybe the arduino_nicla_vision, that uses the video_stm32_dcmi.c
which unless it was just added, I don't believe it has support for video_get_selection,
which I believe will cause both samples to fail.

I am trying west build -b arduino_nicla_vision/stm32h747xx/m7 and notice a lot of broken builds... but 2aa7cb5 (which introduced it) works well.

Many other commits after that did not, including many before introduction of video_set_selection(). It looks like you found another thing we need to fix! :D

It is surprising that this didn't get called by the build test. Maybe this platform / shield is not part of the automatic tests ?

The UVC sample seems to work, however.

Thank you for all the testing!

[KurtE]

Hum, no not having set_selection API availability shouldn't block the sample to run since -ENOSYS (which is what is returned if the API is not supported) is not going to block the sample. Actually upon request of @josuah I tested it with the nativesim and the video-sw-generator snippet and it is well working (even if set_selection is not supported in those case).

Sorry you are correct. It should be handled by the ENOSYS...

Was looking through the examples trying to see the usages of the different get/set settings... To see how hard it would
be to implement some/all at camera level...

[josuah]

[@avolmat-st] It is surprising that this didn't get called by the build test. Maybe this platform / shield is not part of the automatic tests ?

This was ambiguous but by "broken builds" I meant that the firmware was building well, but there was a runtime issue, which can mostly be caught by running it in hardware. I have not tried the automated test yet but could be interesting! Only manual tests so far.

[josuah]

See the usages of the different get/set settings... To see how hard it would be

The idea is that drivers contain the storage for settings: struct video_ctrl, but that the video APIs manage their content.

As soon as an user modify a video control via video_set_ctrl(), the value gets updated on the driver struct directly, the driver is only notified of which control changed, and use the updated value from ctrl.val. This is the Linux video control model (doc) adapted to Zephyr.