lcd/lvgl-display resolution/orientation/rotation problems

[glory-man]

imxrt-1052-based board
Zephyr SHA-1: 0714052
integrated LVGL- SHA-1: 2b498e6f36d6b82ae1da12c8b7742e318624ecf5
LCD-panel 1024/768

My application should use 270 deg rotated orientation of lcd-panel. During lvgl_init() in zephyr\modules\lvgl\lvgl.c lvgl-disp resolution is set based on display_get_capabilities() output. But mcux_elcdif_get_capabilities() returns x/y_resolution of lcd-panel excluding rotation. So my lvgl-disp initilized as W=1024, H=768, instead of W=768, H=1024. imxrt has a built-in GPU/PXP which is used in elcdif-driver to rotate an image but this CONFIG_MCUX_ELCDIF_PXP_ROTATE_90/180/270 considered for data rotation, but not taken into account when determining display capabilities - if rotate 90/270 is active display height and width should be swaped. Is this a feature or bug? I think get_capabilties() should return rotated resolution (lvgl_allocate_rendering_buffers() must take into account display rotation when adjusting disp_driver->hor_res/ver_res).

As far as I know LVGL-lib has built-in rotation mechanism using NXP_PXP. So I tried to enable it by insert CONFIG_LV_USE_GPU_NXP_PXP in my prj.conf, but received error message
modules/lib/gui/lvgl/src/draw/nxp/pxp/lv_draw_pxp_blend.c:37:10: fatal error: lvgl_support.h: No such file or directory
So it seems this option not integrated into Zephyr, or some files missed. Am I missing something?

If CONFIG_MCUX_ELCDIF_PXP_ROTATE_90/180/270 is activated via prj.conf PXP will be used only if x=0,y=0 and full buffer should be outputed in mcux_elcdif_write(). In case partial buffer output such code used

src = buf;
dst = dev_data->fb[dev_data->next_idx];
dst += dev_data->pixel_bytes * (y * config->rgb_mode.panelWidth + x);

for (h_idx = 0; h_idx < desc->height; h_idx++) {
  memcpy(dst, src, dev_data->pixel_bytes * desc->width);
  src += dev_data->pixel_bytes * desc->pitch;
  dst += dev_data->pixel_bytes * config->rgb_mode.panelWidth;
}

This code does not take into account rotation, so the output buffer may contain both rotated and unrotated data. So need to add code to rotate data. I have an example used in previous appl to rotate 270 deg.

[faxe1008]

LVGL offers settings for display rotation. Can you check if this patch resolves the issue:

iff --git a/modules/lvgl/lvgl.c b/modules/lvgl/lvgl.c
index 76df359e7f..d96c6c6ddf 100644
--- a/modules/lvgl/lvgl.c
+++ b/modules/lvgl/lvgl.c
@@ -228,6 +228,7 @@ static int lvgl_init(void)
 #ifdef CONFIG_LV_Z_FULL_REFRESH
        disp_drv.full_refresh = 1;
 #endif
+       lv_disp_set_rotation(lv_disp_get_default(), LV_DISP_ROT_270);

        err = lvgl_allocate_rendering_buffers(&disp_drv);
        if (err != 0) {

Regarding PXP we did some work to get the needed pieces into place:
lvgl/lvgl#5838 (Sidenote: this also needs to be ported to v9 and submitted upstream)
zephyrproject-rtos/hal_nxp#343

What is still missing is the setup in the glue code in the zephyr main repository:
zephyrproject-rtos/hal_nxp#343 (comment)

Hooking up the IRQs up is kinda messy but doable.

[glory-man]

LVGL offers settings for display rotation. Can you check if this patch resolves the issue:

iff --git a/modules/lvgl/lvgl.c b/modules/lvgl/lvgl.c
index 76df359e7f..d96c6c6ddf 100644
--- a/modules/lvgl/lvgl.c
+++ b/modules/lvgl/lvgl.c
@@ -228,6 +228,7 @@ static int lvgl_init(void)
 #ifdef CONFIG_LV_Z_FULL_REFRESH
        disp_drv.full_refresh = 1;
 #endif
+       lv_disp_set_rotation(lv_disp_get_default(), LV_DISP_ROT_270);

        err = lvgl_allocate_rendering_buffers(&disp_drv);
        if (err != 0) {

This code tries to rotate unregistered display driver - lv_disp_get_default() returns NULL, so lv_disp_set_rotation() return without effect. But this is not solution - IMHO - it will activate lvgl built-in rotate engine, and without implemented hardware support SW-rotatre will use, as far as I understand.

EDIT: simpliest way - IMHO - is swap x/y resolution in mcux_elcdif_get_capabilities(), and remove panel height/width checking in lvgl_allocate_rendering_buffers() #ifdef CONFIG_LV_Z_BUFFER_ALLOC_STATIC (by the way, there is no such checking if CONFIG_LV_Z_BUFFER_ALLOC_STATIC not defined)

if (data->cap.x_resolution <= DISPLAY_WIDTH) {
...
if (data->cap.y_resolution <= DISPLAY_HEIGHT) {

In case if LVGL-pxp-rotate engine is worked, it make sense to put this line after lv_disp_drv_register() int lvgl_init(). But I can't understand how to use it.

[faxe1008]

This code tries to rotate unregistered display driver - lv_disp_get_default() returns NULL, so lv_disp_set_rotation() return without effect. But this is not solution - IMHO - it will activate lvgl built-in rotate engine, and without implemented hardware support SW-rotatre will use, as far as I understand.

Yeah my bad, you might need to put this later down in the function. That's correct it will use the software rotation done by LVGL. Not too sure what the best way to go about this is. IMO touching the get_capabilities is not a great option, it should reflect the display "real" resolution. Is the rotation in your case caused by the way that you are mouting the display physically?

PXP support is generally available in 8.4, it needed the IRQs to be connected. A PR was raised zephyrproject-rtos/lvgl#52 and I upstreamed this to v8.4 and v9.1.
Note: from my understanding within v9.1 we have to use the LVGL display rotation since this is the information from which the PXP engine is fed. Of course this means that in the case that PXP is not used there will be a performance hit due to the SW rotation being used.
I think that since we are offering the capability to statically define the drawbuffers we need a LVGL Kconfig flag for setting the display rotation. This is not ideal since you might get weird effects if the display does support rotation on the controller side, but I can not come up with a nicer solution for this. @danieldegrasse @jfischer-no do you have any input on this :^) ?

[danieldegrasse]

I think that since we are offering the capability to statically define the drawbuffers we need a LVGL Kconfig flag for setting the display rotation. This is not ideal since you might get weird effects if the display does support rotation on the controller side, but I can not come up with a nicer solution for this. @danieldegrasse @jfischer-no do you have any input on this :^) ?

I'm a bit unclear here- are you saying add a Kconfig to enable software rotation?

From the PXP side, it is only really useful (as far as I know) if an entire framebuffer is provided for it to rotate. Hence the reason for the issue noted above:

This code does not take into account rotation, so the output buffer may contain both rotated and unrotated data

When using the PXP I set CONFIG_LV_Z_FULL_REFRESH=y and LV_Z_VDB_SIZE=100.

Currently it should be possible to rotate a display buffer with LVGL and Zephyr by calling lv_disp_set_rotation(), and setting CONFIG_MCUX_ELCDIF_PXP_ROTATE_90/180/270 (note that the PXP rotates counterclockwise, so for a rotation setting of LV_DISP_ROT_90 you would set CONFIG_MCUX_ELCDIF_PXP_ROTATE_270), without swapping the width/height of the display.

I'm able to rotate the LVGL widgets sample using the PXP like so:

Add rotation call in sample code:

--- a/samples/modules/lvgl/demos/src/main.c
+++ b/samples/modules/lvgl/demos/src/main.c
@@ -36,6 +36,8 @@ int main(void)
        CONFIG_LV_USE_DEMO_STRESS, or CONFIG_LV_USE_DEMO_WIDGETS
 #endif

+       lv_disp_set_rotation(lv_disp_get_default(), LV_DISP_ROT_90);
+
        lv_task_handler();
        display_blanking_off(display_dev)

Use the following build command:
west build -p always -b mimxrt1050_evk samples/modules/lvgl/demos/ -DCONFIG_LV_Z_DEMO_WIDGETS=y -DCONFIG_MCUX_ELCDIF_PXP_ROTATE_270=y -DCONFIG_DMA=y -DCONFIG_MCUX_ELCDIF_PXP=y -DCONFIG_LV_Z_FULL_REFRESH=y -DCONFIG_LV_Z_VDB_SIZE=100

@glory-man could you try this, and see if you have any luck using the PXP?

As a side note, I'm aware the documentation for the PXP rotation is misleading. I'll open a PR to make it more robust, to make this use case a bit simpler to enable

Edit: here's a PR to (hopefully) improve documentation around this feature a bit, and enable it on the RT1050: #72192

[glory-man]

Is the rotation in your case caused by the way that you are mouting the display physically?

Yes. In my case display mounted in portrait mode (768 x 1204) while original is landscape (1024 x 768). So panel's zero point does not match with lvgl zero point. Previous in zephyr were CONFIG_LVGL_HOR_RES_MAX/CONFIG_LVGL_VER_RES_MAX which configured lvgl resolution, now lvgl resolution matches with panel settings.

[glory-man]

@glory-man could you try this, and see if you have any luck using the PXP?

Yes. It works. As far as understand in this case used display_mcux_elcdif.c PXP realization, disp->driver->sw_rotate = 0 by-default - so lvgl-SW rotation switched off , lvgl-gpu switched off too.

I can suggest such mod

diff --git a/drivers/display/display_mcux_elcdif.c b/drivers/display/display_mcux_elcdif.c
index e4b45f89d40..25b85c76c24 100644
--- a/drivers/display/display_mcux_elcdif.c
+++ b/drivers/display/display_mcux_elcdif.c
@@ -117,13 +117,44 @@ static int mcux_elcdif_write(const struct device *dev, const uint16_t x, const u
 		}
 		/* Now, write the display update into active framebuffer */
 		src = buf;
-		dst = dev_data->fb[dev_data->next_idx];
-		dst += dev_data->pixel_bytes * (y * config->rgb_mode.panelWidth + x);
+		if (!IS_ENABLED(CONFIG_MCUX_ELCDIF_PXP_ROTATE_90) && !IS_ENABLED(CONFIG_MCUX_ELCDIF_PXP_ROTATE_180) && !IS_ENABLED(CONFIG_MCUX_ELCDIF_PXP_ROTATE_270)) {
+			dst = dev_data->fb[dev_data->next_idx];
+			dst += dev_data->pixel_bytes * (y * config->rgb_mode.panelWidth + x);
 
-		for (h_idx = 0; h_idx < desc->height; h_idx++) {
-			memcpy(dst, src, dev_data->pixel_bytes * desc->width);
-			src += dev_data->pixel_bytes * desc->pitch;
-			dst += dev_data->pixel_bytes * config->rgb_mode.panelWidth;
+			for (h_idx = 0; h_idx < desc->height; h_idx++) {
+				memcpy(dst, src, dev_data->pixel_bytes * desc->width);
+				src += dev_data->pixel_bytes * desc->pitch;
+				dst += dev_data->pixel_bytes * config->rgb_mode.panelWidth;
+			}
+		}
+		else {
+			int w_idx;
+			for (h_idx = 0; h_idx < desc->height; h_idx++) {
+				int offs = 0;
+				dst = dev_data->fb[dev_data->next_idx];
+				if (IS_ENABLED(CONFIG_MCUX_ELCDIF_PXP_ROTATE_90)) {
+					offs = ((x + 1) * config->rgb_mode.panelWidth - (y + h_idx) - 1);
+				} else if (IS_ENABLED(CONFIG_MCUX_ELCDIF_PXP_ROTATE_180)) {
+					offs = ((config->rgb_mode.panelHeight - (y + h_idx)) * config->rgb_mode.panelWidth - x - 1);
+				} else if (IS_ENABLED(CONFIG_MCUX_ELCDIF_PXP_ROTATE_270)) {
+					offs = ((config->rgb_mode.panelHeight - x - 1) * config->rgb_mode.panelWidth + (y + h_idx));
+				}
+				dst += (dev_data->pixel_bytes * offs);
+				for (w_idx = 0; w_idx < desc->pitch; w_idx++) {
+					memcpy(dst, src, dev_data->pixel_bytes);
+					if (IS_ENABLED(CONFIG_MCUX_ELCDIF_PXP_ROTATE_90)) {
+						offs = config->rgb_mode.panelWidth;
+					} else if (IS_ENABLED(CONFIG_MCUX_ELCDIF_PXP_ROTATE_180)) {
+						offs = -1;
+					} else if (IS_ENABLED(CONFIG_MCUX_ELCDIF_PXP_ROTATE_270)) {
+						offs = -config->rgb_mode.panelWidth;
+					} else {
+						offs = 0;
+					}
+					dst += (dev_data->pixel_bytes * offs);
+					src += dev_data->pixel_bytes;
+				}
+			}
 		}
 
 		LOG_DBG("Setting FB from %p->%p", (void *)dev_data->active_fb,

in display_mcux_elcdif driver to enable partial framebuffer rotation. In this case demo could be worked with CONFIG_LV_Z_FULL_REFRESH=n.

[danieldegrasse]

Yes. It works. As far as understand in this case used display_mcux_elcdif.c PXP realization, disp->driver->sw_rotate = 0 by-default - so lvgl-SW rotation switched off , lvgl-gpu switched off too.

Yes- this is correct.

I can suggest such mod

Interesting- thanks for looking at how to enable this. Truthfully partial refreshes are not a priority for me in that driver (because performance is very poor using that method, and it isn't how the peripheral was designed to function). However, if you are interested in sending a PR to enable this feature I'd be happy to review it and help get it merged!

[glory-man]

Truthfully partial refreshes are not a priority for me in that driver (because performance is very poor using that method, and it isn't how the peripheral was designed to function).

I also have an example code for rotating (implemented for rotation by kPXP_Rotate90 and kPXP_Rotate270, but not for kPXP_Rotate180) a buffer of arbitrary size using PXP in blocks that are multiples of kPXP_BlockSize16, the rest of the buffer outside the block boundaries is rotated programmatically.