IMX8MP CM7 Zephyr booting by remoteproc

[mbahmani90]

Hi,

Hope you are well.

I compile a sample code for IMX8MP CM7 and a zephyr.elf file is created in build/zephyr directory. On the other hand, in mcuxpresso sdk two types of .elf file is created which are used for TCM and DDR memory respectively,

Do you know if zephyr.elf file is for TCM or DDR? Is it possible to create .elf file for both memory?

Thanks

[DerekSnell]

Hi @mbahmani90 ,
The imx8mp_evk has build configs for both TCM and DDR. Please see the i.MX8MP EVK board page for the details. The default board name uses TCMs. Building for the board variant imx8mp_evk/mimx8ml8/m7/ddr will use DDR.

The memory used by the linker is set in devicetree using the chosen nodes zephyr,flash and zephyr,sram. In imx8mp_evk_mimx8ml8_m7.dts, code (.text) and .rodata is placed in ITCM, and .data and .bss are placed in DTCM. In imx8mp_evk_mimx8ml8_m7_ddr.dts, both are placed in DDR.

Best regards

[mbahmani90]

Hi JiafeiPan,

Thank you for your reply.

As you see imx_rproc_att_imx8mn already has { 0x40000000, 0x40000000, 0x80000000, 0 }.

static const struct imx_rproc_att imx_rproc_att_imx8mn[] = {
	/* dev addr , sys addr  , size	    , flags */
	/* ITCM   */
	{ 0x00000000, 0x007E0000, 0x00020000, ATT_OWN | ATT_IOMEM },
	/* OCRAM_S */
	{ 0x00180000, 0x00180000, 0x00009000, 0 },
	/* OCRAM */
	{ 0x00900000, 0x00900000, 0x00020000, 0 },
	/* OCRAM */
	{ 0x00920000, 0x00920000, 0x00020000, 0 },
	/* OCRAM */
	{ 0x00940000, 0x00940000, 0x00050000, 0 },
	/* QSPI Code - alias */
	{ 0x08000000, 0x08000000, 0x08000000, 0 },
	/* DDR (Code) - alias */
	{ 0x10000000, 0x40000000, 0x0FFE0000, 0 },
	/* DTCM */
	{ 0x20000000, 0x00800000, 0x00020000, ATT_OWN | ATT_IOMEM },
	/* OCRAM_S - alias */
	{ 0x20180000, 0x00180000, 0x00008000, ATT_OWN },
	/* OCRAM */
	{ 0x20200000, 0x00900000, 0x00020000, ATT_OWN },
	/* OCRAM */
	{ 0x20220000, 0x00920000, 0x00020000, ATT_OWN },
	/* OCRAM */
	{ 0x20240000, 0x00940000, 0x00040000, ATT_OWN },
	/* DDR (Data) */
	{ 0x40000000, 0x40000000, 0x80000000, 0 },
};

Linux Version:

Linux imx8mp-lpddr4-evk 6.1.22-rt8 #1 SMP PREEMPT_RT Thu Dec 21 09:48:39 UTC 2023 aarch64 GNU/Linux

[JiafeiPan]

Hi, @mbahmani90 ,

I don't know why your kernel mapping fails, I suggest you can add some debug information in the function imx_rproc_da_to_va() in imx_rproc.c, and find it fails in imx_rproc_da_to_sys() or in finding va address and dig more.

[mbahmani90]

Do you know if any specific configuration is required in device tree or menuconfig?

[JiafeiPan]

As I know you need to reserve the DDR memory to be used by M-Core in device tree.

[mbahmani90]

I checked the device tree

// SPDX-License-Identifier: (GPL-2.0 OR MIT)
/*
 * Copyright 2020 NXP
 */

/dts-v1/;

#include "imx8mp-evk.dts"

/ {
	aliases {
		i2c0 = &i2c1;
		i2c1 = &i2c2;
		i2c2 = &i2c_rpbus_3;
	};

	reserved-memory {
		#address-cells = <2>;
		#size-cells = <2>;
		ranges;

		m4_reserved: m4@80000000 {
			no-map;
			reg = <0 0x80000000 0 0x1000000>;
		};

		vdev0vring0: vdev0vring0@55000000 {
			reg = <0 0x55000000 0 0x8000>;
			no-map;
		};

		vdev0vring1: vdev0vring1@55008000 {
			reg = <0 0x55008000 0 0x8000>;
			no-map;
		};

		vdevbuffer: vdevbuffer@55400000 {
			compatible = "shared-dma-pool";
			reg = <0 0x55400000 0 0x100000>;
			no-map;
		};

		rsc_table: rsc-table@550ff000 {
			reg = <0 0x550ff000 0 0x1000>;
			no-map;
		};

		audio_reserved: audio@81000000 {
			compatible = "shared-dma-pool";
			no-map;
			reg = <0 0x81000000 0 0x10000000>;
		};

		micfil_reserved: mic_rpmsg@91000000 {
			compatible = "shared-dma-pool";
			no-map;
			reg = <0 0x91000000 0 0x100000>;
		};
	};

	sound-wm8960 {
		status = "disabled";
	};

	sound-micfil {
		status = "disabled";
	};

	rpmsg_audio: rpmsg_audio {
		compatible = "fsl,imx8mp-rpmsg-audio";
		model = "wm8960-audio";
		fsl,platform = "rpmsg-audio-channel";
		fsl,enable-lpa;
		fsl,rpmsg-out;
		fsl,rpmsg-in;
		assigned-clocks = <&clk IMX8MP_CLK_SAI3>;
		assigned-clock-parents = <&clk IMX8MP_AUDIO_PLL1_OUT>;
		assigned-clock-rates = <12288000>;
		clocks = <&audio_blk_ctrl IMX8MP_CLK_AUDIO_BLK_CTRL_SAI3_IPG>,
			 <&audio_blk_ctrl IMX8MP_CLK_AUDIO_BLK_CTRL_SAI3_MCLK1>,
			 <&audio_blk_ctrl IMX8MP_CLK_AUDIO_BLK_CTRL_SDMA3_ROOT>,
			 <&clk IMX8MP_AUDIO_PLL1_OUT>,
			 <&clk IMX8MP_AUDIO_PLL2_OUT>;
		clock-names = "ipg", "mclk", "dma", "pll8k", "pll11k";
		audio-codec = <&codec>;
		memory-region = <&audio_reserved>;
		power-domains = <&audiomix_pd>;
		audio-routing =
			"LINPUT1", "MICB",
			"LINPUT3", "MICB";
		status = "okay";
	};

	rpmsg_micfil: rpmsg_micfil {
		compatible = "fsl,imx8mp-rpmsg-audio";
		model = "micfil-audio";
		fsl,platform = "rpmsg-micfil-channel";
		fsl,enable-lpa;
		fsl,rpmsg-in;
		assigned-clocks = <&clk IMX8MP_CLK_PDM>;
		assigned-clock-parents = <&clk IMX8MP_AUDIO_PLL1_OUT>;
		assigned-clock-rates = <196608000>;
		clocks = <&audio_blk_ctrl IMX8MP_CLK_AUDIO_BLK_CTRL_PDM_IPG>,
			 <&audio_blk_ctrl IMX8MP_CLK_AUDIO_BLK_CTRL_PDM_ROOT>,
			 <&audio_blk_ctrl IMX8MP_CLK_AUDIO_BLK_CTRL_SDMA3_ROOT>,
			 <&clk IMX8MP_AUDIO_PLL1_OUT>,
			 <&clk IMX8MP_AUDIO_PLL2_OUT>;
		clock-names = "ipg", "mclk", "dma", "pll8k", "pll11k";
		memory-region = <&micfil_reserved>;
		power-domains = <&audiomix_pd>;
		status = "okay";
	};

	imx8mp-cm7 {
		compatible = "fsl,imx8mn-cm7";
		rsc-da = <0x55000000>;
		clocks = <&clk IMX8MP_CLK_M7_DIV>,
			 <&audio_blk_ctrl IMX8MP_CLK_AUDIO_BLK_CTRL_AUDPLL_ROOT>;
		clock-names = "core", "audio";
		mbox-names = "tx", "rx", "rxdb";
		mboxes = <&mu 0 1
			  &mu 1 1
			  &mu 3 1>;
		memory-region = <&vdevbuffer>, <&vdev0vring0>, <&vdev0vring1>, <&rsc_table>, <&m4_reserved>;
		status = "okay";
		fsl,startup-delay-ms = <500>;
	};
};

/*
 * ATTENTION: M7 may use IPs like below
 * ECSPI0/ECSPI2, FLEXCAN, GPIO1/GPIO5, GPT1, I2C3, I2S3, UART4,
 * PWM4, SDMA1/SDMA2
 */
&ecspi2 {
	status = "disabled";
};

&flexcan1 {
	status = "disabled";
};

&flexspi {
	status = "disabled";
};

/delete-node/ &i2c3;

&i2c_rpbus_3 {
	compatible = "fsl,i2c-rpbus";
	#address-cells = <1>;
	#size-cells = <0>;
	status = "okay";

	pca6416: gpio@20 {
		compatible = "ti,tca6416";
		reg = <0x20>;
		gpio-controller;
		#gpio-cells = <2>;
	};

	ov5640_1: ov5640_mipi@3c {
		compatible = "ovti,ov5640";
		reg = <0x3c>;
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_csi0_pwn>, <&pinctrl_csi0_rst>;
		clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO2>;
		clock-names = "xclk";
		assigned-clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO2>;
		assigned-clock-parents = <&clk IMX8MP_CLK_24M>;
		assigned-clock-rates = <24000000>;
		csi_id = <0>;
		powerdown-gpios = <&gpio4 1 GPIO_ACTIVE_HIGH>;
		reset-gpios = <&gpio4 0 GPIO_ACTIVE_LOW>;
		mclk = <24000000>;
		mclk_source = <0>;
		mipi_csi;
		status = "disabled";

		port {
			ov5640_mipi_1_ep: endpoint {
				remote-endpoint = <&mipi_csi1_ep>;
				data-lanes = <1 2>;
				clock-lanes = <0>;
			};
		};
	};

	codec: wm8960@1a {
		compatible = "wlf,wm8960,lpa";
		reg = <0x1a>;
		wlf,shared-lrclk;
		SPKVDD1-supply = <&reg_audio_pwr>;
	};
};

&pwm4{
	status = "disabled";
};

&sai3 {
	status = "disabled";
};

&micfil {
	status = "disabled";
};

&sdma3{
	status = "disabled";
};

&uart3 {
    status = "disabled";
};

&uart4 {
	status = "disabled";
};

I changed:

memory-region = <&vdevbuffer>, <&vdev0vring0>, <&vdev0vring1>, <&rsc_table>;

to

memory-region = <&vdevbuffer>, <&vdev0vring0>, <&vdev0vring1>, <&rsc_table>, <&m4_reserved>;

I no longer receive any error when loading CM7. However, nothing happened in CM7 side!

root@imx8mp-lpddr4-evk:~# bash test_z_m7_ddr.sh
test_z_m7_ddr.sh: line 3: echo: write error: Invalid argument
[   36.408216] remoteproc remoteproc0: powering up imx-rproc
[   36.419396] remoteproc remoteproc0: Booting fw image ddr_zephyr.elf, size 561780
[   36.419428] remoteproc remoteproc0: No resource table in elf
[   36.928210] remoteproc remoteproc0: remote processor imx-rproc is now up

As far as I know, CM7 should be load in address 0x80000000 with the size of 16MB 0x1000000 but why is it different in { 0x40000000, 0x40000000, 0x80000000, 0 }, ?

[JiafeiPan]

Hi, @mbahmani90,

Currently we have two problems:

The first problem is remoteproc can't load elf issue. If CM7 uses ddr_code and ddr_sys, for example, currently uses 0x80000000 as code start address and uses 0x80200000 as sys start address. we can found currently there is already one mapping entry in the mapping table of imx_rproc_att_imx8mn[] in remoteproc Linux driver imx_rproc.c:
{ 0x40000000, 0x40000000, 0x80000000, 0 },
It can cover the memory range of ddr_code and ddr_sys, but it doesn't set ATT_OWN flag, so the driver can't do devm_ioremap_wc(), the result is that it will get NULL virtual address when calling the function imx_rproc_da_to_sys(), so that remoteproc can't load the elf sections into DDR memory.
One of the solution is just like you mentioned, put m4_reserved region in the memory-region list of the node imx8mp-cm7, another solution is to add a separate section in the mapping table of imx_rproc_att_imx8mn[] in remoteproc Linux driver imx_rproc.c, for example:

diff --git a/drivers/remoteproc/imx_rproc.c b/drivers/remoteproc/imx_rproc.c
index 11347441070d..28a371582d31 100644
--- a/drivers/remoteproc/imx_rproc.c
+++ b/drivers/remoteproc/imx_rproc.c
@@ -193,10 +193,8 @@ static const struct imx_rproc_att imx_rproc_att_imx8mn[] = {
        { 0x20220000, 0x00920000, 0x00020000, ATT_OWN },
        /* OCRAM */
        { 0x20240000, 0x00940000, 0x00040000, ATT_OWN },
-       { 0x80000000, 0x80000000, 0x01000000, ATT_OWN },
        /* DDR (Data) */
-       { 0x40000000, 0x40000000, 0x40000000, 0 },
-       { 0x81000000, 0x81000000, 0x7F000000, 0 },
+       { 0x40000000, 0x40000000, 0x80000000, 0 },
 };

After we fixed the first problem, elf segments can be loaded into related memory correctly, but as you said, no response from M7 Core. the root cause is when we use ddr_code to put the code in the DDR memory, but M7 will still to get the first instruction from ITCM zero address, so we need to put rom_start section to ITCM zero address, the solution is to enabled CONFIG_ROMSTART_RELOCATION_ROM:
For example:

diff --git a/boards/nxp/imx8mp_evk/imx8mp_evk_mimx8ml8_m7_ddr_defconfig b/boards/nxp/imx8mp_evk/imx8mp_evk_mimx8ml8_m7_ddr_defconfig
index 17542cb4eec..8c30c5b6fa3 100644
--- a/boards/nxp/imx8mp_evk/imx8mp_evk_mimx8ml8_m7_ddr_defconfig
+++ b/boards/nxp/imx8mp_evk/imx8mp_evk_mimx8ml8_m7_ddr_defconfig
@@ -12,3 +12,4 @@ CONFIG_CONSOLE=y
 CONFIG_XIP=y
 CONFIG_CODE_DDR=y
 CONFIG_PINCTRL=y
+CONFIG_ROMSTART_RELOCATION_ROM=y

You can refer to discussion in PR: #71684

After the second problem is fixed, you should run ddr zephyr image by using remoteproc correctly.

[JiafeiPan]

Updated board document to address the issues we found in this discussion:

#75426

[JiafeiPan]

May I know the commit ID of Zephyr you are using? and also please execute west update to update the other components accordingly.

BTW, I updated the subject of this discussion to reflect the issue we are talking about.

[mbahmani90]

I update west and it was solved.

Many Thanks @JiafeiPan.

[mbahmani90]

Sorry only one question I know it is out of ZephyrOS scope. Do you have any idea how to solve this problem in mcuxpresso SDK?

[JiafeiPan]

We should follow the same implementation to modify link file of FreeRTOS to move boot start code to TCM zero address, or you can modify remoteproc driver to copy the boot start code to TCM zero address during loading elf image, it has the same implement in uboot bootaux command to copy the start instructions, so uboot bootaux has no such issue, may I know why you want to use remoteproc but not use uboot bootaux to boot m-core?

[mbahmani90]

Thank you for you response.

Reasons:

1. During development I need to change CM7 firmware constantly. To load new firmware by u-boot. I should restart the Linux and it is really time consuming. It is also possible to use JTAG but I don't access it.

2. If in the feature we need to update the firmware of CM7 remotely I think it would be the best way to upload the firmware without restarting.

[JiafeiPan]

Let's close this discussion as it is resolved by PR #75426 which has been merged, thanks.