[mimx93_evk_a55] CONFIG_HEAP_MEM_POOL_SIZE to 65535 cause the zephy bin inscreate 64K

[hakehuang]

Describe the bug

modify prj.conf CONFIG_HEAP_MEM_POOL_SIZE=65535

the zephyr.bin size increase 64K as well

Please also mention any information which could help others to understand
the problem you're facing:

• What target platform are you using?
  mimx93_evk_a55
• What have you tried to diagnose or workaround this issue?
  No
• Is this a regression? If yes, have you been able to "git bisect" it to a
  specific commit?
  No regress

To Reproduce
Steps to reproduce the behavior:

1. change the samples/hello_world/prj.conf, adding
   CONFIG_HEAP_MEM_POOL_SIZE=65535
2. build
   west build -b mimx93_evk_a55
3. check the file size

Expected behavior
as the HEAP is put into noint area, this should not increase the binary size.

Impact
this will cause the imx95 binary increase too much to deploy

Logs and console output

before change

zephyr/samples/hello_world$ ls build/zephyr/zephyr.bin -la
-rwxrwxr-x 1 ubuntu ubuntu 106500  9月 13 16:11 build/zephyr/zephyr.bin

after change

zephyr/samples/hello_world$ ls build/zephyr/zephyr.bin -la
-rwxrwxr-x 1 ubuntu ubuntu 172036  9月 13 16:11 build/zephyr/zephyr.bin

Environment (please complete the following information):

• OS: (e.g. Linux,, )
• Toolchain (e.g Zephyr SDK, ...)
• Commit SHA or Version used: zephyr-v3.4.0-3573-g79158a777b

[ibirnbaum]

This is likely the same issue that should have been fixed for aarch32 with #60368, which got reverted due to issues with the qemu_cortex_r5 target. To fix this issue, .bss and .noinit have to be moved to the end of the image. At the time being, there's non-zero data following those sections, therefore, the linker has to pad them, resulting in a binary that keeps growing as additional zero-initialized/un-initialized arrays, heaps etc. are added in the application code.

[hakehuang]

This is likely the same issue that should have been fixed for aarch32 with #60368, which got reverted due to issued with the qemu_cortex_r5 target. To fix this issue, .bss and .noinit have to be moved to the end of the image. At the time being, there's non-zero data following those sections, therefore, the linker has to pad them, resulting in a binary that keeps growing as additional zero-initialized/un-initialized arrays, heaps etc. are added in the application code.

@ibirnbaum , thanks for this inforamtion, so this issue will not fixed?

[ibirnbaum]

@hakehuang From my point of view, there's still the intention to fix this, although my scope is limited to aarch32.

The memory protection tests eventually failed in the CI after the aarch32 linker command file update on the qemu_cortex_r5 target. Aside from the fact that this is a target from the early days, where certain configuration data isn't yet linked to the device tree as would be the case nowadays, but is hardcoded instead which definitely needs to be fixed, I believe that there's a resonable chance that the tests failing is down to QEMU itself. I have yet to investigate this further. The option of excluding this target from the tests and re-applying the linker command file update is on the table, but I'd like to be sure that the tests failing is actually down to this particular platform's quirks and not my linker command file.

[dleach02]

This is likely the same issue that should have been fixed for aarch32 with #60368, which got reverted due to issues with the qemu_cortex_r5 target. To fix this issue, .bss and .noinit have to be moved to the end of the image. At the time being, there's non-zero data following those sections, therefore, the linker has to pad them, resulting in a binary that keeps growing as additional zero-initialized/un-initialized arrays, heaps etc. are added in the application code.

What were the issues? Because moving the bss/noinit sections to the end seems reasonable.

[hakehuang]

What were the issues? Because moving the bss/noinit sections to the end seems reasonable.

@dleach02 , per my understaing, this fixes fails on qemu_cortex_r5 target, need some more analysis. @ibirnbaum correct me if I get iw wrongly.

[ibirnbaum]

We're talking about problems here that are as far as I can tell 100% related to the target qemu_cortex_r5 itself.
Some of the tests within the kernel.mem_protect suite failed on that target, the simulation appears to have crashed during early boot.

As far as I can tell at the time being, qemu_cortex_r5 is messy regarding the MPU support in general, and QEMU probably adds some extra quirks on top of that. For starters, qemu_cortex_r5 defines the base address of RAM at a location where it shouldn't be, on actual hardware, address 0x0 wouldn't work without at least some tinkering when building the bitstream/first stage bootloader for the UltraScale. With the default memory layout, there's a black hole between 0x10000 and 0x20000. QEMU just doesn't care about that.

Next, the unsuitable base address of course also shows up in the array of MPU region entries. But it's not even obtained via the device tree, it's hardcoded. The MPU entry for standard RAM just opens up 2 GB without any access restrictions. Unlike on MMU-based aarch32 systems, things like .rodata aren't considered at all in the UltraScale's MPU region entries (admittedly, 16 MPU entries aren't a lot, but despite this, read-only data should probably be read-only...).

So there's a lot to fix regarding this particular target (which dates back to the early days, DT integration probably wasn't possible yet back then?). When reproducing the reported test failures, I manually moved the base of RAM to a suitable location both in the DT and the MPU code, only to find out that the test cases in question will actually pass on actual UltraScale hardware (also with the linker command file where .bss/.noinit were moved to the end of the image), but the same test case still crashes in QEMU. If I don't have a MPU entry that covers at least the first 192k starting at address 0x0, Zephyr won't boot in QEMU. So either QEMU sees memory accesses in that range that aren't actually there on real hardware, or on real hardware, they're not considered an access permissions violation. It recently dawned on me that the Ethernet driver (guess who wrote that one...) might be involved due to its DMA area placement, but I haven't had the time to verify this yet.

However, even with the memory area starting at 0x0 covered in the MPU regions array, some test cases will boot and others won't. There's probably something else broken regarding the MPU and/or QEMU. Unfortunately, debugging the R-Cores in qemu_cortex_r5 is a real hassle despite the integrated gdb stub in QEMU, as the gdb stub identifies the system as aarch64 (due to the 64-bit Cortex-A cores in the UltraScale), causing gdb clients to complain about the incompatibility between an aarch64 system and an aarch32 binary for the R-cores. Therefore, I'm focusing the debugging efforts on a real (out-of-tree) hardware target with a Lauterbach debugger attached, however, this won't really help if it's actually QEMU that's behaving incorrectly.

I hope that I'll get around to further investigation soon, but there are also a few more issues I have to dig into regarding the aarch32 arch code, some aarch32 Cortex-A specific stuff, the Xilinx GEM Ethernet driver etc.

Long story short, I'm positive that those test case failures weren't related to the linker command file changes, but to this one particular target. I had run the tests prior to committing on an MMU-based system and didn't have any issues there.