tfm: stack limitation on unsecured application on b_u585i_iot02a_ns

[RomainPelletant]

Describe the bug
When using TFM (with STM32U585), the stack is limited on non-secure application.

First test based on tfm_ipc sample:

• Add several threads in tfm_ipc sample (more than 132KB as total stack)
• Run tfm_ipc sample : reboot looping after TF-M boot (non secure boot)

Second test based on tfm_ipc sample under stack limit:

• Same as first test but limit total stack to 131KB
• Run successfully

Third test without TFM:

• Remove all TFM-related from TFM_IPC sample
• Add several threads (more than 200KB as total stack)
• Run succesfully

Please also mention any information which could help others to understand
the problem you're facing:

• What target platform are you using?
  STM32U585 custom board
• What have you tried to diagnose or workaround this issue?
  I tried to increase NS_STACK_SIZE in region_defs.h but no success. Maybe MSPLIM/PSPLIM registers could trigger stack overflow/protection.
  The stack limit reached is equivalent to memory shift by an offset equal to 0x20000 (131072d)
• Is this a regression? If yes, have you been able to "git bisect" it to a
  specific commit?
  Tested with TF-M 1.7 and TF-M 1.8 with the same result
  Zephyr 3.4.99 (rebased on main)
  MCU: STM32U585QI

To Reproduce
Steps to reproduce the behavior:

1. Open sample TFM_IPC (in sample/tfm_integration/tfm_ipc)
2. Increase main stack size and add K_THREAD_DEFINE to reach up to 131KB of stacks
3. Run with TFM
4. make & flash (regression.sh && west flash)
5. See error

Expected behavior
MCU has 768KB of RAM and I expect to use them, at least 256KB.

Impact
showstopper

Logs and console output

[Sec Thread] Secure image initializing!
Booting TF-M v1.8.0
[INF][Crypto] Initialising HW accelerator... complete.
[WRN] This device was provisioned with dummy keys. This device is NOT SECURE
[Sec Thread] Secure image initializing!
Booting TF-M v1.8.0
[INF][Crypto] Initialising HW accelerator... complete.
[WRN] This device was provisioned with dummy keys. This device is NOT SECURE
[Sec Thread] Secure image initializing!
Booting TF-M v1.8.0
[INF][Crypto] Initialising HW accelerator... complete.
[WRN] This device was provisioned with dummy keys. This device is NOT SECURE
[Sec Thread] Secure image initializing!
Booting TF-M v1.8.0
[INF][Crypto] Initialising HW accelerator... complete.

Environment (please complete the following information):

• OS: MacOS
• Zephyr SDK 0.15.2
• Zephyr main for test (TF-M 1.8)

[RomainPelletant]

To reproduce with TF-M 1.8, it seems like build is broken (as reported in #61503) :

Issue comes from MCUBoot (commit 5b1d511 & 3016d00) introducing a new function prototype (in boot_record.h)

int boot_save_shared_data(const struct image_header *hdr, const struct flash_area *fap,
                          const uint8_t slot, const int max_app_size)

with the same function name of another source file already present in tfm/bl2/src/shared_data.c

int boot_save_shared_data(const struct image_header *hdr,
                          const struct flash_area *fap)

As a workaround for test purpose, I disabled boot measurement capabilities.

[RomainPelletant]

Someone has an idea about that issue? Maybe a direction ?
@erwango FYI

[erwango]

@RomainPelletant Would you be able to provide a map file for each of the 2 first cases ?

[erwango]

@RomainPelletant Here are some hints:
RAM partition is done as follows on U575:

– SRAM1: three 64-Kbyte blocks (total 192 Kbytes)
– SRAM2: 8-Kbyte + 56-Kbyte blocks (total 64 Kbytes). In addition SRAM2 blocks
can be retained in Standby mode.
– SRAM3: eight 64-Kbyte blocks (total 512 Kbytes)
– SRAM4: 16-Kbyte

TFM uses SRAM2, while Zephyr NS application is set to use SRAM1 defined as a continuous RAM of 768K.
So once NS starts using SRAM2, which is TFM assigned area, a reset is triggered (which is, I guess an expected TF-M behavior).

Would you be able to try to get Zephyr NS application to use SRAM3 ?

[RomainPelletant]

@erwango thank you so much for your help!
I will give it a try and come back to you (with map files if the issue is not solved)

[erwango]

@RomainPelletant Did you had a chance to try this lead ?

[RomainPelletant]

@erwango For now, the device is unable to boot at all with the modification. I think the mistake comes from me.
I am trying few fixes and come back to you