Trying to build micropython for esp32s3-wroom with camera

[uraich]

I tried to first simply build micropython with
make board=ESP32_GENERIC_S3
but this creates a build directory build-ESP32_GENERIC (I expected build-ESP32_GENERIC _S3) and it seems to compile for the ESP32 and not the S3.
A fatal error occurred: This chip is ESP32-S3 not ESP32. Wrong --chip argument? when I try to deploy.
How do I build for the ESP32S3 with oct PSRAM?

[jimmo]

I'm not sure if you're running into https://github.com/orgs/micropython/discussions/12294

But perhaps more likely the issue is that you should be using BOARD=ESP32_GENERIC_S3 not board=ESP32_GENERIC_S3 (which will be ignored and therefore use the default, which is ESP32_GENERIC).

If you have octal spiram though, you also need to configure that, using make BOARD=ESP32_GENERIC_S3 BOARD_VARIANT=SPIRAM_OCT.

[uraich]

Hi Jim,
Thanks for the prompt answer. Typing "board" in small letters was a very stupid error of mine. Nevertheless, it was good I asked the question because I did not know about the board variant parameter.
The PSRAM is found and tested without errors:

As I told you, I wanted to create a firmware with the esp32-cam driver integrated and that works fine (with your help!).
Before, configuration of the camera failed when allocating the framebuffer in PSRAM but this error is also gone.
What I do not understand though is the output of gc.mem_free(): 55312
I had expected a much bigger number.

[shariltumin]

This is probably due to the new way of allocating the MP heap, which was introduced by the

mpconfigport.h:#define MICROPY_GC_SPLIT_HEAP               (1)
mpconfigport.h:#define MICROPY_GC_SPLIT_HEAP_AUTO          (1)

and in the main.c

...
// Initial Python heap size. This starts small but adds new heap areas on
// demand due to settings MICROPY_GC_SPLIT_HEAP & MICROPY_GC_SPLIT_HEAP_AUTO
#define MP_TASK_HEAP_SIZE (64 * 1024)
...

The initial MP heap size is 64MB and will grow automatically, read lines 265 to 284 of micropython/py/gc.c

For firmware with camera support, I will probably disable this feature and set a fixed MP heap size in ports/esp32/main.c and to leave enough PSRAM for the camera frame buffer and system use.

I am sure others will find this feature very useful. Thank you, the core MP developers and maintainers have done a great job implementing this.

[jimmo]

@shariltumin yes you're right that's exactly what's going on here.

@uraich I've just written up a bit of an explainer on this feature: https://github.com/orgs/micropython/discussions/12316

[uraich]

Thanks to both of you and particularly @jimmo for the excellent explication.

This makes sense now. ... and as I said, my ESP32S3 board with camera works as expected, access to the camera driver included.
Many thanks again.

[shariltumin]

The following is a short report comparing the differences between two firmware versions for the ESP32-S2FN4R2 (4MB Flash, 2MB PSRAM, 320KB SRAM). Firmware-A enables the new dynamic MP heap allocation feature. Firmware-B disables this feature and the MP heap is fixed and set to the largest allocation block (PSRAM block of course).

Firmware-A with dynamic MP heap run log:

MPY: soft reboot
MicroPython v1.20.0-396-g1dedb65e6 on 2023-08-29; ESP32 S2_MINI WEMOS (KAKI5) with ESP32-S2FN4R2
Type "help()" for more information.
>>> gc.mem_free()
61344
>>> import esp32
>>> esp32.idf_heap_info(esp32.HEAP_DATA)
[(32767, 15643, 15360, 15643), (2097152, 2029576, 1998848, 2029576), (179048, 136740, 131072, 132288), (14864, 2916, 2816, 2784), (8144, 7764, 7680, 7764)]
>>> import micropython as mp
>>> mp.mem_info()
stack: 704 out of 15360
GC: total: 64000, used: 2496, free: 61504, max new split: 1998848
 No. of 1-blocks: 45, 2-blocks: 13, max blk sz: 18, max free sz: 3808
>>> a = b'a'*1998848
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError: memory allocation failed, allocating 1998849 bytes
...
...
>>> a = b'a'*1950000
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError: memory allocation failed, allocating 1950001 bytes
>>> a = b'a'*1940000
>>> esp32.idf_heap_info(esp32.HEAP_DATA)
[(32767, 15643, 15360, 15643), (2097152, 40312, 39936, 40312), (179048, 136740, 131072, 132288), (14864, 2916, 2816, 2784), (8144, 7764, 7680, 7764)]
>>> mp.mem_info()
stack: 704 out of 15360
GC: total: 2007616, used: 1942304, free: 65312, max new split: 131072
 No. of 1-blocks: 37, 2-blocks: 18, max blk sz: 121251, max free sz: 3808
>>> gc.mem_free()
65152

Firmware-B with a fixed size MP heap run log:

MicroPython v1.20.0-396-g1dedb65e6 on 2023-08-29; ESP32 S2_MINI WEMOS (KAKI5) with ESP32-S2FN4R2
Type "help()" for more information.
>>> gc.mem_free()
2015680
>>> import micropython as mp
>>> mp.mem_info()
stack: 704 out of 15360
GC: total: 2017088, used: 2080, free: 2015008
 No. of 1-blocks: 40, 2-blocks: 6, max blk sz: 18, max free sz: 125903
>>> import esp32
>>> esp32.idf_heap_info(esp32.HEAP_DATA)
[(32767, 15643, 15360, 11543), (2097152, 30728, 30720, 30728), (179312, 137004, 131072, 132552), (14864, 2916, 2816, 2784), (8144, 7764, 7680, 7764)]
>>> a = b'a'*2015008
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError: memory allocation failed, allocating 2015009 bytes
...
...
>>> a = b'a'*2010000
>>> esp32.idf_heap_info(esp32.HEAP_DATA)
[(32767, 15643, 15360, 11543), (2097152, 30728, 30720, 30728), (179312, 137004, 131072, 132552), (14864, 2916, 2816, 2784), (8144, 7764, 7680, 7764)]
>>> mp.mem_info()
stack: 704 out of 15360
GC: total: 2017088, used: 2012784, free: 4304
 No. of 1-blocks: 48, 2-blocks: 21, max blk sz: 125626, max free sz: 256
>>> gc.mem_free()
4144

Immediately after rebooting, the largest block of memory that Firmware-A can provide is 1940000 bytes. While Firmware-B can provide 2010000 bytes.

The next largest block of memory accessible for Firmware-A is 131072 bytes, which is the largest block of SRAM available! With Firmware-B there are only 4144 bytes left.

Firmware-A can access both PSRAM and SRAM. Firmware-B can only access PSRAM. In total, Firmware-A can access more than Firmware-B. In Firmware-B, SRAM can only be accessed by esp-idf tasks, system and user C modules.

In Firmware-A, the MP code can initially take up too much memory. This can lead to memory allocation failures for system tasks at a later stage.

In Firmware-B, the MP heap size is fixed and the MP code cannot access SRAM.

These only apply to ESP32 boards with PSRAM. The dynamic MP heap will work better than the fixed heap on ESP32 boards without PSRAM.