ESP32 real time applications and py/gc & esp32 heap autosplit - some documentation

[beyonlo]

Hi all.

Thank you guys for the work in this amazing feature, the py/gc & esp32 heap autosplit. I have an application (like a CLP) that needs to be realtime, I mean, the gc.collect() needs to use the minimum time to run. I know that feature py/gc & esp32: heap autosplit was implemented just to have less time running gc.collect() if applications are not using all memory. I also checked #12141 and #13035.

I would like to know if are there some user documentation about that feature if there are some tip/trick where I can optimize still more my application according with the new heap autosplit feature on ESP32 to have less time running the gc.collect(). I know that that feature is automatic (user transparent), but if the user has something more to do and/or good practices, again, according to the new heap autosplit I will appreciate it.

How my solution works: for this application to work well (like a CLP) I have two ESP32-S3 - I have no problem using two ESP32-S3, which is fine to me. The first one is dedicated just for this work, where it runs in a loop (while True) reading/setting digital/analog inputs/temperature, doing some analysis/calculations, etc. So this dedicated ESP32-S3 does not use asyncio or any ethernet/WiFi network communication and logs, that is done by the second ESP32-S3. The both ESP32-S3 communicating between a UART channel. I was thinking of using some bare metal MicroPython as the first one processor, like the new RP2350 (still not discarded), because the ESP32 uses an OS, and ESP32 is very bad at real time interruptions. But actually I'm reading all inputs in a loop instead of configuring them as IRQ, so ESP32 seems to be good for this job (when I have one chip dedicated to this working in an infinite loop).

Do you have any suggestions for my solution and/or for optimizing this kind of application? Any recommendations are welcome.

Thank you in advance.

[projectgus]

The auto-split should be more or less self-tuning. MicroPython only grows the heap if there's no other option, so it tries to do a gc.collect() on the current smaller heap first, and expands it only if that doesn't work. Similarly, if it finds a new expanded heap chunk is empty after a GC pass then it will permanently release it and shrink the heap back to the original size (this doesn't always happen, but it should happen in cases such as where the heap was expanded to satisfy one giant temporary allocation.)

I think the best approach to minimise time spent in GC (and GC overall) is more or less the same as for any other MicroPython program - allocate any large buffers once and reuse them (with things like .readinto() instead of .read(), or by taking a memoryview and then slicing that instead of slicing a bytearray, etc). More tips in https://docs.micropython.org/en/latest/reference/speed_python.html

The other tip I'd offer is to measure whatever implementation you have. You may find it's performing fine already.

[davefes]

I was thinking of using some bare metal MicroPython as the first one processor, like the new RP2350 (still not discarded), because the ESP32 uses an OS, and ESP32 is very bad at real time interruptions.

After a few years of struggling with random errors on ESP32 I have moved all my non-WiFi projects to the WeACT BlackPill. There has been some current work done towards getting firmware into the mainline.
#15646