ESP32 SPIRAM - How to setup and use

[Rotaclick]

Hi, I have an ESP32 with 32Mb SPIRAM (although I believe only 4Mb is available) - I have been working on a project which was just going to run on the internal flash, however I now need more memory and would like to make use of the attached SPIRAM. I have downloaded this version of Micropython (MicroPython v1.14 on 2021-02-02; ESP32 module (spiram) with ESP32) and flashed it to my ESP32.

My question is how do I access this memory?

I have tried google but can't seem to get a straight answer - I have tried many sample codes that make reference to an SPIRAM module and PSRAM module that don't appear to be in this version of firmware....

Any help would be much appreciated.

Many thanks

[davefes]

First, download v1.19.1 ... a lot has changed since v1.14. I use two of these devices and thought the extra memory was available from the beginning.

[davefes]

Which actual ESP32 SPIRAM chip variant or dev board are you using? Mine are the ESP32 WROVER 1E variant. I am aware of current issues on the ESP32-WROOM-32E-N8R2 variant.

https://github.com/orgs/micropython/discussions/10875

[davefes]

Further checks indicate that I have the N4R8 variant. I only use these because the flash was bigger then the generic ESP32. So, I have no experience using the PSRAM.

Still the first step is to get it to boot up.

[Rotaclick]

Hi, I’m using ESP32 Lolin lite

[davefes]

When you flashed the image, I assume esptool, did it tell you which chip it was programming?

[Rotaclick]

Hi, yes I am using espressif download tool.

These are the screen shots from uploading v1.14 esp32 SPIRAM version (v1.19 causes a crash /

looping error).

I can boot the esp32 but when checking memory using gc.mem_free() I only see 110016 bytes…

[davefes]

from the repl prompt:

import gc
gc.mem_free()

On a generic ESP32 with v1.18 I get 109623

[IlysvlVEizbr]

Please check ESP32 Technical Reference Manual, in 1.3.3 External Memory section.

ESP32 could only access 4MB SPIRAM per CPU core without changing the memory map. In my tests, it seems that for both cpu cores to access the same data, their default memory mapping points to the same 4MB area. (Multithreading in a shared memory manner.)
As far as I know, memory mapping is a MicroPython firmware job, and the related settings are not open to the user layer API (yet). So users really only have access to a 4MB area on SPIRAM without changing the firmware.

One of the possible ways to access the remaining SPIRAM resources is to control the raw SPI connected to SPIRAM, reading and writing data at the lowest level.
In this way, however, these SPIRAM resources become suitable only for storing low-speed data and are unlikely to be used for system RAM. (Accessing them may also take up additional system RAM.)

[peterhinch]

One of the possible ways to access the remaining SPIRAM resources is to control the raw SPI connected to SPIRAM, reading and writing data at the lowest level.

There is probably a bear-trap here in the form of concurrency issues. I think you would need to modify the firmware to use a mutex when accessing the SPIRAM. My gut feeling is that making this work reliably would be very difficult. Much easier to add an external SPIRAM chip for data access.

[Rotaclick]

Hi Everyone, thanks for all your reply’s and looking at this issue for me. But I think I have been barking up the wrong tree! I thought the external IC was 32Mb of SPIRAM when in fact it turns out to be SPI flash…🤦🏻‍♂️ Apologies

[nestorld]

I'm getting this with micropython v1.20 and v1.21 when using the firmware version with SPIRAM downloaded from https://micropython.org/download/ESP32_GENERIC:

stack: 736 out of 15360 GC: total: 4098240, used: 11968, free: 4086272 No. of 1-blocks: 248, 2-blocks: 45, max blk sz: 40, max free sz: 255380
I have tested with these:
- ESP32_GENERIC-SPIRAM-20230426-v1.20.0
- ESP32_GENERIC-SPIRAM-20231005-v1.21.0

The v1.20.0 is way more stable using threads. The "OSError: can't create thread" error occurs much less than with v1.21.0, and the few times that it happens it can be handled with a try-except to retry, but with v1.21.0 it occurs a lot and sometimes it even crashes the ESP32 execution.

I need to create a custom firmware including my code and I have been able to do it with v1.21.0, but because of the stability of v1.20.0, I need to do it with that version, but when I try to do it I get just a few more than 100 KB of free memory when using micropython.mem_info().
I know that v1.20.0 needs esp-idf v4.0.2, v4.2.2 or v4.4 for the build process (I'm using v4.4). And I have used this command to build the firmware:
$ idf.py build $ idf.py -D MICROPY_BOARD=GENERIC_SPIRAM build

and I have also tried with:
$ idf.py build $ idf.py -D MICROPY_BOARD=GENERIC_SPIRAM BOARD_VARIANT=SPIRAM build

...but with the same result, the built firmware is not recognizing the 4 MB of SPIRAM.
Summarizing:
- Downloading prebuilt firmware for ESP32 with SPIRAM, both are recognizing the PSIRAM
- Prebuilt firmware v1.20.0 is way more stable than v1.21.0 using threads
- I have been able to build a custom firmware with v1.21.0, recognizing the PSIRAM with the same unstable behavior as with the prebuilt one
- I have been able to build a custom firmware with v1.20.0 but it is not recognizing the PSIRAM

[jimmo]

@gitcnd The reason we don't publish an ESP32 build that supports both spiram and non-spiram boards, is that due to silicon bugs in the OG ESP32 you must enable workarounds to use spiram which come at a performance and code size penalty.

On S2 and S3 we publish a single build that auto-detects spiram. But it's still not that simple, see e.g. espressif/esp-idf#11971 which we recently had to workaround.

I think you've misunderstood the point of the feature list. Many of the things are e.g. form factors (Feather) or just describing what the board has in the way of connectors (JST-PH, etc) or additional features that could be used from MicroPython (CAN transceiver), and this has no relationship to what's included in the firmware. Also you seem to think that you can do a lot more from a .mpy file than is realistically possible

Three main reasons why we include any given thing in the firmware:

1. It's something that people who use this board are likely to use. Remember: these builds are for people who don't want to compile their own firmware. People are absolutely free to define any board configuration they like and enable/disable any feature they want. Just because we ship default firmware has no bearing on how you have to use MicroPython on any given device.
2. It's something that can't be implemented in Python, or can't be implemented efficiently in Python, or requires deep integration (e.g Ethernet), or it would be crazy to re-implement (e.g. BLE). USB is an example of something on the boundary, and on the TinyUSB ports we are about to merge support for just the very lowest level implemented in C via TinyUSB with the device classes implemented in Python.
3. It's significantly more efficient to load a frozen .mpy (executed directly from ROM) than it is to load a .mpy off the filesystem (has to be copied to RAM first). We will eventually implement Add VfsMap filesystem, mpremote deploy-mapfs, and ability to import .mpy files from ROM (WIP) #8381 (this is a large piece of work) but until then that's the way it is.

Also note that included drivers etc use no RAM until they are loaded.

[nestorld]

My multi threads program is stable now with micropython v1.20.0. I’m only needing to build a custom firmware with that version to include my libraries. Has anyone done it with v1.20.0 for ESP32 with PSIRAM? I have tried the isp.py build commands shown in the documentation, but the resulting firmware is not recognizing the PSIRAM

[gitcnd]

@nestorld - I'm not sure how to tell idy.py to enable and use "features" (RTFM might shed light on it?), but, you can go into the .../boards/ folder, and do this:-

cp -a ESP32_GENERIC ESP32_GENERIC_SPIRAM

and then edit these files:-

micropython/ports/esp32/boards/ESP32_GENERIC_SPIRAM/mpconfigboard.cmake

so it says this

set(SDKCONFIG_DEFAULTS
    boards/sdkconfig.base
    boards/sdkconfig.ble
    boards/sdkconfig.spiram 
)

and maybe mpconfigboard.h to put psram in the name (so you know when it boots that you're using it)

and board.json so it lists psram in the features

    "features": [
        "BLE",
        "External Flash",
        "spiram",
        "WiFi"
    ],

and finally...

idf.py -D MICROPY_BOARD=ESP32_GENERIC_SPIRAM -B build-ESP32_GENERIC_SPIRAM build

(tested, working on psram and nonpsram boards with v1.22, esp-idf 5.xx)

[nestorld]

"(tested, working on psram and nonpsram boards with v1.22, esp-idf 5.xx)"

Thank you @gitcnd, I have built mp firmware for 1.21 and 1.22 with PSIRAM and w/o it too, but for some reason, for 1.20 it seems to be different and I have not been able to create the firmware with PSIRAM support. I will try your procedure and will let you know