How do I start using ROMFS?

[VirtualWolf]

This may be a dense question, but there's something I'm clearly missing here I think. 😅 I've been reading up on ROMFS and wanted to start using it on my WROOM-based board (Adafruit's HUZZAH32), but I cannot for the life of me figure out how one actually gets ROMFS enabled, mpremote romfs query returns ROMFS is not enabled on this device.

I've come across this page in the wiki here, but that seems to involving needing to build your own MicroPython firmware? It's not enabled by default on any of the standard builds that can be downloaded from the website?

[peterhinch]

This does seem to be the current situation. I have experimented with it, and it works very well, but enabling it does involve creating a special build. Compared to frozen bytecode, performance is identical. You only need to compile once as against once each time your code changes.

Precompiled builds with it enabled would open up the feature to many more users.

[Josverl]

I have written a doc page on using RomFS, but decided against adding a full list of individual boards, as that would grow old quick.

So searching GitHub for MICROPY_VFS_ROM in .h files is the current best offer

💡Perhaps it is possible to add a romfs flag in the individual board.json files to allow selecting for that attribute on the download pages.

[VirtualWolf]

Perhaps it is possible to add a romfs flag in the individual board.json files to allow selecting for that attribute on the download pages.

Yeah that would very handy! (And is what I had been expecting, given the already-existing separate builds for OTA support and SPIRAM and so on).

[Josverl]

I do not think it is feasible to add a romfs variant to all existing boards and variants, as that would double the build matrix, but I'll look into making it simpler to identify the firmwares that have enabled romfs.

[shariltumin]

Your firmware must have ROMFS partation. Try

>>> from esp32 import Partition as part
>>> part.find(1,0x8f)
[<Partition type=1, subtype=143, address=2162688, size=2031616, label=romfs, encrypted=0>]

If part.find(1,0x8f) return [] then you have no romfs partation in your flash.

$ mpremote a1 romfs query
ROMFS is not enabled on this device

Having romfs partation is not enough, you also need to enable it.
Build firmware using custom board mpconfigboard.h with:

// Enable ROMFS
#define MICROPY_VFS_ROM                     (1)
#define MICROPY_VFS_ROM_IOCTL               (1)

Now, firmware built with romfs-partation + MICROPY_VFS_ROM,
The import path will have /rom:

>>> import sys
>>> sys.path
['', '.frozen', '/rom', '/rom/lib', '/lib']

$ mpremote a0 romfs query
ROMFS0 partition has size 2031616 bytes (496 blocks of 4096 bytes each)
  Raw contents: ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff ...
  Not a valid ROMFS

then, you can, for example:

$ mpremote a0 romfs deploy util
Building romfs filesystem, source directory: util/
/
\-- what_partations.py -> .mpy
Image size is 166 bytes
ROMFS0 partition has size 2031616 bytes (496 blocks of 4096 bytes each)
Preparing ROMFS0 partition for writing
Writing at offset 0
ROMFS image deployed

$ mpremote a0 romfs query
ROMFS0 partition has size 2031616 bytes (496 blocks of 4096 bytes each)
  Raw contents: d2:cd:31:80:81:20:05:81:1d:13:77:68 ...
  ROMFS image size: 160

Then:

$ mpremote a0
Connected to MicroPython at /dev/ttyACM0
Use Ctrl-] or Ctrl-x to exit this shell

>>> import what_partations
<Partition type=1, subtype=2, address=36864, size=24576, label=nvs, encrypted=0>
<Partition type=1, subtype=1, address=61440, size=4096, label=phy_init, encrypted=0>
<Partition type=1, subtype=143, address=2162688, size=2031616, label=romfs, encrypted=0>
<Partition type=1, subtype=129, address=4194304, size=12582912, label=vfs, encrypted=0>
>>> 

[shariltumin]

If you are just looking for a ESP32 firmware that supports ROMFS, you can download it here:

https://github.com/shariltumin/romfs_tutorial_micropython/tree/main/esp32/firmware

[VirtualWolf]

Build firmware using custom board mpconfigboard.h

This is very "now draw the rest of the fucking owl" vibes, haha. Cheers for the guidance, I think I'll just wait for the hopefully-future-state where this is enabled by default!