freezeFS: Freeze file structures into MicroPython images to mount or deploy

[bixb922]

I needed a way to deploy html, css, jpg, json and binary MIDI files, freezing those together with the rest of the .py files in my manifest.py. This utility program and driver solved that for me, I hope you find this useful.

You put your files in a folder and subfolders, and run freezeFS.py on your PC to convert all subfolders and files to a single .py file, for example: myfolder.py. You add that myfolder.py file and the Virtual File System driver vfsfrozen.py to the manifest.py. You add import myfolder to boot.py or main.py and generate the MicroPython image. When booting that image, the import myfolder statement will automatically mount /myfolder into the file system with os.mount(), and the files will be available read-only in that folder. The file data is accessed directly in flash.

There is also an option of freezeFS to deploy (copy) the folder once to the flash file system. That is useful to deploy files to / and it's subfolders or to deploy read-write files to a certain folder.

It is a Python/MicroPython only solution (no C/C++), with the advantages and disadvantages that has. I tested it on ESP32-S3, ESP32 and Raspberry Pi Pico.

Please see https://github.com/bixb922/freezeFS

I'd be happy to hear if this is useful for someone. Please report issues in that repository.

Thanks to @peterhinch, I based this on the idea of data_to_py, a utility to freeze resources to flash. There cetainly are are more options to do freeze files to MicroPython images, such as creating a littlefs2 image and freezing that to the Python image.

[peterhinch]

This looks extremely useful. I've added a pointer to it in the docs for data_to_py.

There is a plan by the maintainers to provide for a read-only filesystem in Flash which can be created without the need for compilation. This promises to be a superb enhancement which would obsolete our efforts. However it's been a long time not coming, so I'm not holding my breath.

[andrewleech]

For reference this is intended to be the future official method mentioned above, but I agree having tools that work right now are great!
#8381

[bixb922]

I have updated this utility: reezefs saves a file structure with subfolders and builds an self-extractable or self-mountable archive in a .py file, optionally with compression, to be frozen as bytecode or extracted.

It is now installable with pip install freezers

There are several ways to use freezefs:

• Freeze the archive as frozen bytecode in a MicroPython image. Import the archive and it gets mounted as a read-only file system. The files continue to reside in the frozen image.
• Freeze the compressed archive as frozen bytecode in a MicroPython image. Import the archive once to extract the file structure to the flash file system. The purpose is to aid initial deployment of read/write files.

These two options require RAM to import the compressed archive:

• Run a compressed .py archive with mpremote run. The files get extracted to the microprocessor. This is an easy way to install many files at once, and it is quite fast.
• Install a compressed .py (or better: .mpy) archive with mip install and then import the file to extract all files. This aids in installing complete systems over-the-air. The file should be compiled with mpy-cross, to get all the gain from compression.

Overall, it simplifies deploying text and binary files, such as MicroPython code, html pages, json data files, etc.

[peterhinch]

Have you considered making this installable with mip and mpremote?

This is the standard way of installing libraries directly from GitHub and it is easy to implement (and easy for users too).

[bixb922]

Thanks for the comment. Yes, MicroPython packages should be mip installable, that is a very useful mechanism.

However, in this case, the freezefs program runs on the PC, and the output .py file is generated with all the necessary MicroPython code to mount or extract the archive. No separate installation of packages or libraries is needed on the microcontroller. Output archive files are self-mountable or self-extracting.

[peterhinch]

Apologies - I goofed there :(