Where are files stored when building micropython and freezing modules?

[maxi07]

Hello! I am trying to freeze my project into micropython. This is part of my project structure:

.
├── main.py
├── config.json
├── ...
├── css/
│   └── styles.css
└── modules/
    ├── module1.py
    └── module2.py

the modules are copied fine, but my main.py imports the config.json. This now fails with OSError: [Errno 2] ENOENT when freezing the files. Where would that config.json now be accessible when being frozen into the firmware?

[robert-hh]

When freezing, no file system is created. Only .py files are frozen in to the firmware image and being stored in a special data structure. The json file and other file types are ignored.

[dlech]

It depends on how you are using the data, but you could put the json in a string or convert it to a Python data structure. If you still want to keep the actual .json file in the source code, you could create a Python script to automate the conversion.

I.e. like this (a string)...

CONFIG_JSON = """
{
    "option1": "value1",
    "option2": "value2"
}
"""

or like this (a Python dict/list/etc.)...

CONFIG = {
    "option1": "value1",
    "option2": "value2",
}

[maxi07]

While this option could work with the simple json, I have bigger HTML and CSS files which I would like to keep seperate for easier development.
I still don't quite understand what you mean by conversion?

[dlech]

I mean you can write a Python script that runs on your computer at compile time that reads all of the .css/.html/.json files and creates a new .py file and writes all of that information to the new .py file in a format that is valid Python syntax.

[karfas]

You can embed your file content (like @dlech suggested) in .py files and create some (default) config/html/CSS files out of these frozen files, if they don't exist in the filesystem.
Just use your imagination - your program can do anything you want.

[Josverl]

Compiling to bytecode is not the only way to freeze.
It may be an option to use freeze_as_str in your manifest.py

freeze_as_str("foo.html")

micropython/tools/makemanifest.py

Line 181 in cfcce4b

if result.kind == manifestfile.KIND_FREEZE_AS_STR:

Not sure how to reference this runtime though.

Looks like they can be referenced as strings, but hard to test on a phone.

micropython/tools/makemanifest.py

Lines 243 to 252 in cfcce4b

	else:
	output_mpy = (
	b'#include "py/emitglue.h"\n'
	b"extern const qstr_pool_t mp_qstr_const_pool;\n"
	b"const qstr_pool_t mp_qstr_frozen_const_pool = {\n"
	b" (qstr_pool_t*)&mp_qstr_const_pool, MP_QSTRnumber_of, 0, 0\n"
	b"};\n"
	b'const char mp_frozen_names[] = { MP_FROZEN_STR_NAMES "\\0"};\n'
	b"const mp_raw_code_t *const mp_frozen_mpy_content[] = {NULL};\n"
	)

[bixb922]

I wrote this utility program that I use to freeze html, css, json, image, etc files to a .py file. I add these .py files to the manifest.py to be frozen in the MicroPython image. On import, the file structure gets mounted as a read only file system on the microcontroller. For read-write files, the file structure can get deployed (copied) to the controllers flash. I have done testing and it's in active use, but probably more testing may be necessary. If you use it, I'd be very happy to get feedback. Here is the link: https://github.com/bixb922/freezeFS/tree/main

[maxi07]

this looks promising, I will definitely check it out this weekend!

[maxi07]

this works perfectly fine for me, thank you so much!

[jimmo]

Peter wrote a script for doing this: https://github.com/peterhinch/micropython-samples/blob/master/data_to_py/data_to_py.py