📂 MPBridge : MicroPython file system bridge

[AmirHmZz]

I've developed a simple minimal file manager-like tool for MicroPython running devices which uses platforms file manager (i.e. explorer.exe on Windows) to make it easier to manage files on MicroPython devices. It uses mpremote for communication and watchdog to capture file system events and apply changes on connected device. I've tested it on Windows but theoretically it should work on other platforms too. One of the most important advantages of this tool is It doesn't provide any uncomfortable or old-school GUI and uses OS' file manager. This point allows users to use any tool/application that they wish to modify files on devices.

Github repo :
https://github.com/AmirHmZz/mpbridge

[peterhinch]

I attempted to run this under Linux Mint 20.3 with this result which I've never seen before:

$ ./main.py
import-im6.q16: attempt to perform an operation not allowed by the security policy `PS' @ error/constitute.c/IsCoderAuthorized/408.

The system was in a peculiar state with the mouse cursor disabled. I tried to break in with ctrl-c without success. I then tried to close the Bash window, which did break into the program with this traceback:

^C
^C^X
import-im6.q16: attempt to perform an operation not allowed by the security policy `PS' @ error/constitute.c/IsCoderAuthorized/408.
from: can't read /var/mail/typing
import-im6.q16: attempt to perform an operation not allowed by the security policy `PS' @ error/constitute.c/IsCoderAuthorized/408.
./main.py: line 8: syntax error near unexpected token `('
./main.py: line 8: `from watchdog.events import ('

I had installed the latest version of watchdog.

[AmirHmZz]

Would you please open an issue in repo? Thank you :)

[peterhinch]

Now done: AmirHmZz/mpbridge#1

[peterhinch]

This problem proved to be a misconfiguration of my machine. It's important to install mpremote with pip rather than just having the Python file on the path.

This is very useful, especially when the target has a lot of files and directories.

[jimmo]

@AmirHmZz I notice that the file manager uses mpremote via subprocess. It would probably be good to investigate using it as a library instead (in which case you probably actually want to use the lower-level pyboard module), and then you can just make this a dependency of your project. It might be worth looking at how the Belay tool does exactly this -- https://github.com/BrianPugh/belay

[AmirHmZz]

@jimmo Thanks for your recommendation. I've developed the project about 2 month ago and it was a one-day project to facilitate my works. Before I publish this small project I was going to just make it work but now, I'll keep making it better. I've had the exact same idea yesterday and I was going to mention you right here to ask you about mpremote but you've answered all my questions. So thanks and I'll be on it soon.

[AmirHmZz]

@jimmo I've taken a look at mpremote source code but it doesn't seem to be much friendly to be used as a library. Most of functions write their output directly into the stdout. It seems mpremote was developed to be used only as a shell utility and there's lack of a clean API. It could have that API beside the shell utilities which could be built on top of that API. Anyway, I can make a copy of pyboard.py and modify it to achieve what I want as Baley did but I wanted to use mpremote as dependency of my project and avoid monkey patching it's functionality. This also makes my project more stable as it can benefits from bug fixes and updates to mpremote module. What do you suggest?

[jimmo]

You should use pyboard.py, which is what mpremote uses for board access and doing the low level operations, e.g. filesystem. If your project depends on the PyPI mpremote package, then you will get pyboard as part of that. I don't know why Belay has their own copy (I thought last time I looked they were using it via the mpremote dependency).

See also https://github.com/orgs/micropython/discussions/10282#discussioncomment-4463402

(And in particular see the linked PR which might be useful to you)

[AmirHmZz]

@jimmo I've reviewed your PR and that was exactly what I was talking about! If it's possible, please also realese a new version of mpremote on PyPI after merging that PR. Then I can use pyboard.py from mpremote module directly and avoid subprocesses. Thanks again :)

[rhubarbdog]

Have you tried https://github.com/dhylands/rshell

[AmirHmZz]

I've seen that repo before but I would prefer to depend on MicroPython it self to keep my project up to dated. Also what my project needs now is a Python API which allows it to communicate with pyboard, not another shell utility.

[peterhinch]

rshell uses pyboard.py.

[rhubarbdog]

have you seen pyboard.py in repository https://github.com/micropython/micropython in directory tools

[AmirHmZz]

Yes, I've described about it here #10240 (reply in thread)

[Wind-stormger]

WOW
It shocked me when it ran.
It's completely different from my preconceived imagination.
Even if I disconnect the USB, it still works when I connect it again!

I LOVE IT 💘

[Wind-stormger]

Calm down my excitement.
A suggestion that comes to mind at this point.
I only tested it in windows10, it creates a temporary folder for use, but the folder name looks irregular, tmpnwzwz5xg, I think this folder can be given a proper name, such as MPY_PORT_TEMP.

[AmirHmZz]

The project uses tempfile built-in module of standard CPython to create a temporary directory in all platforms. The reason that I choose to create a temp directory instead of a permanent directory with specific name was this:
Temporary directories are placed somewhere to be deleted on every boot of OS. So If anything goes wrong (e.g. process gets killed) It's not important that cleanup part didn't run.

But I would be glad to add another shell option to specify pre created path to be used by utility, but I don't get the point. In which scenario it can help you?

[Wind-stormger]

If there is a specified path, Windows can create and keep its shortcut, put it on the desktop or in a tile, even if the folder window is closed, it can still be opened quickly by clicking the shortcut.
Ubuntu also has a similar feature.
Is it possible to specify a folder name using standard CPython's tempfile built-in module?
If possible, so I can have a path that will not change, and create a shortcut, even if the system is restarted and the temporary folder is deleted, the shortcut will still remain.
And when micropython-filemanager runs again and creates this folder, the shortcut can be restored and does not need to be created again.

[AmirHmZz]

I think you've misunderstood the purpose of this tool. To make it clear, assume the micropython device is just a removable USB flash storage and you want to manage files on it so you can use this tool. But you cannot manage files on USB storage when it's not connected, so you should keep your MicroPython device connected while bridge is running. Stashing changes and uploading them later can be implemented If we find it useful for developers.

[AmirHmZz]

@Wind-stormger I've committed a change according to your suggestion to add mpbridge- prefix to temporary directory names.

[AmirHmZz]

🎉 Version 0.1.0 and release on PyPI

I've renamed the project to mpbridge because it is more than a file manager tool and also released the first version on PyPI. Now it can easily be installed using pip. It also uses mpremote directly as its dependency instead of using subprocesses.

Special thanks to @peterhinch and @jimmo for their helps.

[AmirHmZz]

After 3 weeks from first publish on PyPI, Now in version 1.3.1 mpbridge can sync directories and run dev mode which help you debug your projects faster. Check repo for more information and thanks to @Wind-stormger for their ideas and contributions.

[Wind-stormger]

We've been relying on a few helper tools or IDEs for these workflows:

1. Upload the .py file to the micropython device, and maybe other file operations.
2. Open the REPL to view the running results.
3. Modify the .py file.
4. Repeat 1, 2, 3.

The appearance of mpremote has made me very happy. All operations can be completed in the terminal, and the dependent projects of the development environment are greatly reduced.

However, each operation needs to enter the path of each file in the terminal. Even though most terminals have the function of retaining historical commands and can quickly switch and execute several commands, it is still somewhat inconvenient.

mpbridge dev [port] [local path]This function simplifies all operations and significantly improves the efficiency in actual development.

Still does not need any additional GUI interface, but makes full use of the graphical file management interface that comes with the operating system.

Just keep mpbridge running in the background, and any file changes in the path you specify will be automatically synced to the micropython device, and a REPL is also available in the terminal.

Now you only need to focus on the code development of micropython.

If you ask what code editor should you choose? Feel free! Just you like!