Deleting folder recursively and creating files recursively (creating non-existing folder structures before creating files)

[liudr]

I have a need to delete a folder with subfolders and content on the file system. I wonder if there is any calls that can do that recursively. I've read about creating files with non-existing folder structures. It seems to require some os calls to first create the folder structure:

https://stackoverflow.com/questions/12517451/automatically-creating-directories-with-file-output

Unfortunately I couldn't find the mkdirs('/abc/def') call in mp's os module. That means I must create these mkdir() recursively? I tried mkdir() and only one folder is allowed at a time. Same as rmdir(). So is there any modules I can import to make this easy or do I need to write my own recursive routines to handle this? I can definitely write one but need time to code and debug. If there is already such module, that would be great. Thanks.

[peterhinch]

See shutil.

[peterhinch]

It's a matter of personal taste in my opinion. The "do it and ask for forgiveness" approach is fashionable, but I can't say it's better.

[bixb922]

os.stat() allows to check if file or folder, and throws an exception if not found

[liudr]

Thank you both for your suggestions. I'm going with catching exceptions when calling os.mkdir(). I goal was to set up folder structures and download files into these folders. For the content of each folder, I organized my file names in a list besides the folder name such as /folder/subfolder/. This way I can just do mkdir() on the folder name, one folder level at a time until I finish making the deepest folder level. Then chdir() to the folder and create the files. An interesting tidbit on shutil.rmtree() removing folders, I've been testing my code on a PC, literally running it on a PC (with if switches to separate PC stuff vs mpy stuff). There on a PC, the python IDLE IDE could hold up a folder so I couldn't delete it not because it doesn't exist but because it's been locked until the IDLE quits. No such thing on mpy though.

[peterhinch]

Dave Hylands' rshell does this kind of thing, implementing rsync to synchronise a directory tree on a device with that on a host.

[liudr]

Thanks @peterhinch
I wonder if thonny is using this or rsync as well. rshell seems to use a local connection (UART) and requires RAW REPL, although it's called remote shell. I also don't see how I might automate the work with scripts etc. I need the files to be downloaded from a web server programatically.

I didn't clarify the purpose of my code. It was intended for OTA updates to file systems. Compared against a regular IoT OTA update that usually overwrites a whole flash partition with downloaded firmware images, I'm making several subfolders on the MPY flash VFS as OTA folders, such as Factory, OTA0, and OTA1. So if I request an OTA update, the board will determine which file folder, OTA0 or OTA1 will be used to store the update, and then download a manifest file that lists where on the web server to download all files into the proper OTA folder and maintain the same folder structure as the server, like an HTTP drag and drop. This was the reason for deleting trees to clear the content in the next OTA folder before setting up folder structures for the files to be downloaded. This is much faster than an otherwise partition-wide update that is megabytes in size. Plus, that might be able to only update micropython partition (app partition). I'm also thinking about incremental updates where only a few files must be downloaded to the current OTA folder which will only take a second or two.