Flash corruption after failed file copy with rshell

[camielverdult]

Port, board and/or hardware

STM32 @ v1.22.2

MicroPython version

MicroPython v1.22.2 on 2024-04-16; [REDACTED]_[gt48468c1] with STM32G474

This is a custom board definition for a product with intellectual property. The STM32 version is STM32G473VET3. This is the mpconfigboard.mk:

# MCU settings
MCU_SERIES = g4
CMSIS_MCU = STM32G474xx
MICROPY_FLOAT_IMPL = single
AF_FILE = boards/stm32g474_af.csv

# 
LD_FILES = boards/stm32g474.ld boards/common_basic.ld

Reproduction

1. I used rshell with the rsync command.
2. Some copies would have a error in remote transfer.
3. I would observe less and less available flash space after removing all files from the /flash/ directory (also using rshell) to the point where half way through copying there would be no space left (this was only hinted by ampy after using it instead of rshell).
4. Erased full flash using STM32CubeProgrammer and reprogrammed firmware
5. Checked available flash with the /flash directory being empty.
6. Available flash was now the same as the total fs size

Example of corrupted flash resulting in less available space:

>>> import os
>>> statvfs_data = os.statvfs('/flash')
>>> f_frsize, f_blocks, f_bfree = (
...     statvfs_data[1],
...     statvfs_data[2],
...     statvfs_data[3],
... )
>>> fs_total_size = f_blocks * f_frsize
>>> fs_total_size_kB = fs_total_size / 1024
>>> fs_free_size = f_bfree * f_frsize
>>> fs_free_size_kB = fs_free_size / 1024
>>> fs_total_size_kB
143.0
>>> fs_free_size_kB
53.0

After erasing flash and re-flashing micropython v1.22.2 (main.py and boot.py are present in `/flash/ directory):

>>> import os
>>> statvfs_data = os.statvfs('/flash')
>>> f_frsize, f_blocks, f_bfree = (
...     statvfs_data[1],
...     statvfs_data[2],
...     statvfs_data[3],
... )
>>> fs_total_size = f_blocks * f_frsize
>>> fs_total_size_kB = fs_total_size / 1024
>>> fs_free_size = f_bfree * f_frsize
>>> fs_free_size_kB = fs_free_size / 1024
>>> fs_total_size_kB
143.0
>>> fs_free_size_kB
142.0

Expected behaviour

Handle invalid flash writes to not corrupt flash

Observed behaviour

Flash corruption leading to decreased available flash size, in turn resulting in more write errors and more flash corruption.

Additional Information

No, I've provided everything above.

Code of Conduct

Yes, I agree

Edit: add free size after erasing flash and re-flashing Micropython

[peterhinch]

I'm converting this to a discussion because this is not a MicroPython bug: rshell is not an official MP utility.

I and many others have used rshell extensively with STM32 without issue. I suggest you try the officially supported mpremote to determine whether the fault is with rshell or with your hardware/port.

If you believe the bug is in rshell you could report it here.

[camielverdult]

rshell is not an official MP utility.

I agree. I thought it would be a problem on the Micropython side of things. It does make more sense for writing errors to be handled by the writer than the receiving end. Thanks for the reply.

[dhylands]

rshell is only using standard python calls to write data to the filesystem. So I would argue that the filesystem should be returning errors if it gets full and not corrupting itself.

I certainly agree that it would be better if rshell did something like checked to see if there was enough available space, but the filesystem should be protecting itself. As such this should be reproducible using mpremote.