How to do the correct count of cycles to write/erase flash cycles?

[beyonlo]

Hi all

I Have a question to understand how to consider write/erase flash cycles with MicroPython. I'm using an Espressif ESP32-S3 module (ESP32-S3-WROOM-1) N16R2, so it has 16 MB SPI Flash and 2MB SPI RAM.

Some definitions:

1. The flash block size is 4 kB (4096 bytes), right?
2. I'm using many files (and writing new data periodically on them) with max size of 4096 bytes (to fit with flash block size), that is the minimum flash block size used by filesystem. Even if I write in the file less than 4096 bytes a block of 4096 bytes will be reserved, right?
3. I'm using LittleFS filesystem. As I'm using MicroPython 1.20.0 (official firmware .bin), until I know, the default filesystem in the 1.20.0 is LittleFS right?
4. The SPI Flash integrated on the ESP32-S3 module has, of course, a limit of write/erase cycles. That can be between 10k-100k cycles, that depends temperature that will be run, etc.
5. The write/erase cycles mean how many times a single sector of Flash can be erased before it's no longer guaranteed to work, right?

If some that definitions above is not correct, please, fix me.

Well, now the question:

When I'm writing new data in each one of that files, will the files continue located forever in the same sectors?

My goal with this question is to count how much write cycles I can have using my scenario until reach 10k cycles (for example):
a. Need I to consider in my scenario that each file that I will write new data as +1 cycle?
b. Or the correct is to consider as +1 cycle just when I write new data in all my files (if each file will be in the same sector forever)? Or
c. Or both (a and b is wrong), so how I should be consider?

A complement question: considering my scenario, how many bytes has a sector and how is the relation between a sector and a Flash block size?
This doc say that by default Sector size is 512 bytes (https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/storage/wear-levelling.html#:~:text=Sector%20size%20is%20512%20bytes)

My use:
Espressif module ESP32-S3-WROOM-1-N16R2
MicroPython 1.20.0

Thank you very much!

[peterhinch]

Your definitions are right.

When I'm writing new data in each one of that files, will the files continue located forever in the same sectors?

No - littlefs goes to great lengths to ensure that this doesn't happen, re-mapping sectors to minimise wear. You might like to read this doc which describes the ideas behind its design.

[beyonlo]

@peterhinch Thank you to point me that doc, I saw there that LittleFS does the Wear leveling automatically - that's great :)

What do you think about to use esp nvs instead normal write mode? The nvs does wear leveling too as we can see here

I'm thinking to migrate my normal mode (using LittleFs) to NVS because in my tests the NVS is very very faster than normal write when data is less than 1KB. What do you think about that?

Follow bellow my code with the real test:

import json, time
from esp32 import NVS
nvs = NVS("datax1")

list_data = []
list_data.append('.' * 1)
list_data.append('.' * 10)
list_data.append('.' * 50)
list_data.append('.' * 100)
list_data.append('.' * 200)
list_data.append('.' * 300)
list_data.append('.' * 400)
list_data.append('.' * 500)
list_data.append('.' * 600)
list_data.append('.' * 700)
list_data.append('.' * 800)
list_data.append('.' * 900)
list_data.append('.' * 1000)
list_data.append('.' * 1200)
list_data.append('.' * 1400)
list_data.append('.' * 1600)
list_data.append('.' * 1800)
list_data.append('.' * 2000)
list_data.append('.' * 2500)
list_data.append('.' * 3000)
list_data.append('.' * 3500)
list_data.append('.' * 4000)

for data in list_data:
    datax1 = json.dumps(data)

    key = 'datax1'
    data = datax1

    len_data = len(data)
    start_time = time.ticks_ms()
    nvs.set_blob(key, data)
    nvs.commit()
    end_time = time.ticks_ms()
    delta_time_nvs = time.ticks_diff(end_time, start_time)

    nvs.erase_key(key)

    start_time = time.ticks_ms()
    f = open('test.json', 'w')
    f.write(data)
    f.close()
    end_time = time.ticks_ms()
    delta_time_normal_mode = time.ticks_diff(end_time, start_time)
    print(f'Time to write {len_data} bytes is -> NVS: {delta_time_nvs} ms, Normal Mode: {delta_time_normal_mode} ms')
    print('----------------------')

Output:

$ mpremote run nvs_vs_normal_write_test_01.py 
Time to write 3 bytes is -> NVS: 1 ms, Normal Mode: 6 ms
----------------------
Time to write 12 bytes is -> NVS: 2 ms, Normal Mode: 16 ms
----------------------
Time to write 52 bytes is -> NVS: 2 ms, Normal Mode: 7 ms
----------------------
Time to write 102 bytes is -> NVS: 2 ms, Normal Mode: 8 ms
----------------------
Time to write 202 bytes is -> NVS: 3 ms, Normal Mode: 115 ms
----------------------
Time to write 302 bytes is -> NVS: 3 ms, Normal Mode: 51 ms
----------------------
Time to write 402 bytes is -> NVS: 58 ms, Normal Mode: 47 ms
----------------------
Time to write 502 bytes is -> NVS: 4 ms, Normal Mode: 61 ms
----------------------
Time to write 602 bytes is -> NVS: 4 ms, Normal Mode: 61 ms
----------------------
Time to write 702 bytes is -> NVS: 4 ms, Normal Mode: 59 ms
----------------------
Time to write 802 bytes is -> NVS: 5 ms, Normal Mode: 69 ms
----------------------
Time to write 902 bytes is -> NVS: 63 ms, Normal Mode: 49 ms
----------------------
Time to write 1002 bytes is -> NVS: 5 ms, Normal Mode: 48 ms
----------------------
Time to write 1202 bytes is -> NVS: 5 ms, Normal Mode: 49 ms
----------------------
Time to write 1402 bytes is -> NVS: 70 ms, Normal Mode: 52 ms
----------------------
Time to write 1602 bytes is -> NVS: 14 ms, Normal Mode: 58 ms
----------------------
Time to write 1802 bytes is -> NVS: 68 ms, Normal Mode: 64 ms
----------------------
Time to write 2002 bytes is -> NVS: 102 ms, Normal Mode: 59 ms
----------------------
Time to write 2502 bytes is -> NVS: 70 ms, Normal Mode: 72 ms
----------------------
Time to write 3002 bytes is -> NVS: 104 ms, Normal Mode: 76 ms
----------------------
Time to write 3502 bytes is -> NVS: 62 ms, Normal Mode: 96 ms
----------------------
Time to write 4002 bytes is -> NVS: 182 ms, Normal Mode: 85 ms
----------------------

[peterhinch]

Interesting. There are odd outliers where NVS slows down for certain lengths:

Time to write 902 bytes is -> NVS: 63 ms, Normal Mode: 49 ms

This could occur when NVS triggers an erase cycle.

For small objects, notably integers, NVS wins hands down. Although there might still be occasional slow writes when an erase is triggered.

[bidrohini]

The flash block size is 4 kB (4096 bytes) on the ESP32-S3 module.
You mentioned you are using LittleFS filesystem, and the minimum flash block size used by the filesystem is 4096 bytes. Even if you write less than 4096 bytes to a file, a whole block of 4096 bytes will be reserved for that file. So, you are correct in assuming that a block of 4096 bytes will be reserved for each file, even if the file size is smaller.
LittleFS is the default filesystem in MicroPython 1.20.0.
The ESP32-S3 module's SPI Flash indeed has a limited number of write/erase cycles, which can be between 10,000 to 100,000 cycles, depending on factors such as temperature, usage conditions, etc.

[robert-hh]

Even if you write less than 4096 bytes to a file, a whole block of 4096 bytes will be reserved for that file. So, you are correct in assuming that a block of 4096 bytes will be reserved for each file, even if the file size is smaller.

That does not meet my observation, at least not for LFS2. In my tests, I could easily write more files than blocks. The test writes a series of small files filled with random numbers, then it reads the series of files back and verifies the content. Test script below:

# file write and erase test
import uos
import random
import time

def run(n=10, len=256):
    seed = random.getrandbits(32)
    b=bytearray(len)
    random.seed(seed)
    for i in range (n):
        for j in range(len):
            b[j] = random.getrandbits(8)
        name = "file" + str(i)
        print("create ", name)
        f = open(name, "wb")
        f.write(b)
        f.close()
        time.sleep_ms(10)

    for e in uos.ilistdir(""):
        print(e)

    random.seed(seed)
    for i in range (n):
        for j in range(len):
            b[j] = random.getrandbits(8)
        name = "file" + str(i)
        print("read ", name)
        f = open(name, "rb")
        d = f.read(len)
        f.close()
        if b != d:
            print("Read mismatch")
            input("Next")
        time.sleep_ms(10)

    for i in range (n):
        name = "file" + str(i)
        print("delete ", name)
        uos.remove(name)
        time.sleep_ms(10)

    for e in uos.ilistdir(""):
        print(e)


def xrun(n=40, m=30):
    for i in range(n):
        print("Iteration", i+1)
        time.sleep(1)
        run(m)

run()

[bixb922]

I tested different files with littlefs and files smaller than 128 bytes are stored in the same 4096 byte block.

[beyonlo]

What are the nvs advantages (in addition to being very faster) that you see over normal mode write using littlefs?

And about the nvs disadvantages?

[rkompass]

Working through similar findings with LFS2 previously I found after modifying Roberts code:
The modified filesystem test:

# file write and erase test
import uos
import random
import time

# We change the run funtion so that it allows to have a list/tuple of lengths
#   that it cycles through
#
def run(n=120, flen=200, prn=True):

    read_mismatches = 0
    fsusg_mx = 0
    fscons_rat_mx = 0

    if isinstance(flen, int):
        flen = (flen,)
    elif not isinstance(flen, (tuple, list)):
        raise ValueError('run: flen has to be of type int, tuple or list')
    lfl = len(flen)
    bytes_written = 0
    
    fs_stat = uos.statvfs('/')
    fs_free0 = fs_stat[0] * fs_stat[3]
    if prn:
        print("File system free space: {} KB".format(fs_free0/1024))
    
    seed = random.getrandbits(32)
    random.seed(seed)
    for i in range (n):
        b=bytearray(flen[i%lfl])
        for j in range(len(b)):
            b[j] = random.getrandbits(8)
        name = "file" + str(i)
        if prn:
            if i%10 == 0:
                print("create ", end='')
            else:
                print(name, end=' ')
            if i%10 == 9:
                print()
        f = open(name, "wb")
        f.write(b)
        f.close()
        bytes_written += len(b)
        del(b)
        if i % 10 == 9 or i == n-1:
            fs_stat = uos.statvfs('/')
            fs_free = fs_stat[0] * fs_stat[3]
            fsusg_mx = max(fsusg_mx, fs_free0-fs_free)
            fscons_rat = (fs_free0-fs_free)/bytes_written
            fscons_rat_mx = max(fscons_rat_mx, fscons_rat)
            if prn:
                print("FS free: {} KB. Bytes written: {} B. FS usage per Bytes: {:.2f}".format(
                    fs_free/1024, bytes_written, fscons_rat))
                time.sleep(1)
        time.sleep_ms(2)

    if prn:
        print('-------------------------------------')

    random.seed(seed)
    for i in range (n):
        b=bytearray(flen[i%lfl])
        for j in range(len(b)):
            b[j] = random.getrandbits(8)
        name = "file" + str(i)
        f = open(name, "rb")
        d = f.read(len(b))
        f.close()
        if b != d:
            read_mismatches += 1
            if prn:
                print("Read mismatch")
                input("Next")
        del(b)
        time.sleep_ms(2)

    for i in range (n):
        name = "file" + str(i)
        if prn:
            if i%10 == 0:
                print("delete ", end='')
            else:
                print(name, end=' ')
            if i%10 == 9:
                print()
        uos.remove(name)
        bytes_written -= flen[i%lfl]
        if i % 10 == 9:
            fs_stat = uos.statvfs('/')
            fs_free = fs_stat[0] * fs_stat[3]
            if prn:
                if i < n-1:
                    print("FS free: {} KB. Bytes written: {} B. FS usage per Bytes: {:.2f}".format(
                    fs_free/1024, bytes_written, (fs_free0-fs_free)/bytes_written))
                else:
                    print("File system free space: {} KB".format(fs_free/1024))
                time.sleep(1)

        time.sleep_ms(2)

    print('files: {},  len: {}, mism: {}, fsusg: {}, usgmxrat: {}'.format(n, flen, read_mismatches, fsusg_mx, fscons_rat_mx))
    return (read_mismatches, fsusg_mx, fscons_rat_mx)

yielded:

files: 500,  len: (20,), mism: 0, fsusg: 155648, usgmxrat: 16.384
files: 300,  len: (50,), mism: 0, fsusg: 155648, usgmxrat: 10.73434
files: 200,  len: (100,), mism: 0, fsusg: 155648, usgmxrat: 7.7824
files: 150,  len: (200,), mism: 0, fsusg: 188416, usgmxrat: 6.371555
files: 80,  len: (500,), mism: 0, fsusg: 204800, usgmxrat: 5.149257
files: 50,  len: (1000,), mism: 0, fsusg: 204800, usgmxrat: 4.096
files: 50,  len: (2000,), mism: 0, fsusg: 204800, usgmxrat: 2.048
files: 50,  len: (4000,), mism: 1, fsusg: 204800, usgmxrat: 1.024
files: 30,  len: (5000,), mism: 1, fsusg: 245760, usgmxrat: 1.6384
files: 30,  len: (6000,), mism: 0, fsusg: 245760, usgmxrat: 1.365333
files: 30,  len: (7000,), mism: 0, fsusg: 245760, usgmxrat: 1.170286

So we see that LFS2 behaves badly for very small file sizes, although it is able to accommodate more than one file into a 4096 bytes block.
This is in contrast to the statements in https://github.com/littlefs-project/littlefs/blob/master/DESIGN.md where a graphics is given:

Roughly we observe the first saw-tooth above the 4K file size. But the constant upper limit at a 4-fold space consumption is not there.
As I understand it, this essential feature, aimed at with LFS2, does not work out. Please correct me If I'm wrong.
If this finding is correct perhaps it should be mentioned / raised as an issue? Do we have the original LFS2 code in Micropython or some branch/derivation?

Seems that the nfs advantage is just in this area, regarding file space.