v3-alpha: Rbyds and B-trees and gcksums, oh my! #1111
+195,812
−28,619
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit actually does two things:
1. Opportunistically marks caches as flushed if they were included in
the crystallization region in lfsr_file_crystallize
2. Reroutes lfsr_file_read_ through lfsr_file_crystallize to minimize
stack cost
---
Digging into why lazy-crystallization adds so much stack, it seems the
main reason is because lfsr_file_read_ drags in lfsr_file_flush, which
puts the entirety of the stack hot-path under both lfsr_file_read and
lfsr_file_read_.
But why are we calling lfsr_file_flush? And not just
lfsr_file_crystallize to claim the leaf? Isn't the cache flushed in
lfsr_file_read before reading?
The one concerning case is when reads bypass the cache (read >
cache_size). With cache-bypassing reads, it's entirely possible for
lfsr_file_read_ to end up with unflushed data. lfsr_file_read's logic
gives the cache priority in this case, so it's not like we're going to
read outdated data or anything, but if we crystallize without flushing
we risk wasting erased-state that will need to be recrystallized later.
What's extra humorous is our crystallization logic _does_ correctly
write out the cache, it just doesn't clear the LFS_o_UNFLUSH bit because
it doesn't know if progress has been made.
So to avoid this, all we need to do is add an explicit check to
lfsr_file_crystallize that clears the LFS_o_UNFLUSH bit if our cache
ends up written out as a part of crystallization.
Note this is slightly more powerful than lfsr_file_flush, since we don't
_need_ to flush the cache if it's not in our crystallization region.
As an extra plus this affects all lfsr_file_crystallize calls, so now
lfsr_file_truncate/fruncate also avoid unnecessary recrystallization.
That's some good code reuse right there!
---
Long story short, rerouting lfsr_file_read_ through
lfsr_file_crystallize moves it off the stack hot-path, bringing our
stack down to almost pre-lazy-crystallization levels:
code stack ctx
before: 37140 2304 636
after: 37184 (+0.1%) 2288 (-0.7%) 636 (+0.0%)
At a code cost, but this code also allows lfsr_file_read/truncate/
fruncate to avoid recrystallization with opportunistic flushes in cases
where we need to discard file->leaf.
Not sure how the duplication here went unnoticed. Note we can _not_ reuse block_start/end for the buffer updates, since those depend on the crystallized/aligned result. No code changes though. The good news is the compiler is doing a good job with the dense math in these functions.
- lfsr_rbyd_init
- lfsr_rbyd_claim
- lfsr_btree_claim
- lfsr_mdir_claim
Saves a bit of code, but I think this is just because of a tweak to how
we check for shared btree erased-state that crept in (now only comparing
blocks instead of block + trunk):
code stack ctx
before: 37184 2288 636
after: 37172 (-0.0%) 2288 (+0.0%) 636 (+0.0%)
This should match the internal LFSR_TAG_* values, but was probably missed during a refactor.
The idea here, is we give each lfsr_btree_t an optional leaf rbyd, in
addition to the root rbyd. This leaf rbyd acts as a cache for the most
recent leaf, allowing nearby btree lookups to skip the full btree walk.
Unfortunately, this failed on pretty much every measurable metric...
---
The motivation for this is that we often do a bunch of nearby btree
lookups:
- Btree iteration via lfsr_btree_lookupnext is a bit naive, walking from
the root every step.
- Our crystallization algorithm requires a bunch of nearby lookups to
figure out our crystallization heuristic. Currently at most 4, when
you need to lookup both crystal neighbors and then _also_ both
fragment neighbors for coalescing.
- Checksum collision resolution for dids and (FUTURE) ddkeys can require
an unbounded number of sequential lookups.
Though to be fair, this is an exceptional case if our checksum is any
good.
- Bids with multiple rattrs require nearby lookups to resolve.
Though currently this can be explicitly avoided via
lfsr_btree_lookupleaf + lfsr_rbyd_lookup.
The theory was that cases like these could explicitly keep track of the
leaf rbyd to avoid full btree walks, but in practice this never really
worked out. Tracking if we're still in the relevant leaf rbyd just adds
too much logic/code cost.
But if this leaf tracking logic was implemented once in the btree
layer...
The other theoretical benefit was being able to move more rbyds off the
stack. Sure our btrees take up more RAM, but if that results in stack
savings, that may be a win.
Oh, and this would let our btree API and rbyd API converge without
performance concerns. Internal users could in theory call
lfsr_btree_lookupnext + lfsr_btree_lookup with the same performance as
explicitly tracking the rbyd.
---
But this was a complete failure!
First the good news: There was a modest speedup of around ~2x to linear
reads.
And that's the good news.
Now the bad news:
1. There was no noticeable performance gain in any other benchmarks.
To be fair, we're at the early stages of benchmarking, so the
benchmarks may not be the most thorough, but thinking about it, there
are some explanations:
- In any benchmark that writes, fetch + erase + prog dominates. Being
able to skip fetches during lookups makes our btree lookups
surprisingly cheap!
- Any random read heavy benchmark is likely thrashing this cache,
which is to be expected.
- For small 1-block btrees, the leaf cache is useless because the
entire btree is cache in the root rbyd.
And keep in mind, our blocks are BIG. "Small" here could be on
the order of ~128KiB-1MiB for NAND flash.
- For the mtree, fetched mdirs actually already act as a sort of leaf
cache.
The extra btree leaf cache isn't doing _nothing_, but each layer of
the mtree has diminishing returns due to btree's ridiculous
branching factor.
- For file btrees, we're explicitly caching the leaf fragments/
blocks, so the extra btree leaf cache has diminishing returns for
the same reason.
2. Code cost was bad, stack cost was worse:
code stack ctx
before: 37172 2288 636
after: 38068 (+2.4%) 2416 (+5.6%) 664 (+4.4%)
Tracking the leaf required more code, that's expected. And, to be
fair, the current code has had a lot more time to congeal.
What wasn't expected was the stack cost.
Unfortunately these caches didn't really take any rbyds off the stack
hot-path:
- We _can_ get rid of the rbyd in lfsr_btree_lookup/namelookup, but
we were already hacking our way around the critical one in
lfsr_mtree_lookup/namelookup by reusing the mdir's rbyd!
- We can't even abuse the leaf rbyd in the commit logic, since the
target btree can end up iterated/traversed by lfs_alloc.
That was a fun bug.
And the addition of a second rbyd to lfsr_btree_t increases both ctx
and stack anywhere btrees are allocated.
Maybe this will make more sense when we add the auxiliary btrees, or
after more benchmarking, but for now the theoretical performance
improvements just aren't worth it.
Will probably revert this, but I wanted to commit it in case the idea is
worth resurrecting in the future, if in the future nearby btree lookups
are a bigger penalty than they are now.
See the relevant commit for why. These just added surprisingly little performance benefit for the code/stack cost. Maybe in a future performance-preferring littlefs driver.
Originally adopted during the failed btree-leaf-cache, I just think this is a bit more readable when mixed in with parent, sibling, etc. Also a couple comment tweaks. No code changes.
Not sure why, but this just seems more intuitive/correct. Maybe because LFSR_TAG_NAME is always the first tag in a file's attr set: LFSR_TAG_NAMELIMIT 0x0039 v--- ---- --11 1--1 LFSR_TAG_FILELIMIT 0x003a v--- ---- --11 1-1- Seeing as several parts of the codebase still use the previous order, it seems reasonable to switch back to that. No code changes.
- LFS_CKPARITY -> LFS_CKMETAPARITY - LFS_CKDATACKSUMS -> LFS_CKDATACKSUMREADS The goal here is to provide hints for 1. what is being checked (META, DATA, etc), and 2. on what operation (FETCHES, PROGS, READS, etc). Note that LFS_CKDATACKSUMREADS is intended to eventually be a part of a set of flags that can pull off closed fully-checked reads: - LFS_CKMETAREDUNDREADS - Check data checksums on reads - LFS_CKDATACKSUMREADS - Check metadata redund blocks on reads - LFS_CKREADS - LFS_CKMETAREDUNDREADS + LFS_CKDATACKSUMREADS Also it's probably not a bad idea for LFS_CKMETAPARITY to be harder to use. It's really not worth enabling unless you understand its limitations (<1 bit of error detection, yay). No code changes.
This time to account for the new LFS_o_UNCRYST and LFS_o_UNGRAFT flags.
This required moving the T flags out of the way, which of course
conflicted with TSTATE, so that had to move...
One thing that helped was shoving LFS_O_DESYNC up with the internal
state flags. It's definitely more a state flag than the other public
flags, it just also happens to be user toggleable.
Here's the new jenga:
8 8 8 8
.----++----++----++----.
.-..----..-..-..-------.
o_flags: |t|| f ||o||t|| o |
|-||-.--':-:|-|'--.-.--'
|-||-|.----.|-'--------.
t_flags: |t||f||tstt|| t |
'-''-''----'|----.-----'
.----..-.:-:|----|:-:.-.
m_flags: | m ||c||o|| t ||o||m|
|----||-|'-'|-.--''-''-'
|----||-|---|-|.-------.
f_flags: | m ||c| |t|| f |
'----''-'---'-''-------'
This adds a bit of code, but that's not the end of the world:
code stack ctx
before: 37172 2288 636
after: 37200 (+0.1%) 2288 (+0.0%) 636 (+0.0%)
These mimic the relevant LFS_O_* flags, and allow users to assert
whether or not a traversal will mutate the filesystem:
LFS_T_MODE 0x00000001 The traversal's access mode
LFS_T_RDWR 0x00000000 Open traversal as read and write
LFS_T_RDONLY 0x00000001 Open traversal as read only
In theory, these could also change internal allocations, but littlefs
doesn't really work that way.
Note we _don't_ add related LFS_GC_RDONLY, LFS_GC_RDWR, etc flags. These
are sort of implied by the relevant LFS_M_* flags.
Adds a bit more code, probably because of the slightly more complicated
internal constants for the internal traversals. But I think the
self-documentingness is worth it:
code stack ctx
before: 37200 2288 636
after: 37220 (+0.1%) 2288 (+0.0%) 636 (+0.0%)
This was a simple oversight, we weren't checking recipient file caches
when broadcasting sync!
Fixed by limiting the synced cache to the last n bytes that fit in the
recipient's cache. This is a bit more complicated than first n bytes,
but more intuitive/likely to be relevant to the recipient file.
Adds a bit of code/stack. In theory this shouldn't really affect the
stack, but lfsr_file_sync is a sensitive function on the stack hot-path:
code stack ctx
before: 37220 2288 636
after: 37260 (+0.1%) 2296 (+0.3%) 636 (+0.0%)
This adds lfsr_btree_commitroot_ and lfsr_bshrub_commitroot_, to contain
the root-specific commit logic such that it can be forced off the stack
hot-path if necessary.
---
Note we're not actually using LFS_NOINLINE yet, as the critical
function, lfsr_btree_commitroot_ is implicitly forced off the stack
hot-path via the multiple calls from lfsr_btree_commit and
lfsr_bshrub_commit.
And I'm not sure it makes sense to use LFS_NOINLINE here. It absolutely
wrecks lfsr_bshrub_commitroot_'s stack, which always ends up on the
stack hot-path because of the route through lfsr_mdir_commit.
Is this a big hack? Honestly yeah.
It doesn't even really save that much stack, but I figured it was worth
a try:
code stack ctx
before: 37260 2296 636
after: 37300 (+0.1%) 2280 (-0.7%) 636 (+0.0%)
At least the code organization is a bit better, with lfsr_bshrub_commit
reusing lfsr_btree_commitroot_ for bshrub -> btree migration.
Life's too short to not use this flag.
Not sure what the point of this was, I think it was copied from a d3 example svg at some point. But it forces the svg to always fit in the window, even if this makes the svg unreadable. These svgs tend to end up questionably large in order to fit in the most info, so the unreadableness ends up a real problem for even modest window sizes.
I don't think this hyphen broke anything, but this matches the naming convention of other files in the codebase (lfs_util.h for example).
Removing the vestiges of v2 tests.
Kinda. It's actually only 3 scripts. These have been replaced with the new dbg*.py scripts: - readblock.py -> dbgblock.py - readmdir.py -> dbgrbyd.py - readtree.py -> dbglfs.py
Removing the final vestiges of v2.
This one was a bit more involved. Removes utils that are no longer useful, and made sure some of the name/API changes over time are adopted consistently: - lfs_npw2 -> lfs_nlog2 - lfs_tole32_ -> lfs_tole32 - lfs_fromle32_ -> lfs_fromle32 Also did another pass for lfs_ prefixes on mem/str functions. The habit to use the naked variants of these is hard to break!
Like the LFS_DISK_VERSION, the intention is to mark the current driver as experimental. Just in case...
Member
Author
|
There is high risk this thread drowns in noise. So, please limit this thread to high-level v3-wide discussion. For bugs, low-level questions, concerns, etc, feel free to create issues on littlefs with "v3" in the title. I will label relevant issues with the v3 label when I can. |
geky
added
needs major version
Contributor
|
Fantastic! Great to see this v3 work making it's way into the world. The code size increase would be the biggest concern for MicroPython. Any optimisations to get that down would be welcome. In particular we put read-only littlefs2 inside a 32k bootloader. Do you have any idea how big the read-only version of v3 is? |
Now that lfs3_mtree_traverse_ uses a sort of state matrix, lfs3_rbyd_appendrattr_ is the only function still relying on a big switch-case statement. Replacing it with a series of if-else statements leaves the codebase switch-case free (ignoring test/bench runners, etc). Switch-case statements are extremely error prone in C, with the shared scope, implicit fallthrough, etc. And, with today's compilers, the result still ends up the same, so switch-case statements offer no benefit except maybe a more enjoyable syntax for masochists. Avoiding switch-case statements in code where we care about correctness is probably a good idea. No code changes
With the relaxation of traversal behavior under mutation, I think it
makes sense to bring back LFS3_T_EXCL. If only to allow traversals to
gaurantee termination under mutation. Now that traversals no longer
guarantee forward progress, it's possible to get stuck looping
indefinitely if the filesystem is constantly being mutated.
Non-excl traversals are probably still useful for GC work and debugging
threads, but LFS3_T_EXCL now allows traversals to terminate immediately
with LFS3_ERR_BUSY at the first sign of unrelated filesystem mutation:
LFS3_T_EXCL 0x00000008 Error if filesystem modified
Internally, we already track unrelated mutation to avoid corrupt state
(LFS3_t_DIRTY), so this is a very low-cost feature:
code stack ctx
before: 35944 2280 660
after: 35964 (+0.1%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38916 2296 772
gbmap after: 38940 (+0.1%) 2296 (+0.0%) 772 (+0.0%)
code stack ctx
gc before: 36016 2280 768
gc after: 36036 (+0.1%) 2280 (+0.0%) 768 (+0.0%)
Now that we use LFS3_ERR_BUSY for traversals, we no longer have an excuse for not returning LFS3_ERR_BUSY on root-related errors: - lfs3_remove(&lfs3, "/") => LFS3_ERR_BUSY - lfs3_rename(&lfs3, "/", *) => LFS3_ERR_BUSY - lfs3_rename(&lfs3, *, "/") => LFS3_ERR_BUSY This better aligns with POSIX. Arguably we should have defined LFS3_ERR_BUSY for this case anyways, it's not like additional error codes cost much. No code changes.
LFS3_REVDBG introduced a lot of overhead for something I'm not sure
anyone will actually use (I have enough tooling that the state of an
rbyd is rarely a mystery, see dbgbmap.py). That, and we're running out
of flags!
So this reduces LFS3_REVDBG to just store one of "himb" in the first
(lowest) byte of the revision count; information that is easily
available:
vvvv---- -------- -------- --------
vvvvrrrr rrrrrr-- -------- --------
vvvvrrrr rrrrrrnn nnnnnnnn nnnnnnnn
vvvvrrrr rrrrrrnn nnnnnnnn dddddddd
'-.''----.----''----.- - - '---.--'
'------|----------|----------|---- 4-bit relocation revision
'----------|----------|---- recycle-bits recycle counter
'----------|---- pseudorandom noise (if revnoise)
'---- h, i, m, or b (if revdbg)
-11-1--- - h = mroot anchor
-11-1--1 - i = mroot
-11-11-1 - m = mdir
-11---1- - b = btree node
Some other notes:
- Enabled LFS3_REVDBG and LFS3_REVNOISE to work together, now that
LFS3_REVDBG doesn't consume all unused rev bits.
Note that LFS3_REVDBG has priority over LFS3_REVNOISE, but _not_
recycle-bits, etc. Otherwise problems would happen for recycle-bits
>2^20 (though do we care?).
- Fixed an issue where using the gcksum as a noise source results in
noise=0 when there is only an mroot. This is due to how we xor out
the current mdir cksum during an mdir commit.
Fixed by using gcksum_p instead of gcksum.
- Added missing LFS3_I_REVDBG/REVNOISE flags in the tests, so now you
can actually run the tests with LFS3_REVDBG/REVNOISE (this probably
just fell out-of-date at some point).
---
Curiously, despite LFS3_REVDBG/REVNOISE being disabled by default, this
did save some code. I'm guessing the non-tail-call mtree/gbmap commit
functions prevented some level of inlining?:
code stack ctx
before: 35964 2280 660
after: 35964 (+0.0%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38940 2296 772
gbmap after: 38828 (-0.3%) 2296 (+0.0%) 772 (+0.0%)
Now that we don't need to encode tstate info in our traversal flags, we
can move things around to be a bit more comfortable.
This is also after some tweaking to make space for planned features:
O flags:
O_MODE 0x00000003 ---- ---- ---- ---- ---- ---- ---- --11
O_RDONLY 0x00000000 ---- ---- ---- ---- ---- ---- ---- ----
O_WRONLY 0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
O_RDWR 0x00000002 ---- ---- ---- ---- ---- ---- ---- --1-
O_CREAT 0x00000004 ---- ---- ---- ---- ---- ---- ---- -1--
O_EXCL 0x00000008 ---- ---- ---- ---- ---- ---- ---- 1---
O_TRUNC 0x00000010 ---- ---- ---- ---- ---- ---- ---1 ----
O_APPEND 0x00000020 ---- ---- ---- ---- ---- ---- --1- ----
O_FLUSH 0x00000040 ---- ---- ---- ---- ---- ---- -1-- ----
O_SYNC 0x00000080 ---- ---- ---- ---- ---- ---- 1--- ----
O_DESYNC 0x00100000 ---- ---- ---1 ---- ---- ---- ---- ----
O_DEDAG* 0x00000100 ---- ---- ---- ---- ---- ---1 ---- ----
O_DEDUP* 0x00000200 ---- ---- ---- ---- ---- --1- ---- ----
O_COMPR? 0x00000400 ---- ---- ---- ---- ---- -1-- ---- ----
O_CKMETA 0x00010000 ---- ---- ---- ---1 ---- ---- ---- ----
O_CKDATA 0x00020000 ---- ---- ---- --1- ---- ---- ---- ----
O_REPAIRMETA* 0x00040000 ---- ---- ---- -1-- ---- ---- ---- ----
O_REPAIRDATA* 0x00080000 ---- ---- ---- 1--- ---- ---- ---- ----
o_WRSET 0x00000003 ---- ---- ---- ---- ---- ---- ---- --11
o_TYPE 0xf0000000 1111 ---- ---- ---- ---- ---- ---- ----
o_ZOMBIE 0x08000000 ---- 1--- ---- ---- ---- ---- ---- ----
o_UNCREAT 0x04000000 ---- -1-- ---- ---- ---- ---- ---- ----
o_UNSYNC 0x02000000 ---- --1- ---- ---- ---- ---- ---- ----
o_UNCRYST 0x01000000 ---- ---1 ---- ---- ---- ---- ---- ----
o_UNGRAFT 0x00800000 ---- ---- 1--- ---- ---- ---- ---- ----
o_UNFLUSH 0x00400000 ---- ---- -1-- ---- ---- ---- ---- ----
* Planned
? Hypothetical
T flags:
T_MODE 0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
T_RDONLY 0x00000000 ---- ---- ---- ---- ---- ---- ---- ----
T_RDWR 0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
T_MTREEONLY 0x00000002 ---- ---- ---- ---- ---- ---- ---- --1-
T_EXCL 0x00000008 ---- ---- ---- ---- ---- ---- ---- 1---
T_MKCONSISTENT 0x00000800 ---- ---- ---- ---- ---- 1--- ---- ----
T_RELOOKAHEAD 0x00001000 ---- ---- ---- ---- ---1 ---- ---- ----
T_REGBMAP 0x00002000 ---- ---- ---- ---- --1- ---- ---- ----
T_PREERASE* 0x00004000 ---- ---- ---- ---- -1-- ---- ---- ----
T_COMPACTMETA 0x00008000 ---- ---- ---- ---- 1--- ---- ---- ----
T_CKMETA 0x00010000 ---- ---- ---- ---1 ---- ---- ---- ----
T_CKDATA 0x00020000 ---- ---- ---- --1- ---- ---- ---- ----
T_REPAIRMETA* 0x00040000 ---- ---- ---- -1-- ---- ---- ---- ----
T_REPAIRDATA* 0x00080000 ---- ---- ---- 1--- ---- ---- ---- ----
t_EVICT* 0x00000010 ---- ---- ---- ---- ---- ---- ---1 ----
t_TYPE 0xf0000000 1111 ---- ---- ---- ---- ---- ---- ----
t_ZOMBIE 0x08000000 ---- 1--- ---- ---- ---- ---- ---- ----
t_CKPOINTED 0x04000000 ---- -1-- ---- ---- ---- ---- ---- ----
t_DIRTY 0x02000000 ---- --1- ---- ---- ---- ---- ---- ----
t_STALE 0x01000000 ---- ---1 ---- ---- ---- ---- ---- ----
t_BTYPE 0x00f00000 ---- ---- 1111 ---- ---- ---- ---- ----
* Planned
M/F flags:
M_MODE 0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
M_RDWR 0x00000000 ---- ---- ---- ---- ---- ---- ---- ----
M_RDONLY 0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
M_STRICT? 0x00000002 ---- ---- ---- ---- ---- ---- ---- --1-
M_FORCE? 0x00000004 ---- ---- ---- ---- ---- ---- ---- -1--
M_FORCEWITHRECKLESSABANDON?
0x00000008 ---- ---- ---- ---- ---- ---- ---- 1---
M_FLUSH 0x00000040 ---- ---- ---- ---- ---- ---- -1-- ----
M_SYNC 0x00000080 ---- ---- ---- ---- ---- ---- 1--- ----
M_DEDAG* 0x00000100 ---- ---- ---- ---- ---- ---1 ---- ----
M_DEDUP* 0x00000200 ---- ---- ---- ---- ---- --1- ---- ----
M_COMPR? 0x00000400 ---- ---- ---- ---- ---- -1-- ---- ----
M_REVDBG 0x00000010 ---- ---- ---- ---- ---- ---- ---1 ----
M_REVNOISE 0x00000020 ---- ---- ---- ---- ---- ---- --1- ----
M_CKPROGS 0x00100000 ---- ---- ---1 ---- ---- ---- ---- ----
M_CKFETCHES 0x00200000 ---- ---- --1- ---- ---- ---- ---- ----
M_CKMETAPARITY 0x00400000 ---- ---- -1-- ---- ---- ---- ---- ----
M_CKMETAREDUND* 0x00800000 ---- ---- 1--- ---- ---- ---- ---- ----
M_CKDATACKSUMS 0x01000000 ---- ---1 ---- ---- ---- ---- ---- ----
M_CKREADS* 0x01800000 ---- ---1 1--- ---- ---- ---- ---- ----
M_MKCONSISTENT 0x00000800 ---- ---- ---- ---- ---- 1--- ---- ----
M_RELOOKAHEAD 0x00001000 ---- ---- ---- ---- ---1 ---- ---- ----
M_REGBMAP 0x00002000 ---- ---- ---- ---- --1- ---- ---- ----
M_PREERASE* 0x00004000 ---- ---- ---- ---- -1-- ---- ---- ----
M_COMPACTMETA 0x00008000 ---- ---- ---- ---- 1--- ---- ---- ----
M_CKMETA 0x00010000 ---- ---- ---- ---1 ---- ---- ---- ----
M_CKDATA 0x00020000 ---- ---- ---- --1- ---- ---- ---- ----
M_REPAIRMETA* 0x00040000 ---- ---- ---- -1-- ---- ---- ---- ----
M_REPAIRDATA* 0x00080000 ---- ---- ---- 1--- ---- ---- ---- ----
F_CKFACTORY* 0x00000002 ---- ---- ---- ---- ---- ---- ---- --1-
F_GBMAP 0x02000000 ---- --1- ---- ---- ---- ---- ---- ----
F_GDDTREE* 0x04000000 ---- -1-- ---- ---- ---- ---- ---- ----
F_GPTREE* 0x08000000 ---- 1--- ---- ---- ---- ---- ---- ----
F_METAR1* 0x10000000 ---1 ---- ---- ---- ---- ---- ---- ----
F_METAR2* 0x20000000 --1- ---- ---- ---- ---- ---- ---- ----
F_METAR3* 0x30000000 --11 ---- ---- ---- ---- ---- ---- ----
F_DATAR1* 0x40000000 -1-- ---- ---- ---- ---- ---- ---- ----
F_DATAR2* 0x80000000 1--- ---- ---- ---- ---- ---- ---- ----
F_DATAR3* 0xc0000000 11-- ---- ---- ---- ---- ---- ---- ----
* Planned
? Hypothetical
It's a bit concerning that _all_ 32-bit mount flags end up used, but
what can you do...
Code changes minimal:
code stack ctx
before: 35964 2280 660
after: 35968 (+0.0%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38828 2296 772
gbmap after: 38828 (+0.0%) 2296 (+0.0%) 772 (+0.0%)
These are unlikely to make much progress, but that doesn't seem like a great reason to disallow these flags in lfs3_format: LFS3_F_REGBMAP 0x00002000 Repopulate the gbmap LFS3_F_COMPACTMETA 0x00008000 Compact metadata logs These are actually guaranteed to do _no_ work when formatting _without_ the gbmap, but with the gbmap it's less clear. Looking forward to the planned ckfactory feature, these may be useful for cleaning up any rbyd commits created as a part of building the initial gbmap. --- Also tweaked the formatting for LFS3_F_* flags a bit, including making all ifdefs explicit (mainly ifdef LFS3_RDONLY). Mixed ifdefs are a real pain to read. No code changes.
I mean, why not? These redirect to the same internal lfs3_fs_gc_ function anyways. Might as well keep things consistent. Added: LFS3_F_MKCONSISTENT 0x00000800 Make the filesystem consistent LFS3_F_RELOOKAHEAD 0x00001000 Repopulate lookahead buffer LFS3_F_MKCONSISTENT is guaranteed to be a noop, but LFS3_F_RELOOKAHEAD forces a filesystem traversal, which may have some niche use case. No code changes.
- test_gc_repoplookahead_progress -> test_gc_relookahead_progress - test_gc_repoplookahead_mutation -> test_gc_relookahead_mutation - test_gc_repoplookahead_relaxed -> test_gc_relookahead_relaxed - test_gc_repopgbmap_progress -> test_gc_regbmap_progress - test_gc_repopgbmap_mutation -> test_gc_regbmap_mutation - test_gc_repopgbmap_relaxed -> test_gc_regbmap_relaxed - test_mount_t_repoplookahead -> test_mount_t_relookahead - test_mount_t_repopgbmap -> test_mount_t_regbmap
Yeah, after using these for a bit, the RE* names were not great. Trying LOOK* now, as an alternative that hopefully still implies the similar behavior without needing an additional prefix for LOOKAHEAD: - LFS3_*_RELOOKAHEAD -> LFS3_*_LOOKAHEAD - LFS3_*_REGBMAP -> LFS3_*_LOOKGBMAP - cfg.regbmap_thresh -> cfg.lookgbmap_thresh - cfg.gc_relookahead_thresh -> cfg.gc_lookahead_thresh - cfg.gc_regbmap_thresh -> cfg.gc_lookgbmap_thresh
Other than moving things around to make space for planned features, this
also adopts the idea of allowing compat flags to be ored into a single
32-bit integer, at least in the short-term.
Note though that these are still stored in separate wcompat/rcompat
tags, to make compat tests easier, and we may introduce conflicting
flags in the future if we run out of 32-bits. This is just an indulgence
to potentially make tooling/debugging easier until that happens.
Rcompat flags:
RCOMPAT_NONSTANDARD+
0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
RCOMPAT_WRONLY+ 0x00000004 ---- ---- ---- ---- ---- ---- ---- -1--
RCOMPAT_MMOSS 0x00000010 ---- ---- ---- ---- ---- ---- ---1 ----
RCOMPAT_MSPROUT+ 0x00000020 ---- ---- ---- ---- ---- ---- --1- ----
RCOMPAT_MSHRUB+ 0x00000040 ---- ---- ---- ---- ---- ---- -1-- ----
RCOMPAT_MTREE 0x00000080 ---- ---- ---- ---- ---- ---- 1--- ----
RCOMPAT_BMOSS+ 0x00000100 ---- ---- ---- ---- ---- ---1 ---- ----
RCOMPAT_BSPROUT+ 0x00000200 ---- ---- ---- ---- ---- --1- ---- ----
RCOMPAT_BSHRUB 0x00000400 ---- ---- ---- ---- ---- -1-- ---- ----
RCOMPAT_BTREE 0x00000800 ---- ---- ---- ---- ---- 1--- ---- ----
RCOMPAT_MDIRR1* 0x00001000 ---- ---- ---- ---- ---1 ---- ---- ----
RCOMPAT_MDIRR2* 0x00002000 ---- ---- ---- ---- --1- ---- ---- ----
RCOMPAT_MDIRR3* 0x00003000 ---- ---- ---- ---- --11 ---- ---- ----
RCOMPAT_BTREER1* 0x00004000 ---- ---- ---- ---- -1-- ---- ---- ----
RCOMPAT_BTREER2* 0x00008000 ---- ---- ---- ---- 1--- ---- ---- ----
RCOMPAT_BTREER3* 0x0000c000 ---- ---- ---- ---- 11-- ---- ---- ----
RCOMPAT_GRM 0x00010000 ---- ---- ---- ---1 ---- ---- ---- ----
RCOMPAT_GMV? 0x00020000 ---- ---- ---- --1- ---- ---- ---- ----
RCOMPAT_GDDTREE* 0x00100000 ---- ---- ---1 ---- ---- ---- ---- ----
RCOMPAT_GPTREE* 0x00200000 ---- ---- --1- ---- ---- ---- ---- ----
RCOMPAT_DATAR1* 0x00400000 ---- ---- -1-- ---- ---- ---- ---- ----
RCOMPAT_DATAR2* 0x00800000 ---- ---- 1--- ---- ---- ---- ---- ----
RCOMPAT_DATAR3* 0x00c00000 ---- ---- 11-- ---- ---- ---- ---- ----
rcompat_OVERFLOW+ 0x80000000 1--- ---- ---- ---- ---- ---- ---- ----
* Planned
+ Reserved
? Hypothetical
Wcompat flags:
WCOMPAT_NONSTANDARD+
0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
WCOMPAT_RDONLY+ 0x00000002 ---- ---- ---- ---- ---- ---- ---- --1-
WCOMPAT_GCKSUM 0x00040000 ---- ---- ---- -1-- ---- ---- ---- ----
WCOMPAT_GBMAP 0x00080000 ---- ---- ---- 1--- ---- ---- ---- ----
WCOMPAT_DIR 0x01000000 ---- ---1 ---- ---- ---- ---- ---- ----
WCOMPAT_SYMLINK? 0x02000000 ---- --1- ---- ---- ---- ---- ---- ----
WCOMPAT_SNAPSHOT? 0x04000000 ---- -1-- ---- ---- ---- ---- ---- ----
wcompat_OVERFLOW+ 0x80000000 1--- ---- ---- ---- ---- ---- ---- ----
+ Reserved
? Hypothetical
Ocompat flags:
OCOMPAT_NONSTANDARD+
0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
ocompat_OVERFLOW+ 0x80000000 1--- ---- ---- ---- ---- ---- ---- ----
+ Reserved
Other notes:
- M* and B* struct flags were reordered to match META -> DATA order
elsewhere. This no longer matches the tag ordering, but there's an
argument the B* tags apply more generally (all btrees) than the B*
compat flag (only file btrees).
- MDIR/BTREE/DATA redund flags were moved near relevant flags, rather
than sticking them in the higher-order bits as we are planning to do
in the M_*/F_* flags. The compat flags already won't match because of
the mdir/btree split (which is IMO too much detail to include in
M_*/F_* flags, but hard to argue against in the compat flags), and
this keeps the highest bit free for OVERFLOW, which is useful
internally.
- Moving DIR to the current-highest bit makes it easy to add 6 more file
types (7 if you ignore OVERFLOW), before things start getting cramped.
No code changes.
This was referenced Nov 14, 2025
Having on-disk definitions in one place is useful for referencing them later, even if they aren't relevant for most API users. .h files in C are already forced to expose a bunch of internal details anyways, in order to provide struct size/alignment. Might as well include on-disk information that would have even bigger consequences if it changed. Moved: - Compat flag definitions - Tag definitions - DSIZEs and relevant encoding comments - Note some of these were already required to define lfs3_t
This includes the mask/rm/grow bits: - LFS3_tag_RM - LFS3_tag_GROW - LFS3_tag_MASK0/2/8/12 Our in-device only handle types: - LFS3_tag_ORPHAN - LFS3_tag_TRV - LFS3_tag_UNKNOWN And in-device only tags with special behavior: - LFS3_tag_INTERNAL - LFS3_tag_RATTRS - LFS3_tag_SHRUBCOMMIT - LFS3_tag_GRMPUSH - LFS3_tag_MOVE - LFS3_tag_ATTRS Usually I'm not a big fan of case-sensitive naming patterns, but this has been useful for self-documenting what compat flags are in-device only. Might as well extend the idea to our tag definitions.
I can't think of a reason this should be uint8_t. Bumping it up to uint32_t matches the type used for other flags (even though whence is arguably not flags in a strict sense). No code changes.
This more closely matches behavior of functions like mkdir and remove,
even though mkgbmap/rmgbmap operate on a special object and not files.
Besides, returning an error is more useful as users are always free to
ignore said error.
Adds what appears to be one literal to mkgbmap (curiously not rmgbmap?
snuck into alignment?):
code stack ctx
before: 35968 2280 660
after: 35968 (+0.0%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38824 2296 772
gbmap after: 38828 (+0.0%) 2296 (+0.0%) 772 (+0.0%)
TLDR: Replaced lfs3_file_ckmeta/ckdata and lfs3_fs_ckmeta/ckdata with
flag based ck functions:
- lfs3_file_ckmeta -> lfs3_file_ck + LFS3_CK_CKMETA
- lfs3_file_ckdata -> lfs3_file_ck + LFS3_CK_CKDATA
- lfs3_fs_ckmeta -> lfs3_fs_ck + LFS3_FSCK_CKMETA
- lfs3_fs_ckdata -> lfs3_fs_ck + LFS3_FSCK_CKDATA
Note lfs3_fs_ck is equivalent to lfs3_fs_gc, but:
1. Performs the work in one call (equivalent to littlefs2's lfs2_fs_gc)
2. Takes flags at call time (like lfs3_mount) instead of cfg time (like
lfs3_fs_gc)
3. Avoids the constant RAM necessary to track incremental GC state
---
Motivation:
I've been thinking: It's a bit weird that users are able to one-shot
janitorial work in lfs3_mount, but there's no equivalent function after
the filesystem is mounted.
Originally this is what lfs3_fs_gc was for, but after adding support for
incremental GC, it made sense to hide lfs3_fs_gc behind the opt-in
LFS3_GC ifdef due to the extra (ironically non-gc-able) state.
In theory lfs3_trv_t fills a bit of the gap, but, without the internal
i_flag handling and traversal restarts, it's a bit hard to use. And
basically requires duplicating said log, which we need anyways for
lfs3_mount!
So ideally we'd add an explicit one-shot GC function, but now lfs3_fs_gc
is taken.
While thinking about alternative names, I realized we can just call this
lfs3_fs_ck and completely replace lfs3_fs_ckmeta/ckdata.
This has some extra benefits:
- Avoids an explosion of ckmeta/ckdata/repairmeta/repairdata functions
- Discourages redundant traversals that could accomplish more work
- Makes it less confusing that ckdata implies ckmeta
---
I also tweaked lfs3_file_ck to match, but note that lfs3_file_ck is
internally very different from lfs3_fs_ck. For one, lfs3_file_ck only
supports "actual" check flags (LFS3_CK_*) vs all gc flags (LFS3_FSCK_*):
lfs3_file_ck:
LFS3_CK_CKMETA 0x00010000 Check metadata checksums
LFS3_CK_CKDATA 0x00020000 Check metadata + data checksums
LFS3_CK_REPAIRMETA* 0x00040000 Repair metadata blocks
LFS3_CK_REPAIRDATA* 0x00080000 Repair metadata + data blocks
* Planned
lfs3_fs_ck:
LFS3_FSCK_MKCONSISTENT 0x00000800 Make the filesystem consistent
LFS3_FSCK_LOOKAHEAD 0x00001000 Repopulate lookahead buffer
LFS3_FSCK_LOOKGBMAP 0x00002000 Repopulate the gbmap
LFS3_FSCK_PREERASE* 0x00004000 Pre-erase unused blocks
LFS3_FSCK_COMPACTMETA 0x00008000 Compact metadata logs
LFS3_FSCK_CKMETA 0x00010000 Check metadata checksums
LFS3_FSCK_CKDATA 0x00020000 Check metadata + data checksums
LFS3_FSCK_REPAIRMETA* 0x00040000 Repair metadata blocks
LFS3_FSCK_REPAIRDATA* 0x00080000 Repair metadata + data blocks
* Planned
As a plus, this also saves a bit of code:
code stack ctx
before: 35968 2280 660
after: 35924 (-0.1%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38828 2296 772
gbmap after: 38812 (-0.0%) 2296 (+0.0%) 772 (+0.0%)
Unintentionally arriving at the infamous "fsck" name is a bit funny. But it's probably something we don't want to conflict with if we can help it, on the off chance we want a sort of lfs3_fsck function in the future. (This is all hypothetical, but lfs3_fsck may expect an unmounted filesystem, and have a much larger scope than lfs3_fs_ck. Though typing this out now I'm realizing how confusing that might be...) Since lfs3_file_ck and lfs3_fs_ck share a subset of flags, it's not _entirely_ unreasonable for lfs3_file_ck and lfs3_fs_ck to share the same namespace. There's a risk of confusing users around what flags lfs3_file_ck accepts, but we have asserts, and said flags (LFS3_CK_MKCONSISTENT, LFS3_CK_LOOKAHEAD, etc) just don't really make sense in lfs3_file_ck: fs file y LFS3_CK_MKCONSISTENT 0x00000800 Make the filesystem consistent y LFS3_CK_LOOKAHEAD 0x00001000 Repopulate lookahead buffer y LFS3_CK_LOOKGBMAP 0x00002000 Repopulate the gbmap y LFS3_CK_PREERASE* 0x00004000 Pre-erase unused blocks y LFS3_CK_COMPACTMETA 0x00008000 Compact metadata logs y y LFS3_CK_CKMETA 0x00010000 Check metadata checksums y y LFS3_CK_CKDATA 0x00020000 Check metadata + data checksums y y LFS3_CK_REPAIRMETA* 0x00040000 Repair data blocks y y LFS3_CK_REPAIRDATA* 0x00080000 Repair metadata + data blocks * Planned Another option would be to document that lfs3_fs_ck accepts both LFS3_CK_* _and_ LFS3_GC_* flags, but I worry that would be more confusing. It would also lock us into supporting all LFs3_GC_* flags in lfs3_fs_ck, which may not always be the case. Though this is an argument for doing away with the whole LFS3_M/F/CK/GC/I_* duplication... (tbh another reason for this is to reduce the number of namespaces by at least one). No code changes.
Though note most high-level calls (lfs3_file_close, lfs3_dir_close,
etc), still include an assertion at a higher-level.
Why make the internal APIs harder to use than they need to be? As a
plus this drops the need for a separate bool-returning
lfs3_handle_close_.
Saves a bit of code:
code stack ctx
before: 35924 2280 660
after: 35912 (-0.0%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38812 2296 772
gbmap after: 38800 (-0.0%) 2296 (+0.0%) 772 (+0.0%)
This was motivated by a discussion with a gh user, in which it was noted
that not having a reserved suptype for internal tags risks potential
issues with long-term future tag compatibility.
I think the risk is low, but, without a reserved suptype, it _is_
possible for a future tag to conflict with an internal tag in an older
driver version, potentially and unintentionally breaking compatibility.
Note this is especially concerning during mdir compactions, where we
copy tags we may not understand otherwise.
In littlefs2 we reserved suptype=0x100, though this was mostly an
accident due to saturating the 3-bit suptype space. With the larger tag
space in littlefs3, the reserved suptype=0x100 was dropped.
---
Long story short, this reserves suptype=0 for internal flags (well, and
null, which is _mostly_ internal only, but does get written to disk as
unreachable tags).
Unfortunately, adding a new suptype _did_ require moving a bunch of
stuff around:
LFS3_TAG_NULL 0x0000 v--- ---- +--- ----
LFS3_TAG_INTERNAL 0x00tt v--- ---- +ttt tttt
LFS3_TAG_CONFIG 0x01tt v--- ---1 +ttt tttt
LFS3_TAG_MAGIC 0x0131 v--- ---1 +-11 --rr
LFS3_TAG_VERSION 0x0134 v--- ---1 +-11 -1--
LFS3_TAG_RCOMPAT 0x0135 v--- ---1 +-11 -1-1
LFS3_TAG_WCOMPAT 0x0136 v--- ---1 +-11 -11-
LFS3_TAG_OCOMPAT 0x0137 v--- ---1 +-11 -111
LFS3_TAG_GEOMETRY 0x0138 v--- ---1 +-11 1---
LFS3_TAG_NAMELIMIT 0x0139 v--- ---1 +-11 1--1
LFS3_TAG_FILELIMIT 0x013a v--- ---1 +-11 1-1-
LFS3_TAG_ATTRLIMIT? 0x013b v--- ---1 +-11 1-11
LFS3_TAG_GDELTA 0x02tt v--- --1- +ttt tttt
LFS3_TAG_GRMDELTA 0x0230 v--- --1- +-11 ----
LFS3_TAG_GBMAPDELTA 0x0234 v--- --1- +-11 -1rr
LFS3_TAG_GDDTREEDELTA* 0x0238 v--- --1- +-11 1-rr
LFS3_TAG_GPTREEDELTA* 0x023c v--- --1- +-11 11rr
LFS3_TAG_NAME 0x03tt v--- --11 +ttt tttt
LFS3_TAG_BNAME 0x0300 v--- --11 +--- ----
LFS3_TAG_REG 0x0301 v--- --11 +--- ---1
LFS3_TAG_DIR 0x0302 v--- --11 +--- --1-
LFS3_TAG_STICKYNOTE 0x0303 v--- --11 +--- --11
LFS3_TAG_BOOKMARK 0x0304 v--- --11 +--- -1--
LFS3_TAG_SYMLINK? 0x0305 v--- --11 +--- -1-1
LFS3_TAG_SNAPSHOT? 0x0306 v--- --11 +--- -11-
LFS3_TAG_MNAME 0x0330 v--- --11 +-11 ----
LFS3_TAG_DDNAME* 0x0350 v--- --11 +1-1 ----
LFS3_TAG_DDTOMB* 0x0351 v--- --11 +1-1 ---1
LFS3_TAG_STRUCT 0x04tt v--- -1-- +ttt tttt
LFS3_TAG_BRANCH 0x040r v--- -1-- +--- --rr
LFS3_TAG_DATA 0x0404 v--- -1-- +--- -1--
LFS3_TAG_BLOCK 0x0408 v--- -1-- +--- 1err
LFS3_TAG_DDKEY* 0x0410 v--- -1-- +--1 ----
LFS3_TAG_DID 0x0420 v--- -1-- +-1- ----
LFS3_TAG_BSHRUB 0x0428 v--- -1-- +-1- 1---
LFS3_TAG_BTREE 0x042c v--- -1-- +-1- 11rr
LFS3_TAG_MROOT 0x0431 v--- -1-- +-11 --rr
LFS3_TAG_MDIR 0x0435 v--- -1-- +-11 -1rr
LFS3_TAG_MSHRUB+ 0x0438 v--- -1-- +-11 1---
LFS3_TAG_MTREE 0x043c v--- -1-- +-11 11rr
LFS3_TAG_BMRANGE 0x044u v--- -1-- +1-- ++uu
LFS3_TAG_BMFREE 0x0440 v--- -1-- +1-- ----
LFS3_TAG_BMINUSE 0x0441 v--- -1-- +1-- ---1
LFS3_TAG_BMERASED 0x0442 v--- -1-- +1-- --1-
LFS3_TAG_BMBAD 0x0443 v--- -1-- +1-- --11
LFS3_TAG_DDRC* 0x0450 v--- -1-- +1-1 ----
LFS3_TAG_DDPCOEFF* 0x0451 v--- -1-- +1-1 ---1
LFs3_TAG_PCOEFFMAP* 0x0460 v--- -1-- +11- ----
LFS3_TAG_ATTR 0x06aa v--- -11a +aaa aaaa
LFS3_TAG_UATTR 0x06aa v--- -11- +aaa aaaa
LFS3_TAG_SATTR 0x07aa v--- -111 +aaa aaaa
LFS3_TAG_SHRUB 0x1kkk v--1 kkkk +kkk kkkk
LFS3_TAG_ALT 0x4kkk v1cd kkkk +kkk kkkk
LFS3_TAG_CKSUM 0x300p v-11 ---- ++++ +pqq
LFS3_TAG_NOTE 0x3100 v-11 ---1 ++++ ++++
LFS3_TAG_ECKSUM 0x3200 v-11 --1- ++++ ++++
LFS3_TAG_GCKSUMDELTA 0x3300 v-11 --11 ++++ ++++
* Planned
+ Reserved
? Hypothetical
Some additional notes:
- I was on the fence on keeping the 0x30 prefix on config tags now that
it is not longer needed to differentiate from null, but ultimately
decided to keep it because: 1. it's fun, 2. it decreases the chance
of false positives, 3. it keeps the redund bits readable in hexdumps,
and 4. it reserves some tags < config, which is useful since order
matters.
Instead, I pushed the 0x30 prefix to _more_ tags, mainly gstate.
As a coincidence, meta related tags (MNAME, MROOT, MRTREE) all shifted
to also have the 0x30 prefix, which is a nice bit of unexpected
consistency.
- I also considered reserving the redund bits across the config tags
similarly to what we've done in struct/gstate tags, but decided
against it as 1. it significantly reduces the config tag space
available, and 2. makes alignment with VERSION + R/W/OCOMPAT a bit
awkward.
Instead I think would should relax the redund bit alignment in other
suptypes, though in practice the intermixing of non-redund and redund
tags makes this a bit difficult.
Maybe we should consider including redund bits as a hint for things
like DATA? DDKEY? BSHRUB? etc?
- I created a bit more space for file btree struct tags, allowing for
both the future planned DDKEY, and BLOCK with optional erased-bit. We
don't currently use this, but it may be useful for the future planned
gddtree, which in-theory can track erased-state in partially written
file blocks.
Currently tracking erased-state in file blocks is difficult due to
the potential of multiple references, and inability to prevent ecksum
conflicts in raw data blocks.
- UATTR/SATTR bumped up to 0x600/0x700 to keep the 1-bit alignment,
leaving the suptype 0x500 unused. Though this may be useful if we ever
run out of struct tags (suptype=0x400), which is likely where most new
tags will go.
---
Code changes were minimal, but with a bunch of noise:
code stack ctx
before: 35912 2280 660
after: 35920 (+0.0%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38800 2296 772
gbmap after: 38812 (+0.0%) 2296 (+0.0%) 772 (+0.0%)
Well, kinda. At the moment we don't have any reund support (it's a TODO), so arguably redund=0 and this is just a comment tweak. Though our mdirs _are_ already redund=1... so maybe these should actually set redund=1? It's unclear, so for now I've just tweaked the comment, and we should probably revisit when _actually_ implementing meta/data redundancy. --- Note this only really affects struct tags: LFS3_TAG_STRUCT 0x04tt v--- -1-- +ttt tttt LFS3_TAG_BRANCH 0x040r v--- -1-- +--- --rr LFS3_TAG_DATA 0x0404 v--- -1-- +--- -1rr LFS3_TAG_BLOCK 0x0408 v--- -1-- +--- 1err LFS3_TAG_DDKEY* 0x0410 v--- -1-- +--1 --rr LFS3_TAG_DID 0x0420 v--- -1-- +-1- ---- LFS3_TAG_BSHRUB 0x0428 v--- -1-- +-1- 1-rr LFS3_TAG_BTREE 0x042c v--- -1-- +-1- 11rr LFS3_TAG_MROOT 0x0431 v--- -1-- +-11 --rr LFS3_TAG_MDIR 0x0435 v--- -1-- +-11 -1rr LFS3_TAG_MSHRUB+ 0x0438 v--- -1-- +-11 1-rr LFS3_TAG_MTREE 0x043c v--- -1-- +-11 11rr LFS3_TAG_BMRANGE 0x044u v--- -1-- +1-- ++uu LFS3_TAG_BMFREE 0x0440 v--- -1-- +1-- ---- LFS3_TAG_BMINUSE 0x0441 v--- -1-- +1-- ---1 LFS3_TAG_BMERASED 0x0442 v--- -1-- +1-- --1- LFS3_TAG_BMBAD 0x0443 v--- -1-- +1-- --11 LFS3_TAG_DDRC* 0x0450 v--- -1-- +1-1 ---- LFS3_TAG_DDPCOEFF* 0x0451 v--- -1-- +1-1 ---1 LFs3_TAG_PCOEFFMAP* 0x0460 v--- -1-- +11- ---- This redund hint may be useful for debugging and the theoretical CKMETAREDUND feature.
This changes -w/--wait to sleep _after_ -k/--keep-open, instead of including the time spent waiting on inotifywait in the sleep time. 1. It's easier, no need to keep track of when we started waiting. 2. It's simpler to reason about. 3. It trivially avoids the multiple wakeup noise that plagued watch.py + vim (vim likes to do a bunch of renaming and stuff when saving files, including the file 4913 randomly?) Avoiding this was previously impossible because -~/--sleep was effectively a noop when combined with -k/--keep-open. --- Also renamed from -~/--sleep -> -w/--wait, which is a bit more intuitive and avoids possible shell issues with -~. To make this work, dropped the -w/--block-cycles shortform flag in dbgtrace.py. It's not like this flag is ever used anyways. Though at the moment this is ignoring the possible conflict with -w/--word-bits...
I'm trying to avoid the inevitable conflict with -w/--word, which will probably become important when exploring non-32-bit filesystem configurations. Renaming this to -t/--wait still conflicts with -t/--tree and -t/--tiny, but as a debug-only flag, I think these are less important. Oh, and -t/--trace, but test.py/bench.py are already quite different in their flag naming (see -d/--disk vs -d/--diff). --- Renamed a few other flags while tweaking things: - -t/--tiny -> --tiny (dropped shortform) - -w/--word-bits -> -w/--word/--word-bits - -t/--tree -> -R/--tree/--rbyd/--tree-rbyd - -R/--tree-rbyd -> -Y/--rbyd-all/--tree-rbyd-all - -B/--tree-btree -> -B/--btree/--tree-btree After tinkering with it a bit, I think the -R/-Y/-B set of flags are a decent way to organize the tree renderers. At least --tree-rbyd-all does a better job of describing the difference between --tree-rbyd and --tree-rbyd-all.
Test/bench filters have proven to be mostly non-optional, protecting against bad configuration that doesn't make any sense. It's still valid to want to override test filters sometimes, but using a more, uh, forceful verb probably makes sense here. The shortform would conflict with -f/--fail, so no shortform flag for this, but some argue --force should never have a shortform flag anyways.
Just a bit less typing than --o, and lowers risk of conflicts with actual flags we may care about. To be honest I was procrastinating because I thought this would be a lot more work! I was prepared to write a hacky secondary parser, but argparse already supports this natively with prefix_chars='-+'. Yay!
This is still a hack to make the seek _enum_ appear somewhat readable in our dbg _flags_ script. But the previously internal SEEK_MODE was causing all seek flags to be hidden from -l/--list confusingly.
The idea is this is similar to ~ for the home directory, hopefully simplifying littlefs path parsing in scripts: - /hi -> hi in root dir - ./hi -> hi in current dir - ../hi -> hi in parent dir - ~/hi -> hi in home dir - %/hi -> hi in littlefs root dir Note this only works when standalone: - ~hi != ~/hi - %hi != %/hi And files named % can still be referenced with a ./ prefix: - ./% -> % in current dir - %/% -> % in littlefs root dir --- This is probably overkill for dbglfs3.py, as the arg ordering was already enough to disambiguate disk path vs mroot address vs littlefs path, but eventually I think the idea will be useful for more powerful scripts. A hypothetical: $ mklfs3 cp disk -b4096 -r image_files %/image_files
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note: v3-alpha discussion (#1114)
Unfortunately GitHub made a complete mess of the PR discussion. To try to salvage things, please use #1114 for new comments. Feedback/criticism are welcome and immensely important at this stage.
Table of contents ^
Hello! ^
Hello everyone! As some of you may have already picked up on, there's been a large body of work fermenting in the background for the past couple of years. Originally started as an experiment to try to solve littlefs's$O(n^2)$ metadata compaction, this branch eventually snowballed into more-or-less a full rewrite of the filesystem from the ground up.
There's still several chunks of planned work left, but now that this branch has reached on-disk feature parity with v2, there's nothing really stopping it from being merged eventually.
So I figured it's a good time to start calling this v3, and put together a public roadmap.
NOTE: THIS WORK IS INCOMPLETE AND UNSTABLE
Here's a quick TODO list of planned work before stabilization. More details below:
This work may continue to break the on-disk format.
That being said, I highly encourage others to experiment with v3 where possible. Feedback is welcome, and immensely important at this stage. Once it's stabilized, it's stabilized.
To help with this, the current branch uses a v0.0 as its on-disk version to indicate that it is experimental. When it is eventually released, v3 will reject this version and fail to mount.
Unfortunately, the API will be under heavy flux during this period.
A note on benchmarking: The on-disk block-map is key for scalable allocator performance, so benchmarks at this stage needs to be taken with a grain of salt when many blocks are involved. Please refer to this version as "v3 (no bmap)" or something similar in any published benchmarks until this work is completed.
Wait, a disk breaking change? ^
Yes. v3 breaks disk compatibility from v2.
I think this is a necessary evil. Attempting to maintain backwards compatibility has a heavy cost:
Development time - The littlefs team is ~1 guy, and v3 has already taken ~2.5 years. The extra work to make everything compatible would stretch this out much longer and likely be unsustainable.
Code cost - The goal of littlefs is to be, well, little. This is unfortunately in conflict with backwards compatibility.
Take the new B-tree data-structure, for example. It would be easy to support both B-tree and CTZ skip-list files, but now you need ~2x the code. This cost gets worse for the more enmeshed features, and potentially exceeds the cost of just including both v3 and v2 in the codebase.
So I think it's best for both littlefs as a project and long-term users to break things here.
Note v2 isn't going anywhere! I'm happy to continue maintaining the v2 branch, merge bug fixes when necessary, etc. But the economic reality is my focus will be shifting to v3.
What's new ^
Ok, with that out of the way, what does breaking everything actually get us?
Implemented: ^
Efficient metadata compaction:$O(n^2) \rightarrow O(n \log n)$ ^
v3 adopts a new metadata data-structure: Red-black-yellow Dhara trees (rbyds). Based on the data-structure invented by Daniel Beer for the Dhara FTL, rbyds extend log-encoded Dhara trees with self-balancing and self-counting (also called order-statistic) properties.
This speeds up most metadata operations, including metadata lookup ($O(n) \rightarrow O(\log n)$ ), and, critically, metadata compaction ( $O(n^2) \rightarrow O(n \log n)$ ).
This improvement may sound minor on paper, but it's a difference measured in seconds, sometimes even minutes, on devices with extremely large blocks.
Efficient random writes:$O(n) \rightarrow O(\log_b^2 n)$ ^
A much requested feature, v3 adopts B-trees, replacing the CTZ skip-list that previously backed files.
This avoids needing to rewrite the entire file on random reads, bringing the performance back down into tractability.
For extra cool points, littlefs's B-trees use rbyds for the inner nodes, which makes CoW updates much cheaper than traditional array-packed B-tree nodes when large blocks are involved ($O(n) \rightarrow O(\log n)$ ).
Better logging: No more sync-padding issues ^
v3's B-trees support inlining data directly in the B-tree nodes. This gives us a place to store data during sync, without needing to pad things for prog alignment.
In v2 this padding would force the rewriting of blocks after sync, which had a tendency to wreck logging performance.
Efficient inline files, no more RAM constraints:$O(n^2) \rightarrow O(n \log n)$ ^
In v3, B-trees can have their root inlined in the file's mdir, giving us what I've been calling a "B-shrub". This, combined with the above inlined leaves, gives us a much more efficient inlined file representation, with better code reuse to boot.
Oh, and B-shrubs also make small B-trees more efficient by avoiding the extra block needed for the root.
Independent file caches ^
littlefs's
pcache,rcache, and file caches can be configured independently now. This should allow for better RAM utilization when tuning the filesystem.Easier logging APIs:
lfs3_file_fruncate^Thanks to the new self-counting/order-statistic properties, littlefs can now truncate from both the end and front of files via the new
lfs3_file_fruncateAPI.Before, the best option for logging was renaming log files when they filled up. Now, maintaining a log/FIFO is as easy as:
Sparse files ^
Another advantage of adopting B-trees, littlefs can now cheaply represent file holes, where contiguous runs of zeros can be implied without actually taking up any disk space.
Currently this is limited to a couple operations:
lfs3_file_truncatelfs3_file_fruncatelfs3_file_seek+lfs3_file_writepast the end of the fileBut more advanced hole operations may be added in the future.
Efficient file name lookup:$O(n) \rightarrow O(\log_b n)$ ^
littlefs now uses a B-tree (yay code reuse) to organize files by file name. This allows for much faster file name lookup than the previous linked-list of metadata blocks.
A simpler/more robust metadata tree ^
As a part of adopting B-trees for metadata, the previous threaded file tree has been completely ripped out and replaced with one big metadata tree: the M-tree.
I'm not sure how much users are aware of it, but the previous threaded file tree was a real pain-in-the-ass with the amount of bugs it caused. Turns out having a fully-connected graph in a CoBW filesystem is a really bad idea.
In addition to removing an entire category of possible bugs, adopting the M-tree allows for multiple directories in a single metadata block, removing the 1-dir = 1-block minimum requirement.
A well-defined sync model ^
One interesting thing about littlefs, it doesn't have a strictly POSIX API. This puts us in a relatively unique position, where we can explore tweaks to the POSIX API that may make it easer to write powerloss-safe applications.
To leverage this (and because the previous sync model had some real problems), v3 includes a new, well-defined sync model.
I think this discussion captures most of the idea, but for a high-level overview:
Open file handles are strictly snapshots of the on-disk state. Writes to a file are copy-on-write (CoW), with no immediate affect to the on-disk state or any other file handles.
Syncing or closing an in-sync file atomically updates the on-disk state and any other in-sync file handles.
Files can be desynced, either explicitly via
lfs3_file_desync, or because of an error. Desynced files do not recieve sync broadcasts, and closing a desynced file has no affect on the on-disk state.Calling
lfs3_file_syncon a desynced file will atomically update the on-disk state, any other in-sync file handles, and mark the file as in-sync again.Calling
lfs3_file_resyncon a file will discard its current contents and mark the file as in-sync. This is equivalent toclosing and reopening the file.
Stickynotes, no more 0-sized files ^
As an extension of the littlefs's new sync model, v3 introduces a new file type:
LFS3_TYPE_STICKYNOTE.A stickynote represents a file that's in the awkward state of having been created, but not yet synced. If you lose power, stickynotes are hidden from the user and automatically cleaned up on the next mount.
This avoids the 0-sized file issue, while still allowing most of the POSIX interactions users expect.
A new and improved compat flag system ^
v2.1 was a bit of a mess, but it was a learning experience. v3 still includes a global version field, but also includes a set of compat flags that allow non-linear addition/removal of future features.
These are probably familiar to users of Linux filesystems, though I've given them slightly different names:
rcompat flags- Must understand to read the filesystem (incompat_flags)wcompat flags- Must understand to write to the filesystem (ro_compat_flags)ocompat flags- No understanding necessary (compat_flags)This also provides an easy route for marking a filesystem as read-only, non-standard, etc, on-disk.
Error detection! - Global-checksums ^
v3 now supports filesystem-wide error-detection. This is actually quite tricky in a CoBW filesystem, and required the invention of global-checksums (gcksums) to prevent rollback issues caused by naive checksumming.
With gcksums, and a traditional Merkle-tree-esque B-tree construction, v3 now provides a filesystem-wide self-validating checksum via
lfs3_fs_cksum. This checksum can be stored external to the filesystem to provide protection against last-commit rollback issues, metastability, or just for that extra peace of mind.Funny thing about checksums. It's incredibly cheap to calculate checksums when writing, as we're already processing that data anyways. The hard part is, when do you check the checksums?
This is a problem that mostly ends up on the user, but to help, v3 adds a large number checksum checking APIs (probably too many if I'm honest):
LFS3_M_CKMETA/CKDATA- Check checksums during mountLFS3_O_CKMETA/CKDATA- Check checksums during file openlfs3_fs_ckmeta/ckdata- Explicitly check all checksums in the filesystemlfs3_file_ckmeta/ckdata- Explicitly check a file's checksumsLFS3_T_CKMETA/CKDATA- Check checksums incrementally during a traversalLFS3_GC_CKMETA/CKDATA- Check checksums during GC operationsLFS3_M_CKPROGS- Closed checking of data during progsLFS3_M_CKFETCHES- Optimistic (not closed) checking of data during fetchesLFS3_M_CKREADS(planned) - Closed checking of data during readsBetter traversal APIs ^
The traversal API has been completely reworked to be easier to use (both externally and internally).
No more callback needed, blocks can now be iterated over via the dir-like
lfs3_trv_readfunction.Traversals can also perform janitorial work and check checksums now, based on the flags provided to
lfs3_trv_open.Incremental GC ^
GC work can now be accomplished incrementally, instead of requiring one big go. This is managed by
lfs3_fs_gc,cfg.gc_flags, andcfg.gc_steps.Internally, this just shoves one of the new traversal objects into
lfs3_t. It's equivalent to managing a traversal object yourself, but hopefully makes it easier to write library code.However, this does add a significant chunk of RAM to
lfs3_t, so GC is now an opt-in feature behind theLFS3_GCifdef.Better recovery from runtime errors ^
Since we're already doing a full rewrite, I figured let's actually take the time to make sure things don't break on exceptional errors.
Most in-RAM filesystem state should now revert to the last known-good state on error.
The one exception involves file data (not metadata!). Reverting file data correctly turned out to roughly double the cost of files. And now that you can manual revert with
lfs3_file_resync, I figured this cost just isn't worth it. So file data remains undefined after an error.In total, these changes add a significant amount of code and stack, but I'm of the opinion this is necessary for the maturing of littlefs as a filesystem.
Standard custom attributes ^
Breaking disk gives us a chance to reserve attributes
0x80-0xbffor future standard custom attributes:0x00-0x7f- Free for user-attributes (uattr)0x80-0xbf- Reserved for standard-attributes (sattr)0xc0-0xff- Encouraged for system-attributes (yattr)In theory, it was technically possible to reserve these attributes without a disk-breaking change, but it's much safer to do so while we're already breaking the disk.
v3 also includes the possibility of extending the custom attribute space from 8-bits to ~25-bits in the future, but I'd hesitate to to use this, as it risks a significant increase in stack usage.
More tests! ^
v3 comes with a couple more tests than v2 (+~6812.2%):
You may or may not have seen the test framework rework that went curiously under-utilized. That was actually in preparation for the v3 work.
The goal is not 100% line/branch coverage, but just to have more confidence in littlefs's reliability.
Simple key-value APIs ^
v3 includes a couple easy-to-use key-value APIs:
lfs3_get- Get the contents of a filelfs3_size- Get the size of a filelfs3_set- Set the contents of a filelfs3_remove- Remove a file (this one already exists)This API is limited to files that fit in RAM, but if it fits your use case, you can disable the full file API with
LFS3_KVONLYto save some code.If your filesystem fits in only 2 blocks, you can also define
LFS3_2BONLYto save more code.These can be useful for creating small key-value stores on systems that already use littlefs for other storage.
Planned: ^
Efficient block allocation, via optional on-disk block-map (bmap) ^
The one remaining bottleneck in v3 is block allocation. This is a tricky problem for littlefs (and any CoW/CoBW filesystem), because we don't actually know when a block becomes free.
This is in-progress work, but the solution I'm currently looking involves 1. adding an optional on-disk block map (bmap) stored in gstate, and 2. updating it via tree diffing on sync. In theory this will drop huge file writes:$O(n^2 \log n) \rightarrow O(n \log_b^2 n)$
There is also the option of using the bmap as a simple cache, which doesn't avoid the filesystem-wide scan but at least eliminates the RAM constraint of the lookahead buffer.
As a plus, we should be able to leverage the self-counting property of B-trees to make the on-disk bmap compressible.
Bad block tracking ^
This is a much requested feature, and adding the optional on-disk bmap finally gives us a place to track bad blocks.
Pre-erased block tracking ^
Just like bad-blocks, the optional on-disk bmap gives us a place to track pre-erased blocks. Well, at least in theory.
In practice it's a bit more of a nightmare. To avoid multiple progs, we need to mark erased blocks as unerased before progging. This introduces an unbounded number of catch-22s when trying to update the bmap itself.
Fortunately, if instead we store a simple counter in the bmap's gstate, we can resolve things at the mrootanchor worst case.
Error correction! - Metadata redundancy ^
Note it's already possible to do error-correction at the block-device level outside of littlefs, see ramcrc32cbd and ramrsbd for examples. Because of this, integrating in-block error correction is low priority.
But I think there's potential for cross-block error-correction in addition to the in-block error-correction.
The plan for cross-block error-correction/block redundancy is a bit different for metadata vs data. In littlefs, all metadata is logs, which is a bit of a problem for parity schemes. I think the best we can do is store metadata redundancy as naive copies.
But we already need two blocks for every mdir, one usually just sits unused when not compacting. This, combined with metadata usually being much smaller than data, makes the naive scheme less costly than one might expect.
Error correction! - Data redundancy ^
For raw data blocks, we can be a bit more clever. If we add an optional dedup tree for block -> parity group mapping, and an optional parity tree for parity blocks, we can implement a RAID-esque parity scheme for up to 3 blocks of data redundancy relatively cheaply.
Transparent block deduplication ^
This one is a bit funny. Originally block deduplication was intentionally out-of-scope, but it turns out you need something that looks a lot like a dedup tree for error-correction to work in a system that allows multiple block references.
If we already need a virtual -> physical block mapping for error correction, why not make the key the block checksum and get block deduplication for free?
Though if this turns out to not be as free as I think it is, block deduplication will fall out-of-scope.
Stretch goals: ^
These may or may not be included in v3, depending on time and funding:
lfs3_migratefor v2->v3 migration ^16-bit and 64-bit variants ^
Config API rework ^
Block device API rework ^
Custom attr API rework ^
Alternative (cheaper) write-strategies (write-once, global-aligned, eager-crystallization) ^
Advanced file tree operations (
lfs3_file_punchhole,lfs3_file_insertrange,lfs3_file_collapserange,LFS3_SEEK_DATA,LFS3_SEEK_HOLE) ^Advanced file copy-on-write operations (shallow
lfs3_cowcopy+ opportunisticlfs3_copy) ^Reserved blocks to prevent CoW lockups ^
Metadata checks to prevent metadata lockups ^
Integrated block-level ECC (ramcrc32cbd, ramrsbd) ^
Disk-level RAID (this is just data redund + a disk aware block allocator) ^
Out-of-scope (for now): ^
If we don't stop somewhere, v3 will never be released. But these may be added in the future:
Alternative checksums (crc16, crc64, sha256, etc) ^
Feature-limited configurations for smaller code/stack sizes (
LFS3_NO_DIRS,LFS3_KV,LFS3_2BLOCK, etc) ^lfs3_file_openatfor dir-relative APIs ^lfs3_file_opennfor non-null-terminated-string APIs ^Transparent compression ^
Filesystem shrinking ^
High-level caches (block cache, mdir cache, btree leaf cache, etc) ^
Symbolic links ^
100% line/branch coverage ^
Code/stack size ^
littlefs v1, v2, and v3, 1 pixel ~= 1 byte of code, click for a larger interactive codemap (commit)
littlefs v2 and v3 rdonly, 1 pixel ~= 1 byte of code, click for a larger interactive codemap (commit)
Unfortunately, v3 is a little less little than v2:
On one hand, yes, more features generally means more code.
And it's true there's an opportunity here to carve out more feature-limited builds to save code/stack in the future.
But I think it's worth discussing some of the other reasons for the code/stack increase:
Runtime error recovery ^
Recovering from runtime errors isn't cheap. We need to track both the before and after state of things during fallible operations, and this adds both stack and code.
But I think this is necessary for the maturing of littlefs as a filesystem.
Maybe it will make sense to add a sort of
LFS3_GLASSmode in the future, but this is out-of-scope for now.B-tree flexibility ^
The bad news: The new B-tree files are extremely flexible. Unfortunately, this is a double-edged sword.
B-trees, on their own, don't add that much code. They are a relatively poetic data-structure. But deciding how to write to a B-tree, efficiently, with an unknown write pattern, is surprisingly tricky.
The current implementation, what I've taken to calling the "lazy-crystallization algorithm", leans on the more complicated side to see what is possible performance-wise.
The good news: The new B-tree files are extremely flexible.
There's no reason you need the full crystallization algorithm if you have a simple write pattern, or don't care as much about performance. This will either be a future or stretch goal, but it would be interesting to explore alternative write-strategies that could save code in these cases.
Traversal inversion ^
Inverting the traversal, i.e. moving from a callback to incremental state machine, adds both code and stack as 1. all of the previous on-stack state needs to be tracked explicitly, and 2. we now need to worry about what happens if the filesystem is modified mid-traversal.
In theory, this could be reverted if you don't need incremental traversals, but extricating incremental traversals from the current codebase would be an absolute nightmare, so this is out-of-scope for now.
Benchmarks ^
A note on benchmarking: The on-disk block-map is key for scalable allocator performance, so benchmarks at this stage needs to be taken with a grain of salt when many blocks are involved. Please refer to this version as "v3 (no bmap)" or something similar in any published benchmarks until this work is completed.
First off, I would highly encourage others to do their own benchmarking with v3/v2. Filesystem performance is tricky to measure because it depends heavily on your application's write pattern and hardware nuances. If you do, please share in this thread! Others may find the results useful, and now is the critical time for finding potential disk-related performance issues.
Simulated benchmarks ^
To test the math behind v3, I've put together some preliminary simulated benchmarks.
Note these are simulated and optimistic. They do not take caching or hardware buffers into account, which can have a big impact on performance. Still, I think they provide at least a good first impression of v3 vs v2.
To find an estimate of runtime, I first measured the amount of bytes read, progged, and erased, and then scaled based on values found in relevant datasheets. The options here were a bit limited, but WinBond fortunately provides runtime estimates in the datasheets on their website:
NOR flash - w25q64jv
NAND flash - w25n01gv
SD/eMMC - Also w25n01gv, assuming a perfect FTL
I said optimistic, didn't I? I could't find useful estimates for SD/eMMC, so I'm just assuming a perfect FTL here.
These also assume an optimal bus configuration, which, as any embedded engineer knows, is often not the case.
Full benchmarks here: https://benchmarks.littlefs.org (repo, commit)
And here are the ones I think are the most interesting:
Note that SD/eMMC is heavily penalized by the lack of on-disk block-map! SD/eMMC breaks flash down into many small blocks, which tends to make block allocator performance dominate.
Linear writes, where we write a 1 MiB file and don't call sync until closing the file. ^
This one is the most frustrating to compare against v2. CTZ skip-lists are really fast at appending! The problem is they are only fast at appending:
Random writes, note we start with a 1MiB file. ^
As expected, v2 is comically bad at random writes. v3 is indistinguishable from zero in the NOR case:
Logging, write 4 MiB, but limit the file to 1 MiB. ^
In v2 this is accomplished by renaming the file, in v3 we can leverage
lfs3_file_fruncate.v3 performs significantly better with large blocks thanks to avoiding the sync-padding problem:
Funding ^
If you think this work is worthwhile, consider sponsoring littlefs. Current benefits include:
I joke, but I truly appreciate those who have contributed to littlefs so far. littlefs, in its current form, is a mostly self-funded project, so every little bit helps.
If you would like to contribute in a different way, or have other requests, feel free to reach me at geky at geky.net.
As stabilization gets closer, I will also be open to contract work to help port/integrate/adopt v3. If this is interesting to anyone, let me know.
Thank you @micropython, @fusedFET for sponsoring littlefs, and thank you @Eclo, @kmetabg, and @nedap for your past sponsorships!
Next steps ^
For me, I think it's time to finally put together a website/wiki/discussions/blog. I'm not sure on the frequency quite yet, but I plan to write/publish the new DESIGN.md in chapters in tandem with the remaining work.
EDIT: Pinned codemap/plot links to specific commits via benchmarks.littlefs.org/tree.html
EDIT: Updated with rdonly code/stack sizes
EDIT: Added link to #1114
EDIT: Implemented simple key-value APIs
EDIT: Added lfs3_migrate stretch goal with link to #1120
EDIT: Adopted lfs3_traversal_t -> lfs3_trv_t rename
EDIT: Added link to #1125 to clarify "feature parity"