This effectively reverts 1f824a0:
- LFS3_T_COMPACTMETA -> LFS3_T_COMPACT
- gc_compactmeta_thresh -> gc_compact_thresh
And friends.
After using LFS3_T_COMPACTMETA for a bit, I think it just adds noise
without much value. Especially when next to LFS3_T_LOOKAHEAD,
LFS3_GC_PREERASE, LFS3_M_SYNC, etc.
It's interesting that we already have some very distinct verbs for this
sort of thing based on data type (compact => metadata, garbage-collect
=> disk, compress => data).
dbgflags.py now uses +flags to indicate flag namespaces, but our gdb
script only forwarded -f/--flags, which made dbgflags a bit of a pain
to use in the debugger!
Fortunately an easy fix.
Now this works:
(gdb) dbgflags +t trv.gc.t.h.flags
LFS3_T_RDWR 0x00000000 Open traversal as read and write
LFS3_T_MKCONSISTENT 0x00000100 Make the filesystem consistent
LFS3_T_LOOKAHEAD 0x00000200 Repopulate lookahead buffer
LFS3_t_TRAVERSAL 0x60000000 Type = traversal
LFS3_t_MDIR 0x00010000 Btype = mdir
Our flag space is already really packed, and I'm not sure having these
as separate flags is meaningful or useful for users. They both indicate
to repopulate allocators, and most users probably won't care that there
are two subtly different allocators operating under the hood.
There's an argument that LOOKAHEAD not touching disk is a useful
distinction, but in practice you really only need LOOKAHEAD work when
mounted RDWR.
So, merged the behaviors of LOOKAHEAD + LOOKGBMAP such that
LFS3_*_LOOKAHEAD requests repopulation of all allocators based on
gc_lookahead_thresh and gc_lookgbmap_thresh.
In priority order (some notes below):
1. If max(lookahead, gbmap) < gc_lookahead_thresh => repop lookahead
2. If gbmap < gc_lookgbmap_thresh => repop gbmap
As a plus, this makes it easier to avoid LFS3_IFDEF_GBMAP mess.
---
It's interesting to note LFS3_*_LOOKAHEAD will still repopulate the
lookahead buffer when the gbmap is present, but only if this would gain
more knowledge than was is currently in the gbmap.
I considered disabling lookahead scans completely when we have a gbmap,
but repopulating the lookahead buffer is still useful if the gbmap is at
risk of exhaustion. This is what gc_lookahead_thresh is for anyways, and
users can set gc_lookahead_thresh=0 if they want to disable this
behavior.
Relatedly, lookahead scans are actually prioritized over gbmap scans
(when they would gain knowledge). In theory this minimizes gc latency,
as gbmap scans risk triggering a full lookahead scan when building the
new gbmap.
---
Code changes minimal:
code stack ctx
before: 35152 2136 660
after: 35152 (+0.0%) 2136 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38076 2136 776
gbmap after: 38080 (+0.0%) 2136 (+0.0%) 776 (+0.0%)
The idea is this is similar to ~ for the home directory, hopefully
simplifying littlefs path parsing in scripts:
- /hi -> hi in root dir
- ./hi -> hi in current dir
- ../hi -> hi in parent dir
- ~/hi -> hi in home dir
- %/hi -> hi in littlefs root dir
Note this only works when standalone:
- ~hi != ~/hi
- %hi != %/hi
And files named % can still be referenced with a ./ prefix:
- ./% -> % in current dir
- %/% -> % in littlefs root dir
---
This is probably overkill for dbglfs3.py, as the arg ordering was
already enough to disambiguate disk path vs mroot address vs littlefs
path, but eventually I think the idea will be useful for more powerful
scripts.
A hypothetical:
$ mklfs3 cp disk -b4096 -r image_files %/image_files
This is still a hack to make the seek _enum_ appear somewhat readable in
our dbg _flags_ script. But the previously internal SEEK_MODE was
causing all seek flags to be hidden from -l/--list confusingly.
Just a bit less typing than --o, and lowers risk of conflicts with
actual flags we may care about.
To be honest I was procrastinating because I thought this would be a lot
more work! I was prepared to write a hacky secondary parser, but argparse
already supports this natively with prefix_chars='-+'. Yay!
Test/bench filters have proven to be mostly non-optional, protecting
against bad configuration that doesn't make any sense.
It's still valid to want to override test filters sometimes, but using a
more, uh, forceful verb probably makes sense here.
The shortform would conflict with -f/--fail, so no shortform flag for
this, but some argue --force should never have a shortform flag anyways.
I'm trying to avoid the inevitable conflict with -w/--word, which will
probably become important when exploring non-32-bit filesystem
configurations.
Renaming this to -t/--wait still conflicts with -t/--tree and -t/--tiny,
but as a debug-only flag, I think these are less important.
Oh, and -t/--trace, but test.py/bench.py are already quite different in
their flag naming (see -d/--disk vs -d/--diff).
---
Renamed a few other flags while tweaking things:
- -t/--tiny -> --tiny (dropped shortform)
- -w/--word-bits -> -w/--word/--word-bits
- -t/--tree -> -R/--tree/--rbyd/--tree-rbyd
- -R/--tree-rbyd -> -Y/--rbyd-all/--tree-rbyd-all
- -B/--tree-btree -> -B/--btree/--tree-btree
After tinkering with it a bit, I think the -R/-Y/-B set of flags are a
decent way to organize the tree renderers. At least --tree-rbyd-all does
a better job of describing the difference between --tree-rbyd and
--tree-rbyd-all.
This changes -w/--wait to sleep _after_ -k/--keep-open, instead of
including the time spent waiting on inotifywait in the sleep time.
1. It's easier, no need to keep track of when we started waiting.
2. It's simpler to reason about.
3. It trivially avoids the multiple wakeup noise that plagued
watch.py + vim (vim likes to do a bunch of renaming and stuff when
saving files, including the file 4913 randomly?)
Avoiding this was previously impossible because -~/--sleep was
effectively a noop when combined with -k/--keep-open.
---
Also renamed from -~/--sleep -> -w/--wait, which is a bit more intuitive
and avoids possible shell issues with -~.
To make this work, dropped the -w/--block-cycles shortform flag in
dbgtrace.py. It's not like this flag is ever used anyways.
Though at the moment this is ignoring the possible conflict with
-w/--word-bits...
This was motivated by a discussion with a gh user, in which it was noted
that not having a reserved suptype for internal tags risks potential
issues with long-term future tag compatibility.
I think the risk is low, but, without a reserved suptype, it _is_
possible for a future tag to conflict with an internal tag in an older
driver version, potentially and unintentionally breaking compatibility.
Note this is especially concerning during mdir compactions, where we
copy tags we may not understand otherwise.
In littlefs2 we reserved suptype=0x100, though this was mostly an
accident due to saturating the 3-bit suptype space. With the larger tag
space in littlefs3, the reserved suptype=0x100 was dropped.
---
Long story short, this reserves suptype=0 for internal flags (well, and
null, which is _mostly_ internal only, but does get written to disk as
unreachable tags).
Unfortunately, adding a new suptype _did_ require moving a bunch of
stuff around:
LFS3_TAG_NULL 0x0000 v--- ---- +--- ----
LFS3_TAG_INTERNAL 0x00tt v--- ---- +ttt tttt
LFS3_TAG_CONFIG 0x01tt v--- ---1 +ttt tttt
LFS3_TAG_MAGIC 0x0131 v--- ---1 +-11 --rr
LFS3_TAG_VERSION 0x0134 v--- ---1 +-11 -1--
LFS3_TAG_RCOMPAT 0x0135 v--- ---1 +-11 -1-1
LFS3_TAG_WCOMPAT 0x0136 v--- ---1 +-11 -11-
LFS3_TAG_OCOMPAT 0x0137 v--- ---1 +-11 -111
LFS3_TAG_GEOMETRY 0x0138 v--- ---1 +-11 1---
LFS3_TAG_NAMELIMIT 0x0139 v--- ---1 +-11 1--1
LFS3_TAG_FILELIMIT 0x013a v--- ---1 +-11 1-1-
LFS3_TAG_ATTRLIMIT? 0x013b v--- ---1 +-11 1-11
LFS3_TAG_GDELTA 0x02tt v--- --1- +ttt tttt
LFS3_TAG_GRMDELTA 0x0230 v--- --1- +-11 ----
LFS3_TAG_GBMAPDELTA 0x0234 v--- --1- +-11 -1rr
LFS3_TAG_GDDTREEDELTA* 0x0238 v--- --1- +-11 1-rr
LFS3_TAG_GPTREEDELTA* 0x023c v--- --1- +-11 11rr
LFS3_TAG_NAME 0x03tt v--- --11 +ttt tttt
LFS3_TAG_BNAME 0x0300 v--- --11 +--- ----
LFS3_TAG_REG 0x0301 v--- --11 +--- ---1
LFS3_TAG_DIR 0x0302 v--- --11 +--- --1-
LFS3_TAG_STICKYNOTE 0x0303 v--- --11 +--- --11
LFS3_TAG_BOOKMARK 0x0304 v--- --11 +--- -1--
LFS3_TAG_SYMLINK? 0x0305 v--- --11 +--- -1-1
LFS3_TAG_SNAPSHOT? 0x0306 v--- --11 +--- -11-
LFS3_TAG_MNAME 0x0330 v--- --11 +-11 ----
LFS3_TAG_DDNAME* 0x0350 v--- --11 +1-1 ----
LFS3_TAG_DDTOMB* 0x0351 v--- --11 +1-1 ---1
LFS3_TAG_STRUCT 0x04tt v--- -1-- +ttt tttt
LFS3_TAG_BRANCH 0x040r v--- -1-- +--- --rr
LFS3_TAG_DATA 0x0404 v--- -1-- +--- -1--
LFS3_TAG_BLOCK 0x0408 v--- -1-- +--- 1err
LFS3_TAG_DDKEY* 0x0410 v--- -1-- +--1 ----
LFS3_TAG_DID 0x0420 v--- -1-- +-1- ----
LFS3_TAG_BSHRUB 0x0428 v--- -1-- +-1- 1---
LFS3_TAG_BTREE 0x042c v--- -1-- +-1- 11rr
LFS3_TAG_MROOT 0x0431 v--- -1-- +-11 --rr
LFS3_TAG_MDIR 0x0435 v--- -1-- +-11 -1rr
LFS3_TAG_MSHRUB+ 0x0438 v--- -1-- +-11 1---
LFS3_TAG_MTREE 0x043c v--- -1-- +-11 11rr
LFS3_TAG_BMRANGE 0x044u v--- -1-- +1-- ++uu
LFS3_TAG_BMFREE 0x0440 v--- -1-- +1-- ----
LFS3_TAG_BMINUSE 0x0441 v--- -1-- +1-- ---1
LFS3_TAG_BMERASED 0x0442 v--- -1-- +1-- --1-
LFS3_TAG_BMBAD 0x0443 v--- -1-- +1-- --11
LFS3_TAG_DDRC* 0x0450 v--- -1-- +1-1 ----
LFS3_TAG_DDPCOEFF* 0x0451 v--- -1-- +1-1 ---1
LFs3_TAG_PCOEFFMAP* 0x0460 v--- -1-- +11- ----
LFS3_TAG_ATTR 0x06aa v--- -11a +aaa aaaa
LFS3_TAG_UATTR 0x06aa v--- -11- +aaa aaaa
LFS3_TAG_SATTR 0x07aa v--- -111 +aaa aaaa
LFS3_TAG_SHRUB 0x1kkk v--1 kkkk +kkk kkkk
LFS3_TAG_ALT 0x4kkk v1cd kkkk +kkk kkkk
LFS3_TAG_CKSUM 0x300p v-11 ---- ++++ +pqq
LFS3_TAG_NOTE 0x3100 v-11 ---1 ++++ ++++
LFS3_TAG_ECKSUM 0x3200 v-11 --1- ++++ ++++
LFS3_TAG_GCKSUMDELTA 0x3300 v-11 --11 ++++ ++++
* Planned
+ Reserved
? Hypothetical
Some additional notes:
- I was on the fence on keeping the 0x30 prefix on config tags now that
it is not longer needed to differentiate from null, but ultimately
decided to keep it because: 1. it's fun, 2. it decreases the chance
of false positives, 3. it keeps the redund bits readable in hexdumps,
and 4. it reserves some tags < config, which is useful since order
matters.
Instead, I pushed the 0x30 prefix to _more_ tags, mainly gstate.
As a coincidence, meta related tags (MNAME, MROOT, MRTREE) all shifted
to also have the 0x30 prefix, which is a nice bit of unexpected
consistency.
- I also considered reserving the redund bits across the config tags
similarly to what we've done in struct/gstate tags, but decided
against it as 1. it significantly reduces the config tag space
available, and 2. makes alignment with VERSION + R/W/OCOMPAT a bit
awkward.
Instead I think would should relax the redund bit alignment in other
suptypes, though in practice the intermixing of non-redund and redund
tags makes this a bit difficult.
Maybe we should consider including redund bits as a hint for things
like DATA? DDKEY? BSHRUB? etc?
- I created a bit more space for file btree struct tags, allowing for
both the future planned DDKEY, and BLOCK with optional erased-bit. We
don't currently use this, but it may be useful for the future planned
gddtree, which in-theory can track erased-state in partially written
file blocks.
Currently tracking erased-state in file blocks is difficult due to
the potential of multiple references, and inability to prevent ecksum
conflicts in raw data blocks.
- UATTR/SATTR bumped up to 0x600/0x700 to keep the 1-bit alignment,
leaving the suptype 0x500 unused. Though this may be useful if we ever
run out of struct tags (suptype=0x400), which is likely where most new
tags will go.
---
Code changes were minimal, but with a bunch of noise:
code stack ctx
before: 35912 2280 660
after: 35920 (+0.0%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38800 2296 772
gbmap after: 38812 (+0.0%) 2296 (+0.0%) 772 (+0.0%)
Unintentionally arriving at the infamous "fsck" name is a bit funny.
But it's probably something we don't want to conflict with if we can
help it, on the off chance we want a sort of lfs3_fsck function in the
future. (This is all hypothetical, but lfs3_fsck may expect an unmounted
filesystem, and have a much larger scope than lfs3_fs_ck. Though typing
this out now I'm realizing how confusing that might be...)
Since lfs3_file_ck and lfs3_fs_ck share a subset of flags, it's not
_entirely_ unreasonable for lfs3_file_ck and lfs3_fs_ck to share the
same namespace.
There's a risk of confusing users around what flags lfs3_file_ck
accepts, but we have asserts, and said flags (LFS3_CK_MKCONSISTENT,
LFS3_CK_LOOKAHEAD, etc) just don't really make sense in lfs3_file_ck:
fs file
y LFS3_CK_MKCONSISTENT 0x00000800 Make the filesystem consistent
y LFS3_CK_LOOKAHEAD 0x00001000 Repopulate lookahead buffer
y LFS3_CK_LOOKGBMAP 0x00002000 Repopulate the gbmap
y LFS3_CK_PREERASE* 0x00004000 Pre-erase unused blocks
y LFS3_CK_COMPACTMETA 0x00008000 Compact metadata logs
y y LFS3_CK_CKMETA 0x00010000 Check metadata checksums
y y LFS3_CK_CKDATA 0x00020000 Check metadata + data checksums
y y LFS3_CK_REPAIRMETA* 0x00040000 Repair data blocks
y y LFS3_CK_REPAIRDATA* 0x00080000 Repair metadata + data blocks
* Planned
Another option would be to document that lfs3_fs_ck accepts both
LFS3_CK_* _and_ LFS3_GC_* flags, but I worry that would be more
confusing. It would also lock us into supporting all LFs3_GC_* flags in
lfs3_fs_ck, which may not always be the case.
Though this is an argument for doing away with the whole
LFS3_M/F/CK/GC/I_* duplication... (tbh another reason for this is to
reduce the number of namespaces by at least one).
No code changes.
TLDR: Replaced lfs3_file_ckmeta/ckdata and lfs3_fs_ckmeta/ckdata with
flag based ck functions:
- lfs3_file_ckmeta -> lfs3_file_ck + LFS3_CK_CKMETA
- lfs3_file_ckdata -> lfs3_file_ck + LFS3_CK_CKDATA
- lfs3_fs_ckmeta -> lfs3_fs_ck + LFS3_FSCK_CKMETA
- lfs3_fs_ckdata -> lfs3_fs_ck + LFS3_FSCK_CKDATA
Note lfs3_fs_ck is equivalent to lfs3_fs_gc, but:
1. Performs the work in one call (equivalent to littlefs2's lfs2_fs_gc)
2. Takes flags at call time (like lfs3_mount) instead of cfg time (like
lfs3_fs_gc)
3. Avoids the constant RAM necessary to track incremental GC state
---
Motivation:
I've been thinking: It's a bit weird that users are able to one-shot
janitorial work in lfs3_mount, but there's no equivalent function after
the filesystem is mounted.
Originally this is what lfs3_fs_gc was for, but after adding support for
incremental GC, it made sense to hide lfs3_fs_gc behind the opt-in
LFS3_GC ifdef due to the extra (ironically non-gc-able) state.
In theory lfs3_trv_t fills a bit of the gap, but, without the internal
i_flag handling and traversal restarts, it's a bit hard to use. And
basically requires duplicating said log, which we need anyways for
lfs3_mount!
So ideally we'd add an explicit one-shot GC function, but now lfs3_fs_gc
is taken.
While thinking about alternative names, I realized we can just call this
lfs3_fs_ck and completely replace lfs3_fs_ckmeta/ckdata.
This has some extra benefits:
- Avoids an explosion of ckmeta/ckdata/repairmeta/repairdata functions
- Discourages redundant traversals that could accomplish more work
- Makes it less confusing that ckdata implies ckmeta
---
I also tweaked lfs3_file_ck to match, but note that lfs3_file_ck is
internally very different from lfs3_fs_ck. For one, lfs3_file_ck only
supports "actual" check flags (LFS3_CK_*) vs all gc flags (LFS3_FSCK_*):
lfs3_file_ck:
LFS3_CK_CKMETA 0x00010000 Check metadata checksums
LFS3_CK_CKDATA 0x00020000 Check metadata + data checksums
LFS3_CK_REPAIRMETA* 0x00040000 Repair metadata blocks
LFS3_CK_REPAIRDATA* 0x00080000 Repair metadata + data blocks
* Planned
lfs3_fs_ck:
LFS3_FSCK_MKCONSISTENT 0x00000800 Make the filesystem consistent
LFS3_FSCK_LOOKAHEAD 0x00001000 Repopulate lookahead buffer
LFS3_FSCK_LOOKGBMAP 0x00002000 Repopulate the gbmap
LFS3_FSCK_PREERASE* 0x00004000 Pre-erase unused blocks
LFS3_FSCK_COMPACTMETA 0x00008000 Compact metadata logs
LFS3_FSCK_CKMETA 0x00010000 Check metadata checksums
LFS3_FSCK_CKDATA 0x00020000 Check metadata + data checksums
LFS3_FSCK_REPAIRMETA* 0x00040000 Repair metadata blocks
LFS3_FSCK_REPAIRDATA* 0x00080000 Repair metadata + data blocks
* Planned
As a plus, this also saves a bit of code:
code stack ctx
before: 35968 2280 660
after: 35924 (-0.1%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38828 2296 772
gbmap after: 38812 (-0.0%) 2296 (+0.0%) 772 (+0.0%)
Other than moving things around to make space for planned features, this
also adopts the idea of allowing compat flags to be ored into a single
32-bit integer, at least in the short-term.
Note though that these are still stored in separate wcompat/rcompat
tags, to make compat tests easier, and we may introduce conflicting
flags in the future if we run out of 32-bits. This is just an indulgence
to potentially make tooling/debugging easier until that happens.
Rcompat flags:
RCOMPAT_NONSTANDARD+
0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
RCOMPAT_WRONLY+ 0x00000004 ---- ---- ---- ---- ---- ---- ---- -1--
RCOMPAT_MMOSS 0x00000010 ---- ---- ---- ---- ---- ---- ---1 ----
RCOMPAT_MSPROUT+ 0x00000020 ---- ---- ---- ---- ---- ---- --1- ----
RCOMPAT_MSHRUB+ 0x00000040 ---- ---- ---- ---- ---- ---- -1-- ----
RCOMPAT_MTREE 0x00000080 ---- ---- ---- ---- ---- ---- 1--- ----
RCOMPAT_BMOSS+ 0x00000100 ---- ---- ---- ---- ---- ---1 ---- ----
RCOMPAT_BSPROUT+ 0x00000200 ---- ---- ---- ---- ---- --1- ---- ----
RCOMPAT_BSHRUB 0x00000400 ---- ---- ---- ---- ---- -1-- ---- ----
RCOMPAT_BTREE 0x00000800 ---- ---- ---- ---- ---- 1--- ---- ----
RCOMPAT_MDIRR1* 0x00001000 ---- ---- ---- ---- ---1 ---- ---- ----
RCOMPAT_MDIRR2* 0x00002000 ---- ---- ---- ---- --1- ---- ---- ----
RCOMPAT_MDIRR3* 0x00003000 ---- ---- ---- ---- --11 ---- ---- ----
RCOMPAT_BTREER1* 0x00004000 ---- ---- ---- ---- -1-- ---- ---- ----
RCOMPAT_BTREER2* 0x00008000 ---- ---- ---- ---- 1--- ---- ---- ----
RCOMPAT_BTREER3* 0x0000c000 ---- ---- ---- ---- 11-- ---- ---- ----
RCOMPAT_GRM 0x00010000 ---- ---- ---- ---1 ---- ---- ---- ----
RCOMPAT_GMV? 0x00020000 ---- ---- ---- --1- ---- ---- ---- ----
RCOMPAT_GDDTREE* 0x00100000 ---- ---- ---1 ---- ---- ---- ---- ----
RCOMPAT_GPTREE* 0x00200000 ---- ---- --1- ---- ---- ---- ---- ----
RCOMPAT_DATAR1* 0x00400000 ---- ---- -1-- ---- ---- ---- ---- ----
RCOMPAT_DATAR2* 0x00800000 ---- ---- 1--- ---- ---- ---- ---- ----
RCOMPAT_DATAR3* 0x00c00000 ---- ---- 11-- ---- ---- ---- ---- ----
rcompat_OVERFLOW+ 0x80000000 1--- ---- ---- ---- ---- ---- ---- ----
* Planned
+ Reserved
? Hypothetical
Wcompat flags:
WCOMPAT_NONSTANDARD+
0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
WCOMPAT_RDONLY+ 0x00000002 ---- ---- ---- ---- ---- ---- ---- --1-
WCOMPAT_GCKSUM 0x00040000 ---- ---- ---- -1-- ---- ---- ---- ----
WCOMPAT_GBMAP 0x00080000 ---- ---- ---- 1--- ---- ---- ---- ----
WCOMPAT_DIR 0x01000000 ---- ---1 ---- ---- ---- ---- ---- ----
WCOMPAT_SYMLINK? 0x02000000 ---- --1- ---- ---- ---- ---- ---- ----
WCOMPAT_SNAPSHOT? 0x04000000 ---- -1-- ---- ---- ---- ---- ---- ----
wcompat_OVERFLOW+ 0x80000000 1--- ---- ---- ---- ---- ---- ---- ----
+ Reserved
? Hypothetical
Ocompat flags:
OCOMPAT_NONSTANDARD+
0x00000001 ---- ---- ---- ---- ---- ---- ---- ---1
ocompat_OVERFLOW+ 0x80000000 1--- ---- ---- ---- ---- ---- ---- ----
+ Reserved
Other notes:
- M* and B* struct flags were reordered to match META -> DATA order
elsewhere. This no longer matches the tag ordering, but there's an
argument the B* tags apply more generally (all btrees) than the B*
compat flag (only file btrees).
- MDIR/BTREE/DATA redund flags were moved near relevant flags, rather
than sticking them in the higher-order bits as we are planning to do
in the M_*/F_* flags. The compat flags already won't match because of
the mdir/btree split (which is IMO too much detail to include in
M_*/F_* flags, but hard to argue against in the compat flags), and
this keeps the highest bit free for OVERFLOW, which is useful
internally.
- Moving DIR to the current-highest bit makes it easy to add 6 more file
types (7 if you ignore OVERFLOW), before things start getting cramped.
No code changes.
Yeah, after using these for a bit, the RE* names were not great.
Trying LOOK* now, as an alternative that hopefully still implies the
similar behavior without needing an additional prefix for LOOKAHEAD:
- LFS3_*_RELOOKAHEAD -> LFS3_*_LOOKAHEAD
- LFS3_*_REGBMAP -> LFS3_*_LOOKGBMAP
- cfg.regbmap_thresh -> cfg.lookgbmap_thresh
- cfg.gc_relookahead_thresh -> cfg.gc_lookahead_thresh
- cfg.gc_regbmap_thresh -> cfg.gc_lookgbmap_thresh
I mean, why not? These redirect to the same internal lfs3_fs_gc_
function anyways. Might as well keep things consistent.
Added:
LFS3_F_MKCONSISTENT 0x00000800 Make the filesystem consistent
LFS3_F_RELOOKAHEAD 0x00001000 Repopulate lookahead buffer
LFS3_F_MKCONSISTENT is guaranteed to be a noop, but LFS3_F_RELOOKAHEAD
forces a filesystem traversal, which may have some niche use case.
No code changes.
These are unlikely to make much progress, but that doesn't seem like a
great reason to disallow these flags in lfs3_format:
LFS3_F_REGBMAP 0x00002000 Repopulate the gbmap
LFS3_F_COMPACTMETA 0x00008000 Compact metadata logs
These are actually guaranteed to do _no_ work when formatting _without_
the gbmap, but with the gbmap it's less clear. Looking forward to the
planned ckfactory feature, these may be useful for cleaning up any rbyd
commits created as a part of building the initial gbmap.
---
Also tweaked the formatting for LFS3_F_* flags a bit, including making
all ifdefs explicit (mainly ifdef LFS3_RDONLY). Mixed ifdefs are a real
pain to read.
No code changes.
LFS3_REVDBG introduced a lot of overhead for something I'm not sure
anyone will actually use (I have enough tooling that the state of an
rbyd is rarely a mystery, see dbgbmap.py). That, and we're running out
of flags!
So this reduces LFS3_REVDBG to just store one of "himb" in the first
(lowest) byte of the revision count; information that is easily
available:
vvvv---- -------- -------- --------
vvvvrrrr rrrrrr-- -------- --------
vvvvrrrr rrrrrrnn nnnnnnnn nnnnnnnn
vvvvrrrr rrrrrrnn nnnnnnnn dddddddd
'-.''----.----''----.- - - '---.--'
'------|----------|----------|---- 4-bit relocation revision
'----------|----------|---- recycle-bits recycle counter
'----------|---- pseudorandom noise (if revnoise)
'---- h, i, m, or b (if revdbg)
-11-1--- - h = mroot anchor
-11-1--1 - i = mroot
-11-11-1 - m = mdir
-11---1- - b = btree node
Some other notes:
- Enabled LFS3_REVDBG and LFS3_REVNOISE to work together, now that
LFS3_REVDBG doesn't consume all unused rev bits.
Note that LFS3_REVDBG has priority over LFS3_REVNOISE, but _not_
recycle-bits, etc. Otherwise problems would happen for recycle-bits
>2^20 (though do we care?).
- Fixed an issue where using the gcksum as a noise source results in
noise=0 when there is only an mroot. This is due to how we xor out
the current mdir cksum during an mdir commit.
Fixed by using gcksum_p instead of gcksum.
- Added missing LFS3_I_REVDBG/REVNOISE flags in the tests, so now you
can actually run the tests with LFS3_REVDBG/REVNOISE (this probably
just fell out-of-date at some point).
---
Curiously, despite LFS3_REVDBG/REVNOISE being disabled by default, this
did save some code. I'm guessing the non-tail-call mtree/gbmap commit
functions prevented some level of inlining?:
code stack ctx
before: 35964 2280 660
after: 35964 (+0.0%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38940 2296 772
gbmap after: 38828 (-0.3%) 2296 (+0.0%) 772 (+0.0%)
With the relaxation of traversal behavior under mutation, I think it
makes sense to bring back LFS3_T_EXCL. If only to allow traversals to
gaurantee termination under mutation. Now that traversals no longer
guarantee forward progress, it's possible to get stuck looping
indefinitely if the filesystem is constantly being mutated.
Non-excl traversals are probably still useful for GC work and debugging
threads, but LFS3_T_EXCL now allows traversals to terminate immediately
with LFS3_ERR_BUSY at the first sign of unrelated filesystem mutation:
LFS3_T_EXCL 0x00000008 Error if filesystem modified
Internally, we already track unrelated mutation to avoid corrupt state
(LFS3_t_DIRTY), so this is a very low-cost feature:
code stack ctx
before: 35944 2280 660
after: 35964 (+0.1%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38916 2296 772
gbmap after: 38940 (+0.1%) 2296 (+0.0%) 772 (+0.0%)
code stack ctx
gc before: 36016 2280 768
gc after: 36036 (+0.1%) 2280 (+0.0%) 768 (+0.0%)
This solves the previous gc-needs-block-queue-so-we-can-clobber-block-
queue issue by adding an additional LFS3_t_STALE flag to indicate when
any block queues would be invalid.
So instead of clearing block queues in lfs3_alloc_ckpoint, we just set
LFS3_t_STALE, and any lfs3_trv_ts can clear their block queues in
lfs3_trv_read. This allows lfs3_mgc_ts to be allocated without a block
queue when doing any LFS3_M_*/LFS3_F_*/LFS3_GC_* work.
LFS3_t_STALE is set at the same time as LFS3_t_CKPOINT and LFS3_t_DIRTY,
but we need a separate bit so lfs3_trv_read can clear the flag after
flushing without losing ckpoint/dirty information.
---
Unfortunately, none of the stack-allocated lfs3_mgc_ts are on the stack
hot-path, so we don't immediate savings. But note the 2-words saved in
ctx when compiling in LFS3_GC mode:
code stack ctx
before: 35940 2280 660
after: 35944 (+0.0%) 2280 (+0.0%) 660 (+0.0%)
code stack ctx
gbmap before: 38916 2296 772
gbmap after: 38916 (+0.0%) 2296 (+0.0%) 772 (+0.0%)
code stack ctx
gc before: 36012 2280 776
gc after: 36016 (+0.0%) 2280 (+0.0%) 768 (-1.0%)
The main idea here is to drop the flag-encoded tstate state machine, and
replace it with a matrix controlled by special mid + bid values:
-- mid ->
-5 -4 -3 -2 >=-1
bid -2 x x x --> mdir
v >=-1 x gbm gbm x --> bshrub/btree
'----|----|----|----|----> mroot anchor
'----|----|----|----> mroot chain + mtree
'----|----|----> gbmap (in-ram gbmap)
'----|----> gbmap_p (on-disk gbmap)
'----> file bshrubs/btrees
This was motivated by the observation that everything in our filesystem
can be modeled as mdir + bshrub/btree tuples, as long as some states are
noops. And we can cleanly encode these tuples in the unused negative
mid + bid ranges without needing an explicit state machine.
Well, that and the previous tstate state machine approach being an ugly
pile of switch cases and messy logic.
Note though that some mids may need to traverse multiple mdirs/bshrub/
btrees:
- The mroot chain + mtree (mid=-4) needs to traverse all mroots in the
mroot chain, and detect any cycles.
- File mdirs (mid>=-1) need to traverse both the on-disk bshrub/btree
and any opened file handles' bshrubs/btrees before moving onto the
next mid.
This grows O(n^2) because all file handles are in one big unsorted
linked-list, but as usual we don't care.
In addition to the greatly simplified traversal logic, the new state
matrix simplifies traversal clobbering: Setting bid=-2 always forces a
bshrub/btree refetch.
This comes at the cost of traversal _precision_, i.e. we can now revisit
previously visited bshrub/btree nodes. But I think this is well worth it
for more robust traversal clobbering. Traversal clobbering is delicate
and difficult to get right.
Besides, we can already revisit blocks due to CoW references, so what's
the harm in revisiting blocks when under mutation?
---
The simpler traversal logic leads to a nice amount of code savings
across the board:
code stack ctx
before: 36476 2304 660
after: 35940 (-1.5%) 2280 (-1.0%) 660 (+0.0%)
code stack ctx
gbmap before: 39524 2320 772
gbmap after: 38916 (-1.5%) 2296 (-1.0%) 772 (+0.0%)
code stack ctx
gc before: 36548 2304 804
gc after: 36012 (-1.5%) 2280 (-1.0%) 776 (-3.5%)
Note the ctx savings in LFS3_GC mode. Most of the stack/ctx savings
comes from the smaller lfs3_mtrv_t struct, which no longer needs to
stage bshrubs (we no longer care about bshrubs across mdir commit as a
part of the above clobbering simplifications):
before after
lfs3_mtrv_t: 128 100 (-21.9%)
lfs3_mgc_t: 128 100 (-21.9%)
lfs3_trv_t: 136 108 (-20.6%)
Unfortunately, the simpler clobbering means now any gc work needs the
block queue (i.e. lfs3_trv_t), solely so clobbering the block queue
doesn't clobber unallocated memory. Not great but hopefully fixable.
---
Some other notes:
- As a part of simplifying traversal clobbering, everything is triggered
by lfs3_alloc_ckpoint (via lfs3_trv_ckpoint_).
This may clobber traversals more than is strictly necessary, but
that's kinda the idea. Better safe than sorry.
And no more need to explicit lfs3_handle_clobber calls is nice.
- Opened file handle iteration is now tracked by the traversal handle's
position in the handle linked-list, instead of a separate handle
pointer. This means one less thing to disentangle and makes traversals
no longer a special case for things like lfs3_handle_close.
You may think this bumps traversals up to O(n^3) in-ram, but because
we only ever visit each unique handle + mid once, we can keep the
total O(n^2) if we're smart about linked-list updates!
- lfs3_mdir_commit needed to be tweaked to accept mids<=-1, instead of
just mid=-1 for the mroot. Unfortunately I don't know how much this
costs on its own.
- The reorganization of lfs3_mtrv_t means lfs3_mtortoise_t gets its own
struct again!
- No more tstate state machine also frees up a big chunk of the
traversal flag space, which was getting pretty cramped.
- lookupleaf -> lookupnext_
- namelookupleaf -> namelookup_
I want to move away from lookupleaf usage in general in the dbg scripts,
like we have in lfs3.c, but I also just really don't want to touch these
scripts again unless I need to. They've been useful, but also a big time
sink.
Maybe I should actually learn Python's new type system. That would
probably help here...
So:
- cfg.gc_repoplookahead_thresh -> cfg.gc_relookahead_thresh
- cfg.gc_repopgbmap_thresh -> cfg.gc_regbmap_thresh
- cfg.gbmap_repop_thresh -> cfg.gbmap_re_thresh
- LFS3_*_REPOPLOOKAHEAD -> LFS3_*_RELOOKAHEAD
- LFS3_*_REPOPGBMAP -> LFS3_*_REGBMAP
Mainly trying to reduce the mouthful that is REPOPLOOKAHEAD and
REPOPGBMAP.
As a plus this also avoids potential confusion of "repop" as a push/pop
related operation.
This should make tag editing less tedious/error-prone. We already used
self-parsing to generate -l/--list in dbgtag.py, but this extends the
idea to tagrepr (now Tag.repr), which is used in quite a few more
scripts.
To make this work the little tag encoding spec had to become a bit more
rigorous, fortunately the only real change was the addition of '+'
characters to mark reserved-but-expected-zero bits.
Example:
TAG_CKSUM = 0x3000 ## v-11 ---- ++++ +pqq
^--^----^----^--^-^-- valid bit, unmatched
'----|----|--|-|-- matches 1
'----|--|-|-- matches 0
'--|-|-- reserved 0, unmatched
'-|-- perturb bit, unmatched
'-- phase bits, unmatched
dbgtag.py 0x3000 => cksumq0
dbgtag.py 0x3007 => cksumq3p
dbgtag.py 0x3017 => cksumq3p 0x10
dbgtag.py 0x3417 => 0x3417
Though Tag.repr still does a bit of manual formatting for the
differences between shrub/normal/null/alt tags.
Still, this should reduce the number of things that need to be changed
from 2 -> 1 when adding/editing most new tags.
This required a bit of a hack: LFS3_seek_MODE, which is marked internal
to try to minimize confusion, but really doesn't exist in the code at
all.
But a hack is probably good enough for now.
This has just proven much easier to tweak in dbgtag.py, so adopting the
same self-parsing pattern in dbgflags.py/dbgerr.py. This makes editing
easier by (1) not needing to worry about parens/quotes/commas, and
(2) allowing for non-python expressions, such as the mode flags in
dbgflags.py.
The only concern is script startup may be slightly slower, but we really
don't care.
This drops LFS3_t_MUTATED in favor of just using LFS3_t_CKPOINTED
everywhere:
1. These meant roughly the same thing, with LFS3_t_MUTATED being a bit
tighter at the cost of needing to be explicitly set.
2. The implicit setting of LFS3_t_CKPOINTED by lfs3_alloc_ckpoint -- a
function that already needs to be called before mutation -- means we
have one less thing to worry about.
Implicit properties like LFS3_t_CKPOINTED are great for building a
reliable system. Manual flags like LFS3_t_MUTATED, not so much.
3. Why use two flags when we can get away with one?
The only downside is we may unnecessarily clobber gc/traversal work when
we don't actually mutate the filesystem. Failed file open calls are a
good example.
However this tradeoff seems well worth it for an overall simpler +
more reliable system.
---
Saves a bit of code:
code stack ctx
before: 37220 2352 688
after: 37160 (-0.2%) 2352 (+0.0%) 688 (+0.0%)
code stack ctx
gbmap before: 40184 2368 856
gbmap after: 40132 (-0.1%) 2368 (+0.0%) 856 (+0.0%)
Note: This affects the blocking lfs3_alloc_repopgbmap as well as
incremental gc/traversal repopulations. Now all repop attempts return
LFS3_ERR_NOSPC when we don't have space for the gbmap, motivation below.
This reverts the previous LFS3_t_NOSPC soft error, in which traversals
were allowed to continue some gc/traversal work when encountering
LFS3_ERR_NOSPC. This results in a simpler implementation and fewer error
cases to worry about.
Observation/motivation:
- The main motivation is noticing that when we're in low-space
conditions, we just start spamming gbmap repops even if they all fail.
That's really not great! We might as well just mark the flash as dead
if we're going to start spamming erases!
At least with an error the user can call rmgbmap to try to make
progress.
- If we're in a low-space condition, something else will probably return
LFS3_ERR_NOSPC anyways. Might as well report this early and simplify
our system.
- It's a simpler model, and littlefs3 is already much more complicated
than littlefs2. Maybe we should lean more towards a simpler system
at the cost of some niche optimizations.
---
This had the side-effect of causing more lfs3_alloc_ckpoints to return
errors during testing, which revealed a bug in our uz/uzd_fuzz tests:
- We weren't flushing after writes to the opened RDWR files, which could
cause delayed errors to occur during the later read checks in the
test.
Fortunately LFS3_O_FLUSH provides a quick and easy fix!
Note we _don't_ adopt this in all uz/uzd_fuzz tests, only those that
error. It's good to test both with and without LFS3_O_FLUSH to test
that read-flushing also works under stress.
Saves a bit of code:
code stack ctx
before: 37260 2352 688
after: 37220 (-0.1%) 2352 (+0.0%) 688 (+0.0%)
code stack ctx
gbmap before: 40220 2368 856
gbmap after: 40184 (-0.1%) 2368 (+0.0%) 856 (+0.0%)
This relaxes error encountered during lfs3_mtree_gc to _not_ propagate,
but instead just log a warning and prevent the relevant work from being
checked off during EOT.
The idea is this allows other work to make progress in low-space
conditions.
I originally meant to limit this to gbmap repopulations, to match the
behavior of lfs3_alloc_repopgbmap, but I think extending the idea to all
filesystem mutating operations makes sense (LFS3_T_MKCONSISTENT +
LFS3_T_REPOPGBMAP + LFS3_T_COMPACTMETA).
---
To avoid incorrectly marking traversal work as completed, we need to
track if we hit any ENOSPC errors, thus the new LFS3_t_NOSPC flag:
LFS3_t_NOSPC 0x00800000 Optional gc work ran out of space
Not the happiest just throwing flags at problems, but I can't think of a
better solution at the moment.
This doesn't differentiate between ENOSPC errors during the different
types of work, but in theory if we're hitting ENOSPC errors whatever
work returns the error is a toss-up anyways.
---
Adds a bit of code:
code stack ctx
before: 37208 2352 688
after: 37248 (+0.1%) 2352 (+0.0%) 688 (+0.0%)
code stack ctx
gbmap before: 40120 2368 856
gbmap after: 40204 (+0.2%) 2368 (+0.0%) 856 (+0.0%)
- LFS3_T_COMPACT -> LFS3_T_COMPACTMETA
- gc_compact_thresh -> gc_compactmeta_thresh
And friends:
LFS3_M_COMPACTMETA 0x00000800 Compact metadata logs
LFS3_GC_COMPACTMETA 0x00000800 Compact metadata logs
LFS3_I_COMPACTMETA 0x00000800 Filesystem may have uncompacted metadata
LFS3_T_COMPACTMETA 0x00000800 Compact metadata logs
---
This does two things:
1. Highlights that LFS3_T_COMPACTMETA only interacts with metadata logs,
and has no effect on data blocks.
2. Better matches the verb+noun names used for other gc/traversal flags
(REPOPGBMAP, CKMETA, etc).
It is a bit more of a mouthful, but I'm not sure that's entirely a bad
thing. These are pretty low-level flags.
And friends:
LFS3_M_REPOPLOOKAHEAD 0x00000200 Repopulate lookahead buffer
LFS3_GC_REPOPLOOKAHEAD 0x00000200 Repopulate lookahead buffer
LFS3_I_REPOPLOOKAHEAD 0x00000200 Lookahead buffer is not full
LFS3_T_REPOPLOOKAHEAD 0x00000200 Repopulate lookahead buffer
To match LFS3_T_REPOPGBMAP, which is more-or-less the same operation.
Though this does turn into quite the mouthful...
And tweaked a few related comments.
I'm still on the fence with this name, I don't think it's great, but it
at least betters describes the "repopulation" operation than
"rebuilding". The important distinction is that we don't throw away
information. Bad/erased block info (future) is still carried over into
the new gbmap snapshot, and persists unless you explicitly call
rmgbmap + mkgbmap.
So, adopting gbmap_repop_thresh for now to see if it's just a habit
thing, but may adopt a different name in the future.
As a plus, gbmap_repop_thresh is two characters shorter.
A bit less simplified than I hoped, we don't _strictly_ need both
LFS3_t_DIRTY + LFS3_t_MUTATED if we're ok with either (1) making
multiple passes to confirm fixorphans succeeded or (2) clear the COMPACT
flag after one pass (which may introduce new uncompacted metadata). But
both of these have downsides, and we're not _that_ stressed for flag
space yet...
So keeping all three of:
LFS3_t_DIRTY 0x04000000 Filesystem modified outside traversal
LFS3_t_MUTATED 0x02000000 Filesystem modified during traversal
LFS3_t_CKPOINTED 0x01000000 Filesystem ckpointed during traversal
But I did manage to get rid of the bit swapping by tweaking LFS3_t_DIRTY
to imply LFS3_t_MUTATED instead of being exclusive. This removes the
"failed" gotos in lfs3_mtree_gc and makes things a bit more readable.
---
I also split lfs3_fs/handle_clobber into separate lfs3_fs/handle_clobber
and lfs3_fs/handle_mutate functions. This added a bit of code, but I
think is worth it for a simpler internal API. A confusing internal API
is no good.
In total these simplifications saved a bit of code:
code stack ctx
before: 37208 2360 684
after: 37176 (-0.1%) 2360 (+0.0%) 684 (+0.0%)
code stack ctx
gbmap before: 40100 2432 848
gbmap after: 40060 (-0.1%) 2432 (+0.0%) 848 (+0.0%)
This adds LFS3_T_REBUILDGBMAP and friends, and enables incremental gbmap
rebuilds as a part of gc/traversal work:
LFS3_M_REBUILDGBMAP 0x00000400 Rebuild the gbmap
LFS3_GC_REBUILDGBMAP 0x00000400 Rebuild the gbmap
LFS3_I_REBUILDGBMAP 0x00000400 The gbmap is not full
LFS3_T_REBUILDGBMAP 0x00000400 Rebuild the gbmap
On paper, this is more or less identical to repopulating the lookahead
buffer -- traverse the filesystem, mark blocks as in-use, adopt the new
gbmap/lookahead buffer on success -- but a couple nuances make
rebuilding the gbmap a bit trickier:
- Unlike the lookahead buffer, which eagerly zeros in allocation, we
need an explicit zeroing pass before we start marking blocks as
in-use. This means multiple traversals can potentially conflict with
each other, risking the adoption of a clobbered gbmap.
- The gbmap, which stores information on disk, relies on block
allocation and the temporary "in-flight window" defined by allocator
ckpoints to avoid circular block states during gbmap rebuilds. This
makes gbmap rebuilds sensitive to allocator ckpoints, which we
consider more-or-less a noop in other parts of the system.
Though now that I'm writing this, it might have been possible to
instead include gbmap rebuild snapshots in fs traversals... but that
would probably have been much more complicated.
- Rebuilding the gbmap requires writing to disk and is generally much
more expensive/destructive. We want to avoid trying to rebuild the
gbmap when it's not possible to actually make progress.
On top of this, the current trv-clobber system is a delicate,
error-prone mess.
---
To simplify everything related to gbmap rebuilds, I added a new
internal traversal flag: LFS3_t_CKPOINTED:
LFS3_t_CKPOINTED 0x04000000 Filesystem ckpointed during traversal
LFS3_t_CKPOINTED is set, unconditionally, on all open traversals in
lfs3_alloc_ckpoint, and provides a simple, robust mechanism for checking
if _any_ allocator checkpoints have occured since a traversal was
started. Since lfs3_alloc_ckpoint is required before any block
allocation, this provides a strong guarantee that nothing funny happened
to any allocator state during a traversal.
This makes lfs3_alloc_ckpoint a bit less cheap, but the strong
guarantees that allocator state is unmodified during traversal are well
worth it.
This makes both lookahead and gbmap passes simpler, safer, and easier to
reason about.
I'd like to adopt something similar+stronger for LFs3_t_MUTATED, and
reduce this back to two flags, but that can be a future commit.
---
Unfortunately due to the potential for recursion, this ended up reusing
less logic between lfs3_alloc_rebuildgbmap and lfs3_mtree_gc than I had
hoped, but at like the main chunks (lfs3_alloc_remap,
lfs3_gbmap_setbptr, lfs3_alloc_adoptgbmap) could be split out into
common functions.
The result is a decent chunk of code and stack, but the value is high as
incremental gbmap rebuilds are the only option to reduce the latency
spikes introduced by the gbmap allocator (it's not significantly worse
than the lookahead buffer, but both do require traversing the entire
filesystem):
code stack ctx
before: 37164 2352 684
after: 37208 (+0.1%) 2360 (+0.3%) 684 (+0.0%)
code stack ctx
gbmap before: 39708 2376 848
gbmap after: 40100 (+1.0%) 2432 (+2.4%) 848 (+0.0%)
Note the gbmap build is now measured with LFS3_GBMAP=1, instead of
LFS3_YES_GBMAP=1 (maybe-gbmap) as before. This includes the cost of
mkgbmap, lfs3_f_isgbmap, etc.
The gbmap introduces quite a bit of complexity with how it interacts
with config: block_count => gbmap weight, and wcompat => gbmap enabled.
On one hand this means fewer sources of truth, on the other hand it
makes the gbmap logic cross subsystems and a bit messy.
To avoid trying to parse a bunch of disabled/garbage gstate, this adds
wcompat/rcompat checks to our Gstate class, exposed via __bool__.
This also means we actually need to parse wcompat/rcompat/ocompat flags,
but that wasn't to difficult (though currently only supports 32-bits).
---
I added conditional repr logic for the grm and gbmap, but didn't bother
with the gcksum. The gcksum is used too many other places in these
scripts to expect a nice rendering when disabled.
In lfs3_fs_grow, we need to update any gbmaps to match the new disk
size. The actual patch to the gbmap is easy, but it does get a bit
delicate since we need to feed the gbmap with an allocator in the new
disk size.
Fortunately, the opportunistism of the gbmap allocator avoids any
catch-22 issues, as long as we make sure to not trigger any gbmap
rebuilds.
Adds a bit of code, but not much:
code stack ctx
before: 37168 2352 684
after: 37168 (+0.0%) 2352 (+0.0%) 684 (+0.0%)
code stack ctx
gbmap before: 39000 2456 800
gbmap after: 39116 (+0.3%) 2456 (+0.0%) 800 (+0.0%)
TLDR: This drops the idea of different bmap strategies/modes, and sorts
out most of the compile-time/runtime conditional bmap interactions.
---
Motivation: Benchmarking (at least up to the 32-bit word limit) has
shown the bmap will unlikely be a significant bottleneck, even on large
disks. The largest disks tend to be NAND, and NAND's ridiculous block
size limits pressure on block allocation.
There are still concerns for areas I haven't measured yet:
- SD/eMMC/FTL - Small blocks, so more pressure on block allocation. In
theory the logical block size can be artificially increased, but this
comes with a granularity tradeoff.
- I've only measured throughput, latency is a whole other story.
However, users have reported lfs3_fs_gc is useful for mitigating this,
so maybe latency is less of a concern now?
But while there may still be room for improvement via alternative bmap
strategies, the risk a concerning amount of complexity. Yes,
configuration gets more complicated, but the real issue is any bmap
strategies that try to track _deallocations_ (the original idea being
treediffing) risk falling leaking blocks if all cases aren't covered.
The current "bmap cache" strategy strikes a really nice balance where it
reduces _amortized_ block allocation -> ~O(log n) without RAM, while
retaining the safe, bug-resistant, single-source-of-truth properties
that come with lookahead-based allocation.
---
So, long story short, dropping other strategies, and now the presence of
the bmap is a boolean flag.
This is also the first format-specific flag:
- Define LFS3_BMAP to enable the bmap logic, but note by default the
bmap will still not be used.
- Define LFS3_YES_BMAP to force the bmap to be used.
- With LFS3_BMAP, passing LFS3_F_GBMAP to lfs3_format will include the
on-disk block-map.
- No flag is needed during mount, the presence of the bmap is determined
by the on-disk wcompat flags (LFS3_WCOMPAT_GBMAP). This also prevents
rw mounting if the bmap is not supported, but rdonly mounting is
allowed.
- Users can check if the bmap is in use via lfs3_fs_stat, which reports
LFS3_I_GBMAP in the flags field.
There's still some missing pieces, but these will be a bit more
involved:
- lfs3_fs_grow needs to be made bmap aware!
- We probably want something like lfs3_fs_mkgbmap and lfs3_fs_rmgbmap to
allow converting between bmap backed/not-backed filesystem images.
Code changes minimal:
code stack ctx
before: 37172 2352 684
after: 37172 (+0.0%) 2352 (+0.0%) 684 (+0.0%)
code stack ctx
bmap before: 38844 2456 800
bmap after: 38852 (+0.0%) 2456 (+0.0%) 800 (+0.0%)
New bmap range tags:
LFS3_TAG_BMRANGE 0x033u v--- --11 --11 uuuu
LFS3_TAG_BMFREE 0x0330 v--- --11 --11 ----
LFS3_TAG_BMINUSE 0x0331 v--- --11 --11 ---1
LFS3_TAG_BMERASED 0x0332 v--- --11 --11 --1-
LFS3_TAG_BMBAD 0x0333 v--- --11 --11 --11
Note 0x334-0x33f are still reserved for future bmap tags, but the new
encoding fits in the surprisingly common 2-bit subfield that may
deduplicate some decoding code.
Fitting in 2-bits is the main reason for this, now that in-flight ranges
look like they won't be worth exploring further. Worst case we can
always add more bm tags in the future. And it may even make sense to use
an entire bit for in-flight tags, since in theory the concept can apply
to more than just in-use blocks.
---
Another benefit of this encoding: In-use vs free is a bit check, and I
like the implication that an in-use + erased block can only be a bad
block.
No code changes:
code stack ctx
before: 37172 2352 684
after: 37172 (+0.0%) 2352 (+0.0%) 684 (+0.0%)
code stack ctx
bmap before: 38844 2456 800
bmap after: 38844 (+0.0%) 2456 (+0.0%) 800 (+0.0%)
Too much room for confusion, and potential flag conflicts in the future.
Note it already conflicted with -e/--error-* flags.
--exec is a rather technical flag anyways, and will probably be wrapped
in other ci/script scaffolding most of the time.
I'm not sure if this was ever implemented, or broken during a refactor,
but we were ignoring -s/-S flags when writing .csv/.json output with
-o/-O.
Curious, because the functionality _was_ implemented in fold, just
unused. All this required was passing -s/-S to fold correctly.
Note we _don't_ sort diff_results, because these are never written to
.csv/.json output.
At some point this behavior may have been a bit more questionable, since
we use to allow mixing -o/-O and table rendering. But now that -o/-O is
considered an exclusive operation, ignoring -s/-S doesn't really make
sense.
---
Why did this come up? Well imagine my frustration when:
1. In tikz/pgfplots, \addplot table only really works with sorted data
2. csv.py has a -s/-S flag for sorting!
3. -s/-S doesn't work!
- Added --small-total. Like --small-header, this omits the first column
which usually just has the informative text TOTAL.
- Tweaked -Q/--small-table so it renders with --small-total if
-Y/--summary is provided.
- Added --total as an alias for --summary + --no-header + --small-total,
i.e. printing only the totals (which may be multiple columns) and no
other decoration.
This is useful for scripting, now it's possible to extract just, say,
the sum of some csv and embed with $():
echo $(./scripts/code.py lfs3.o --total)
- Tweaked total to always output a number (0) instead of a dash (-),
even if we have no results.
This relies on Result() with no args, which risks breaking scripts
where the Result type expects an argument. To hopefully catch this
early, the table renderer currently creates a Result() before trying
to fold the total result.
- If first column is empty (--small-total + --small-header, --no-header,
etc) collapse width to zero. This avoids a bunch of extra whitespace,
but still includes the two spaces normal used to separate names from
fields.
But I think those spaces are a good thing. It makes it hard to miss
the implicit padding in the table renderer that risks breaking
dependent scripts.
Found when trying to measure ctx of yaffs2, which relies on incomplete
structs to hide some internal state (yaffs_summary_tags, yaffs_DIR).
This is less common in microcontroller filesystems since almost all
structs end up statically/stack allocated, and you can't statically
allocate incomplete structs.
It's not too surprising, but incomplete structs have no associated
DW_AT_byte_size in the relevant dwarf info, which broke ctx.py and
structs.py...
As a workaround, I'm now defaulting to size=0 if DW_AT_byte_size is
missing.
---
With this fix, at least structs.py is able to pick up the later internal
definition of yaffs_summary_tags. ctx.py doesn't because it only looks
at the unique dwarf offset referenced by the function definition, but
I'm hesitant to try anything more clever here.
yaffs_DIR is noteworthy in that there is simply no complete definition.
Internally, yaffs_DIR pointers alias yaffsfs_DirSearchContext structs.
In this case I think returning size=0 is the only reasonable option.
I've been struggling to keep plots readable with --x/ylim-stddev, it may
have been the wrong tool for the job.
This adds --x/ylim-ratio as an alternative, which just sets the limit to
include x-percent of the data (I avoided "percen"t in the name because
it should be --x/ylim-ratio=0.98, not 98, though I'm not sure "ratio" is
great either...).
Like --x/ylim-stddev, this can be used in both one and two argument
forms:
$ ./scripts/plot.py --ylim-ratio=0.98
$ ./scripts/plot.py --ylim-=-0.98,+0.98
So far, --x/ylim-ratio has proven much easier to use, maybe because our
amortized results don't follow a normal distribution? --x/ylim-ratio
seems to do a good job of clipping runaway amortized results without too
much information loss.
Continued benchmarking efforts are indicating this isn't really an
optional optimization.
This brings back lazy grafting, where the file leaf is allowed to fall
out-of-date to minimize bshrub/btree updates. This is controlled by
LFS3_o_UNGRAFT, which is similar, but independent from LFS3_o_UNCRYST:
- LFS3_o_UNCRYST - File's leaf not fully crystallized
- LFS3_o_UNGRAFT - File's leaf does not match disk
Note it makes sense for files to be UNGRAFT only, in the case where the
current crystal terminates at the end-of-file but future appends are
likely. And it makes sense for files to be UNCRYST only, in cases where
we graft uncrystallized blocks so the bshrub/btree makes sense.
Which brings us to the main change from the previous lazy-grafting
implementation: lfs3_file_lookupnext no longer includes ungrafted
leaves.
Instead, functions should call lfs3_file_graft if they need
lfs3_file_lookupnext to make sense.
This significantly reduces the code cost of lazy grafting, at the risk
of needing to graft more frequently. Fortunately we don't actually need
to call lfs3_file_graft all that often:
- lfs3_file_read already flushes caches/leaves before attempting any
bshrub/btree reads for simplicity (heavy are not currently considered
a priority, if you need this consider opening two file handles).
- lfs3_file_flush_ _does_ need to call lfs3_file_graft before the
crystallization heuristic pokes, but if we can't resume
crystallization, we would probably need to graft the crystal to
satisfy the flush anyways.
---
Lazy grafting, i.e. procrastinating on bshrub/btree updates during block
appends, is an optimization previously dropped due to perceived
nicheness:
- We can only lazily graft blocks, inlined data fragments always require
bshrub/btree updates since they live in the bshrub/btree.
- Sync forces bshrub/btree updates anyways, so lazy grafting has no
benefit for most logging applications.
- This performance penalty of eagerly grafting goes away if your caches
are large enough.
Note that the last argument is a non-argument in littlefs's case. They
whole point of littlefs is that you _don't_ need RAM to fix things.
However these arguments are all moot when you consider that the "niche
use case" -- linear file writes -- is the default bottleneck for most
applications. Any file operation becomes a linear write bottleneck when
the arguments are large enough. And this becomes a noticeable issue when
benchmarking.
So... This brings back lazy grafting. But with a more limited scope
w.r.t. internal file operations (the above lfs3_file_lookupnext/
lfs3_file_graft changes).
---
Long story short, lazy grafting is back again, reverting the ~3x
performance regression for linear file writes.
But now with quite a bit less code/stack cost:
code stack ctx
before: 36820 2368 684
after: 37032 (+0.6%) 2352 (-0.7%) 684 (+0.0%)
This adds %i and %I as punescape modifiers for limited printing of
integers with SI prefixes:
- %(field)i - base-10 SI prefixes
- 100 => 100
- 10000 => 10K
- 0.01 => 10m
- %(field)I - base-2SI prefixes
- 128 => 128
- 10240 => 10Ki
- 0.125 => 128mi
These can also easily include units as a part of the punescape string:
- %(field)iops/s => 10Kops/s
- %(field)IB => 10KiB
This is particularly useful in plotmpl.py for adding explicit
x/yticklabels without sacrificing the automatic SI-prefixes.
So now these lex correctly:
- 1e9 => 1000000000
- 1e+9 => 1000000000
- 1e-9 => -1000000000
A bit tricky when you think about how these could be confused for binary
addition/subtraction. To fix we just eagerly grab any signs after the e.
These are particularly useful for manipulating simulated benchmarks,
where we need to convert things to/from nanoseconds.
Christopher Haster