The tests highlighted that the LFS_I_DIRTY flag in lfsr_tinfo approach
is insufficient. Consider what happens if our filesystem is mutated
while traversing the last mdir:
1. Traversal traverses last mdir, populate blocks, return first block
2. Filesystem mutated, maybe mdir was compacted, clobbers traversal and
sets LFS_I_DIRTY
3. Traversal return LFS_ERR_NOENT immediately, last block never
returned (and out of date), LFS_I_DIRTY never returned
Not only do we miss the LFS_I_DIRTY flag, but we completely miss the
last block in the mdir pair without any warning.
This is _not_ a problem for the actual lookahead buffer, since we still
internally check the LFS_I_DIRTY flag before marking it as complete, but
it is an issue for any external logic that depends on the traversal
being complete...
---
We could revert to LFS_T_EXCL, but, to be honest, I just really don't
know a good name for this flag...
LFS_T_EXCL is a bad name because it conflicts with LFS_O_EXCL. These
flags have very different behaviors, which risks confusing users, and
risks potential name conflicts down the line if we ever want
LFS_T_EXCL-esque semantics for open dirs/files (not unreasonable, though
quite fancy).
My current best contender is LFS_T_WATCH, but while scratching my head
on this, I starting to wonder why we're even providing LFS_T_EXCL in the
first place...
We err on the side of forcing users to implement filesystem-external
features themselves when possible elsewhere, and LFS_T_EXCL technically
_can_ be implemented entirely outside of the filesystem. Though to be
fair it is quite annoying/tedious.
It's not like there's any equivalent feature for dir/file reads anyways.
And a background thread calling lfsr_traversal_read with LFS_T_LOOKAHEAD
will still _eventually_ make progress, even if it takes a bit longer.
Don't get me wrong, I understand it is significantly easier to implement
this inside the filesystem than outside. But it's also easier to
implement this later than right now. And if we implement this later,
hopefully we'll have a better idea what exactly will be useful for
users.
---
Removing LFS_T_EXCL/LFS_I_DIRTY has no real impact on code cost. We were
really just exposing internal logic that we need for lookahead
correctness anyways:
code stack
before: 35224 2680
after: 35220 (-0.0%) 2680 (+0.0%)
This just forwards the internal LFS_I_DIRTY flag to the user via the
lfsr_tinfo flags field.
Benefits of this approach:
- Gives the user more flexibility on what to do if the filesystem is
modified, maybe you want to keep traversing depending on some other
logic.
- Can eventually add other flags to tinfo.flags, such as
LFS_I_COMPACTED, LFS_I_REPAIRED, LFS_I_INCONSISTENT, etc.
- Avoids confusion around the very different behaviors of LFS_O_EXCL and
LFS_T_EXCL.
I tried to come up with a better name (maybe LFS_T_WATCH?) but it was
a bit of a struggle... Switching to a flags approach sidesteps the
issue.
- Can drop the LFS_ERR_BUSY error code for now.
Code changes were fairly insignificant:
code stack
before: 35244 2680
after: 35224 (-0.1%) 2680 (+0.0%)
The only concern is that the tests highlighted it's possible for our
flag scheme to miss mutation if it happens after/during the last set of
blocks... Not sure how to handle this yet...
LFS_O_SYNC always implies LFS_O_FLUSH, otherwise what exactly are you
syncing? Making this explicit in the bit pattern should hopefully make
this clear for curious users, though lfsr_file_flush would be called
anyways because of how lfsr_file_sync is implemented.
This also moves the LFS_O_DESYNC bit pattern around so SYNC/FLUSH are
neighbors. SYNC/DESYNC may seem related, but in lfsr_file_open they
actually are quite different:
LFS_O_FLUSH 0x0040 ---- ---- -1-- ----
LFS_O_SYNC 0x00c0 ---- ---- 11-- ----
LFS_O_DESYNC 0x0100 ---- ---1 ---- ----
Code changes, mostly just noise from moving bits around:
code stack
before: 35228 2680
after: 35244 (+0.0%) 2680 (+0.0%)
It still doesn't make sense to check data without checking metadata, but
keeping this named LFS_T_CKDATA should hopefully clarify what it does
differently from LFS_T_CKMETA.
This implication is also now encoded in the bit pattern:
LFS_T_CKMETA 0x0100 ---- ---1 ---- ----
LFS_T_CKDATA 0x0300 ---- --11 ---- ----
In theory a clever user could force only the CKDATA bit to be set, and
such a configuration would _probably_ work fine, but it won't be
supported just to cut down on possible configurations to test.
No code changes:
code stack
before: 35228 2680
after: 35228 (+0.0%) 2680 (+0.0%)
It's probably a bad reason, but this avoids wasting too much time
figuring out how to name things.
Now most traversal functions return an lfsr_tag_t + lfsr_bptr_t pair,
which is enough to describe the current relevant traversal objects:
tag=LFSR_TAG_MDIR => (lfsr_mdir_t*)bptr.data.u.buffer
tag=LFSR_TAG_BRANCH => (lfsr_rbyd_t*)bptr.data.u.buffer
tag=LFSR_TAG_DATA => bptr.data
tag=LFSR_TAG_BPTR => bptr
This would be a bit better if lfsr_data_t's buffer field was a void*,
but that would mess with byte-level arithmetic, which is more common
with lfsr_data_ts.
This also adopts the fragmented/optional out-params used elsewhere in
the codebase. I thought this would add quite a bit more stack cost,
since we need redundant tags/bptrs to make lfsr_mtree_traverse/
lfsr_mtree_gc work, but surprisingly not:
code stack
before: 35256 2680
after: 35228 (-0.1%) 2680 (+0.0%)
It seems we make up the extra stack cost of redundant tags/bptrs by
giving the compiler more stack-alloc flexibility, tighter per-function
return types, and opting-out of tags/bptrs in most low-level traversals:
lfs_alloc mainly.
But if the fragmented/optional out-params is net harmful for code/stack
size, we should reconsider the pattern system-wide. This does probably
deserve a second look in the future...
This could go either way, it's a case of the classic C strchr type
conundrum.
But unlike iteration, we're more likely to mutate things when doing a
full traversal, so requiring everything to be mutable makes a bit more
sense.
Note that even readonly operations, fetchck for example, need access to
a mutable rbyd struct.
No code changes:
code stack
before: 35256 2680
after: 35256 (+0.0%) 2680 (+0.0%)
This solves the issue of multiple mdirs/rbyds in lfsr_mtree_gc, where
it's easy for traversal state to fall out of sync when mutating parts of
the filesystem.
Is it good design, with self-referential pointers making everything more
entangled? Not sure!
This saves a bit of stack, but adds a bit of code, which makes sense,
pointer chasing can be costly. But both of these changes are well below
the compiler noise floor:
code stack
before: 35228 2688
after: 35256 (+0.1%) 2680 (-0.3%)
Just in case any tags leak through. If an orphan tag ended up in an
lfsr_stat call, it could be quite confusing to users...
Current types:
// user facing
LFS_TYPE_REG 1 ---1
LFS_TYPE_DIR 2 --1-
LFS_TYPE_SYMLINK* 3 --11
// internal
LFS_TYPE_BOOKMARK 4 -1-- -.
LFS_TYPE_ORPHAN 5 -1-1 +- on-disk only
LFS_TYPE_COMPR* 6 -11- -'
LFS_TYPE_TRAVERSAL 9 1--1 <-- in-ram only
* Hypothetical
This has no impact on code size:
code stack
before: 35228 2688
after: 35228 (+0.0%) 2688 (+0.0%)
We probably want a common function to tell us if a given omdir is also
an obshrub.
Conveniently all types with bshrubs happen to share a common bit, and
this pattern will probably continue for a while:
// user facing
LFS_TYPE_REG 1 ---1 <-- bshrub
LFS_TYPE_DIR 2 --1-
LFS_TYPE_SYMLINK* 3 --11 <-- bshrub
// internal
LFS_TYPE_BOOKMARK 4 -1--
LFS_TYPE_TRAVERSAL 5 -1-1 <-- bshrub
* Hypothetical
Maybe this is overspecialized, but at least our tests will break quite
quickly if this ever turns out not to be true...
Code changes:
code stack
before: 35256 2688
after: 35228 (-0.1%) 2688 (+0.0%)
So now files and traversals contain several nested structs:
file <-- lfsr_file_t
file.o <-- lfsr_obshrub_t
file.o.o <-- lfsr_omdir_t
This gets a bit ugly, but it's really the only way to make the compiler
happy when also with C's annoying strict aliasing rules.
This also makes lfsr_traversal_t a simple alias of lfsr_mtraversal_t,
with lfsr_mtraversal_t now including all of the obshrub/omdir state.
This simplifies things internally, and allows lfsr_mtree_gc to assert on
opened-list enrollment, but risks increased stack cost for all of the
unused fields.
Fortunately this stack cost turned out to not be that significant:
code stack
before: 35264 2680 (+0.0%)
after: 35256 (-0.0%) 2688 (+0.3%)
Does what it says on the tin.
This simplifies lfsr_omdir_clobber, and can be used in more places. It's
a bit more flexible than implicitly mkdirtying in lfs_alloc_ckpoint, but
does add another function call.
But thanks to better code reuse this ends up saving a bit of code:
code stack
before: 35304 2688
after: 35264 (-0.1%) 2680 (-0.3%)
This brings back clobbering individual omdirs, so modifying an unsynced
file should leave all other traversals intact. This is the most precise
level of clobbering that I think is reasonable to implement.
This means we should be able to, say, check all currently-committed
checksums while writing to unsynced files at the same time. Which might
be useful? Maybe?
To make this work, lfsr_traversal_clobber now relies on the current
traversal state to know how to clobber correctly. This is more verbose,
but likely safer/more flexible.
Curiously, this actually ended up saving a bit of code, which is a bit
surprising:
code stack
before: 35356 2688
after: 35304 (-0.1%) 2688 (+0.0%)
Maybe manipulating the state machine directly gives the compiler more
info to work with? Not sure.
Since we're clobbering at the mid-level now, our mtraversals can only
ever point to unsynced reg file handles.
This means we can limit traversal clobbering to lfsr_file_close, and
move it out of the common/simple lfsr_omdir_close.
Look like any code changes canceled out perfectly:
code stack
before: 35356 2688
after: 35356 (-0.0%) 2688 (+0.0%)
This adds an indirect pointer to lfsr_btraversal_t, so references to the
btree/bshrub root point to the actual btree/bshrub root rbyd struct.
This means if our bshrub root is mutated due to, say, mdir compaction,
this doesn't necessarily invalidate our btraversal.
But note this is strictly limited to bshrub roots. If you modify any
other part of the bshrub/btree, expect the traversal to be broken.
This means we can do whatever we want with mdirs and not worry about
invaliding bshrub traversals, which is quite nice! It also fixes our
failing bshrub-traversal-mutation tests.
This adds a bit of stack cost, but because we are moving fewer rbyd
structs around in lfsr_btree_traverse_, actually ends up saving a bit of
code. Though we are well below the compiler noise floor:
code stack
before: 35368 2680
after: 35356 (-0.0%) 2688 (+0.3%)
Implementing gc_compact_thresh over bshrubs highlighted that it's really
not that difficult, and probably required, for traversal bshrubs to be
tracked correctly during mdir commits/compacts/splits/etc. And if we
track bshrubs across mdir commits, we might as well clobber traversals
at the mid level, allowing traversals to always reach btrees/bshrubs not
under active mutation.
One key thing to note: we should never be traversing a bshrub that is
not referenced elsewhere, either on-disk in an mdir or in-ram via an
opened file. So any compacted traversal bshrubs are not wasted prog
cycles.
This moves most of the clobbering logic back up into the high-level
functions (lfsr_remove/rename mainly), where we know which mids may be
clobbered.
This has a code cost, but it's really not all that much for more
thorough/correct filesystem traversals under mutation:
code stack
before: 35268 2680
after: 35368 (+0.3%) 2680 (+0.0%)
Unfortunately, lingering rbyd references in our btraversal structs are
still an issue, and some bshrub tests are failing... Though I do have
some ideas on how to fix this.
This does a few things:
- Deduplicates bshrub/btree rbyd lookups when rbyd is not explicitly
provided -- note this is by far the most common case.
- Moves the mid-level rbyd allocation out of the stack-hot-path.
Assuming lfsr_mtree_gc is never in the stack-hot-path, which seems
unlikely.
- Reduces pointer chasing in lfsr_btree_commit__, giving the compiler
more flexibility + assumptions and hopefully allowing it to optimize
better.
This may also be disentangling several cross-layer struct references,
which is probably a good thing.
The result is rather significant stack savings for what is a minor
refactor:
code stack
before: 35440 2800
after: 35268 (-0.5%) 2680 (-4.3%)
This doesn't quite get us back to pre-commit_-rbyd levels, but it's
pretty close:
code stack
before commit_-rbyd: 34652 2640
commit_-rbyd-explicit (before): 35440 (+2.3%) 2800 (+6.1%)
commit_-rbyd-implicit (after): 35268 (+1.8%) 2680 (+1.5%)
This gets a bit muddled now with traversals mutating inner btree nodes
directly.
Except for some asserts, we can accept any bid in the relevant rbyd
here, and accepting any bid is better than accepting only one bid
(left-leaning) inconsistent with the rest of the btree API
(right-leaning)...
Code changes minimal:
code stack
before: 35448 2800
after: 35440 (-0.0%) 2800 (+0.0%)
These aren't really different than btree nodes, except bshrubs need to
be enrolled in our opened list for commits to work.
Fortunately this is already true for explicit traversals, which are
currently the only traversals where we need to simultaneously mutate the
filesystem. This mainly just required adding additional checks for
LFS_TYPE_TRAVERSAL bshrubs, tests, and making sure traversal.bshrub is
never in an invalid state.
This continues to add code/stack cost for what is ultimately a
relatively niche feature:
code stack
before: 35268 2776
after: 35448 (+0.5%) 2800 (+0.9%)
Maybe btree/bshrub compactions should be disabled by default?
The internal btree node traversal here is a bit different than in
dbgbtree.py, dbgmtree.py, etc, because we want to include both inner and
leaf nodes. We could decode the branch tags multiple times, but checking
for bid changes is a bit simpler.
It's interesting to note this went missed for so long, because, well,
it's hard to actually have more than one btree node. And the tests
explicitly covering large btree structures, test_btree, aren't parsable
by dbgbmap.py since they don't create real filesystem images.
Note, gc_compact_thresh over bshrubs is not yet implemented... That's
_another_ can of worms since we need to be able to commit to non-tracked
bshrubs somehow...
But at least this proves gc_compact_thresh over btrees is possible.
Now, if LFS_T_COMPACT is provided, any btree nodes > gc_compact_thresh
will be compacted during traversal/gc operations.
To make this work required a rather deep modification to the
lfsr_btree_commit/lfsr_bshrub_commit code paths to expose direct-rbyd
commit functions that can commit to arbitrary btree nodes:
- lfsr_btree_commit - bid, attrs, attr_count
- lfsr_bshrub_commit - bid, attrs, attr_count
- lfsr_btree_commit_ - bid, rbyd, rid, attrs, attr_count
- lfsr_bshrub_commit_ - bid, rbyb, rid, attrs, attr_count
- lfsr_btree_commit__ - bscratch, bid, rbyd, rid, attrs, attr_count
These are good to have, and will also be useful for implementing
metadata redundancy in the future.
Unfortunately, all of this comes at a significant code/stack cost:
code stack
before: 34652 2640
after: 35268 (+1.8%) 2776 (+5.2%)
lfs_fs_gc is still not reimplemented, but this is accessible through the
traversal API with LFS_T_COMPACT.
This is also the first traversal operation that can mutate the
filesystem, which brings its own set of problems:
- We need to set LFS_F_DIRTY in lfsr_mtree_gc now, which really
highlights how much of a mess having two flag fields is...
We do _not_ clobber in this case, since we assume lfsr_mtree_gc knows
what it's doing.
- We can now commit to an mroot in the mroot chain outside of the normal
mroot chain update logic.
This is a bit scary, but should just work.
The only issue so far is that we need to allow mdirs to follow the
mroot during mroot splits if mid=-1, even if they aren't lfs_t's mroot
mdir.
This should now be decently tested with the new
test_traversal_compact_* tests.
- It's easy for mtraversal's mdir and mtinfo's mdir to fall out of sync
when mutating... Why do we have two of these?
The actual compaction itself is pretty straightforward: just mark as
unerased, eoff=-1, and call lfsr_mdir_commit with an empty commit. This
is now wrapped up in lfsr_mdir_compact.
Code changes:
code stack
before: 34528 2640
after: 34652 (+0.4%) 2640 (+0.0%)
Though the real hard part will be implementing gc_compact_thresh over
btree nodes...
It really doesn't make sense to check data and not check metadata. We're
already traversing the metadata, so validating it adds very little
overhead, and how can we trust our data if we can't trust our metadata?
This renames LFS_T_CKDATA -> LFS_T_CK, which now also implies
LFS_T_CKMETA. This implication is done explicitly in lfsr_mtree_traverse
instead of doing anything fancy with flags.
Implying LFS_T_CKMETA also means one less configuration to support.
Code changes:
code stack
before: 34524 2640
after: 34528 (+0.0%) 2640 (+0.0%)
To hopefully hint that these are not pure-output pointers, unlike
underscore arguments elsewhere (though this isn't really an intentional
convention).
Cksums need to be initialized with zero, so that multiple prog/read
operations can chain cksum updates.
This has already tripped me up a couple times.
Separated out omdir/mdir and mtraversal. You still need to allocate an
mdir for mtraversal to work, but this avoids the extra cost of omdir's
linked-list.
To avoid _too_ many pointers, I duplicated the flags field into both
lfsr_traversal_t and lfsr_mtraversal_t. This is basically free since we
end up with a bunch of padding for mtraversal's state field, but comes
with the risk of getting confused when the two flag fields don't match
in the future.
I also merged the intermediary btype field into flags to avoid yet
another single-byte field, where it fits comfortably in 3-bits.
Note that the mdir can be uninitialized in cases where we don't need to
worry about traversal clobbering.
---
This has the same problems as separating out mdirs/bshrubs in bshrub
functions: more stack/code to move the multiple pointers around, but is
necessary to avoid strict aliasing issues. There's no way to represent
overlapping omdir/mdir/mtraversal struct in standard C99 otherwise.
The end result saves a bit of code, but adds a bit of stack:
code stack
before: 34576 2632
after: 34524 (-0.2%) 2640 (+0.3%)
Though these numbers may be close enough to the compiler noise floor to
not really care about...
This is how littlefs used to be organized, and I found it a bit easier
to navigate: low-level => go to front, high-level => go to end.
This also moves lfs_alloc immediately after the mtree logic, which is
sort of related, and before anything high-level.
lfs_init/lfs_deinit are also now immediately before lfs_mount/
lfs_unmount/lfs_format, which are closely intertwined.
This did actually affect our code cost, which is interesting. No logic
was changed, only moved:
code stack
before: 34566 2632
after: 34576 (+0.0%) 2632 (+0.0%)
There is a cyclic dependency between the bshrub and mdir logic, so
there's no obvious order, but grouping up bshrubs and btrees makes a lot
of sense since they share many low-level operations (lfsr_btree_commit_,
lfsr_btree_traverse_, etc).
Bshrubs really are just inlined btrees after all.
lfs.c is now roughly organized into three large sections:
1. Raw on-disk data structures (rbyds, btrees, bshrubs, etc)
2. The mess that is metadata (mtree, mdirs, etc)
3. High-level types/functions (files, dirs, mount, traversals, etc)
No code changes.
This gets a bit messy, since lfsr_bshrub_commit really requires the
bshrub to be enrolled in the opened mdir list to stage correctly.
To make this work, our internal SHRUBCOMMIT and SHRUBTRUNK attrs now
take a pointer to the active shrub, and assume it is followed by a
staging shrub in memory. This is a big hack/assumption that leaks
through lfsr_bshrub_commit, but it at gets the job done in our current
system.
Note some functions were renamed instead, these didn't really make sense
as pure-bshrub functions:
- lfsr_bshrub_readnext -> lfsr_file_readnext
- lfsr_bshrub_read -> lfsr_file_read_
---
The main reason for this is to comply with C99's strict aliasing rules,
which can be a real PIA sometimes.
We need to track a bshrub in lfsr_mtraversal_t, but we really don't want
to pay the RAM cost for an entire lfsr_file_t. The best option I've
found is to pass around multiple pointers to the relevant internal
structs (mdir+bshrub), but this adds a stack+code cost.
So far, strict aliasing is a net downside:
code stack
before: 34478 2624
-fno-strict-aliasing: 34502 (+0.1%) 2616 (-0.3%)
after: 34566 (+0.3%) 2632 (+0.3%)
But it's baked into the standard and we can't always rely on
-fno-strict-aliasing being available.
Been leaning towards this naming scheme. Now lfsr_omdir_* functions
match the lfsr_omdir_t type they operate on.
- Renamed lfs.opened -> lfs.omdirs
- Renamed lfsr_opened_isopen -> lfsr_omdir_isopen
- Renamed lfsr_opened_add -> lfsr_omdir_open
- Renamed lfsr_opened_remove -> lfsr_omdir_close
- Renamed lfsr_mid_isopen -> lfsr_omdir_ismidopen
Similar to the lfsr_mtree_* functions, the theory is making the grm
implicit saves some code needing to carry that extra bit of context
around.
Most of these functions don't really make sense without filesystem
context anyways.
In practice, most of these functions are inlined, so any savings the
compiler would have probably already figured out:
code stack
before: 34478 2624
after: 34478 (+0.0%) 2624 (+0.0%)
So instead of looking for another bookmark, we check to see if the next
mid's did matches.
This is in theory more robust, and can handle neighboring
directories/files without bookmarks, but adds a bit of code cost.
Deduplicating these checks into lfsr_grm_pushdid helps reduce this a
bit:
code stack
before: 34406 2624
after-no-dedup: 34526 (+0.3%) 2624 (+0.0%)
after-dedup: 34470 (+0.2%) 2624 (+0.0%)
lfsr_mtree_seek is a bit of an odd function, a hammer for too many
nails.
Using lfsr_mtree_lookup directly with manual mdir.mid manipulation gives
the internal layers more flexibility and room for optimizations.
Code changes:
code stack
before: 34426 2624
after: 34406 (-0.1%) 2624 (+0.0%)
This makes mtree implicit in most of littlefs's core functions, which
simplifies things. It also makes lfsr_mtree_traverse naming consistent
with other mtree-esque operation.
Renames:
- Renamed lfsr_fs_weight -> lfsr_mtree_weight (implicit mtree)
- Renamed lfsr_mtree_weight -> lfsr_mtree_weight_ (explicit mtree)
- Renamed lfsr_fs_traverse* -> lfsr_mtree_traverse*
- Renamed LFSR_TSTATE_* -> LFSR_MTRAVERSAL_*
Implicit mtree functions, note these are pretty much the backbone of
littlefs:
- lfsr_mtree_weight
- lfsr_mtree_lookup
- lfsr_mtree_seek
- lfsr_mtree_namelookup
- lfsr_mtree_pathlookup
- lfsr_mtree_traverse
This makes the naming is a bit inconsistent with lfsr_btree_*,
lfsr_rbyd_*, etc, but sometimes rules needs to bend a bit.
Besides, most of these functions needed access to the mroot anyways, so
it's not like they were really ever able to operate on independent
mtrees correctly.
And you can't complain about the code savings:
code stack
before: 34562 2624
after: 34426 (-0.4%) 2624 (+0.0%)
This, in theory, deduplicates this bounds check, and matches
lfsr_btree_lookup/lfsr_rbyd_lookup in behavior.
There's was only one non-asserting bounds check that could actually be
"deduplicated" (ignoring lfsr_mtree_seek for now), so this just resulted
in compiler noise:
code stack
before: 34558 2624
after: 34562 (+0.0%) 2624 (+0.0%)
So now lfsr_traversal_read will only return LFS_ERR_BUSY if LFS_T_EXCL
was provided to lfsr_traversal_open.
This means it's no longer possible to opportunistically traverse blocks,
_and_ detect mutation in the same traversal (though I suppose you could
open multiple traversals for this?), but on the flipside this
potentially frees up the implementation a bit.
This motivation for this is that LFS_ERR_BUSY is potentially confusing
and annoying to handle if you don't care about mutation.
code stack
before: 34566 2624
after: 34558 (-0.0%) 2624 (+0.0%)
blocks={0,0} technically worked, but only because the only mdirs allowed
at block 0 are mroot blocks. In theory, an mdir at block 0 could match
blocks={0,0} and clobber incorrectly, but it's not possible for such an
mdir to be in a non-trivial mtree, since blocks={0,1} are reserved for
the mrootanchor.
But bleh, that's complicated.
Setting blocks={-1,-1} makes the mdir truely invalid/unmatchable and
provides a stronger invariant at a minor code cost.
Code changes:
code stack
before: 34554 2624
after: 34566 (+0.0%) 2624 (+0.0%)
There can always be more tests, but I think these give a nice set of
coverage over corner-cases in our traversal clobbering scheme.
These did find a couple bugs:
- If we clobber an inlined mroot, we need to adjust the mid by two
mdirs, but only if there is no mtree/mdirs.
To avoid this and other mid-related headaches, we just provide the new
mid in lfsr_mdir_commit, since we always know it here.
- lfsr_mdir_commit compares mdirs by mptr, which means we need to
clobber traversal's mdir's mptrs or else lfsr_mdir_commit will clobber
already-clobbered traversals.
There may be a better way to solve this, but it will probably get into
the weeds with how lfsr_mdir_commit relies on mids vs mptrs...
Code changes:
code stack
before: 34570 2624
after: 34566 (-0.0%) 2624 (+0.0%)
Now that the dust has settled and we sort of know what the traversal
implementation will look like, we can look at the before and after to
get a rough idea of how much the traversal API actually costs:
code stack
no-traversal (before): 33886 2560
yes-traversal (after): 34566 (+2.0%) 2624 (+2.5%)
Note this still includes the annoying lfsr_btree_traverse inlining stack
cost, which isn't really the traversal API's fault and may be avoidable
in the future.
This splits LFSR_TSTATE_BTREE into separate LFSR_TSTATE_MTREE/BTREE/
OBTREE states that indicate what to do next after traversing the btree.
This removes the need to point indirectly to file's o.next pointer,
since we can just point to the file struct itself.
I've also simplified opened-file clobbering to just move to the next
opened mdir, instead of searching for another unsynced file. This
simplifies things but does mean we now need to clobber traversals when
closing non-file objects. Implicitly calling lfsr_opened_clobber in
lfsr_opened_remove solves this with very little extra code cost,
deduplicated, and gives us a stronger invariant for traversal references
to closed objects. So win win?
Oh, and all the explicit open-file clobber checks are now deduplicated
into lfsr_opened_clobber again.
These tweaks save quite a bit of code:
code stack
before: 34740 2624
after: 34570 (-0.5%) 2624 (+0.0%)
Now, lfsr_mdir_commit just clobbers all traversals associated with the
current mdir, irrespective of mid.
This makes our traversal clobbering model quite a bit simpler, drops any
mess related to bshrub staging, and allows lfsr_mdir_commit to handle
most of the clobbering logic with the exception of opened file handles.
This also fits mtree/mroot clobbering a bit better, with mtree
clobbering behaving the same as a file btree in the mroot.
The downside is we will miss more blocks during clobbered traversals,
but clobbered traversals are best effort anyways. The saved code cost
and simpler/more robust clobbering model are probably worth it.
Traversal clobbering is already complicated enough...
Code/stack changes:
code stack
before: 34716 2648
after: 34740 (+0.1%) 2624 (-0.9%)
A number of traversal changes:
- Traversal now traverses the mtree's btree (the inner btree nodes)
separately from iterating over mdirs in the mtree.
This makes resuming clobbered traversals more robust as there's less
state to worry about. It also reduces all btree traversals to a single
state which simplifies the traversal logic and _in theory_ reduces
code/RAM costs.
This does add a second O(n logbn) pass through the mtree, but this
takes the fast path since we already validated btree nodes. mtree
traversal is probably dominated by mdir fetching anyways...
- lfsr_mdir_commit no longer clobbers mid-related traversals. This was a
bit too complicated with attrs potentially inserting new mids.
Instead, it's up to upper layers to explicitly clobber traversals.
Most of these already need to update dir positions, so it's not that
much extra code, but it does add cost.
lfsr_mdir_commit still clobbers mroot/mtree related traversals.
- We now stage bshrubs in traversals during mdir compaction, so we
shouldn't need to clobber traversals when the mdir compacts.
In theory as long as we clobber traversals that reference opened
files, we should never end up being the only reference to a bshrub. So
we should be able to stage bshrubs without cost.
This is _not_ working at the moment, because we aren't updating the
actual btraversal state correctly... not sure how to fix this yet...
Code/stack changes:
code stack
before: 34682 2544
after: 34716 (+0.1%) 2648 (+4.1%)
The surprise stack cost is _very_ interesting. Where is this coming
from?
It turns out when we reduce all btree traversals to a single state, and a
single function call, GCC is happy to inline lfsr_btree_traverse
directly into lfsr_fs_traverse.
This is great for code cost, but now lfs_fs_traverse contains the entire
stack frame of lfsr_btree_traverse, which is quite large. When we called
lfsr_btree_traverse twice, this stack frame was never nested with
lfsr_mtree_lookup, but now our tools think it is...
I'm not sure how to fix this. Maybe improving our tooling to understand
shrinkwrap optimizations will find this doesn't actually cost as much?
Or maybe not since this is in a complicated switch case state machine?
We could use an explicit __attribute__((noinline)), but this sort of
heavy-handed optimization guidance has been out-of-scope for littlefs up
until now...
I'm leaving this as-is for now, but it may be worth looking this again
in the future.
This adds lfsr_opened_clobber which can be called to clobber any open
traversals related to an mid, or all traversals if mid=-1. Clobbering
here means throw away any in-progress btraversals and move to the next
mid. We need to do this in several places to avoid outdated references
to btrees.
The other option would be to treat traversals like additional unsynced
file handles, add them to the lookahead buffer, copy shrubs during
compaction, etc, but I don't think we want to pay this cost since the
underlying data is otherwise inaccessible. No reason to check/repair
blocks we're not using anymore...
To make this work, LFS_BTRAVERSAL(bid) now supports resuming from a
specific bid, in lfsr_mtraversal_t we use this to resume mtree traversal
from a specific mid when clobbered.
---
Other changes:
- lfsr_mdir_commit now marks all removed mdirs with LFS_F_ZOMBIE, and
updating related dir positions is done in lfsr_remove/lfsr_rename.
I was originally planning to use LFS_F_ZOMBIE to clobber traversals as
well, but it didn't work out.
- Added lfsr_fs_weight, which returns the effective mdir/mtree weight,
including inlined-in-mroot mdirs.
- Fixed did-mask miscalculation in lfsr_mkdir where fs/mtree weight
wasn't shifted by mdir_bits. This probably just went unnoticed during
some mid refactoring.
- Changed traversals to only traverse _unsynced_ opened files. No reason
to traverse files we know match disk. This also makes is so only
unsynced files need to worry about clobbering traversals.
This has the catch that we need to point to the traversing file handle
somehow so we can clobber correctly. The (hacky?) solution is to point
to the next pointer itself, which tells us both where to go next, and
what file handle we are currently traversing.
- Moved LFS_F_UNSYNC flags to before file operations, instead of after.
This is needed for the above traverse-unsync-only logic in case we
alloc in the middle of a file operation.
Code changes:
code stack
before: 34454 2544
after: 34682 (+0.7%) 2544 (+0.0%)
Also added some specific tests over corner cases caused by traversing
and mutating the filesystem at the same time.
Unfortunately these aren't passing yet. Our mid-clobbering logic doesn't
handle mid insertion correctly, so we end up clobbering more traversals
than we need to...
The traversal logic is a bit simpler if everything can pass around/
populate the same struct, so this reverts some changes made when
implementing lfsr_traversal_t, bringing back bid as a side-channel and
making btinfo/mtinfo typedef aliases.
btinfo/mtinfo are also required arguments for lfsr_btree_traverse/
lfsr_fs_traverse now, so it's even easier to forward these to lower
layers if they alias.
What return-pointers should/shouldn't be optional is still an open
question, but at least for btinfo/mtinfo matching lfs_stat makes sense.
This saves a bit of code/stack:
code stack
before: 34474 2552
after: 34454 (-0.0%) 2544 (-0.3%)
This sort of turned into a complete refactor of lfs_alloc in order to
move/reuse the lookahead buffer filling logic into lfsr_fs_traverse.
lfs_alloc now calls lfsr_fs_traverse to fill the lookahead buffer when
no more blocks are available, but also you can too with lfsr_traversal_t
+ LFS_T_LOOKAHEAD.
The one big caveat being if any mutation happens to the filesystem, any
incomplete lookahead needs to be tossed out. To help with this,
lfsr_traversal_read now returns LFS_ERR_BUSY (-16) instead of
LFS_ERR_NOENT (-2) if the filesystem has been modified since the
traversal was opened.
Note that by default lfsr_traversal_t will still try to keep traversing
blocks, but can be told to terminate immediately with LFS_T_EXCL.
Continuing the traversal is probably desired for checking checksums,
debugging, etc, as otherwise you could end up looping over only the
first couple blocks in a write-heavy system, but if you are trying to
populate the lookahead buffer you probably want to just abort and start
over.
I considered adding a flags field to lfs_tinfo for this, but decided
against it since it would be the only place in the current API where we
don't use error codes to convey behavior-changing information. Though
this may be worth reconsidering at some point...
---
In reworking lfs_alloc, a lot of the internal logic was broken up into
specific functions:
- lfs_alloc_ckpoint - checkpoint the allocator
- lfs_alloc_discard - discard any lookahead
- lfs_alloc_shift - discard/shift lookahead if progress can be made
- lfs_alloc_markinuse - mark a block as in-use
- lfs_alloc_markfree - mark any remaining blocks as free
- lfs_alloc_findnext - find the next free block in lookahead
If anything this probably makes lfs_alloc more readable, though the
original motivation was to allow lfsr_traversal_t to only shift/zero the
lookahead buffer if there's a chance we can make progress.
This was based on upstream work by opilat and myself.
Code changes:
code stack
before: 34226 2560
after: 34474 (+0.7%) 2552 (-0.3%)
This adds the lfsr_traversal_t object, which encapsulates a traversal
over all blocks in the filesystem.
This replaces the earlier lfs_fs_traverse function, but is sort of
"inside-out" in that instead of taking a callback, an lfsr_traversal_t
object can be read from to return lfs_tinfo structs that describe the
blocks in our system:
lfsr_traversal_open(&lfs, &t) => 0;
lfsr_traversal_read(&lfs, &t, &tinfo) => 0;
tinfo.btype => LFS_BTYPE_MDIR;
tinfo.block => 0x0;
lfsr_traversal_read(&lfs, &t, &tinfo) => 0;
tinfo.btype => LFS_BTYPE_MDIR;
tinfo.block => 0x1;
lfsr_traversal_read(&lfs, &t, &tinfo) => 0;
tinfo.btype => LFS_BTYPE_DATA;
tinfo.block => 0x42;
lfsr_traversal_read(&lfs, &t, &tinfo) => LFS_ERR_NOENT;
lfsr_traversal_close(&lfs, &t) => 0;
This is more flexible, allowing for aborted traversals, yielding,
rewinding, etc, but also more complicated to implement, since it
requires all traversal state to be stored explicitly.
Fortunately, since we needed to reimplement filesystem traversals
anyways, I was able to build this into the new system from the start
using a small state machine to drive the traversal internally. So all
that was really needed was a bit of window dressing, adding
LFS_TYPE_TRAVERSAL to track open traversals, logic to handle
invalidating traversals on file close, mutation, etc...
Which, uh, that last one is not implemented yet. Interactions with other
filesystem operations gets messy, so I figured I'd go ahead and commit
what is currently working.
Ugh, and tests. The biggest downside of adding lfsr_traversal_t is how
many more corner-cases it adds to the system...
lfsr_traversal_t is going to be a work-in-progress for a bit...
---
lfsr_traversal_t also adds a really interesting path towards more access
to advanced low-level operations, such as checking metadata/data
checksums, incrementally progressing the garbage collector, even
repairing bad metadata/data blocks eventually.
Currently implemented is LFS_T_CKMETADATA and LFS_T_CKDATA to check
metadata and data checksums respectively. This is the first feature that
actually allows you to validate data checksums.
Code changes so far:
code stack
before: 33886 2560
after: 34226 (+1.0%) 2560 (+0.0%)
We don't actually need these, all we need are utils defined for the
largest integer size we operate on, currently uint32_t.
Counterintuitively this should make it easier to adopt different integer
widths in the future.
Or maybe this will bite us when lfs_off_t >> lfs_size_t? Oh well, if
that's the case we can fix it then.
No code changes:
code stack
before: 33886 2560
after: 33886 (+0.0%) 2560 (+0.0%)
Christopher Haster