With the quantity of data being output by bench.py now, filtering ASAP
while parsing CSV files is a valuable optimization. And thanks to how
CSV files are structured, we can even avoid ever loading the full
contents into RAM.
This does end up with use filtering for defines redundantly in a few
places, but this is well worth the saved overhead from early filtering.
Also tried to clean up the plot.py/plotmpl.py's data folding path,
though that may have been wasted effort.
This is mainly to allow bench_runner to at least compile after moving
benches out of tree.
Also cleaned up lingering runner/suite munging leftover from the change
to an optional -R/--runner parameter.
This is based on how bench.py/bench_runners have actually been used in
practice. The main changes have been to make the output of bench.py more
readibly consumable by plot.py/plotmpl.py without needing a bunch of
hacky intermediary scripts.
Now instead of a single per-bench BENCH_START/BENCH_STOP, benches can
have multiple named BENCH_START/BENCH_STOP invocations to measure
multiple things in one run:
BENCH_START("fetch", i, STEP);
lfsr_rbyd_fetch(&lfs, &rbyd_, rbyd.block, CFG->block_size) => 0;
BENCH_STOP("fetch");
Benches can also now report explicit results, for non-io measurements:
BENCH_RESULT("usage", i, STEP, rbyd.eoff);
The extra iter/size parameters to BENCH_START/BENCH_RESULT also allow
some extra information to be calculated post-bench. This infomation gets
tagged with an extra bench_agg field to help organize results in
plot.py/plotmpl.py:
- bench_meas=<meas>+amor, bench_agg=raw - amortized results
- bench_meas=<meas>+div, bench_agg=raw - per-byte results
- bench_meas=<meas>+avg, bench_agg=avg - average over BENCH_SEED
- bench_meas=<meas>+min, bench_agg=min - minimum over BENCH_SEED
- bench_meas=<meas>+max, bench_agg=max - maximum over BENCH_SEED
---
Also removed all bench.tomls for now. This may seem counterproductive in
a commit to improve benchmarking, but I'm not sure there's actual value
to keeping bench cases committed in tree.
These were alway quick to fall out of date (at the time of this commit
most of the low-level bench.tomls, rbyd, btree, etc, no longer
compiled), and most benchmarks were one-off collections of scripts/data
with results too large/cumbersome to commit and keep updated in tree.
I think the better way to approach benchmarking is a seperate repo
(multiple repos?) with all related scripts/state/code and results
committed into a hopefully reproducible snapshot. Keeping the
bench.tomls in that repo makes more sense in this model.
There may be some value to having benchmarks in CI in the future, but
for that to make sense they would need to actually fail on performance
regression. How to do that isn't so clear. Anyways we can always address
this in the future rather than now.
Like many prngs, xorshift breaks down when the internal state is 0. The
common fix is to explicitly check for this and replace with a 1 when
this happens (usually when seeding, in this API we have to check every
update, this is less efficient but I don't think we really care).
As a slight tweak, this now checks for 0 but replaces it with -1. This
makes seed=0 different from seed=1, which is nice when using
seed=range(0,n) in tests/benches.
This gives the mtree a dedicated type, with direct mptrs (single mdirs)
being stored decoded, instead of encoding into leb128s. This avoids
encoding/decoding in some cases.
This change is currently a net downgrade, but only because we still have
all of the inlined btree code. Eventually this inlined btree code should
be removed:
code stack
before: 31316 2064
after: 31480 (+0.5%) 2072 (+0.4%)
Also tweaked the tests to no longer test dropping the mtree down to
zero size. Thanks to root bookmarks, we never actually do this, and it
simplifies lfsr_mdir_commit to not support this.
Before:
littlefs v2.0 0x{0,1}.232, rev 99, weight 9.256, bd 4096x256
{00a3,00a4}: 0.1 file0000 reg 32768, trunk 0xa3.a8 32768, btree 0x1a.846 32704
0.2 file0001 reg 32768, trunk 0xa3.16c 32768, btree 0xa2.be1 32704
After:
littlefs v2.0 0x{0,1}.232, rev 99, weight 9.256, bd 4096x256
{00a3,00a4}: 0.1 file0000 reg 32768, trunk 0xa3.a8, btree 0x1a.846
0.2 file0001 reg 32768, trunk 0xa3.16c, btree 0xa2.be1
Most files will have both a shrub and a btree, which makes the previous
output problematically noisy.
Unfortunately, this does lose some information: the size of the
shrub/tree, both of which may be less than the full file. But 1. this
is _technically_ redundant since you only need the block/trunk to fetch an
rbyd (though the weight is useful), and 2. The weight can still be
viewed with -s -i.
dbgbmap.py parses littlefs's mtree/btrees and displays that status of
every block in use:
$ ./scripts/dbgbmap.py disk -B4096x256 -Z -H8 -W64
bd 4096x256, 7.8% mdir, 10.2% btree, 78.1% data
mmddbbddddddmmddddmmdd--bbbbddddddddddddddbbdddd--ddddddmmdddddd
mmddddbbddbbddddddddddddddddbbddddbbddddddmmddbbdddddddddddddddd
bbdddddddddddd--ddddddddddddddddbbddddmmmmddddddddddddmmmmdddddd
ddddddddddbbdddddddddd--ddddddddddddddmmddddddddddddddddddddmmdd
ddddddbbddddddddbb--ddddddddddddddddddddbb--mmmmddbbdddddddddddd
ddddddddddddddddddddbbddbbdddddddddddddddddddddddddddddddddddddd
dddddddddd--ddddbbddddddddmmbbdd--ddddddddddddddbbmmddddbbdddddd
ddmmddddddddddmmddddddddmmddddbbbbdddddddd--ddbbddddddmmdd--ddbb
(ok, it looks a bit better with colors)
dbgbmap.py matches the layout and has the same options as tracebd.py,
allowing the combination of both to provide valuable insight into what
exactly littlefs is doing.
This required a bit of tweaking of tracebd.py to get right, mostly
around conflicting order-based arguments. This also reworks the internal
Bmap class to be more resilient to out-of-window ops, and adds an
optional informative header.
In the hack where we wait for multiple updates to fill out a full
braille/dots line we store the current pixels in a temporary array.
Unfortunately, in some cases, this is the array we modify with
updates...
A copy fixes this.
- Tried to do the rescaling a bit better with truncating divisions, so
there shouldn't be weird cross-pixel updates when things aren't well
aligned.
- Adopted optional -B<block_size>x<block_count> flag for explicitly
specifying the block-device geometry in a way that is compatible with
other scripts. Should adopt this more places.
- Adopted optional <block>.<off> argument for start of range. This
should match dbgblock.py.
- Adopted '-' for noop/zero-wear.
- Renamed a few internal things.
- Dropped subscript chars for wear, this didn't really add anything and
can be accomplished by specifying the --wear-chars explicitly.
Also changed dbgblock.py to match, this mostly affects the --off/-n/--size
flags. For example, these are all the same:
./scripts/dbgblock.py disk -B4096 --off=10 --size=5
./scripts/dbgblock.py disk -B4096 --off=10 -n5
./scripts/dbgblock.py disk -B4096 --off=10,15
./scripts/dbgblock.py disk -B4096 -n10,15
./scripts/dbgblock.py disk -B4096 0.10 -n5
Also also adopted block-device geometry argument across scripts, where
the -B flag can optionally be a full <block_size>x<block_count> geometry:
./scripts/tracebd.py disk -B4096x256
Though this is mostly unused outside of tracebd.py right now. It will be
useful for anything that formats littlefs (littlefs-fuse?) and allowing
the format everywhere is a bit of a nice convenience.
This avoids needing to return the mostly-redundant weight in the
readnext functions, and allows passing the returned data directly to
lfsr_data_read/lfsr_bd_progdata when needed.
code stack
before: 31396 2064
after: 31316 (-0.3%) 2064 (+0.0%)
- Ripped out outdated file-data representation. We don't need this.
- Changed lfsr_data_add/read/cmp to just assert when data is
concatenated data. Theoretically this is possible to implement, but
it's complicated and we never use it, so all it is is a waste of
code size...
- Added implicitly zero-filled hole representation, though this isn't
adopted in the code yet.
- Added lfsr_data_truncate/fruncate, these are really useful for
shrub/tree carving/coalescing.
---
New lfsr_data_t encoding, sign(size) indicates if the data is
on-disk/in-device, and a mode field indicates how in-device data should
be parsed:
sign(size)=1 => on-disk:
.---+---+---+---. .....
|1| size | ..'' ''..
+---+---+---+---+ : : :
| block ------+->| ..:|
+---+---+---+---+ | |......( )::::::|
| off -------' |:::' : |
'---+---+---+---' :' : :
''.. :.''
'''''
sign(size)=0, mode=0 => in-device buffer:
.---+---+---+---. .---+---+---+---.
|0| size | .>| data... |
+---+---+---+---+ | ' . '
|m=0| | | ' . '
+---+---+---+---+ | ' '
| ptr -------' ' '
'---+---+---+---' '---+---+---+---'
sign(size)=0, mode=1 => hole
.---+---+---+---.
|0| size |
+---+---+---+---+
|m=1| |
+---+ +
| |
'---+---+---+---'
sign(size)=0, mode=2 => inlined
.---+---+---+---.
|0| size |
+---+---+---+---+
|m=2| inlined d |
+---+ +
| ata... |
'---+---+---+---'
sign(size)=0, mode=3 => concatenated datas:
.---+---+---+---. .---+---+---+---.
|0| size | .>| data |
+---+---+---+---+ | + +
|m=3| c | | | | |
+---+---+---+---+ | + +
| ptr -------' | |
'---+---+---+---' +---+---+---+---+
| data |
+ +
| |
+ +
| |
+---+---+---+---+
' . '
' . '
' . '
' '
' '
'---+---+---+---'
---
Code/RAM changes:
code stack
before: 31952 2056
after: 31396 (-1.7%) 2064 (+0.4%)
I think the increased RAM cost is due to lfsr_data_add/truncate/fruncate
passing lfsr_data_t around by value, and GCC not being able to optimize
this very well since it's 3 words. I think most move optimizations stop
after 2-words...
- Added lfsr_shrub_lookupnext/lfsr_tree_lookupnext to deduplicate
the various tree lookups that need to support inlined sprouts/bptrs.
Also moved implicit bptr dereferencing here, though this may need
to be tweaked a bit to support data checksumming.
- Moved inlined sprouts/bptrs into readnext (well, lookupnext really).
This simplifies things anywhere we just need to read data from these
trees.
- Renamed lfsr_file_carveshrub/lfsr_file_carvetree ->
lfsr_shrub_carve/lfsr_tree_carve and changed parameters appropriately.
Though these aren't so clear cut. lfsr_shrub_carve still needs the
related file structure to know which mdir to commit to.
lfsr_shrub_carve will also need significant tweaking to support
recovery from failed file writes.
This also ends up losing the shrub/tree lookup reuse. There might still
be a way to deduplicate the read logic after shrub/tree lookup, but it's
probably not worth it considering this logic has become a rather small
part of the lookupnext/readnext machinery
Both shrubs and trees end up calling rbyd/btree lookupnext anyways...
In theory, it should be perfectly fine to read from a filesystem with an
invalid grm mode, so lfsr_data_readgrm has been tweaked to return
LFS_ERR_INVAL in that case.
That being said, we don't actually support read-only mounts, so the end
behavior is still the same, but this lays the groundwork for readonly
mounts in the future.
This also makes it so invalid grm modes result in LFS_ERR_INVAL when
read-only mounts aren't supported, which is what it should be anyways.
The biggest change here is the breaking up of the FLAGS config into
RFLAGS/WFLAGS/OFLAGS. This is directly inspired by, and honestly not
much more than a renaming, of the compat/ro_compat/incompat flags found
in Linux/Unix/POSIX filesystems.
I think these were first introduced in ext2? But I need to do a bit more
research on that.
RFLAGS/WFLAGS/OFLAGS provide a much more flexible, and extensible,
feature flag mechanism than the previous minor version bumps.
The (re)naming of these flags is intended to make their requirements
more clear. In order to do the relevant operation, you must understand
every flag set in the relevant flag:
- RFLAGS / incompat flags - All flags must be understood to read the
filesystem, if not understood the only possible behavior is to fail.
- WFLAGS / ro-compat flags - All flags must be understood to write to the
filesystem, if not understood the filesystem may be mounted read-only.
- OFLAGS / compat flags - Optional flags, if not understood the relevant
flag must be cleared before the filesystem can be written to, but other
than that these flags can mostly be ignored.
Some hypothetical littlefs examples:
- RFLAGS / incompat flags - Transparent compression
Is this the same as a major disk-version break? Yes kinda? An
implementation that doesn't understand compression can't read the
filesystem.
On the other hand, it's useful to have a filesystem that can read both
compressed and uncompressed variants.
- WFLAGS / ro-compat flags - Closed block-map
The idea behind a closed block-map (currently planned), is that
littlefs maintains in global space a complete mapping of all blocks in
use by the filesystem.
For such a mapping to remain consistent means that if you write to the
filesystem you must understand the closed block-map. Or in other
words, if you don't understand the closed block-map you must not write
to the filesystem.
Reading, on the other hand, can ignore many such write-related
auxiliary features, so the filesystem can still be read from.
- OFLAGS / compat flags - Global checksums
Global checksums (currently planned) are extra checksums attached to
each mdir that when combined self-validate the filesystem.
But if you don't understand global checksums, you can still read and
write the filesystem without them. The only catch is that when you write
to the filesystem, you may end up invalidating the global checksum.
Clearing the global checksum bit in the OFLAGS is a cheap way to
signal that the global checksum is no longer valid, allowing you to
still write to the filesystem without this optional feature.
Other tweaks to note:
- Renamed BLOCKLIMIT/DISKLIMIT -> BLOCKSIZE/BLOCKCOUNT
Note these are still the _actual_ block_size/block_count minus 1. The
subtle difference here was the original reason for the name change,
but after working with it for a bit, I just don't think new, otherwise
unused, names are worth it.
The minus 1 stays, however, since it avoids overflow issues at
extreme boundaries of powers of 2.
- Introduces STAGLIMIT/SATTRLIMIT, sys-attribute parallels to
UTAGLIMIT/UATTRLIMIT.
These may be useful if only uattrs are supported, or vice-versa.
- Dropped UATTRLIMIT/SATTRLIMIT to 255 bytes.
This feels extreme, but matches NAMELIMIT. These _should_ be small,
and limiting the uattr/sattr size to a single-byte leads to really
nice packing of the utag+uattrsize in a single integer.
This can always be expanded in the future if this limit proves to be a
problem.
- Renamed MLEAFLIMIT -> MDIRLIMIT and (re?)introduced MTREELIMIT.
These may be useful to limiting the mtree when needed, though it's not
clear the exact use case quite yet.
It's probably better to have a separate names for a tag category and any
specific name, but I can't think of a better name for this tag, and I
hadn't noticed that I was already ignoring the C prefix for CCKSUM tags
in many places.
NAME/CKSUM now mean both the specific tag and tag category, which is a
bit of a hack since both happen to be the 0th-subtype of their
categories.
I may be overthinking things, but I'm guessing of all the possible tag
modes we may want to add in the future, we will mostly like want to add
something that looks vaguely tag like. Like the shrub tags, for example.
It's beneficial, ordering wise, for these hypothetical future tags to
come before the cksum tags.
Current tag modes:
0x0ttt v--- tttt -ttt tttt normal tags
0x1ttt v--1 tttt -ttt tttt shrub tags
0x3tpp v-11 tttt ---- ---p cksum tags
0x4kkk v1dc kkkk -kkk kkkk alt tags
Unfortunately, the tests are starting to take a painfully long time to
run. Some of this is because, in order to get interesting file
topologies, we need to move a ton of data around, but some of this is
also because our current write implementation has some problematically
expensive corner cases.
I have quite a few ideas on how to improve this, but in the meantime the
tests needed to be aggressively trimmed in order to keep development
tolerable (A happy developer is a productive developer).
This mainly meant:
- Disabled powerloss testing on file tests for now.
The reality is that naivly powerloss testing the file tests, i.e.
just truncating the file after each restart, provides very little
value and adds an extreme amount of runtime.
Removed for now. Most of the powerloss file creation concerns are
covered in the dtree tests, and we should eventually add powerloss
tests tailored to recovering files after powerloss instead of just
truncating.
- Avoided tiny fragment sizes with large file sizes.
Tiny fragments are a degenerate case and end up with excessive
overhead (1 byte fragment => 41x overhead!). But they are useful for
revealing subtle bugs. Still, it just doesn't make sense time-wise to
test with tiny fragments once the file size exceeds ~1 block.
- Limited fuzz tests to cover fewer random seeds.
We can increase these if performance improves, but even if not, we can
run these individually with a high number of seeds in CI.
Also fixed a number of bugs found by the extended testing, which is
always a good sign:
- Yet another `lfsr_data_size(&data)` vs `data.u.disk.size` typo.
This is the first time I've seen a real world argument for private
struct/class fields, but I am still against the concept.
- Fixed delta/weight miscalculation when tree-carving a left sibling.
- Fixed missing offset in hole writing during block writes.
- Worked around lfsr_file_readnext's reliance on file->size when we are
using it to write to a block. This may be more a hack than a good
long term solution though.
- Checkpointed the allocator in both lfsr_file_write and lfsr_file_sync.
Otherwise calling lfsr_file_write repeatedly can easily trigger an
incorrect ENOSPC.
- Correctly reverted both shrubs and btrees in truncate/fruncate
This gets a bit more complicated in fruncate, since either one of the
two, or both, can revert.
truncate/fruncate probably deserve a bit more work around reversions
to simpler data structures, as is.
- Added handling of shrub overflows during fruncate.
Notably not possible with truncate, shrub overflows require that we
1. flush the shrub, 2. fruncate the tree, 3. and make sure any side
effects to the buffer are handled correctly.
Yes, erases are the more costly operation that we should highlight. But,
aside from broken code, you can never prog more than you erase.
This makes it more useful to priortize progs over erases, so erases
without an overlaying prog show up as a relatively unique blue,
indicating regions of memory that have been erased but not progged.
Too many erased-but-not-progged regions indicate a potentially wastefull
algorithm.
The original name was a bit of a mouthful.
Also dropped the default crystal_size in the test/bench runners
block_size/4 -> block_size/8. I'm already noticing large amounts of
inflation when blocks are fragmented, though I am experimenting with a
rather small fragment_size right now.
Future benchmarks/experimentation is required to figure out good values
for these.
Note this is really just a proof of concept, and tests are not passing.
There's also a number of hacks holding everything together and really
need to be cleaned up.
I was hoping it would be possible to deduplicate the carveshrub/carvetree
functions the same way shrub/tree readnext functions were deduplicated.
These both share a lot of subtle logic, and in theory operated on minor
variations of the same underlying rbyd structure, but in practice
several issues get in the way:
- While the logic is the same, the way changes are played out is very
different: btrees commit attributes to the btree immediately, whereas
shrubs build up a bounded attr list to commit to the shrub via an mdir
commit.
In theory shrubs could be committed immediately, but it would be
wasteful. And btrees can't commit a bounded attribute list because 1.
rm attrs may need to be split into an unbounded number accross
multiple rbyds, 2. fragmenting blocks may create an unbounded
headache, and 3. attribute lists can't span multiple rbyds so we'd
need to manually play them out anyways.
- We need to allocate a new btree in carvetree, but in carveshrub we
defer allocation to mdir commit time (because of the potential for
failed commits). This complicates things.
- The unions with sprouts/direct bptrs are often very similar, but need
different handling when carving. This gets a bit tricky.
- In theory you could switch between building attrs for shrubs and
immediate commits for btrees, but since the immediate commits _change
the tree_, the carving math changes subtlely.
- carveshrub needs to do several auxilary things: track the shrub estimate,
build attrs in RAM, etc. carvetree needs to do several auxilary
things: dereference bptrs, fragment bptrs, allocate new btrees, etc.
If these can be deduplicated it would likely result in code savings,
but also risks increased RAM costs from trying to do too many things
at once.
The cost of two functions may also be more cognitive than real, since
the subtletly here is just math. And computers happen to be pretty
good at math.
Though this concern may be unfounded, and deduplicated these functions
is still enticing and an interesting idea to explore.
I've already noticed some concerning performance once a write exceeds
our crystallization threshold. This makes sense, as our current strategy
is to completely rewrite any data region over our crystallization
threshold. But I wonder if there's a way to exclude the first block in
our region from the crystallization heuristic...
Anyways, some good progress here, but more work to be done.
The attempt to implement in-rbyd data slicing, being lazily coalesced
during rbyd compaction, failed pretty much completely.
Slicing is a very enticing write strategy, getting both minimal overhead
post-compaction and fast random write speeds, but the idea has some
fundamental conflicts with how we play out attrs post-compaction.
This idea might work in a more powerful filesystem, but brings back the
need to simulate rbyds in RAM, which is something I really don't want to
do (complex, bug-prone, likely adds code cost, may not even be tractable).
So, third time's the charm?
---
This new write strategy writes only datas and bptrs, and avoids dagging
by completely rewriting any regions of data larger than a configurable
crystallization threshold.
This loses most of the benefits of data crystallization, random writes
will now usually need to rewrite a full block, but as a tradeoff our
data at rest is always stored with optimal overhead.
And at least data crystallization still saves space when our data isn't
block aligned, or in sparse files. From reading up on some other
filesystem designs it seems this is a desirable optimization sometimes
referred to as "tail-packing" or "block suballocation"
Some other changes from just having more time to think about the
problem:
1. Instead of scanning to figure out our current crystal size, we can
use a simple heuristic of 1. look up left block, 2. look up right
block, 3. assume any data between these blocks contribute to our
current crystal.
This is just a heuristic, so worst case you write the first and last
byte of a block which is enough to trigger compaction into a block.
But on the plus side this avoids issues with small holes preventing
blocks from being formed.
This approach brings the number of btree lookups down from
O(crystallize_size) to 2.
2. I've gone ahead and dropped the previous scheme of coalesce_size
+ fragment_size and instead adopted a single fragment_size that
controls the size of, well, fragments, i.e. data elements stored
directly in trees.
This affects both the inlined shrub as well as fragments stored in
the inner nodes of the btree. I believe it's very similar to what is
often called "pages" in logging filesystems, though I'm going to
avoid that term for now because it's a bit overloaded.
Previously, neighboring writes that, when combined, would exceed our
coalesce_size, they just weren't combined. Now they are combined up
to our fragment size, potentially splitting the right fragment.
Before (fragment_size=8):
.---+---+---+---+---+---+---+---.
| 8 bytes |
'---+---+---+---+---+---+---+---'
+
.---+---+---+---+---.
| 5 bytes |
'---+---+---+---+---'
=
.---+---+---+---+---+---+---+---+---+---.
| 5 bytes | 5 bytes |
'---+---+---+---+---+---+---+---+---+---'
After:
.---+---+---+---+---+---+---+---.
| 8 bytes |
'---+---+---+---+---+---+---+---'
+
.---+---+---+---+---.
| 5 bytes |
'---+---+---+---+---'
=
.---+---+---+---+---+---+---+---+---+---.
| 8 bytes |2 bytes|
'---+---+---+---+---+---+---+---+---+---'
This leads to better fragment alignment (much like our block
strategy), and minimizes tree overhead.
Any neighboring data to the right is only coalesced if it fits in the
current fragment, or would be rewritten (carved) anyways, to avoid
unnecessary data rewriting.
For example (fragment_size=8):
.---+---+---+---+---+---+---+---+---+---+---+---+---+---.
| 6 bytes | 6 bytes |2 bytes|
'---+---+---+---+---+---+---+---+---+---+---+---+---+---'
+
.---+---+---+---+---.
| 5 bytes |
'---+---+---+---+---'
=
.---+---+---+---+---+---+---+---+---+---+---+---+---+---.
| 8 bytes | 4 bytes |2 bytes|
'---+---+---+---+---+---+---+---+---+---+---+---+---+---'
Other than these changes this commit is mostly a bunch of carveshrub
rewriting again, which continues to be nuanced and annoying to get
bug free.
- -> lfsr_shrub_t
- -> lfsr_tree_t
The idea here is to adopt "shrub" as an umbrella term for the
shrub/sprout union, and "tree" as an umbrella term for the bptr/btree
union. I think this is a bit better than calling shrub/sprout "inlined"
which is a _very_ overloaded term in this codebase (inlined in the tree?
the mdir? inlined in the C struct?).
But already there are some pretty fundamental problems.
The main issue is that, while we correctly dereference slices during
compaction, pending commits that get delayed after compaction still
point to the old block. I'm not sure there's an easy way around this
aside from aborting compaction commits or fully simulating commits,
both of which seem too costly to implement...
Also coalescing during compaction is flawed as well, since our
attributes will be outdated by the time they are committed if there is a
compaction...
Looks like it's back to the drawing board. Either our approach to
compaction needs to change, or this slice/coalescing work needs to be
reverted/redesigned...
Note this is already showing better code reuse, which is a good sign,
though maybe that's just the benefit of reimplementing similar logic
multiple times.
Now both reading and carving end up in the same lfsr_btree_readnext and
lfsr_btree_buildcarve functions for both btrees and shrubs. Both btrees
and shrubs are fundamentally rbyds, so we can share a lot of
functionality as long as we redirect to the correct commit function at
the last minute. This surprising opportunity for deduplication was
noticed while putting together the dbg scripts.
Planned logic (not actual function names):
lfsr_file_readnext -> lfsr_shrub_readnext
| |
| v
'---------> lfsr_btree_readnext
lfsr_file_flushbuffer -> lfsr_shrub_carve ------------.
.---------------------' |
v v
lfsr_file_flushshrub -> lfsr_btree_carve -> lfsr_btree_buildcarve
Though the btree part of the above statement is only a hypothetical at
the moment. Not even the shrubs can survive compaction now.
The reason is the new SLICE tag which needs low-level support in rbyd
compact. SLICE introduces indirect refernces to data located in the same
rbyd, which removes any copying cost associated with coalescing.
Previously, a large coalesce_size risked O(n^2) runtime when
incrementally append small amounts of data, but with SLICEs we can defer
coalescing to compaction time, where the copy is effectively free.
This compaction-time-coalescing is also hypothetical, which is why our
tests are failing. But the theory is promising.
I was originally against this idea because of how it crosses abstraction
layers, requiring some very low-level code that absolutely can not be
omitted in a simpler littlefs driver. But after working on the actual
file writing code for a while I've become convinced the tradeoff is
worth it.
Note coalesce_size will likely still need to be configurable. Data in
fragmenting/sparse btrees is still susceptible to coalescing, and it's
not clear the impacts of internal fragmentation when data sizes approach
the hard block_size/2 limit.
My current thinking is that these are conceptually different types, with
BTREE tags representing the entire btree, and BRANCH tags representing
only the inner btree nodes. We already have multiple btree tags anyways:
btrees attached to files, the mtree, and in the future maybe a bmaptree.
Having separate tags also makes it possible to store a btree in a btree,
though I don't think we'll ever use this functionality.
This also removes the redundant weight field from branches. The
redundant weight field is only a minor cost relative to storage, but it
also takes up a bit of RAM when encoding. Though measurements show this
isn't really significant.
New encodings:
btree encoding: branch encoding:
.---+- -+- -+- -+- -. .---+- -+- -+- -+- -.
| weight | | blocks |
+---+- -+- -+- -+- -+ ' '
| blocks | ' '
' ' +---+- -+- -+- -+- -+
' ' | trunk |
+---+- -+- -+- -+- -+ +---+- -+- -+- -+- -'
| trunk | | cksum |
+---+- -+- -+- -+- -' '---+---+---+---'
| cksum |
'---+---+---+---'
Code/RAM changes:
code stack
before: 30836 2088
after: 30944 (+0.4%) 2080 (-0.4%)
Also reordered other on-disk structs with weight/size, so such structs
always have weight/size as the first field. This may enable some
optimizations around decoding the weight/size without needing to know
the specific type in some cases.
---
This change shouldn't have affected functionality, but it revealed a bug
in a dtree test, where a did gets caught in an mdir split and the split
name makes the did unreachable.
Marking this as a TODO for now. The fix is going to be a bit involved
(fundamental changes to the opened-mdir list), and similar work is
already planned to make removed files work.
Since we need an bptr type internally, a block pointer, which is a bit
more complicated than just a single address, calling our mdir pairs
mptrs makes sense.
This is a pretty big rewrite, but is necessary to avoid "dagging".
"Dagging" (I just made this term up) is when you transform a pure tree
into a directed acyclic graph (DAG). Normally DAGs are perfectly fine in
a copy-on-write system, but in littlefs's cases, it creates havoc for
future block allocator plans, and it's interaction with parity blocks
raises some uncomfortable questions.
How does dagging happen?
Consider an innocent little btree with a single block:
.-----.
|btree|
| |
'-----'
|
v
.-----.
|abcde|
| |
'-----'
Say we wanted to write a small amount of data in the middle of our
block. Since the data is so small, the previous scheme would simply
inline the data, carving the left and right sibling (in the case the
same block) to make space:
.-----.
|btree|
| |
'-----'
.' v '.
| c' |
'. .'
v v
.-----.
|ab de|
| |
'-----'
Oh no! A DAG!
With the potential for multiple pointers to reference the same block in
our btree, some invariants break down:
- Blocks no longer have a single reference
- If you remove a reference you can no longer assume the block is free
- Knowing when a block is free requires scanning the whole btree
- This split operation effectively creates two blocks, does that mean
we need to rewrite parity blocks?
---
To avoid this whole situation, this commit adopts a new crystallization
algorithm.
Instead of allowing crystallization data to be arbitrarily fragmented,
we eagerly coalesce any data under our crystallization threshold, and if
we can't coalesce, we compact everything into a block.
Much like a Knuth heap, simply checking both siblings to coalesce has
the effect that any data will always coalesce up to the maximum size
where possible. And when checking for siblings, we can easily find the
block alignment.
This also has the effect of always rewriting blocks if we are writing a
small amount of data into a block. Unfortunately I think this is just
necessary in order to avoid dagging.
At the very least crystallization is still useful for files not quite
block aligned at the edges, and sparse files. This also avoids concerns
of random writes inflating a file via sparse crystallization.
- Merged lfsr_file_read_ back into lfsr_file_read, I don't think we need
stateless reads in the end.
- Tweaked reads to use conservative hints instead of just filling all
cache lines with whatever is in the retrieved datas.
- Switched to if/else for sprout/shrub and bptr/btree checks. Though
this had no affect on code size, which isn't too surprising.
Now when you mount littlefs, the debug print shows a bit more info:
lfs.c:7881:debug: Mounted littlefs v2.0 0x{0,1}.c63 w43.256, bd 4096x256
To dissassemble this a bit:
littlefs v2.0 0x{0,1}.c63 w43.256, bd 4096x256
^ ^ '-+-' ^ ^ ^ ^ ^
'-|-----|----|---|---|--------|---|-- major version
'-----|----|---|---|--------|---|-- minor version
'----|---|---|--------|---|-- mroot blocks
| | | | | (1st is active)
'---|---|--------|---|-- mroot trunk
'---|--------|---|-- mtree weight
'--------|---|-- mleaf weight
'---|-- block size
'-- block count
dbglfs.py also shows the block device geometry now, as read from the
mroot:
$ ./scripts/dbglfs.py disk -B4096
littlefs v2.0 0x{0,1}.c63, rev 1, weight 43.256, bd 4096x256
...
This may be over-optimizing for testing, but the reason the mount debug
is only one line is to avoid slowing down/messying test output. Both
powerloss testing and remounts completely fill the output with mount
prints that aren't actually all that useful.
Also switching to prefering parens in debug info mainly for mismatched
things.
Mainly aligning things, it was easy for the previous repr to become a
visual mess.
This also represents the config more like how we represent other tags,
since they've changed from a monolithic config block to separate
attributes.
This a compromise between padding the tag repr correctly and parsing
speed.
If we don't have to traverse an rbyd (for, say, tree printing), we don't
want to since parsing rbyds can get quite slow when things get big
(remember this is a filesystem!). This makes tag padding a bit of a hard
sell.
Previously this was hardcoded to 22 characters, but with the new file
struct printing it quickly became apparently this would be a problematic
limit:
12288-15711 block w3424 0x1a.0 3424 67 64 79 70 61 69 6e 71 gdypainq
It's interesting to note that this has only become an issue for large
trees, where the weight/size in the tag can be arbitrarily large.
Fortunately we already have the weight of the rbyd after fetch, so we
can use a heuristic similar to the id padding:
tag padding = 21 + nlog10(max(weight,1)+1)
---
Also dropped extra information with the -x/--device flag. It hasn't
really been useful and was implemented inconsistently. Maybe -x/--device
should just be dropped completely...
You can now pass -s/--structs to dbglfs.py to show any file data
structures:
$ ./scripts/dbglfs.py disk -B4096 -f -s -t
littlefs v2.0 0x{0,1}.9cf, rev 3, weight 0.256
{0000,0001}: -1.1 hello reg 128, trunk 0x0.993 128
0000.0993: .-> 0-15 shrubinlined w16 16 6b 75 72 65 65 67 73 63 kureegsc
.-+-> 16-31 shrubinlined w16 16 6b 65 6a 79 68 78 6f 77 kejyhxow
| .-> 32-47 shrubinlined w16 16 65 6f 66 75 76 61 6a 73 eofuvajs
.-+-+-> 48-63 shrubinlined w16 16 6e 74 73 66 67 61 74 6a ntsfgatj
| .-> 64-79 shrubinlined w16 16 70 63 76 79 6c 6e 72 66 pcvylnrf
| .-+-> 80-95 shrubinlined w16 16 70 69 73 64 76 70 6c 6f pisdvplo
| | .-> 96-111 shrubinlined w16 16 74 73 65 69 76 7a 69 6c tseivzil
+-+-+-> 112-127 shrubinlined w16 16 7a 79 70 61 77 72 79 79 zypawryy
This supports the same -b/-t/-i options found in dbgbtree.py, with the
one exception being -z/--struct-depth which is lowercase to avoid
conflict with the -Z/--depth used to indicate the filesystem tree depth.
I think this is a surprisingly reasonable way to show the inner
structure of files without clobbering the user's console with file
contents.
Don't worry, if clobbering is desired, -T/--no-truncate still dumps all
of the file content.
Though it's still up to the user to manually apply the sprout/shrub
overlay. That step is still complex enough to not implement in this
tool yet.
I
Oh hey, it's that piece of complexity I was worried about.
The problem was that the position calculation for new appended
right_data depended on left_overlap, which fell out of sync when
transitioning from sprout->shrub.
The fix here is to keep left_overlap/right_overlap up to date with the
model that the sprout->shrub transition is effectively doing a
shrub-wide rm first.
Hacky, but hopefully avoids bugs in the future by keeping all of these
variables in a reasonable state...
There may be a simpler way to think about how this code should function,
but I just can't see it. This may deserve a rewrite in the future.
Noticed a lot of duplicate conditions, so tried merging these two code
paths. This does risk a difficult to read/maintain function, since there
are some rather tricky subtleties with the sprout -> shrub transition.
On the other hand, the code reuse does mean less conditions to worry
about.
Merging these code paths also saves a bit of code:
code stack
before: 30960 2256
after: 30700 (-0.8%) 2256 (+0.0%)
Note we still end up with a shrub, even if the file could revert back to
a sprout. This is just a simplification for the inlined file logic. We
never implicitly revert to a sprout.
This mostly figures out how things might work with coalescing, without
fully implementing coalescing yet.
One thing noteworthy, previously when carving right data, we would
remove the right data and rewrite it. This was to accomidate implicit
splits:
buf: [bbbb]
shrub: [llrrrrrr]
1. rm [ll]
2. append [llrr]
3. append [llbbbbrr]
An implicit split being when the left sibling and right sibling are the
same data
buf: [bbbb]
shrub: [llllllll]
1. carve [ll]
2. append [llbbbb]
3. append [llbbbbll]
By separating out the split logic, this rm can be avoided:
buf: [bbbb]
shrub: [llrrrrrr]
1. carve [llrr]
2. append [llbbbbrr]
At the cost of making our implicit split have more steps (in code),
though, I believe it does have less subtle/more understandable behavior:
buf: [bbbb]
shrub: [llllllll]
1. carve [ll]
2. append [llll]
3. append [llbbbbll]
As a plus, we avoid looking up the same sibling twice when doing
implicit splits.
lfsr_file_carveinlined writes data into a shrub, while handling both the
carving logic of data we might be overlapping, and any hole logic we
need to fill out the tree.
This provides a nice, relatively simple but flexible, operation for all
shrub updates:
static int lfsr_file_carveinlined(lfs_t *lfs, lfsr_file_t *file,
lfs_off_t pos, lfs_off_t weight, lfs_soff_t delta,
lfsr_data_t data);
I'm quite happy with how these internal carveinlined/carvebtree
functions are coming together. It's nice to have all of that logic in
one place, even if it's a bit complex.
Ended up changing the name of lfsr_mtree_traversal_t -> lfsr_traversal_t,
since this behaves more like a filesytem-wide traversal than an mtree
traversal (it returns several typed objects, not mdirs like the other
mtree functions for one).
As a part of this changeset, lfsr_btraversal_t (was lfsr_btree_traversal_t)
and lfsr_traversal_t no longer return untyped lfsr_data_ts, but instead
return specialized lfsr_{b,t}info_t structs. We weren't even using
lfsr_data_t for its original purpose in lfsr_traversal_t.
Also changed lfsr_traversal_next -> lfsr_traversal_read, you may notice
at this point the changes are intended to make lfsr_traversal_t look
more like lfsr_dir_t for consistency.
---
Internally lfsr_traversal_t now uses a full state machine with its own
enum due to the complexity of traversing the filesystem incrementally.
Because creating diagrams is fun, here's the current full state machine,
though note it will need to be extended for any
parity-trees/free-trees/etc:
mrootanchor
|
v
mrootchain
.-' |
| v
| mtree ---> openedblock
'-. | ^ | ^
v v | v |
mdirblock openedbtree
| ^
v |
mdirbtree
I'm not sure I'm happy with the current implementation, and eventually
it will need to be able to handle in-place repairs to the blocks it
sees, so this whole thing may need a rewrite.
But in the meantime, this passes the new clobber tests in test_alloc, so
it should be enough to prove the file implementation works. (which is
definitely is not fully tested yet, and some bugs had to be fixed for
the new tests in test_alloc to pass).
---
Speaking of test_alloc.
The inherent cyclic dependency between files/dirs/alloc makes it a bit
hard to know what order to test these bits of functionality in.
Originally I was testing alloc first, because it seems you need to be
confident in your block allocator before you can start testing
higher-level data structures.
But I've gone ahead and reversed this order, testing alloc after
files/dirs. This is because of an interesting observation that if alloc
is broken, you can always increase the test device's size to some absurd
number (-DDISK_SIZE=16777216, for example) to kick the can down the
road.
Testing in this order allows alloc to use more high-level APIs and
focus on corner cases where the allocator's behavior requires subtlety
to be correct (e.g. ENOSPC).
This was a cludge due to needing lfs->mtree initialized to traverse the
mtree, the assumption being that future traversals should strictly
update the mtree/mroot to the existing state.
Moving code around (and adopting an actual state machine, which will be
needed for btree traversal) made this no longer necessary.
Now the mtree/mroot is only initialized in lfsr_mountinited, as it
should be.
Still needs testing, though the byte-level fuzz tests were already causing
blocks to crystallize. I noticed this because of test failures which are
fixed now.
Note the block allocator currently doesn't understand file btrees. To
get the current tests passing requires -DDISK_SIZE=16777216 or greater.
It's probably also worth noting there's a lot that's not implemented
yet! Data checksums and write validation for one. Also ecksums. And we
should probably have some sort of special handling for linear writes so
linear writes (the most common) don't end up with a bunch of extra
crystallizing writes.
Also the fact that btrees can become DAGs now is an oversight and a bit
concerning. Will that work with a closed allocator? Block parity?
First, realized the the LFS_UNREACHABLE logic was flipped after a
confusing test bug (damn double negatives). But also realized LFS_ASSERT
could be tweaked to "call" __builtin_unreachable() on assert failure to
act as a sort of compiler hint.
Turns out this hint saves a little bit of code, note both builds have
LFS_UNREACHABLE fixed:
code stack
without __builtin_unreachable: 28408 1928
with __builtin_unreachable: 28324 (-0.3%) 1920 (+0.0%)
Since __builtin_unreachable is a compiler extension, its usage respects
LFS_NO_INTRINSICS.
Added lfsr_bptr_t to represent block pointers (maybe we should rename
mblocks back to mptr), added fetching of btrees/bptrs in
lfsr_file_opencfg, added estimate tracking to our shrubs so we actually
know when to create a btree, and implemented most of the high-level
btree logic.
It's not working yet, but the biggest idea introduced here is how we
handle block alignment.
See, we really don't want awkward btree topologies to form where small
amounts of data get stuck between blocks:
.-----.--.-----.
| | | |
| | | |
'-----'--'-----'
This is wasteful, as the middle bit of data either gets represented as a
full block with its data partially covered, or as data inlined in the
btree, which comes with ~2x overhead.
The solution here is to scan for a block on either the left or right to
derive our block alignment from.
Unfortunately, since our sibling blocks could have been carved, this
requires scanning all the way from pos-2*B+1 to pos+2*B-1, a total of
4*B-2, to make sure we find a sibling if there is one.
worst case left worst case right
.-----.-----. .-----.-----.
| xxxx| | |p |xxxxx|
|xxxxx| p| | |xxxx |
'-----'-----' '-----'-----'
'----+----' '----+----'
pos-2*bs+1 pos+2*bs-1
Fortunately, at this stage, data should have had many chances to
coalesce, so hopefully the actual scan overhead should be much smaller
in practice.
Writing data to a file linearly, for example, only needs a single lookup
to find the previous block.
So now instead of needing:
./scripts/test.py ./runners/test_runner test_dtree
You can just do:
./scripts/test.py test_dtree
Or with an explicit path:
./scripts/test.py -R./runners/test_runner test_dtree
This makes it easier to run the script manually. And, while there may be
some hiccups with the implicit relative path, I think in general this will
make the test/bench scripts easier to use.
There was already an implicit runner path, though only if the test suite
was completely omitted. I'm not sure that would ever have actually
been useful...
---
Also increased the permutation field size in --list-*, since I noticed it
was overflowing.
Christopher Haster