Good news! test_wl_orphanzombie_fuzz found a rare and difficult to reach
bug. Bad news, it found the bug only after changing littlefs's initial
revision count, which is about as unrelated a change as you can possibly
have...
Oh well, at least now we can add specialized tests targeting this (and
push them to hopefully cover anything similar):
- test_files_mv_split
- test_files_mv_split_backwards
- test_forphans_rename_split
- test_forphans_rename_split_backwards
The bug occurs when a rename of a file to/from the same mdir triggers an
mdir split, and you have that file opened, and the opened file handle
tracks a bshrub or bsprout. Oh, and if that wasn't unlikely enough, this
only breaks when the rename crosses from the new-right-sibling to the
new-left-sibling (inverse order of mdir split compacts), left-to-right
is fine.
The problem is how we stage bshrubs/bsprouts. bshrubs/bsprouts are a bit
tricky in that several unrelated operations can change their location,
sometimes multiple times in the same lfsr_mdir_commit call:
- mdir compaction - move bshrub/bsprout to new mdir
- bshrub commit - append a new shrub trunk
- rename commit - move bshrub/bsprout to a new mdir/mid
To keep track of all of this, lfsr_file_t has a dedicated field,
file.bshrub_, that holds the bshrub/bsprout's new location during
lfsr_mdir_commit. This may be changed multiple times, but the last
change wins.
This works as long as changes occur in an expected order. Importantly,
commits that change the bshrub, such as rename, need to play out after
compactions.
It turns out this is violated when splitting an mdir.
Because we have single pcache, we need to write out the entire compact +
commit of each mdir at a time. When we split, we arbitrarily do this
left-to-right, which results in left commits being played out before
right compactions.
Here's how things play out when we rename right-to-left:
1. commit rename -> bshrub = src mid, orig mdir
2. commit fails because of ERANGE
3. compact left mdir -> bshrub = src mid, left mdir
4. commit left mdir -> bshrub = dst mid, left mdir
5. compact right mdir -> bshrub = src mid, right mdir
6. commit right mdir (skips rename)
Oh no! Our staged bshrub ends up with the wrong location.
---
This is quite tricky to solve. We can't just play out the rename again
on the right mdir, because we've already lost the new bshrub trunk at
this point. Other solutions involving the grm or extra "moved" flags get
messy because, well, lfsr_mdir_commit's internals are quite messy.
The solution here, which is a bit hacky, but also obnoxiously elegant in
a way, is to reorder the split mdir compactions such that the new mdir
containing the commit mid is always compacted last. The means any
related attrs are played out after both compactions, allowing renames to
resolve correctly:
1. commit rename -> bshrub = src mid, orig mdir
2. commit fails because of ERANGE
3. right mdir contains mid
4. compact right mdir -> bshrub = src mid, right mdir
5. commit right mdir (skips rename)
6. compact left mdir -> bshrub = src mid, left mdir
7. commit left mdir -> bshrub = dst mid, left mdir
This only works as long as such commits only span a single mid, though
we already rely on mdir commits being single-mid elsewhere, so maybe
this won't be a problem?
The only real remaining concern is how much complexity this adds to
lfsr_mdir_commit. And while this feels logically messy, the resulting
code cost is surprisingly little:
code stack
before: 33458 2640
after: 33482 (+0.1%) 2640 (+0.0%)
Still, I'll have to scratch my head to see if there's a better way to
solve this...
A much requested feature, this allows much finer control of how RAM is
allocated for the system.
It was difficult to introduce this in previous versions of littlefs due
to how we steal caches during certain file operations, but now we don't
do that and treat the caches much more transparently.
Managing separate cache sizes does add a bit of code, but this is well
worth the potential for RAM savings due to increased flexibility:
code stack
before: 33656 2632
after: 33714 (+0.2%) 2640 (+0.3%)
Also interesting to note this reduces alignment requirements for the
rcache/pcache, since they don't need to share alignment, and completely
removes any alignment requirement from the file buffers.
The main idea here is that diverse tests are better than many similar
tests.
Sure, if we throw fuzz tests at the system all day we'll eventually find
more bugs, but if a developer is in the loop that time is going to be
better spent writing specific tests targeting the fragile parts of the
system.
And don't worry, we can still throw fuzz tests at the system all day by
specifying explicit seeds with -DSEED=blah.
Changes:
- Limited dir-related powerloss fuzz testing to N <= 16.
These tests were the biggest culprit of excessive test runtime,
requiring O(n^2) redundant operations to recover from powerlosses
(they just replay the full sequence on powerloss).
- As a tradeoff, bumped most fuzz tests to a minimum of 20 seeds.
The big exception being the test_fwrite tests, which are heavily
parameterized and already take the most time to run. Each parameter
combination also multiplies the effective number of seeds, so
increasing the number of base seeds will probably have diminishing
returns.
- Limited test_fwrite_reversed to SIZE <= 4*1024*CHUNK.
Writing a file backwards is just about the worst way you could write a
file, since all buffering/coalescing expect writes to eventually make
forward progress. On the flip side, because it's uncommon, writing a
file backwards is also a great way to find bugs. But at some point a
compromise needs to be made.
Impacted test runtimes:
case otime ntime dtime
test_btree_push_fuzz 0.3 0.5 +0.2 (+60.2%)
test_btree_push_sparse_fuzz 0.4 3.3 +2.9 (+720.4%)
test_btree_update_fuzz 0.4 0.9 +0.6 (+141.6%)
test_btree_update_sparse_fuzz 0.5 4.5 +4.1 (+857.4%)
test_btree_pop_fuzz 0.6 2.3 +1.7 (+314.7%)
test_btree_pop_sparse_fuzz 1.2 5.7 +4.4 (+356.2%)
test_btree_split_fuzz 0.5 1.4 +0.8 (+150.2%)
test_btree_split_sparse_fuzz 0.4 5.6 +5.1 (+1163.2%)
test_btree_find_fuzz 0.5 0.7 +0.2 (+50.7%)
test_btree_find_sparse_fuzz 1.0 3.0 +2.0 (+189.8%)
test_btree_traversal_fuzz 0.6 2.3 +1.6 (+260.4%)
test_dirs_mkdir_many 3.3 2.1 -1.3 (-37.8%)
test_dirs_mkdir_many_backwards 3.5 2.1 -1.4 (-39.9%)
test_dirs_mkdir_fuzz 115.3 106.4 -8.9 (-7.7%)
test_dirs_rm_many 283.9 76.8 -207.0 (-72.9%)
test_dirs_rm_many_backwards 216.1 80.6 -135.5 (-62.7%)
test_dirs_rm_fuzz 647.0 68.5 -578.5 (-89.4%)
test_dirs_mv_many 14.2 15.4 +1.1 (+7.9%)
test_dirs_mv_many_backwards 16.5 14.5 -2.1 (-12.5%)
test_dirs_mv_fuzz 1932.5 156.7 -1775.8 (-91.9%)
test_dirs_general_fuzz 561.9 74.5 -487.4 (-86.7%)
test_dread_recursive_rm 336.6 46.2 -290.4 (-86.3%)
test_dread_recursive_mv 55.5 44.6 -11.0 (-19.8%)
test_fsync_rrrr_fuzz 0.4 0.3 -0.1 (-18.4%)
test_fsync_wrrr_fuzz 8.0 12.4 +4.5 (+56.0%)
test_fsync_wwww_fuzz 13.2 33.4 +20.2 (+152.6%)
test_fsync_wwrr_fuzz 5.4 50.9 +45.5 (+841.6%)
test_fsync_rwrw_fuzz 2.4 8.4 +6.0 (+253.9%)
test_fsync_rwrw_sparse_fuzz 3.2 7.5 +4.2 (+129.9%)
test_fsync_rwtfrwtf_sparse_fuzz 6.1 8.5 +2.4 (+39.3%)
test_fsync_drrr_fuzz 11.8 9.2 -2.6 (-21.8%)
test_fsync_wddd_fuzz 9.3 11.9 +2.6 (+28.0%)
test_fsync_rwdrwd_fuzz 1.6 33.1 +31.5 (+1963.4%)
test_fsync_rwdrwd_sparse_fuzz 0.3 1.8 +1.4 (+418.8%)
test_fsync_rwtfdrwtfd_sparse_fuzz 0.3 1.1 +0.8 (+260.2%)
test_fwrite_reversed 728.5 345.2 -383.3 (-52.6%)
TOTAL 7587.5 3792.3 -3795.2 (-50.0%)
The size field in lfs_info doesn't really make sense for stat/dir_read
when the file is a directory. Still, we should probably set it to 0 os
it's not uninitialized.
Fortunately we were already setting size=0 in _most_ cases, this commit
is mostly just checking for size=0 in more test cases.
This avoids implicit info, mid.mid=-1 implying a bad path, and mid.mid=0
implying the root directory, at a tradeoff of potentially making the
returned error codes a bit confusing (0 means the file is NOT found!).
Here are the now possible return codes, aside from lower-level errors
(IO, CORRUPT, etc):
- 0 => path is valid, file NOT found
- EXIST => path is valid, file found
- INVAL => path is valid, but points to root
- NOENT => path is NOT valid, intermediate dir missing
- NOTDIR => path is NOT valid, intermediate dir is not a dir
Since the root has no real mdir entry, I think the special INVAL return
code is warranted. It needs special behavior in relevant functions
anyways.
Note that orphaned files still need special handling.
Code changes:
code stack
before: 33944 2944
after: 34036 (+0.3%) 2944 (+0.0%)
Fortunately these operations are heavily tested in test_dirs. The only
difference with files is the possibility for shrubs to need to be
copied.
Bugs fixed:
- It's counterintuitive, but lfsr_rbyd_appendcompactattr _can_ error
with LFS_ERR_RANGE when we are copying a shrub. This can happen if the
underlying mdir needs compaction itself.
- It's possible to null-trunk bshrubs to appear in our filesystem
traversal. Null-trunk bshrubs don't usually appear in any stable
state, but they are created by lfsr_bshrub_alloc and lfsr_btree_commit
to represent new, yet-uncommitted shrubs.
This gets a bit tricky because we also use null-trunks to indicate if
lfsr_btree_traversal has traversed the root. We can't rely on
bid >= weight for this because zero-weight btrees are allowed.
The solution here, though maybe temporary (famous last words), is to
treat null-trunk btrees as not having a root. Which isn't really true,
but null-trunk btree roots only exist between allocator checkpoints,
so they are allowed to be unreachable.
We really need more asserts that this is the case though... At least
added an assert that we never commit/read null trunks on disk.
POSIX is notoriously full of subtle and confusing nuances. Not through
any fault of POSIX, but as a result of trying to describe a complex
system with simple and easy to use operations.
Corner cases fixed here:
- rename("dir", "file") => ENOTDIR
This is the main surprise to me, and a mistake on my part. I thought
EISDIR would be appropriate for any renames with mismatched types,
since both involve a directory. It would be simpler code-wise, and
avoid ambiguity around if "file" is not a dir, or some other file
exists in the file's path. But I guess ENOTDIR makes more sense if you
think of the destination as the target being operated on.
- remove("/") => EINVAL
- rename("/", "x") => EINVAL
- rename("x", "/") => ENOTEMPTY
- open("/") => EISDIR
It's a bit difficult to lookup what error codes around root operations
should be, since they mostly end up as EPERM on modern systems, but
this doesn't really make sense for littlefs.
The solution chosen here is to prefer directory-related errors (EISDIR,
ENOTEMPTY) when possible, and fall back to EINVAL when the only issue
is that the target is the root directory.
Also I tweaked lfsr_mtree_pathlookup a bit so mid=0 indicates the target
is the root and mid=-1 indicates the target can't be created (because of
a missing directory). I think using mid=0 for the latter is a leftover
from when mid=-1 was a bit of a mess...
Unfortunately, the tests are starting to take a painfully long time to
run. Some of this is because, in order to get interesting file
topologies, we need to move a ton of data around, but some of this is
also because our current write implementation has some problematically
expensive corner cases.
I have quite a few ideas on how to improve this, but in the meantime the
tests needed to be aggressively trimmed in order to keep development
tolerable (A happy developer is a productive developer).
This mainly meant:
- Disabled powerloss testing on file tests for now.
The reality is that naivly powerloss testing the file tests, i.e.
just truncating the file after each restart, provides very little
value and adds an extreme amount of runtime.
Removed for now. Most of the powerloss file creation concerns are
covered in the dtree tests, and we should eventually add powerloss
tests tailored to recovering files after powerloss instead of just
truncating.
- Avoided tiny fragment sizes with large file sizes.
Tiny fragments are a degenerate case and end up with excessive
overhead (1 byte fragment => 41x overhead!). But they are useful for
revealing subtle bugs. Still, it just doesn't make sense time-wise to
test with tiny fragments once the file size exceeds ~1 block.
- Limited fuzz tests to cover fewer random seeds.
We can increase these if performance improves, but even if not, we can
run these individually with a high number of seeds in CI.
Also fixed a number of bugs found by the extended testing, which is
always a good sign:
- Yet another `lfsr_data_size(&data)` vs `data.u.disk.size` typo.
This is the first time I've seen a real world argument for private
struct/class fields, but I am still against the concept.
- Fixed delta/weight miscalculation when tree-carving a left sibling.
- Fixed missing offset in hole writing during block writes.
- Worked around lfsr_file_readnext's reliance on file->size when we are
using it to write to a block. This may be more a hack than a good
long term solution though.
- Checkpointed the allocator in both lfsr_file_write and lfsr_file_sync.
Otherwise calling lfsr_file_write repeatedly can easily trigger an
incorrect ENOSPC.
- Correctly reverted both shrubs and btrees in truncate/fruncate
This gets a bit more complicated in fruncate, since either one of the
two, or both, can revert.
truncate/fruncate probably deserve a bit more work around reversions
to simpler data structures, as is.
- Added handling of shrub overflows during fruncate.
Notably not possible with truncate, shrub overflows require that we
1. flush the shrub, 2. fruncate the tree, 3. and make sure any side
effects to the buffer are handled correctly.
The attempt to implement in-rbyd data slicing, being lazily coalesced
during rbyd compaction, failed pretty much completely.
Slicing is a very enticing write strategy, getting both minimal overhead
post-compaction and fast random write speeds, but the idea has some
fundamental conflicts with how we play out attrs post-compaction.
This idea might work in a more powerful filesystem, but brings back the
need to simulate rbyds in RAM, which is something I really don't want to
do (complex, bug-prone, likely adds code cost, may not even be tractable).
So, third time's the charm?
---
This new write strategy writes only datas and bptrs, and avoids dagging
by completely rewriting any regions of data larger than a configurable
crystallization threshold.
This loses most of the benefits of data crystallization, random writes
will now usually need to rewrite a full block, but as a tradeoff our
data at rest is always stored with optimal overhead.
And at least data crystallization still saves space when our data isn't
block aligned, or in sparse files. From reading up on some other
filesystem designs it seems this is a desirable optimization sometimes
referred to as "tail-packing" or "block suballocation"
Some other changes from just having more time to think about the
problem:
1. Instead of scanning to figure out our current crystal size, we can
use a simple heuristic of 1. look up left block, 2. look up right
block, 3. assume any data between these blocks contribute to our
current crystal.
This is just a heuristic, so worst case you write the first and last
byte of a block which is enough to trigger compaction into a block.
But on the plus side this avoids issues with small holes preventing
blocks from being formed.
This approach brings the number of btree lookups down from
O(crystallize_size) to 2.
2. I've gone ahead and dropped the previous scheme of coalesce_size
+ fragment_size and instead adopted a single fragment_size that
controls the size of, well, fragments, i.e. data elements stored
directly in trees.
This affects both the inlined shrub as well as fragments stored in
the inner nodes of the btree. I believe it's very similar to what is
often called "pages" in logging filesystems, though I'm going to
avoid that term for now because it's a bit overloaded.
Previously, neighboring writes that, when combined, would exceed our
coalesce_size, they just weren't combined. Now they are combined up
to our fragment size, potentially splitting the right fragment.
Before (fragment_size=8):
.---+---+---+---+---+---+---+---.
| 8 bytes |
'---+---+---+---+---+---+---+---'
+
.---+---+---+---+---.
| 5 bytes |
'---+---+---+---+---'
=
.---+---+---+---+---+---+---+---+---+---.
| 5 bytes | 5 bytes |
'---+---+---+---+---+---+---+---+---+---'
After:
.---+---+---+---+---+---+---+---.
| 8 bytes |
'---+---+---+---+---+---+---+---'
+
.---+---+---+---+---.
| 5 bytes |
'---+---+---+---+---'
=
.---+---+---+---+---+---+---+---+---+---.
| 8 bytes |2 bytes|
'---+---+---+---+---+---+---+---+---+---'
This leads to better fragment alignment (much like our block
strategy), and minimizes tree overhead.
Any neighboring data to the right is only coalesced if it fits in the
current fragment, or would be rewritten (carved) anyways, to avoid
unnecessary data rewriting.
For example (fragment_size=8):
.---+---+---+---+---+---+---+---+---+---+---+---+---+---.
| 6 bytes | 6 bytes |2 bytes|
'---+---+---+---+---+---+---+---+---+---+---+---+---+---'
+
.---+---+---+---+---.
| 5 bytes |
'---+---+---+---+---'
=
.---+---+---+---+---+---+---+---+---+---+---+---+---+---.
| 8 bytes | 4 bytes |2 bytes|
'---+---+---+---+---+---+---+---+---+---+---+---+---+---'
Other than these changes this commit is mostly a bunch of carveshrub
rewriting again, which continues to be nuanced and annoying to get
bug free.
But already there are some pretty fundamental problems.
The main issue is that, while we correctly dereference slices during
compaction, pending commits that get delayed after compaction still
point to the old block. I'm not sure there's an easy way around this
aside from aborting compaction commits or fully simulating commits,
both of which seem too costly to implement...
Also coalescing during compaction is flawed as well, since our
attributes will be outdated by the time they are committed if there is a
compaction...
Looks like it's back to the drawing board. Either our approach to
compaction needs to change, or this slice/coalescing work needs to be
reverted/redesigned...
Oh hey, it's that piece of complexity I was worried about.
The problem was that the position calculation for new appended
right_data depended on left_overlap, which fell out of sync when
transitioning from sprout->shrub.
The fix here is to keep left_overlap/right_overlap up to date with the
model that the sprout->shrub transition is effectively doing a
shrub-wide rm first.
Hacky, but hopefully avoids bugs in the future by keeping all of these
variables in a reasonable state...
There may be a simpler way to think about how this code should function,
but I just can't see it. This may deserve a rewrite in the future.
Ended up changing the name of lfsr_mtree_traversal_t -> lfsr_traversal_t,
since this behaves more like a filesytem-wide traversal than an mtree
traversal (it returns several typed objects, not mdirs like the other
mtree functions for one).
As a part of this changeset, lfsr_btraversal_t (was lfsr_btree_traversal_t)
and lfsr_traversal_t no longer return untyped lfsr_data_ts, but instead
return specialized lfsr_{b,t}info_t structs. We weren't even using
lfsr_data_t for its original purpose in lfsr_traversal_t.
Also changed lfsr_traversal_next -> lfsr_traversal_read, you may notice
at this point the changes are intended to make lfsr_traversal_t look
more like lfsr_dir_t for consistency.
---
Internally lfsr_traversal_t now uses a full state machine with its own
enum due to the complexity of traversing the filesystem incrementally.
Because creating diagrams is fun, here's the current full state machine,
though note it will need to be extended for any
parity-trees/free-trees/etc:
mrootanchor
|
v
mrootchain
.-' |
| v
| mtree ---> openedblock
'-. | ^ | ^
v v | v |
mdirblock openedbtree
| ^
v |
mdirbtree
I'm not sure I'm happy with the current implementation, and eventually
it will need to be able to handle in-place repairs to the blocks it
sees, so this whole thing may need a rewrite.
But in the meantime, this passes the new clobber tests in test_alloc, so
it should be enough to prove the file implementation works. (which is
definitely is not fully tested yet, and some bugs had to be fixed for
the new tests in test_alloc to pass).
---
Speaking of test_alloc.
The inherent cyclic dependency between files/dirs/alloc makes it a bit
hard to know what order to test these bits of functionality in.
Originally I was testing alloc first, because it seems you need to be
confident in your block allocator before you can start testing
higher-level data structures.
But I've gone ahead and reversed this order, testing alloc after
files/dirs. This is because of an interesting observation that if alloc
is broken, you can always increase the test device's size to some absurd
number (-DDISK_SIZE=16777216, for example) to kick the can down the
road.
Testing in this order allows alloc to use more high-level APIs and
focus on corner cases where the allocator's behavior requires subtlety
to be correct (e.g. ENOSPC).
This is an exciting new function, made possible by the order-statistic
nature of our rbyds and btrees.
lfsr_file_fruncation is like truncate, but from the front. It can trim
data off of the front of files, and grow files from the front,
effectively prefixing files with zeros cheaply.
This may have some niche use cases for prefixing files with headers, but
the real killer is making logging files trivial. Up until now logging
into a file has always resulted in awkward file-swapping code when a
file gets full. Now maintaining a log is just a single fruncate call.
---
Implementation wise, lfsr_file_fruncate is very similar to
lfsr_file_truncate, except we need to always inject holes into all file
trees to adjust file contents correctly.
What do you think a file's size becomes when you:
1. seek past the end of a file
2. call write with zero data!
POSIX/etc has this case explicitly mentioned, noting that zero-sized
writes should never update the file size.
This clashes with the assumption that file writes always update the file
position, but I suppose it makes a bit of practical sense if you want
zero-sized file writes to be idempotent.
Currently limited to inlined files and only simpler truncate-writes.
But still this lets us test file creation/deletion.
This is also enough logic to make it clear that, even though we have
some powerful high-level primitives, mapping file operations onto these
is still going to be non-trivial.
This makes it easier to evaluate the code/stack/etc sizes and run tests
without bringing in all of the outdated code.
I guess this officially makes this branch more-or-less a full rewrite,
though the benefit of commenting vs deleting this code is that it can be
easily pulled back in when useful.
When you add a function to every benchmark suite, you know if should
probably be provided by the benchmark runner itself. That being said,
randomness in tests/benchmarks is a bit tricky because it needs to be
strictly controlled and reproducible.
No global state is used, allowing tests/benches to maintain multiple
randomness stream which can be useful for checking results during a run.
There's an argument for having global prng state in that the prng could
be preserved across power-loss, but I have yet to see a use for this,
and it would add a significant requirement to any future test/bench runner.
The main benefit is small test ids everywhere, though this is with the
downside of needing longer names to properly prefix and avoid
collisions. But this fits into the rest of the scripts with globally
unique names a bit better. This is a C project after all.
The other small benefit is test generators may have an easier time since
per-case symbols can expect to be unique.
This mostly required names for each test case, declarations of
previously-implicit variables since the new test framework is more
conservative with what it declares (the small extra effort to add
declarations is well worth the simplicity and improved readability),
and tweaks to work with not-really-constant defines.
Also renamed test_ -> test, replacing the old ./scripts/test.py,
unfortunately git seems to have had a hard time with this.
- Removed old tests and test scripts
- Reorganize the block devices to live under one directory
- Plugged new test framework into Makefile
renamed:
- scripts/test_.py -> scripts/test.py
- tests_ -> tests
- {file,ram,test}bd/* -> bd/*
It took a surprising amount of effort to make the Makefile behave since
it turns out the "test_%" rule could override "tests/test_%.toml.test"
which is generated as part of test.py.
Christopher Haster