The previous system of relying on test name prefixes for ordering was
simple, but organizing tests by dependencies and topologically sorting
during compilation is 1. more flexible and 2. simplifies test names,
which get typed a lot.
Note these are not "hard" dependencies, each test suite should work fine
in isolation. These "after" dependencies just hint an ordering when all
tests are ran.
As such, it's worth noting the tests should NOT error of a dependency is
missing. This unfortunately makes it a bit hard to catch typos, but
allows faster compilation of a subset of tests.
---
To make this work the way tests are linked has changed from using custom
linker section (fun linker magic!) to a weakly linked array appended to
every source file (also fun linker magic!).
At least with this method test.py has strict control over the test
ordering, and doesn't depend on 1. the order in which the linker merges
sections, and 2. the order tests are passed to test.py. I didn't realize
the previous system was so fragile.
This didn't really work out as well as I had hoped. There were a few
ideas on how to encode the bid/rid tuple without sacrificing the
(currently 31-bit) integer limit, but these just introduced too much
complexity.
Ideas:
1. In theory, as the mdirs increase in size, the quantity of mdirs needed
for a given number of files decreases. If we say the number of files
fits in an integer of a given size, than we can model the mapping to
mdirs and rids roughly as the number of bits in that integer split
between the two.
Since the block_size is known, the we can find a rather conservative,
yet useful, estimate of the upper bound of rids, which ends up
being ~16 bytes ((2 alts + 1 null + 1 tag) * 4 bytes).
And since our btrees are perfectly balanced, this encoding should only
waste 1 or 2 bits due to rounding to rounding and sign encoding for
special values.
bbbbbbbb bbbbbbbb bbbbbbbr rrrrrrrr
'-----------+-----------''----+---'
| '-- log2(block_size/32)-bit rid
'-------------------- remaining-bit bid
Unfortunately, while this works ok on paper, and maximize the use of
the bits we have available for the mid, the implementation ended up
awkward and difficult to use.
We need to either calculate the relatively complciated log2 of the
block_size on the fly, or cache the value, and use it to shift the
mid around to extract the bid/rid when needed.
Unfortunately, perhaps due to the it being easy to use the bid/rid
directly, we use and mutate the bid/rid quite a bit. We mutate when
updating the mdirs, when decoding grms, when seeking mdirs, etc. If
anything, updating the mid in total is rarer than updating the
bid/rid component in complicated situations.
Note to mention this required access to the lfs config to even begin
decoding, complicating the API and making the result less efficient.
Initial (unoptimized, and not even tested) code size showed ~+800
bytes. So I decided to scrap this.
Maybe it will be worth investigating dynamic rid sizes later, to
increase the possible mtree size for a given mid width. Not sure.
2. Probably one of the worst ideas I've had so far, but it would solve
the mid encoding problem, is to use some form a floating point to
encode the bid/rid pair:
.----------.
v .+-.
bbbbbbbb bbbbbbbb bbbrrrrr rrrrssss
'-----------+-------''----+---''-+'
| | '-- rid bits
| '--------- variable rid
'----------------------- variable bid
An even worse idea would be to use IEEE floating point here. Yes it
would work, and probably work annoyingly well, but we it risk
bringing in a lot of standard conforming backbending that we really
don't care about.
The idea here is to sacrifice some bits to encode the ratio of rid
bits to bid bits. The value of this over the using the block_size is
that we can decode the bid and rid using all of the bits in the
integer alone. Avoiding memory access (and worse debugging) to load
any external constants.
As a plus, all mids in the system would have the same exponent,
simplifying comparisons and other operations.
But this is just trying to solve complexity by adding more
complexity, so I'm not even going to try implementing it.
Still, it's an interesting idea...
In the end I've gone with the KISS implementation. Use half-width
integers, in this case uint16s, for both the bid and rid:
bbbbbbbb bbbbbbbb rrrrrrrr rrrrrrrr
'-------+-------' '-------+-------'
| '-- 16-bit rid
'-------------------- 16-bit bid
This suffers from weakened limits around the number of rids in a block
and number of mdirs in the mtree, which is unfortunate. Still it is
probably worth the tradeoff for the RAM savings and encoding simplicity.
If the mdir is reasonably sized, this does probably approach a decent
distribution of rids and bids in 32-bits. But for outlier cases with
very small and very large mdirs, it risks premature out of bounds
errors.
To protect against mtree errors, we will probably need an additional
configuration option in the form of an mdir limit. Conveniently this
would also provide a way to enforce 2-block mode.
rid errors, on the other hand, depend on block_size/32, so we may not
need another configuration option and can rely on the block_size
to determine if the rids can overflow.
This is probably worth revisiting in the future. Fortunately, with
mdir_limit and block_size configuration options, it should be possible
to increase these limits in the future if this mid bid/rid design
changes.
code stack
before: 22126 2136
after: 22326 (+0.9%) 2088 (-2.2%)
This code size increase was unexpected. Maybe non-32-bit-aligned integers
cost more to load in thumb? Unsure.
The main intention here was to make the tracking of opened mdirs,
mostly opened lfsr_dir_t structs, simpler and more resilient to weird
corner cases. I'm not entirely sure this was successful.
The main changes:
- lfsr_dir_t now contains a full mdir for the dstart entry.
This makes it so that dstarts are not a special case when it comes
to mdir updates, though the fact that directories have 2 mdirs is
still an awkward case on its own.
I considered using two entries in the opened linked-list for this, but
it wouldn't have worked out that well. Both entries need to update the
directory position, so it would have required a third file type. We
would also have needed to make sure removed mdirs mark both mdirs as
removed, otherwise the position mdir would move around arbitrary into
possibly erronous values.
Instead the current solution treats the directory mdirs as a small
array of 2 mdirs, which is as hacky as it is hacky, but does get the
job done with little code duplication.
- Directory positions are updated a bit more intellegently.
Instead of checking if in range before updating, which requires access
to both mdirs and duplicate mid/rid comparison logic, position is
updated without regard for the beginning of the directory, and
un-updated if it was actually out of range of the directory.
This means we only need to compare the mids/rids for each mdir once.
This changes make it so that lfsr_dir_rewind is much cheaper, and
doesn't even need to go to disk. Though I'm not sure it's worth the RAM
increase...
Expanding the lfsr_dir_t dstart entry to a full mdir does a lot for
making mdir updates more consistent, but increases the lfsr_dir_t size
from 52 bytes to 76 bytes (+46.2%).
This is mostly to make it easier to merge mids/rids. Having a special
constant here is tricky when the mid/rid split point is dynamic.
Currently using rbyd.trunk=0 to indicate when an mdir is dropped. This
is nice as it preserves the last mid/rid, which is needed by the readdir
code, and it implicitly returns NOENT to all queries in
lfsr_rbyd_lookup.
When updating any opened mdirs to keep things in sync, we need to know
what rid the mdir is targeting in order to know which on-disk mdir it
should follow in the case of splits. Making this rid an actual member of
the mdir struct simplifies things.
This adds some RAM cost, though the plan is to merge the mid/rid into a
single integer, which requires this change and should actually save RAM
in the long run.
code stack
before: 22342
after: 22204 (-0.6%) 2144 (+1.1%)
Instead of reading eagerly and retreating with the hopes of terminating
early (which almost never happens when compacting, since we need to find
the split_id). lfsr_rbyd_estimate now works inward from the first and
last id to find both the dsize and split_id.
One thing that helps this is the addition of a separate per-id
lfsr_rbyd_estimate, which will be useful for checking if the quantity of
file attributes overflows our mdir limitations.
lfsr_rbyd_estimate also now ignores the -1 id for split_id calculation,
since -1 ids are always cleaned up during splitting, though it does
include it in the calculated dsize so that the condition to split is
determined correctly.
---
This also required rebalance changes. Fortunately, one improvement here
is that we can make a simplifying assumption tha the number of tags
can't exceed the maximum possible number of tags in the calculated
dsize. So worst case, if every tag is empty, the maximum possible dsize
becomes 4*(2*log2(dsize/4))+dsize.
Though it's still unclear if rebalance is worth keeping. Current
comparison:
code stack
rebalance: 22362 2120
no_rebalance: 21922 (-2.0%) 2120 (+0.0%)
This means no special case for uninling-but-not-splitting, but allows
the entire split route to be deduplicated, simplifying things.
The main downside is that for littlefs to go from a single inlined mdir
filesystem to an mtree filesystem it requires a minimum of 2 mdir
allocations (4 blocks) in all cases. This can be avoided, but I think is
worth the tradeoff since it generally occurs once in a filesystem's
lifetime.
This does make 4 block block devices a bit awkward, but those geometries
are always going to be a bit awkward with littlefs's design. At least
this implementation avoids an unecessary B-tree node where possible...
code stack
before: 22586 2320
after: 22414 (-0.8%) 2120 (-8.6%)
I _think_, but haven't verified, the significant stack saving comes from
the fact that since there's one route through lfsr_mtree_split_,
lfsr_mtree_split_ can be inlined into lfsr_mdir_commit. This avoids the
marshalling of all its arguments for the function call, which I've
noticed can have a surprising cost.
---
Also fixed a bug where dstart was not updated with mid changes after
splits/drops. The mdir commit cleanup code has a lot of duplication now,
makes me wonder if there's a better way to structure this.
It turned out our dir-read-idempotent test never created non-dstart
neighbors. This was a bit of a problem since we relied on dstart entries
to know when our dir read terminates. If we seek to an invalid position
(in theory undefined behavior, but easily possible with concurrent
modifications to the directory), we can end up reading an unrealted,
non-dstart entry, and incorrectly reporting that entry as in our current
dir.
This fix reintroduces the did into the lfsr_dir_t struct and uses the
did to determine end-of-dir. This adds some RAM cost, but is more
resilient to any seeks that overshoot the end of the directory.
Using did is also a stronger guarantee we will never accidentally report
unrelated entries as a part of the current directory.
This is entirely a pragmatic change, lfsr_mdir_commit already does
several hairy things with grm tags, decoding, fixing, reencoding, etc,
so it makes sense to move all the encoding logic into lfsr_mdir_commit.
This leads to a couple optimizations:
- We don't need to decode the grm to apply any last minute fixes.
- By allowing the grm arugment to be mutated (they are just sitting on
the stack anyways, we need a copy in case we back out of change due to
error), we can apply and save any grm fixes in the grm argument
itself.
This means we only need to fix the grm at most once, after any mtree
modifications.
Which in turn saves some code and stack cost:
code stack
before: 22930 2392
after: 22706 (-1.0%) 2344 (-2.0%)
Unfortunately the previous attempt to fix the dir seek system didn't
really work. Using a packed mid/rid integer for the offset is tempting,
but since mid/rid can change with any metadata id change in the
filesystem, dir tell offsets would become invalidated if you modified
files in unrelated directories, which isn't great and likely to catch
users by surprise.
This solution builds on the previous dir offset design, which tracks the
dstart-relative position independently from the current mid/rid in our
directory. To update this correctly when there are unrelated changes to
the filesystem, we need to know if metadata id changes are in the range
between our directories dstart and current mid/rid. This in turn means
we need to track our dstart. So our opened directories need three
separate pointers we need to update on every mdir commit:
dir->pos
|
.-------+-------.
a b c d e f g h i j k l m n o p
^ ^
| |
dir->dstart dir->mdir
This has quite a few moving parts, which I was hoping to avoid.
Fortunately we don't need a second mdir, so the RAM cost is pretty
small.
We can also drop dir->did, since the dstart mid/rid render it redundant,
which is interesting.
This is an attempt to fix issues with dir seeking in a filesystem
undergoing changes. The problem with the previous dstart-relative
position encoding is that if we deleted/created new entries outside of
our current directory, we didn't if they were inside or outside of the
current directory, so we couldn't always update our position correctly.
Instead of using a dstart-relative position, this solution crams both
the mid and rid into a single 31-bit integer. Things get a bit tight
here, so we use the current block_size as a heuristic for how many
possible rids we can ever have in a single mdir. The idea is the larger
the rid encoding needs to be, the smaller the mid encoding needs to be,
and we should, _roughly_, approach the same encoding limitation we would
have to dstart-relative position anyways.
Making some assumptions about the maximum possible number of rids in a
block gives us at most ~block_size/8 rids per mdir.
So for 4096 byte blocks (note the exact encoding is dynamic):
sbbbbbbb bbbbbbbb bbbbbbbr rrrrrrrr
^'-----------+----------''----+---'
'------------|----------------|----- sign bit (used for errors)
'----------------|----- 22-bit metadata bid
'----- 9-bit metadata rid
Note this introduced as new, significant limitation on the number of
total mdirs in the system. Normally I would be against this solution for
that reason, however if we adopt this encoding elsewhere in the system it
may improve some RAM cost and in general simplify things by being able to
store any mid in a single integer. More work needs to be done here...
This approach needs some fleshing out and has its own issues (the
offset returned by tell quickly becomes out of date if the filesystem
is modified, but is that really a problem?), but it improves over the
previous implementation by making tell always correct at that moment.
- Prevented removing and renaming of the root directory. This is done by
repurposing the INVAL error in lfsr_mtree_lookup to indicate the
found entry is the root.
The root entry has special behavior in almost every function, owing to
the fact it doesn't really have an mid/rid. So I think this is a
reasonable approach.
- Added support for lfsr_stat of the root directory.
- Fixed off-by-two in lfsr_dir_seek thanks to the "." and ".." entries.
Humorously there is a comment noting this but the code didn't
actually match the comment.
Unfortunately the powerloss testing risks being a big time sink.
Figuring out the best scale of powerloss testing during normal testing
is probably going to be a constant balancing act.
Mainly trying to match the tests over mkdir/rm, which seem to have a
good amount of coverage.
- Fixed issue where move's desination rid wasn't updated correctly if
the destination split.
- Prevented renaming into nonexistant directories.
- Fixed neighboring rid adjustment in rename (+1 not -1 silly).
- Fixed erronously updating the grm's rid during lfsr_fs_fixgrm. In the
"I can't believe this ever worked" category, it seems this usually
didn't cause issues since mid was often marked as removed, making the
erronously updated rid ignored.
Only simple tests right now, but the theory is sound.
This mainly required the addition of the fancy in-device move attribute,
which copies all tags associated with an rid from one rbyd to another in
a single transaction.
This is a carryover from the previous littlefs implementation, though it
is easier to implement here since it is effectively a range query on the
rbyd tree, which trees are really good at. This was intentional.
Oh and I suppose this also required implementing lfsr_rename, which has
a few corner cases to watch out for.
It is nice that both lfsr_remove and lfsr_rename can rely on
lfsr_fs_fixgrm to finish all of the removes, which wasn't previously
reasonable due to the overhead of deorphaning.
Hopefully third times the charm.
The previous solution pretty bluntly did not work outside of the
recursive remove case, because the moment we mark the rid as deleted,
the directory positions no longer get updates. It's not possible to
update the directory position because we don't know how it maps into our
mtree without a full seek from the dstart.
After staring at it a bit, I think this solution should work:
1. Instead of marking the mid/rid as removed when dropping an mdir, we
set the weight to zero and the trunk to zero, causing mdir lookups to
return NOENT without actually going to disk.
This is very important since later mdirs could be allocated on the
same block, and going to disk can result in a corrupted lookup.
2. Eagerly seek to the next mid/rid after every lfsr_dir_read call. This
puts us in a position where rid can be >= the current mdir weight
without issues, and avoids degenerate cases that may be caused by
recursive removes.
3. If we remove an opened dir, instead of marking the mdir as deleted,
move the rid to the next rid. If the mdir was dropped, this leaves us
with rid == mdir weight, and the mdir trunk == 0.
The rid == mdir weight also occurs when we are creating a new file, so
we have a bit of common behavior we can rely on. We just need to make
sure that mdir updates respect the rid == mdir weight situation.
4. On each lfsr_dir_read call, we do an mtree seek of zero. This just
serves to fix our mdir if our rid == mdir weight, without much
additional code (yay for code reuse).
The use of weight=0, trunk=0, for a dropped mdir here is key, and makes
me wonder if this is a better indicator of a dropped mdir than another
reserved mid value. This probably deserves some investigation later.
"Recursion" here just refers to the ability to remove entries in a
directory while iterating over it. This is very useful when you just
want a directory gone, and can be extended to a "true" recursive remove
straightforwardly. This mainly tests that mid/rid updates in opened
mdirs are correct.
To make this work, we need to update opened dirs differently than files,
since opened dirs do not get marked as removed when its rid is removed
and contain an additional position in the dir that needs to be updated.
To keep track of the different types, littlefs now contains 2
linked-lists for opened mdirs. Maybe these should be correctly typed,
but by hiding the specific types behind an array of mdir linked-lists,
we can more efficiently iterate over both lists when necessary.
We should probably compare this approach to the type-tagged approach in
the previous littlefs implementation, but I think the idea of an array
of type-hidden linked-lists just didn't come to me then. There was also
a bit more room in the mdir structs to hide a 1-bit type field. The mdir
structs here are getting pretty squeezed since they are used everywhere.
These mirror the lfsr_mkdir tests, but backwards.
It's interesting to note the rm powerloss testing is much slower than
mkdir powerloss testing. This is because the rm tests can make
significant backwards progress if power is lost (these tests both make
and remove dirs), but mkdir tests always make forward progress (by only
making dirs).
In theory this is pretty much the same as lfsr_mkdir, but backwards.
The main work was making the interactions between removing mids/rids and
the grm correct. This ends up meaning we just need to update the grm on
any mid/rid update the same way we update the list of opened mdirs.
On the plus side, it turned out to be possible to deduplicate the mdir
uninlining route a bit, by adding range argument to lfsr_mdir_commit_
and changing the write of the newly uninlined mtree/mdir to marking
mtree as dirty and then joining the common path.
This lets us move the pre-commit round of grm updates into a single
location in lfsr_mdir_commit, removing and extra function definition and
the related state marshalling while also simplifying the control-flow.
This also raises the question, can more lfsr_mdir_commit be deduplicated
more? Uninlining is a infrequent operation we don't really need to
optimize for.
---
Testing lfsr_remove also found a bug related to incorrect propagation of
when the mroot becomes "unerased" (when rbyd overflows). This raises the
concern that we're not propagating unerased-states very rigorously, and
unexpected errors may not allow the filesystem to resume.
This has never been in a very good place for littlefs, but would be
worth improving in the future.
Instead of iterating over a number of seeds in the test itself, the
seeds are now permuted as a part of normal test defines.
This lets each seed take advantage of other test features, mainly the
ability to test powerlosses heuristically.
This is probably how it should have been done in the first place, but
the permutation tests can't do this since the number of permutations
changes as the size of the test input changes. The test define system
can't handle that very well.
The tradeoffs here are:
- We can't do cross-fuzz checks, such as the balance checks in the rbyd
tests, though those really should be moved to benchmarks anyways.
- The large number of cheap fuzz permutations skews the total
permutation count, though I'm not sure this matters.
before: 3083 permutations (-Gnor)
after: 409893 permutations (-Gnor)
To help with this, added TEST_PL, which is set to true when powerloss
testing. This way tests can check for stronger conditions (no EEXIST)
when not powerloss testing.
With TEST_PL, there's really no reason every test in t5_dirs shouldn't
be reentrant, and this gives us a huge improvement of test coverage very
cheaply.
---
The increased test coverage caught a bug, which is that gstate wasn't
being consumed properly when mtree uninlining. Humorously, this went
unnoticed because the most common form of mtree uninlining, mdir splitting,
ended up incorrectly consuming the gstate twice, which canceled itself
out since the consume operation is basically just xor.
Also added support for printing dstarts to dbglfs.py, to help debugging.
The grm bugs were mostly issues with:
1. Not maintaining the on-disk grm state in RAM (lfs->grm) correctly,
this needs to be updated correctly after every commit or littlefs
gets a confused.
2. lfsr_fs_fixgrm got a bit confused when it was missed when changing
the no-rm encoding from 0 to -2. Added some inline functions to help
avoid this in the future.
3. Leaking information due to mixing fixed sized and variable sized
encodings of the grm delta in places. This is a bit tricky to write
an assert for as we don't parse the full grm when we see a no-rm grm.
This implementation is in theory correct, but of course, being untested,
who knows?
Though this does come with remounting added to all of the directory
tests. This effectively tests that all of the directory creation tests
we have so far maintain grm=0 after each unmount-mount cycle. Which is
valuable.
This bug was just overlooked in testing the mtree, fortunately dir
fuzzing found it. Though since this depends on neighboring mdirs, it
probably would have been found quicker with smaller block sizes. At the
moment I am only testing on NOR-liked geometry (4KiB blocks).
The fix is easy, we can use the difference in the mtree size to
determine if a split or drop happened in mdir commit, since at most one
of these can happen on any mdir commit.
Also added an explicit test for mid updates when splitting and dropping.
lfsr_stat is really a directory operation underneath, so it's good to
add to our testing while we are building up the dir tests.
It's interesting to note lfsr_stat and lfsr_dir_read are less
deduplicatable than their previous versions, since lfsr_stat can get
most of it's info from lfsr_mtree_pathlookup. Though there will probably
need to be some code sharing when we get to files with sizes.
- Checksum collisions
- Collisions with root did
- Collisions needing wraparound
- Possible leb128 encoding issues
Sure enough the last one caught an off-by-one error in our calculation
of the leb128 encoded size. I sort of expected a bug there, since it's
rather nuanced math, so it's good to have test coverage now.
The main issues:
- The addition of the root's dstart entry during lfsr_format throws off
our mtree tests. It's a bit of a hack, but for now I am just manually
deleting the root's dstart entry at the beginning of each tests.
It might be possible to make the mtree tests work around the root's
dstart, but it seems to cause problems for when exactly the mtree
splits.
- btree dnamelookup and mdir dnamelookup need different things from
the rbyd dnamelookup when the dname is not found. The btree lookup
needs the largest branch smaller than the dname, since this is the
"bucket" containing our dname, while the mdir dnamelookup needs
the id that _follows_ the id smaller than the dname, since insertion
causes all ids >= the inserting id to shift up.
The solution here is to make rbyd dnamelookup behave as expected by
btree dnamelookup. btree needs more info about the branch (weight
mostly), so this avoids more issues. mdir dnamelookup adjusts the
id as needed, which costs a bit of code, but makes things work.
Fortunately, mdir dnamelookup can assume the weight is 1, which
simplifies things a bit.
This makes it now possible to create directories in the new system.
The new system now uses a single global "mtree" to store all metadata
entries in the filesystem. In this system, a directory is simply a range
of metadata entries. This has a number of benefits, but does come with
its own problems:
1. We need to indicate which directory each file belongs to. To do this
the file's name entry has been changed to a tuple of leb128-encoded
directory-id + actual file name:
01 66 69 6c 65 2e 74 78 74 .file.txt
^ '----------+----------'
'------------|------------ leb128 directory-id
'------------ ascii/utf8 name
If we include the directory-id as part of filename comparison, files
should naturally be next to other files in the same directory.
2. We need a way allocate directory-ids for new directories. This turns
out to be a bit more tricky than I expected.
We can't use any mid/bid/rid inherent to the mtree, because these
change on any file creation/deletion. And since we commit the did
into the tree, that's not acceptable.
Initially I though you could just find the largest did and increment,
but this gives you no way to reclaim deleted dids. And sure, deleted
dids have no storage consumption, but eventually you will overflow
the did integer. Since this can suddenly happen in a filesystem
that's been in a steady-state for years, that's pretty unnacceptable.
One solution is to do a simple linear search over the mtree for an
unused did. But with a runtime of O(n^2 log(n)), this raises
performance concerns.
Sidenote: It's interesting to note that the Linux kernel's allocation
of process-ids, a very similar problem, is surprisingly complex and
relies on a radix-tree of bitmaps (struct idr). This suggests I'm not
missing an obvious solution somewhere.
The solution I settled on here is to instead treat the set of dids as
a sort of hash table:
1. Hash the full directory path into a did.
2. Perform a linear search until we have no collision.
leb128(truncate28(crc32c("dir")))
.--------'
v
9e cd c8 30 66 69 6c 65 2e 74 78 74 ...0file.txt
'----+----' '----------+----------'
'-----------------|------------ leb128 directory-id
'------------ ascii/utf8 name
Worst case, this can still exhibit the worst case O(n^2 log(n))
performance when we are close to full dids. However that seems
unlikely to happen in practice, since we don't truncate our hashes,
unlike normal hash tables. An additional 32-bit word for each file
is a small price to pay for a low-chance of collisions.
In the current implementation, I do truncate the hash to 28-bits.
Since we encode the hash with leb128, and hashes are statistically
random, this gives us better usage of the leb128 encoding. However
it does limit a 32-bit littlefs to 256 Mi directories.
Maybe this should be a configurable limit in the future.
But that highlights another benefit of this scheme. It's easy to
change in the future without disk changes.
3. We need a way to know if a directory-id is allocated, even if the
directory is empty.
For this we just introduce a new tag: LFSR_TAG_DSTART, which
is an empty file entry that indicates the directory at the given did
in the mtree is allocated.
To create/delete these atomically with the reference in our parent
directory, we can use the GRM system for atomic renames.
Note this isn't implemented yet.
This is also the first time we finally get around to testing all of the
dname lookup functions, so this did find a few bugs, mostly around
reporting the root correctly.
The plan is that names in littlefs now include a directory-id prefixed
as a single leb128.
01 66 69 6c 65 2e 74 78 74 .file.txt
^ '----------+----------'
'------------|------------ leb128 directory-id
'------------ ascii/utf8 name
Unfortunately, while this is easy for read/compare operations to implement,
it creates a bit of a problem for writes. We can't allocate a new buffer
for each name, so we need some sort of extra mechanism.
The solution here is to just add a did member to lfsr_data_t that is
written when non-negative. This works, though it does introduce some
complexity.
Fortunately, did in lfsr_data_t is somewhat free when
sizeof(void*) == sizeof(lfs_size_t), due to the union with disk
references.
It doesn't make sense to test more complex logic, such as t2_btree.toml,
when the logic it is built on, t1_rbyd.toml, does not past testing. The
test runner already guarantees a consistent lexicographic order, so all
we need to do is renamed these from test_* -> tn_*.
Note, if we every have more than 10 tests, we will need to bump up the
number of digits for all tests, so t1_rbyd.toml -> t01_rbyd.toml. This
is the main downside of lexicographic ordering. But we'll cross that
bridge when we get to it.
Took the opportunity to make some allocator tweaks:
- Renamed lfs.free -> lfs.lookahead, it's previous name did cause some
confusion.
- Renamed lfs.free.off -> lfs.lookahead.start
- Renamed lfs.free.i -> lfs.lookahead.next
- Renamed lfs.free.ack -> lfs.lookahead.acked
- Changed bitmap from using 32-bit words to using 8-bit bytes, dropping
the alignment requirement. One of the reasons for 32-bit alignment was
an attempt at future proofing for some sort of free-list.
This never landed, and if it did, it could have been provided without
breaking backwards compatiblity via an additional config option, at a
minor RAM cost.
We never used ffs/clz instructions for this bitmap, so I don't think
using 32-bit words offers much advantage. It just creates another
potential issue for users if their lookahead buffer is unaligned.
These changes should probably also be upstreamed to the current version.
They don't depend on anything rbyd specific.
Note, at some point lfs_alloc will need to be extended to mark block tags,
etc, as in-use during traversal.
- Renamed mpair -> mptr, may have >2 blocks in the future.
- Renamed branch -> bptr for consistency.
- Renamed other_block -> redund_rbyd.
- Changed comparison functions to use -1, 0, +1, even for unordered
types.
- Added lfs_cmp function for unioning comparisons with signed errors.
- Since both trunks emit altle tags now, reworked the trunk merging to
reuse more code.
- Changed lfsr_mdir_fetch to rely on trunk=0 to detect the no-commit
state. This is purely for consistency.
This actually broke some tests that committed nothing, resulting in
trunk-less rbyds, which is a bit concerning, but I don't think
trunk-less rbyds will ever be valid in our system?
- Simplified lfsr_rbyd_estimate calculation, merged lfsr_rbyd_bisect
since this is almost always needed after an estimated failure, and
that way dependent function have to call fewer things to implement
rbyd splitting.
- Dropped vestigial names for now, though need to revisit this later.
After these changes the code size difference between rebalancing and
appending is a bit smaller at ~392 bytes: 16208 -> 16600 (+2.4%). It's
interesting to note this is mostly because the conservative overhead
calculation is easier with rebalancing.
In theory this also saves some stack usage, but since I'm measuring
maximum stack usage it doesn't show up since it's not on the deepest
path.
This isn't actually for performance reasons, but to reduce storage
overhead of the rbyd metadata tree, which was showing signs of being
problematic for small block sizes.
Originally, the plan for compaction was to rely on the self-balancing
rbyd append algorithm and simply append each tag to a new tree.
Unfortunately, since each append requires a rewrite of the trunk
(current search path), this introduces ~n*log(n) alts but only uses ~n alts
for the final tree. This really starts to put pressure on small blocks,
where the exponential-ness of the log doesn't kick in and overhead
limits are already tight.
Measuring lfsr_mdir_commit code size, this shows a ~556 byte cost on
thumb: 16416 -> 16972 (+3.4%). Though there are still some optimizations
on the table, this implementation needs a cleanup pass.
alt overhead code cost
rebalance: <= 28*n 16972
append: <= 24*n*log(n) 16416
Note these all assume worst case alt overhead, but we _need_ to assume
worst case for our rbyd estimations, or else the filesystem can get
stuck in unrecoverable compaction states.
Because of the code cost I'm not sure if rebalancing will stay, be
optional, or replace append-compaction completely yet.
Some implementation notes:
- Most tree balancing algorithms rely on true recursion, I suspect
recursion may be a hard requirement in general, but it's hard to find
bounded-ram algorithms.
This solution gets around the ram requirement by leveraging the fact
that our tags exist in a log to build up each layer in the tree
tail-recursively. It's interesting to note that this is a special
case of having little ram but lots of storage.
- Humorously this shouldn't result in a performance improvement. Rbyd
trees result in a worst case 2*log(n) height, and rebalancing gives us
a perfect worst case log(n) height, but, since we need an additional
alt pointer for each node in our tree, things bump back up to 2*log(n).
- Originally the plan was to terminate each node with an alt-always tag,
but during implementation I realized there was no easy way to get the
key that splits the children with awkward tree lookups. As a
workaround each node is terminated with an altle tag that contains the
key followed by an unreachable null tag. This is redundant information,
but makes the algorithm easier to implement.
Fortunately null tags use the smallest tag encoding, which isn't that
small, but that means this wastes at most 4*n bytes.
- Note this preserves the first-tag-always-ends-up-at-off=0x4 rule, which
is necessary for the littlefs magic to end up in a consistent place.
- I've dropped dropping vestigial names for now, which means vestigial
names can remain in btrees indefinitely. Need to revisit this.
- Finally figured out how to test multiple mroot extensions without an
allocator, though hopefully forcing PROG_SIZE doesn't break test
framework things at some point...
- Added tests that magic string is always in the same place. This isn't
strictly required for littlefs to work, but is a nice feature to have.
Of course, the new tests found a bug, but it was in a surprisingly
place. Accidentally allowed the revision count to be uninitialized when
compacting the mroot. At least there's a test that covers this now.
Now we read the revision count on-demand, trading off some extra reads
for a smaller lfsr_rbyd_t struct.
I believe this is worth it because:
1. We're created a lot of lfsr_rbyd_t structs as a part of the relatively
complicated mdir/btree commit logic in order to safely fallback on errors.
2. We don't really need the revision count for our Cow btrees, so we
only need to read the revision count on mdir fetch (which we were
already reading too many times), on mdir compact, and on rbyd fetch
as a part of checksum calculation.
This really only adds a O(1) cost when we are compacting, which is rather
small.
Current measurements:
code: 8980 -> 9036 (+0.6%)
stack: 1024 -> 1000 (-2.3%)
Though note this is currently without any mdir/btree commit code being
dragged in.
Wide tags are a happy accident that fell out of the realization that we
can view all subtypes of a given tag suptype as a range in our rbyd.
Combining this with how natural it is to operate on ranges in an rbyd
allows us to perform operations on an entire range of subtypes as though
it were a single tag.
- lookup wide tag => find the smallest tag with this tag's suptype, O(log(n))
- remove wide tag => remove all tags with this tag's suptype, O(log(n))
- append wide tag => remove all tags with this tag's suptype, and then
append our tag, O(log(n))
This is very useful for littlefs, where we've already been using tag's
subtypes to hold extra type info, and have had to rely on awkward
alternatives such as deleting existing subtypes before writing our new
subtype.
For example, when committing file metadata (not yet implemented), we can
append a wide struct tag to update the metadata while also clearing out any
lingering struct tags from previous commits, all in one rbyd append
operation.
This uses another mode bit in-device to change the behavior of
lfsr_rbyd_commit, of which we have a couple:
vwgrtttt 0TTTTTTT
^^^^---^--------^- valid bit (currently unused, maybe errors?)
'||---|--------|- wide bit, ignores subtype (in-device)
'|---|--------|- grow bit, don't create new id (in-device)
'---|--------|- rm bit, remove this tag (in-device)
'--------|- 4-bit suptype
'- leb128 subtype
This only affects the in-device tags, not the on-disk tags.
The mk variant of tags was seeing much more use than the grow variant,
since the grow variant is really only used by the btree internals. But
since the default encoding of tags cleared the mk-bit, this led to a
bunch of extra lfsr_tag_setmk calls just to reserialize things correctly
during compact, split, etc.
Flipping the logic so the bit needs to be set to grow tags simplified
things quite a bit.
Note that mk tags do nothing when their delta is zero, so zero-delta
tags are the same in both mk/grow mode.
Originally I thought doing a linear search during fetch was going to be
the best route for name lookups, since we already needed a O(b) fetch,
which set a hard ceiling for name lookup performance.
But it turns out we don't need to fetch during btree name lookups
unless we're also validating! Now that validation and lookups are
disentangled, we can do a binary search over the rbyd to drop our
name lookup down to O(log(b)^2).
Two other motivations for this change:
1. This removes the name search from lfsr_rbyd_fetch, which has been
surprisingly tricky to get right.
2. Now that lfsr_rbyd_fetch doesn't need to follow the create/delete
history to find and reconstruct the state of names, we only need to
know the create/delete state of tags post-fetch, freeing up the rbyd
encoding to be more flexible.
The next thing I plan to do is drop on-disk remove tags, for example.
This drops the total btree namelookup cost from O(b log_b(n)) to
O(log(b)^2 log_b(n)).
This makes it easier to evaluate the code/stack/etc sizes and run tests
without bringing in all of the outdated code.
I guess this officially makes this branch more-or-less a full rewrite,
though the benefit of commenting vs deleting this code is that it can be
easily pulled back in when useful.
I intended to also add a test for cycles in the btree that backs the
mtree (and eventually other btrees), but something really curious
happened.
It turns out it's actually really hard to create a btree cycle, even
intentionally.
This is because each CoW btree pointer includes the expected CRC of
the branch's rbyd. To succesfully create a cycle that isn't trivially
detected in a validating mtree traversal, you would somehow need to
solve for a cyclic set of dependent CRCs that are still valid.
I suspect this is slightly easier than a hash-based construction, due to
the linear nature of CRCs, but still I think it's unreasonable to expect
these sort of cycles to occur in the wild. Even with filesystem bugs.
---
Note this isn't true for the mdirs, which are mutable so storing a
checksum in the pointer isn't possible. For this reason, cycle detection
is kept for mdirs during mtree traversal. This may not be strictly
necessary for the mtree, but it needed for the mroot chain.
Nonetheless, this does simplify things. Specifically it reduces the
cycle detection's tortoise state to only mdir pairs.
Validating btree nodes during lfsr_btree_lookup was useful as a
proof-of-concept, but it's not really needed if we validate btree nodes
during mtree traversal.
mtree traversal provides the first reads into the filesystem. It's how
we find the real mroot, and (in theory at the moment) it provides the core
operation for error detection in correction. With this in mind,
implementing btree node validation in mtree traversal makes a lot of
sense, with lfsr_btree_lookup leveraging an assumed successful
validation for faster/smaller btree walks.
Note that btree node validation during traversal is still optional. We
really don't want to pay this cost during block allocation for example.
---
It may look concerning that there's no related validation in btree traversal
layer itself.
It turns out that a quirk of btree traversal returning inner btree nodes on
first visit, before actually traversing the btree node, is that it's
safe for us to validte the btree node in only the mtree traversal layer.
As long as we don't continue traversing on finding a corrupted btree,
the btree traversal layer will never traverse an unvalidated btree node.
This keeps all the validation logic in the same place, mtree traversal.
I don't know if this will stay this way if/when more error correction
features are added, but it's convenient in the meantime.
Just like lfsr_btree_traversal_t, lfsr_mtree_traversal_t provides a
mechanism for traversing the mtree incrementally, including any inner
btree nodes.
This is one level more complex than btree traversal because we also need
to handle the mroot chain and traversal of rids in each mdir.
Again, mtree traversal returns temporary decoded rbyd structs for inner
nodes. Actually, mtree traversal only returns inner nodes... so maybe
using lfsr_data_t here is the wrong choice:
- tag=LFSR_TAG_BTREE => lfsr_rbyd_t
- tag=LFSR_TAG_MDIR => lfsr_mdir_t
The main thing to note is that traversal here != iteration.
Thanks to the right-leaning nature of our btrees, iteration is already
provided by lfsr_btree_lookupnext, using the bid as the current
iteration state.
What btree traversal provides is traversal over every rbyd + entries
used in the btree, include the inner btree nodes. This is useful for
things like garbage collection and error detection that need to operate
on the raw rbyds.
Note that both btree traversal and iteration are still O(n log_b(n)). We
can't do any better than that without recursion.
One non-intuitive implementation detail, we return a tag describing each
entry, but instead of returning an on-disk data reference for inner
btree nodes, we return a pointer to a temporarily decoded rbyd struct.
This simplifies root handling, and we probably want the decoded version
anyways:
- tag=LFSR_TAG_BTREE => lfsr_rbyd_t
- tag=anything else => lfsr_data_t
The reason for making btree traversal incremental, and not just use a
callback like we've done previously, is to eventually use this as a part
of high-level incremental garbage-collection/error-correction. For this
to work, all of the lower-levels also need to be incremental.
littlefs uses an invasive linked-list in open mdirs to keep any open
files/dirs (and some special mdirs) in sync during filesystem
operations. The main benefit of this is that the filesystem doesn't need
to know the number of open files at compile time.
The implementation here introduces a new type, lfsr_openedmdir_t, for
mdirs that want to participate in the opened-mdir linked-list. This
saves a couple words of memory in the cases where the mdir does not need
to participate in the opend-mdir linked-list.
Since we are creating quite a few more mdir structs in lfsr_mdir_commit now,
the size of this struct is valuable.
The implementation of lfsr_mdir_commit knew this was coming, so aside
from the new type, adding this feature was straightforward:
1. Update opened-mdirs based on in-flight attrs.
2. Update opened-mdirs rbyd state.
3. Mark any deleted opened-mdirs with the reserved mid -2.
4. Test.
It's interesting to note the different performance characteristics of
purely CoW btrees vs our mutable mtree.
The main downside of our mtree is the need to fetch leaf mdirs. This
fetch is expensive, and can be avoided in CoW btrees by storing the
trunk in each branch's parent.
On the other hand, btrees need to propagate all changes upwards to the
root.
An interesting takeaway is that a sort of mdir-trunk cache may be a very
interesting optimization for relatively little RAM cost. This may be
something to explore in the future.
- lfsr_btree_isnull still used tag and not only weight for null trees
- relocation forgot the mid
- missed relocation when uninlining, though this fix should be cleaned up
- made revision count behavior a bit more consistent
Note that the new tests may be -Gnor exclusive, they rely quite a bit on
exactly when compaction happens...
Christopher Haster