The attempt to implement in-rbyd data slicing, being lazily coalesced
during rbyd compaction, failed pretty much completely.
Slicing is a very enticing write strategy, getting both minimal overhead
post-compaction and fast random write speeds, but the idea has some
fundamental conflicts with how we play out attrs post-compaction.
This idea might work in a more powerful filesystem, but brings back the
need to simulate rbyds in RAM, which is something I really don't want to
do (complex, bug-prone, likely adds code cost, may not even be tractable).
So, third time's the charm?
---
This new write strategy writes only datas and bptrs, and avoids dagging
by completely rewriting any regions of data larger than a configurable
crystallization threshold.
This loses most of the benefits of data crystallization, random writes
will now usually need to rewrite a full block, but as a tradeoff our
data at rest is always stored with optimal overhead.
And at least data crystallization still saves space when our data isn't
block aligned, or in sparse files. From reading up on some other
filesystem designs it seems this is a desirable optimization sometimes
referred to as "tail-packing" or "block suballocation"
Some other changes from just having more time to think about the
problem:
1. Instead of scanning to figure out our current crystal size, we can
use a simple heuristic of 1. look up left block, 2. look up right
block, 3. assume any data between these blocks contribute to our
current crystal.
This is just a heuristic, so worst case you write the first and last
byte of a block which is enough to trigger compaction into a block.
But on the plus side this avoids issues with small holes preventing
blocks from being formed.
This approach brings the number of btree lookups down from
O(crystallize_size) to 2.
2. I've gone ahead and dropped the previous scheme of coalesce_size
+ fragment_size and instead adopted a single fragment_size that
controls the size of, well, fragments, i.e. data elements stored
directly in trees.
This affects both the inlined shrub as well as fragments stored in
the inner nodes of the btree. I believe it's very similar to what is
often called "pages" in logging filesystems, though I'm going to
avoid that term for now because it's a bit overloaded.
Previously, neighboring writes that, when combined, would exceed our
coalesce_size, they just weren't combined. Now they are combined up
to our fragment size, potentially splitting the right fragment.
Before (fragment_size=8):
.---+---+---+---+---+---+---+---.
| 8 bytes |
'---+---+---+---+---+---+---+---'
+
.---+---+---+---+---.
| 5 bytes |
'---+---+---+---+---'
=
.---+---+---+---+---+---+---+---+---+---.
| 5 bytes | 5 bytes |
'---+---+---+---+---+---+---+---+---+---'
After:
.---+---+---+---+---+---+---+---.
| 8 bytes |
'---+---+---+---+---+---+---+---'
+
.---+---+---+---+---.
| 5 bytes |
'---+---+---+---+---'
=
.---+---+---+---+---+---+---+---+---+---.
| 8 bytes |2 bytes|
'---+---+---+---+---+---+---+---+---+---'
This leads to better fragment alignment (much like our block
strategy), and minimizes tree overhead.
Any neighboring data to the right is only coalesced if it fits in the
current fragment, or would be rewritten (carved) anyways, to avoid
unnecessary data rewriting.
For example (fragment_size=8):
.---+---+---+---+---+---+---+---+---+---+---+---+---+---.
| 6 bytes | 6 bytes |2 bytes|
'---+---+---+---+---+---+---+---+---+---+---+---+---+---'
+
.---+---+---+---+---.
| 5 bytes |
'---+---+---+---+---'
=
.---+---+---+---+---+---+---+---+---+---+---+---+---+---.
| 8 bytes | 4 bytes |2 bytes|
'---+---+---+---+---+---+---+---+---+---+---+---+---+---'
Other than these changes this commit is mostly a bunch of carveshrub
rewriting again, which continues to be nuanced and annoying to get
bug free.
But already there are some pretty fundamental problems.
The main issue is that, while we correctly dereference slices during
compaction, pending commits that get delayed after compaction still
point to the old block. I'm not sure there's an easy way around this
aside from aborting compaction commits or fully simulating commits,
both of which seem too costly to implement...
Also coalescing during compaction is flawed as well, since our
attributes will be outdated by the time they are committed if there is a
compaction...
Looks like it's back to the drawing board. Either our approach to
compaction needs to change, or this slice/coalescing work needs to be
reverted/redesigned...
Note this is already showing better code reuse, which is a good sign,
though maybe that's just the benefit of reimplementing similar logic
multiple times.
Now both reading and carving end up in the same lfsr_btree_readnext and
lfsr_btree_buildcarve functions for both btrees and shrubs. Both btrees
and shrubs are fundamentally rbyds, so we can share a lot of
functionality as long as we redirect to the correct commit function at
the last minute. This surprising opportunity for deduplication was
noticed while putting together the dbg scripts.
Planned logic (not actual function names):
lfsr_file_readnext -> lfsr_shrub_readnext
| |
| v
'---------> lfsr_btree_readnext
lfsr_file_flushbuffer -> lfsr_shrub_carve ------------.
.---------------------' |
v v
lfsr_file_flushshrub -> lfsr_btree_carve -> lfsr_btree_buildcarve
Though the btree part of the above statement is only a hypothetical at
the moment. Not even the shrubs can survive compaction now.
The reason is the new SLICE tag which needs low-level support in rbyd
compact. SLICE introduces indirect refernces to data located in the same
rbyd, which removes any copying cost associated with coalescing.
Previously, a large coalesce_size risked O(n^2) runtime when
incrementally append small amounts of data, but with SLICEs we can defer
coalescing to compaction time, where the copy is effectively free.
This compaction-time-coalescing is also hypothetical, which is why our
tests are failing. But the theory is promising.
I was originally against this idea because of how it crosses abstraction
layers, requiring some very low-level code that absolutely can not be
omitted in a simpler littlefs driver. But after working on the actual
file writing code for a while I've become convinced the tradeoff is
worth it.
Note coalesce_size will likely still need to be configurable. Data in
fragmenting/sparse btrees is still susceptible to coalescing, and it's
not clear the impacts of internal fragmentation when data sizes approach
the hard block_size/2 limit.
My current thinking is that these are conceptually different types, with
BTREE tags representing the entire btree, and BRANCH tags representing
only the inner btree nodes. We already have multiple btree tags anyways:
btrees attached to files, the mtree, and in the future maybe a bmaptree.
Having separate tags also makes it possible to store a btree in a btree,
though I don't think we'll ever use this functionality.
This also removes the redundant weight field from branches. The
redundant weight field is only a minor cost relative to storage, but it
also takes up a bit of RAM when encoding. Though measurements show this
isn't really significant.
New encodings:
btree encoding: branch encoding:
.---+- -+- -+- -+- -. .---+- -+- -+- -+- -.
| weight | | blocks |
+---+- -+- -+- -+- -+ ' '
| blocks | ' '
' ' +---+- -+- -+- -+- -+
' ' | trunk |
+---+- -+- -+- -+- -+ +---+- -+- -+- -+- -'
| trunk | | cksum |
+---+- -+- -+- -+- -' '---+---+---+---'
| cksum |
'---+---+---+---'
Code/RAM changes:
code stack
before: 30836 2088
after: 30944 (+0.4%) 2080 (-0.4%)
Also reordered other on-disk structs with weight/size, so such structs
always have weight/size as the first field. This may enable some
optimizations around decoding the weight/size without needing to know
the specific type in some cases.
---
This change shouldn't have affected functionality, but it revealed a bug
in a dtree test, where a did gets caught in an mdir split and the split
name makes the did unreachable.
Marking this as a TODO for now. The fix is going to be a bit involved
(fundamental changes to the opened-mdir list), and similar work is
already planned to make removed files work.
Since we need an bptr type internally, a block pointer, which is a bit
more complicated than just a single address, calling our mdir pairs
mptrs makes sense.
Oh hey, it's that piece of complexity I was worried about.
The problem was that the position calculation for new appended
right_data depended on left_overlap, which fell out of sync when
transitioning from sprout->shrub.
The fix here is to keep left_overlap/right_overlap up to date with the
model that the sprout->shrub transition is effectively doing a
shrub-wide rm first.
Hacky, but hopefully avoids bugs in the future by keeping all of these
variables in a reasonable state...
There may be a simpler way to think about how this code should function,
but I just can't see it. This may deserve a rewrite in the future.
Ended up changing the name of lfsr_mtree_traversal_t -> lfsr_traversal_t,
since this behaves more like a filesytem-wide traversal than an mtree
traversal (it returns several typed objects, not mdirs like the other
mtree functions for one).
As a part of this changeset, lfsr_btraversal_t (was lfsr_btree_traversal_t)
and lfsr_traversal_t no longer return untyped lfsr_data_ts, but instead
return specialized lfsr_{b,t}info_t structs. We weren't even using
lfsr_data_t for its original purpose in lfsr_traversal_t.
Also changed lfsr_traversal_next -> lfsr_traversal_read, you may notice
at this point the changes are intended to make lfsr_traversal_t look
more like lfsr_dir_t for consistency.
---
Internally lfsr_traversal_t now uses a full state machine with its own
enum due to the complexity of traversing the filesystem incrementally.
Because creating diagrams is fun, here's the current full state machine,
though note it will need to be extended for any
parity-trees/free-trees/etc:
mrootanchor
|
v
mrootchain
.-' |
| v
| mtree ---> openedblock
'-. | ^ | ^
v v | v |
mdirblock openedbtree
| ^
v |
mdirbtree
I'm not sure I'm happy with the current implementation, and eventually
it will need to be able to handle in-place repairs to the blocks it
sees, so this whole thing may need a rewrite.
But in the meantime, this passes the new clobber tests in test_alloc, so
it should be enough to prove the file implementation works. (which is
definitely is not fully tested yet, and some bugs had to be fixed for
the new tests in test_alloc to pass).
---
Speaking of test_alloc.
The inherent cyclic dependency between files/dirs/alloc makes it a bit
hard to know what order to test these bits of functionality in.
Originally I was testing alloc first, because it seems you need to be
confident in your block allocator before you can start testing
higher-level data structures.
But I've gone ahead and reversed this order, testing alloc after
files/dirs. This is because of an interesting observation that if alloc
is broken, you can always increase the test device's size to some absurd
number (-DDISK_SIZE=16777216, for example) to kick the can down the
road.
Testing in this order allows alloc to use more high-level APIs and
focus on corner cases where the allocator's behavior requires subtlety
to be correct (e.g. ENOSPC).
This is an exciting new function, made possible by the order-statistic
nature of our rbyds and btrees.
lfsr_file_fruncation is like truncate, but from the front. It can trim
data off of the front of files, and grow files from the front,
effectively prefixing files with zeros cheaply.
This may have some niche use cases for prefixing files with headers, but
the real killer is making logging files trivial. Up until now logging
into a file has always resulted in awkward file-swapping code when a
file gets full. Now maintaining a log is just a single fruncate call.
---
Implementation wise, lfsr_file_fruncate is very similar to
lfsr_file_truncate, except we need to always inject holes into all file
trees to adjust file contents correctly.
What do you think a file's size becomes when you:
1. seek past the end of a file
2. call write with zero data!
POSIX/etc has this case explicitly mentioned, noting that zero-sized
writes should never update the file size.
This clashes with the assumption that file writes always update the file
position, but I suppose it makes a bit of practical sense if you want
zero-sized file writes to be idempotent.
This turned out to have limited use for the tests themselves. I was
hoping to avoid the mount->format->mount fallback when powerloss
testing, but we still need it in case format was interrupted.
Still, TEST_PLS is very useful for debugging.
Previouly it was difficult to set a breakpoint at a specific location,
and after a specific powerloss event. Now all you need is this in gdb:
b <line> if test_pls == <pls>
Turns out it's hard to test file holes without seek.
It's interesting to note most of seek's buffer flush work actually
occurs lazily in lfsr_file_write, so lfsr_file_seek turns out to be a
relatively simple function.
The main purpose of this change is to introduce LFSR_DATA_CAT, a
generalized way to concatenated various data references internally.
As a side-effect lfsr_data_t has been completely restructured. Now,
lfsr_data_t can be in one of 4 modes:
If the size field's sign bit=0, the lfsr_data_t points in-device. A new,
count field, determines the encoding:
sign(size)=0, count=0 => inlined:
.---+---+---+---.
| size |
|---+---+---+---|
|c=0| inlined d | note inlined data is just enough to hold
|---+ | one encoded leb128
| ata... |
'---------------'
sign(size)=1, count=1 => direct:
.---+---+---+---. .---+---+---+---.
| size | .>| data... |
|---+---+---+---| | | . |
|c=1| | | . . .
|---+---+---+---| | . . .
| direct ptr -----' . .
'---------------'
sign(size)=1, count>=2 => indirect:
.---+---+---+---. .---+---+---+---. .---+---+---+---.
| size | .>| size | .>| data... |
|---+---+---+---| | |---+---+---+---| | | . |
|c>1| | | |c=1| | | . . .
|---+---+---+---| | |---+---+---+---| | . . .
| indirect ptr ---' | direct ptr -----' . .
'---------------' '---------------' .---+---+---+---.
| size | .>| data... |
|---+---+---+---| | | . |
|c=1| | | . . .
|---+---+---+---| | . . .
| direct ptr -----' . .
'---+---+---+---'
| . |
| . |
. . .
. .
. .
note only one indirect layer is allowed due to no recursion
If the size field's sign bit=1, the lfsr_data_t points on-disk:
sign(size)=0 => on-disk:
.---+---+---+---. .....
| size | ..'' ''..
|---+---+---+---| : : :
| block ------+->| ..:|
|---+---+---+---| | |......( )::::::|
| off -------' |:::' : |
'---------------' :' : :
''.. :.''
'''''
My goal with this commit was to test the new implementation and see how
it would impact code/RAM size before adopting it in the actual file
handling code, and the results are... not great...
code stack
before: 24668 1840
after: 25552 (+3.5%) 1920 (+4.2%)
I think most of the new cost comes from the now correct handling of
read/cmp with concatentated datas, which previously would just assert.
This change gives us LFSR_DATA_CAT, so I will be working with it for
now, but this may be worth looking at again in the future. Maybe the
correct handling of read/cmp should just be reverted to an assert...
This will stop being a problem when we actually have btrees, but for now
the fragmentation caused by byte-level syncs was easily enough to
overflow an mdir when cache size is big.
A smaller cache size is also nicer for debugging, since smaller cache
sizes results in data getting flushed to disk earlier, which is easier
to inspect than in-device buffers. And a 16-byte cache still provides
decent test coverage over cache interactions.
---
Also dropped inline_size to block_size/8. I realized while debugging
that opened shrubs take up additional space until we sync, so we need to
expect up to 2 temporary copies of shrubs when writing files.
The main improvement is moving the special inlined-file compaction logic
up into lfsr_mdir_compact__. We only need this logic for files stored in
mdirs, and thanks to its recursive nature, we weren't getting any
benefit from handling this at a lower level anyways.
This is a nice logical restructuring that probably saves a bit of code
cost in the end.
Another significant improvement is moving the staging copy of the
inlined tree's state up into the file struct itself. This solves the
problem of needed N copies of temporary inlined state when you have N
open files.
It also provides a central place to stage changes when compacting
inlined trees, which happens across several different places in the mdir
commit logic. Though some may see this as more a hack than a feature.
Also note-worthy, but minor: these changes required an additional
opened-mdir linked-list to know when the mdir is a file and may contain
an inlined tree.
Inlined files are unfortunately turning out to have more cost than
expected, mainly due to our strict no-recursion requirement.
It turns out recursively nesting (bounded) trees in a system without
recursion is a recipe for duplicating code. Though there may be other
ways to structure this.
One interesting hiccup during development is the need to have both NULL
tags and DEFERREDNULL tags in order to tell inlined trees apart from the
main tree during compaction.
And by working, I mean you can create inlined trees, just don't
compact/split/move/etc anything. But this does outline the path files
take when writing buffers into inlined trees.
"Inlined trees" in littlefs are entire small rbyd trees embedded as
secondary trees in an mdir's main rbyd tree. When fetching, we can
indicate if a given trunk belongs to the main tree or secondary tree by
setting one of the unused mode bits in the trunk's tag, now called the
"deferred" bit. This bit doesn't need to be included in the alt's "key"
field, so there's no issue with it conflicting with the alt's mode bits.
This requires a bit of tweaking lfsr_rbyd_fetch, since it needs to fall
back to the previous trunk if it discovers the most recent trunk belongs
to an inlined tree. But as a benefit we can leverage the full power of
rbyds in inlined files, including holes, partial updates, etc.
One downside is it looks like these inlined trees may involve more work
in maintining their state correctly, since they need to be sort of
"brought along" when mdirs are compacted, even if they don't actually
have a reference in the mdir yet. But the sheer amount of flexibility
this gives inlined files may make this overhead worth it.
Ran into an interesting macro-related bug. Turns out the way we are
doing implicit prefixing in TAG/ATTR macros sort of breaks how C macros
work a bit. The following does not compile:
lfsr_mdir_commit(lfs, &file->m.mdir, LFSR_ATTRS(
LFSR_ATTR(file->m.mdir.mid, DEFER, 0, DEFER(
(lfsr_rbyd_t*)&file->inlined,
LFSR_ATTR(file->buffer_pos,
DEFERRED(INLINED), +file->buffer_size, BUF(
file->buffer, file->buffer_size))))));
Or to distill it down, this does not compile:
#define LFSR_ATTR(_data) (LFSR_##_data)
#define LFSR_DEFER(_data) (LFSR_##_data)
#define LFSR_DATA(_data) (_data)
int a = LFSR_ATTR(DEFER(ATTR(DATA(1))));
But this does:
#define LFSR_ATTR(_data) (_data)
#define LFSR_DEFER(_data) (_data)
#define LFSR_DATA(_data) (_data)
int a = LFSR_ATTR(LFSR_DEFER(LFSR_ATTR(LFSR_DATA(1))));
Why? Well it turns out the whole way nested C macro's work is a big
hack.
A very reasonable design decision in C is to disallow recursive macro
expansions. Unlike C++, we don't want our preprocessor to suddenly stack
overflow. This rule is enforced by stopping macro expansion when a macro
contains itself. For example:
#define A() B()
#define B() A()
A()
Expands to:
A()
-> B()
-> A() (stops, probably erroring with 'A' undeclared)
But it _is_ common to want to recursively expand macro arguments. Macros
are a part of C's syntax after all, and users usually expect
expressions, such as arguments, to be context-free:
#define A(x) (x) + 1
A(A(A(A(A(0)))))
Naively this would expand to:
A(A(A(A(A(0)))))
-> (A(A(A(A(0))))) + 1 (stops)
The big hack that makes this work in C's preprocessor is the "Argument
prescan". Instead of expanding the "called" macro first, we expand any macro
inside our argument list, _then_ expand the "called" macro, and _then_
expand any new macros produced as a result of the expansion again just
for good measure.
So the above actually expands to:
A(A(A(A(A(0)))))
-> A(A(A(A((0) + 1))))
-> A(A(A(((0) + 1) + 1)))
-> A(A((((0) + 1) + 1) + 1))
-> A(((((0) + 1) + 1) + 1) + 1)
-> (((((0) + 1) + 1) + 1) + 1) + 1
This is still recursive actually! But the recursion is limited to the
actual length of the source code, so the developers likely thought this
was a reasonable tradeoff.
But what does this mean for our implicit prefixing?
#define P_A(x) P_##x
#define P_B(x) P_##x
#define P_C(x) (x)
P_A(B(A(C(0))))
None of A, B, C are in scope without prefixes, so they get expanded
after the "called" macro's expansion:
P_A(B(A(C)))
-> P_B(A(C(0)))
-> P_A(C(0)) (stops)
But this breaks when we hit the nested P_A macro.
---
For now I've gone with the temporary, and extra hacky, solution of
introducing a second LFSR_ATTR_ macro. This nesting of ATTR macros only
happens because of shrubs, and only ever goes 2 layers deep.
In the future maybe we should move away from implicit prefixing. They
have a few rough corners and may be a bit confusing for anyone new to
the code.
Currently limited to inlined files and only simpler truncate-writes.
But still this lets us test file creation/deletion.
This is also enough logic to make it clear that, even though we have
some powerful high-level primitives, mapping file operations onto these
is still going to be non-trivial.
- mbits -> mleaf_bits
- mlimit -> mleaf_limit
- mweight -> mleaf_weight
- lfsr_mridmask -> lfsr_midrmask
- lfsr_mbidmask -> lfsr_midbmask
This is a bit tricky to name, since we want to clarify it's not the
mtree limit and not the mdir's actual rbyd weight. But this also risks
confusing around the difference between mdirs/mleaves (mdirs are
mtree's leaves).
- Fixed LFSR_GRM_DSIZE upper bound, since our mids now fit in a single
leb128.
- Renamed pgrm -> ggrm. To be honest I don't have a great name for this
variable.
Taking advantage of the fact that these functions should never error,
changing the return type to lfsr_data_t allows all of the encoding
information to be passed around quite easily.
And, by giving each lfsr_data_from* function an LFSR_DATA_FROM* macro,
these functions can participate in our attr-list generating macros:
LFSR_ATTR(-1, MTREE, 0, FROMBTREE(lfs, mtree, mtree_buf))
Though one thing to watch out for is the borrowed buffer that stores the
actual data. This might welcome use-after-free bugs since it's not super
clear the buffer remains borrowed. Will need to watch out for this.
Adopted lfsr_rid/bid/mid/did_t where appropriate. This includes using
lfsr_rid_t for tag/rbyd weights. Although I am using lfsr_srid_t for
rbyd weights now, since it both captures the use of the sign bit and
reduces the number of casts a bit in the code.
I learned recently Zig has any-bit integers (e.g. uint31_t), and I'm
realizing how nice it would be to have those in this codebase.
Also tried to use lfs_size_t/lfs_off_t more correctly. In Linux/BSD,
only off_t is used for file-size-related operations and is usually much
larger than size_t. These were used interchangably in littlefs and their
original meaning kind of fell by the wayside. Getting their use right
will be important if littlefs ever supports different integer widths.
This only matters for developers, not users, but it still helps a lot to
get debug representations right.
Since the exact mid encoding depends on the block_size in an unintuitive
manner, it's tricky to render in a debug-friendly way that is useful
both with and without tools.
Previously, I avoided shifting the bid representation, since this would
be closer to the value in the device, but this hides the actual
structure of the mtree. Now the bid is shifted, showing the underlying
mtree/mdir structure, at the cost of needing to know the number of mbits
to encode the mid back into an integer.
So for example, on a device with 4KiB blocks, or 8 mbits:
mid=1
mid=258
mid=515
Becomes:
mid=0.1
mid=1.2
mid=2.3
This continues to make the mbits a more fundamental part of littlefs,
but that's probably just how that's going to be.
This is a simpler way to track dropped mids. Setting trunk=0 was more a
workaround that worked but added more purpose to the trunk field than
originally needed. The mdir's trunk usually still exists after all.
Using mid=-1 previously didn't work due to conflict with mid=-1 to
indicate an mdir is an mroot, but since removed mids only appear in the
opened-mdir list, and the opened-mdir list stores inlined mdirs as
mid=0, this is no longer a problem.
One downside of this change is we no longer get implicit NOENT behavior
from lfsr_rbyd_lookup when attempting to lookup a removed mid, but it
wasn't clear this behavior was going to be very useful...
These tests serve as a direct example of why we can't just return the
difference between the dir's bookmark mid and position mid, which is
unfortunate.
This bit of code allows us to mount an "inconsistent" filesystem after
powerloss and behave as though we've fixed any pending grms without
actually fixing the grms. This lets the filesystem appear consistent
without needing to modify the disk, and allows truely readonly mounts
without sacrificing powerloss-resilience.
This works by just checking any readonly mid operations against pending
grms and returning NOENT if a fix would remove the mid. Fortunately the
more complex mid operations occur when mutating the filesystem, which we
can ignore as any mutation must be preceded by fixing pending grms.
This check has been added to lfsr_mtree_namelookup and lfsr_mtree_seek,
which should propagate the behavior to high-level functions with minimal
code impact.
This leaves only lfsr_mtree_lookup ignoring pending grms, which is useful
because we need it to actually fix the grms. I don't believe this
function will ever be called by a high-level function directly...
Coverage of readonly grms have also been added to the tests.
- Added lfsr_mdir_lookupnext, for iteration through only a single mid.
This is useful for MOVE attributes.
- Renamed LFSR_MDIR_MROOTANCHOR -> LFSR_MROOTANCHOR.
- Renamed functions that operate on mdir blocks lfsr_mdir_* ->
lfsr_mblocks_*.
- Reordered arguments in lfsr_mdir_fetch.
- Renamed mrid_bits/mbid_weight -> mbits/mweight.
This format for mids is a compromise in readability vs debugability.
For example, if our mbid weight is 256 (4KiB blocks), the 19th entry
in the second mdir would be the raw integer 275. With this mid format,
we would print it as 256.19.
The idea is to make it easy to see it's the 19th entry in the mdir while
still making it relatively easy to see that 256.19 and 275 are
equivalent when debugging.
---
The scripts also took some tweaking due to the mid change. Tried to keep
the names consistent, but I don't think it's worthwhile to change too
much of the scripts while they are working.
This adopts a previously discarded idea for compressed mids with a few
tweaks to avoiding decoding the bid/rid portions as much as possible.
The idea of compressed mids is to shove both the mid bid and mid rid
into a single integer, saving RAM and potentially helping filesystem
integration where a unique per-file integer is useful.
Unfortunately this has proven tricky. littlefs fundamentally needs two
ids, one "bid" to lookup which mdir our entry resides on, and one "rid"
to lookup the entry in the mdir. It's tempting to use two half-sized
integers (16-bit for example), but this risks surprising limitations
around the number of files when blocks are either really large or
really small.
Optimally, we'd limit the number of bits reserved for the rid to the
upper bound of number of rids that can fit in a single mdir. This would
allows for more bids when the block size is small, and more rids when
the block size is large. This should roughly approximate the limits of
a per-file integer.
With a bit of math we can estimate the upper bound to be <=block_size/16
with our current compaction strategy.
This idea was previously discarded due to the overhead of extracting the
bids/rids when we need them, but the RAM savings and file-to-integer
mapping is too useful to give up. When it became clear half-width
integers wasn't really going to work, compressed mids became the new
plan:
0bbbbbbb bbbbbbbb bbbbbbbb rrrrrrrr
^'-----------+-----------' '---+--'
'------------|-----------------|---- sign-bit, reserved for driver
| '---- nlog2(bs/16) bits for rid
| (8-bits for 4KiB blocks)
'---------------------- remaining bits for bid
(23-bits for 4KiB blocks)
To reduce the overhead of encoding/decode bids/rids a few extra features
were added to the internal mdir APIs:
1. The mtree has been changed to store mids directly. Giving each mdir
the upper bound as a weight. This allows direct lookup of mids
without any sort of bid decoding, though does bake the upper bound
estimate into the metadata of the filesystem, which isn't the
cleanest design, but if it works it works.
On the plus side, with this upper bound baked in to the filesystems,
GRMs can be encoded in a single leb128, which is nice. This may have
other savings if we ever store mids anywhere else in the filesystem.
2. rids are now mid relative in lfsr_mdir_lookup when non-negative. This
is implemented with a simple condition that is hopefully optimized
out when inlined, though there may be some room for improvement here.
3. rids are now mid relative in lfsr_mdir_commit. This was a bit tricky,
but we can leverage the existing mechanisms for bid-relative rids
used in the btree implementation.
The above changes make it so you can pass the mid around directly for
most of the mdir functions, hopefully reducing the mid decoding
overhead. This savings should only grow as more high-level filesystem
APIs are added.
Here is the resulting code/RAM changes for this entire change (from
before we adopted the mroot bit):
code stack structs
before: 20590 1784 908
after: 20890 (+1.4%) 1744 (-2.3%) 864 (-5.1%)
We weren't comparing mid=-1/mid=0 correctly in lfsr_mdir_commit, which
can happend now thanks to inlining mids in our mroot. This went
unnoticed because we were just copying mroot.mid in our tests so we
never actually tested with mid=0. This is fixed now and the tests test
with a literal mid=0.
This also changes the mbids to be left-leaning, carving out an
mrid-sized number of bits from the mbid, making the route to compressed
mids easier.
This greatly simplifies mid handling at the cost of increased subtlety
around determining if a given mdir is an mroot.
Fortunately it turns out we can rely on context to determine if an mdir
is an mroot or not:
1. If an mdir's mid.bid == -1, it's an mroot. This is always true for
fake mroots, since they can't hold any inlined mids.
2. If the mtree is inlined (mtree.weight == 0), all mdirs are mroots.
This lets us use mid.bid=0 for inlined mids. We just need to check
if the mtree is inlined before deciding if the mdir is an mroot or
not.
The makes it so that for any non-mroot mdir, mid.bid=-1 is always a
reserved value. Which is very useful for compressed mids.
More on this when explaining compressed mids, but basically the idea is
instead of just storing all mdirs in our mtree as single element
entries, store each mdir in as a weighted entry, where the weight is a
known upper bound on the possible number of mid entries in a single
mdir.
With the current mid representation, this just complicates things
without much benefits. But with compressed mids it allows us to lookup
mdirs with the mid directly, and avoid decoding the bid from the mid in
some cases.
The mid-per-mdir upper bound is derived from the block size. We know:
1. Each tag needs <=2 alts+null with our current compaction strategy
2. Each tag/alt encodes to a minimum of 4 bytes
This gives us ~4*4 or ~16 bytes per mid at minimum. If we cram an mdir
with the smallest possible mids, this gives us at most ~block_size/16
mids in a single mdir before the mdir runs out of space.
Note we can't assume ~1/2 block utilization here, as an mdir may
temporarily fill with more mids before compaction occurs.
This is an intermediate commit as a part of a tangent into compressed
mids.
The idea here, is instead of using bid=-1 as a special value for mroots,
use only the top bit to indicate mroots. This allows you to compare
against the grm/other uninlined mids by masking instead of signed
comparison.
This is valuable for compressed mids since extracting bids relies on
knowledge of the block size, and becomes quite a bit more expensive.
mroot bid mroot cmp
before: 0xffffffff lfs_smax32(a, 0) == lfs_smax32(b, 0)
after: 0x80000000 (a & 0x7fffffff) == (b & 0x7fffffff)
The implementation here is a bit clumsy. I think GCC may be not that
great at optimizing out copies of structs being passed around via
inlined functions. But this is only a proof-of-concept.
There is a bit of redundancy here, as we already know the weights of
btree's inner-branches from their parents. But in theory sharing the
same encoding for both the top level btree reference and inner-branches
should offer more chance for deduplication and hopefully less code.
This also moves some members around in the btree encoding so that the
redund blocks are at the beginning. This _might_ simplify decoding of
the variable-length redund blocks at some point.
Current btree encoding:
.----+----+----+----.
| blocks ... redund leb128s (1-20 bytes)
: :
|----+----+----+----|
| trunk ... 1 leb128 (1-5 bytes)
|----+----+----+----|
| weight ... 1 leb128 (1-5 bytes)
|----+----+----+----|
| cksum | 1 le32 (4 bytes)
'----+----+----+----'
This also partially reverts some tag name changes:
- BNAME -> BRANCH
- DMARK -> BOOKMARK
Our rbyds support changing the weight of a tag without knowing the
actual tag. This is useful for btrees, which always make weight changes
without knowing if the leading tag is a name or a branch (it depends on
the type of btree).
But to make this work, it needs the rm-bit to be set. This is because
internally the rm-bit indicates we don't want to write-out a tag. Which
we don't for grow tags, because, well, they're not real tags.
Previously this was done by putting LFSR_TAG_GROW(RM) everywhere a
generic grow as needed, but since this is so common we might as well
just set the rm-bit in LFSR_TAG_GROW.
Note that LFSR_TAG_GROW(tag) (the macro) does not set the rm-bit.
This makes the code a bit more readable at the risk of an unintuitive
relationship between LFSR_TAG_GROW and LFSR_TAG_GROW(tag).
Thanks to lazy merging, our btree nodes can drop to zero weight at
pretty much any time. Unfortunately, we can't really represent non-root
zero weight btree nodes, so things break. (Though even if we could,
those nodes would become unreachable).
Previously we relied on fuzz testing to try to catch these cases, but
that turned out to be insufficient.
This adds explicit tests covering the cases where btree drops can occur,
thanks to the realy-big-attr trick used in similar mtree tests.
Sure enough this revealed a bug that can occur when we split a btree
node at the same time one of the siblings goes to zero weight. (Remember
splits carried out before playing attr-lists).
---
Fortunately this is pretty easy to fix. We can just reroute our split
code to the normal commit/compact recursion handling if one of our
siblings drops to zero, at the cost of some spaghetti.
xkcd.com/292 seems relevant here.
With the introduction of lfsr_data_t, these stopped being useful
functions for littlefs internally.
Maybe these tests should be rewritten to use the *_lookup functions
directly? Unfortunately with the quantity of tests we have now this adds
non-trivial amount of work with questionable benefit.
These functions are no longer needed in lfs.c. They are still needed for
the tests as they are written, but that's not a reason to pollute the
littlefs source code.
Maybe these tests should be rewritten to use lfsr_btree_commit directly?
Unfortunately with the quantity of tests we have now this adds
non-trivial amount of work with questionable benefit.
Note this required changing the INLINED tag to REG in most of the tests,
because our mtree now explicitly requires some sort of NAME tag.
We can also finally see the impact on code and RAM from this restructure:
code stack
before (push/set/pop/split): 21750 1928
after (commit): 20970 (-3.7%) 1744 (-10.6%)
Not too shabby if I say so myself.
This adds an extra bid parameter to lfsr_rbyd_appendall so that attrs
relative to a bid can be adjusted correctly.
This allows us to make attr-lists const again, which is generally a good
things. Passing around complex mutable state is just asking for bugs.
Though since these attr-lists are generally just passed as temporary
arguments, maybe it's not that bad?
The idea here: Instead of having unique functionality for each
individual btree operation (push/set/pop/split), we treat btrees sort of
like rbyds, with a single commit entry point that operates on attr-lists.
This adds code cost, due to needing to parse the attr-list for properties
that can affect inlined btrees (tag changes mostly), but, in theory, comes
with some advantages:
1. A single btree commit entry point with all of the inlined/uninlining
logic should offer better chances for code deduplication, vs
spreading this logic out in each btree operation.
2. Higher-levels should know what the current weight of the branch is,
so we may be able to avoid the implicit math needed to calculate
deltas.
3. Higher-levels have more knowledge about the state of the btree in
general, so there may be other shortcuts. The mtree, for example,
only operates on weight=1 entries, which greatly simplifies a lot of
the related math.
Note that btrees still have strict limits in what's possible in an
attr-list. Btree operations can't cross leaf-rbyd boundaries for
example.
---
A notable omission in this change is the loss of reinlining btrees.
This wase dropped for a couple reasons. It may be worth adding back at a
later time, maybe after we actually have files implemented, but for now
does not seem worth it:
1. Reinlining adds code cost. Reinlining is more complex than you might
expect because we only reinline on compaction. And because we compact
before playing out our attr-list, we need to know if a commit makes
the btree inlinable before committing to the btree.
This is still doable with our attr-lists. We already derive the
change in tags, since we need this to know when to uninline. But it
adds a kind of complex bailing out of btree commits.
2. The benefits of reinlining may not be that great. In most systems, a
tree that is uninlined once is likely to be uninlined again. It's
only if there is a bigger state change in a system that it makes
sense to reinline.
Though, to be fair, waiting for compaction to reinline handled this
quite well. Only reinlining when all erased storage is used up...
3. Thanks to our roots did entry, our mtree can never reinline.
It would be nice to change this, but this would require explicit
handling in lfsr_mdir_commit. Future work?
4. Files are another can of worms, with more complex interactions with
inlinability thanks to (at least on paper right now) always having
inlined data even when uninlined.
If reinlining is valuable for files this can change during that work.
5. Even if files never support reinlinability, truncating files (via
either lfsr_file_truncate or LFSR_O_TRUNC) should give the file a
blank slate, effectively reinlining the file in that case.
---
The current implementation also changes the attr-list to be mutable so
we can adjust attr-list based on the current btree node. This is a
temporary hack! We should add the appropriate functionality to our rbyd
utilities to revert this eventually.
Christopher Haster