These emulate powerloss behavior where only some of the bits being
progged are actually progged if there is a powerloss. This behavior was
the original motivation for our ecksums/fcrcs, so it's good to have this
tested.
As a simplification, these only test the extremes:
- LFS_EMUBD_POWERLOSS_SOMEBITS => one bit progged
- LFS_EMUBD_POWERLOSS_MOSTBITS => all-but-one bit progged
Also they flips bits instead of preserving exact partial prog behavior,
but this is allowed (progs can have any intermediate value), has the
same effect as partial progs, and should encourage failed progs.
This required a number of tweaks in emubd: moved powerloss before prog,
moved mutate after powerloss, etc, but these shouldn't affect other
powerloss behaviors. Handling powerloss after prog was only to avoid
power_cycles=1 being useless, it's not strictly required.
Good news is testing so far suggests our ecksum design is sound.
This sort of reverts the addition of lfsr_bd_unprog, but with a slightly
better API. lfsr_bd_unprog was too much of a hack, and isn't really
generalizable. The align flag isn't necessarily any better, but at least
it's the simplest/least-confusing solution available.
And it's net savings, code-wise:
code stack lfs_t
before: 33690 2608 164
after: 33678 (-0.0%) 2600 (-0.3%) 160 (-2.4%)
This acts as a marker to indicate a fuzz test. It should reference a
define, usually SEED, that can be randomized to get interesting test
permutations.
This is currently unused, but could lead to some interesting uses such
as time-based fuzz testing. It's also just useful for inspecting the
tests (make test-list).
This turned into a sort of system-wide refactor based on learned
knowledge of what we can do with lfsr_attr_t.
The big changes:
- Reverted LFSR_ATTR to mainly take lfsr_data_t again, keeping
lfsr_data_t as the default data representation in the codebase.
Now that we know
LFSR_ATTR_CAT_ still provides concatenation mechanics, and LFSR_ATTR_
provides a way to edit in-flight lfsr_attr_ts.
- Dropped lfsr_cat_t, replaced with explicit const void* + uint16_t,
tried to limit to low-level operations and prefer passing aroud
lfsr_attr_t and lfsr_data_t at a high-level.
Note this cat + cat_count pair is quite similar to the common attrs +
attr_count and buffer + size arguments.
- Adopted lfsr_attr_t more in mid-level functions, lfsr_rbyd_appendattr,
lfsr_rbyd_appendcompactattr, lfsr_file_carve, etc. This is a bit more
ergonomical, allows for use of LFSR_ATTR* macros, and in theory might
even save a bit of stack.
Unfortunately this seems to have resulted in a net hit to code cost,
though I still think it's worth it for the internal ergonomics:
code stack
before: 33652 2624
after: 33780 (+0.4%) 2640 (+0.4%)
Investigating further suggests this may just be the result of compiler
noise and changes to argument placement. lfsr_attr_t does touch a lot of
code...
It's interesting to note the adoption of lfsr_attr_t in
lfsr_rbyd_appendattr* and friends prevents their transformation into
.isra functions, though this doesn't seem to impact code cost too much:
function (5 added, 5 removed) osize nsize dsize
lfsr_cat_size - 48 +48 (+100.0%)
lfsr_file_carve - 1600 +1600 (+100.0%)
lfsr_rbyd_appendattr - 2120 +2120 (+100.0%)
lfsr_rbyd_appendattr_ - 244 +244 (+100.0%)
lfsr_rbyd_appendcompactattr - 68 +68 (+100.0%)
lfsr_rbyd_appendcompactrbyd 144 152 +8 (+5.6%)
lfsr_file_truncate 298 314 +16 (+5.4%)
lfsr_mdir_commit__ 1056 1112 +56 (+5.3%)
lfsr_mdir_compact__ 502 526 +24 (+4.8%)
lfsr_rbyd_appendattrs 132 138 +6 (+4.5%)
lfsr_file_fruncate 386 402 +16 (+4.1%)
lfsr_data_frombtree 84 86 +2 (+2.4%)
lfsr_rbyd_appendcksum 512 520 +8 (+1.6%)
lfsr_file_opencfg 572 580 +8 (+1.4%)
lfsr_rename 608 616 +8 (+1.3%)
lfsr_mkdir 500 504 +4 (+0.8%)
lfsr_bd_prog 278 280 +2 (+0.7%)
lfsr_mdir_commit 2364 2360 -4 (-0.2%)
lfsr_bshrub_commit 716 712 -4 (-0.6%)
lfsr_file_sync 526 514 -12 (-2.3%)
lfsr_file_flush_ 1868 1820 -48 (-2.6%)
lfsr_remove 456 436 -20 (-4.4%)
lfsr_fs_fixgrm 168 160 -8 (-4.8%)
lfsr_cat_size.isra.0 42 - -42 (-100.0%)
lfsr_file_carve.isra.0 1596 - -1596 (-100.0%)
lfsr_rbyd_appendattr.isra.0 2088 - -2088 (-100.0%)
lfsr_rbyd_appendattr_.isra.0 232 - -232 (-100.0%)
lfsr_rbyd_appendcompactattr.isra.0 56 - -56 (-100.0%)
TOTAL 33652 33780 +128 (+0.4%)
So now, instead of one data type trying to do everything, we have two:
1. lfsr_data_t - Readable data, either in-RAM or on-disk
2. lfsr_cat_t - Concatenated data for progging, may be either a simple
in-RAM buffer or an indirect list of lfsr_data_ts
This comes from an observation that most lfsr_attr_t datas were either
simple buffers, NULL, or required the indirect concatenated datas
anyways (concatendated file fragments). By separating lfsr_cat_t and
lfsr_data_t, maybe we can save RAM in lfsr_attr_t by not needing the
three words necessary for the less-common disk references.
Note the interesting tradeoff:
Simple in-RAM buffers/NULL decrease by 1 word (4 bytes):
lfsr_data_t lfsr_cat_t
.---+---+---+---. .---+---+---+---.
|0| size | => |0| size |
+---+---+---+---+ +---+---+---+---+
| ptr | | ptr |
+---+---+---+---+ '---+---+---+---'
| (unused) |
'---+---+---+---'
'-------.-------' '-------.-------'
12 bytes 8 bytes
While on-disk references increase by 2 words (8 bytes):
lfsr_data_t lfsr_cat_t lfsr_data_t
.---+---+---+---. .---+---+---+---. .---+---+---+---.
|1| size | => |1| size | .>|1| size |
+---+---+---+---+ +---+---+---+---+ | +---+---+---+---+
| block | | ptr -------' | block |
+---+---+---+---+ '---+---+---+---' +---+---+---+---+
| off | | off |
'---+---+---+---' '---+---+---+---'
'-------.-------' '-----------------.-----------------'
12 bytes 20 bytes
Unless the on-disk references also need concatenation, in which case
this still saves 1 word (4 bytes).
Note I'm not sure this type split is generalizable to other systems. In
littlefs we can't use recursion, so progging concatenated datas already
required two nested functions, and we happen to never need to read
concatenated data, allowing us to completely omit that functionality. In
other systems, where maybe disk-reference attrs are more common, this
tradeoff may not make sense.
Some other things to note:
- We're also losing the inlined-data representation in this change.
Unfortunately earlier lfsr_data_t measurements showed that this didn't
really contribute much. It saved RAM in name attrs but added quite a
bit of complexity to lfsr_data_t operations.
- By separating simple/cat and RAM/disk, we reduce the abused size bits
from 2-bits down to 1-bit. This doesn't really matter for our current
31/28-bit littlefs impl, but is nice in that it reenables the
theoretical 31/31-bit littlefs impl without in-RAM data-structure
changes.
There are a few temporary hacks that need to be figured out, but this is
already showing code/stack savings. Which is fascinating considering the
new lfsr_cat_* functions and increased temporary allocations:
code stack
before: 33856 2824
after: 33812 (-0.1%) 2800 (-0.8%)
Now that reproducibility issues with erase_value=-1 (erase=noop) are
fixed, this much more useful to test than erase_value=0x1b. Especially
since erase=noop is filled with so many sharp corners.
These tests already found that we were being too confident with our
leb128/lleb128/tag parsing. Since we need to partially parse unfinished/
old commits, lfsr_dir_read* can easily encounter invalid leb128s during
normal operation. If this happens we should not assert.
Doing things correctly has a bit of a cost:
code stack
before: 33928 2824
after: 33976 (+0.1%) 2824 (+0.0%)
At least we haven't seen any issues with our valid bit invalidating
logic yet.
Long story short, rbyd checksums are now fully reproducible. If you
write the same set of tags to any block, you will end up with the same
checksum.
This is actually a bit tricky with littlefs's constraints.
---
The main problem boils down to erased-state. littlefs has a fairly
flexible model for erased-state, and this brings some challenges. In
littlefs, storage goes through 2 states:
1. Erase - Prepare storage for progging. Reads after an erase may return
arbitrary, but consistent, values.
2. Prog - Program storage with data. Storage must be erased and no progs
attempted. Reads after a prog must return the new data.
Note in this model erased-state may not be all 0xffs, though it likely
will be for flash. This allows littlefs to support a wide range of
other storage devices: SD, RAM, NVRAM, encryption, ECC, etc.
But this model also means erased-state may be different from block to
block, and even different on later erases of the same block.
And if that wasn't enough of a challenge, _erased-state can contain
perfectly valid commits_. Usually you can expect arbitrary valid cksums
to be rare, but thanks to SD, RAM, etc, modeling erase as a noop, valid
cksums in erased-state is actually very common.
So how do we manage erased-state in our rbyds?
First we need some way to detect it, since we can't prog if we're not
erased. This is accomplished by the forward-looking erased-state cksum
(ecksum):
.---+---+---+---. \
| commit | |
| | |
| | |
+---+---+---+---+ +-.
| ecksum -------. | | <-- ecksum - cksum of erased state
+---+---+---+---+ | / |
| cksum --------|---' <-- cksum - cksum of commit,
+---+---+---+---+ | including ecksum
| padding | |
| | |
+---+---+---+---+ \ |
| erased | +-'
| | /
. .
. .
You may have already noticed the start of our problems. The ecksum
contains the erased-state, which is different per-block, and our rbyd
cksum contains the ecksum. We need to include the ecksum so we know if
it's valid, but this means our rbyd cksum changes block to block.
Solving this is simple enough: Stop the rbyd's canonical cksum before
the ecksum, but include the ecksum in the actual cksum we write to disk.
Future commits will need to start from the canonical cksum, so the old
ecksum won't be included in new commits, but this shouldn't be a
problem:
.---+---+---+---. . . \ . \ . . . . .---+---+---+---. \ \
| commit | | | | commit | | |
| | | +- rbyd | | | |
| | | | cksum | | | |
+---+---+---+---+ +-. / +---+---+---+---+ | |
| ecksum -------. | | | ecksum | . .
+---+---+---+---+ | / | +---+---+---+---+ . .
| cksum --------|---' | cksum | . .
+---+---+---+---+ | +---+---+---+---+ . .
| padding | | | padding | . .
| | | | | . .
+---+---+---+---+ \ | . . . . . . . +---+---+---+---+ | |
| erased | +-' | commit | | |
| | / | | | +- rbyd
. . | | | | cksum
. . +---+---+---+---+ +-. /
| ecksum -------. | |
+---+---+---+---+ | / |
| cksum ------------'
+---+---+---+---+ |
| padding | |
| | |
+---+---+---+---+ \ |
| erased | +-'
| | /
. .
. .
The second challenge is the pesky possibility of existing valid commits.
We need some way to ensure that erased-state following a commit does not
accidentally contain a valid old commit.
This is where are tag's valid bits come into play: The valid bit of each
tag must match the parity of all preceding tags (equivalent to the
parity of the crc32c), and we can use some perturb bits in the cksum tag
to make sure any tags in our erased-state do _not_ match:
.---+---+---+---. \ . . . . . .---+---+---+---. \ \ \
|v| tag | | |v| tag | | | |
+---+---+---+---+ | +---+---+---+---+ | | |
| commit | | | commit | | | |
| | | | | | | |
+---+---+---+---+ +-----. +---+---+---+---+ +-. | |
|v|p| tag | | | |v|p| tag | | | | |
+---+---+---+---+ / | +---+---+---+---+ / | | |
| cksum | | | cksum | | . .
+---+---+---+---+ | +---+---+---+---+ | . .
| padding | | | padding | | . .
| | | | | | . .
+---+---+---+---+ . . . | . . +---+---+---+---+ | | |
|v---------------- != --' |v------------------' | |
| erased | +---+---+---+---+ | |
. . | commit | | |
. . | | | |
+---+---+---+---+ +-. +-.
|v|p| tag | | | | |
+---+---+---+---+ / | / |
| cksum ----------------'
+---+---+---+---+ |
| padding | |
| | |
+---+---+---+---+ |
|v---------------- != --'
| erased |
. .
. .
New problem! The rbyd cksum contains the valid bits, which contain the
perturb bits, which depends on the erased-state!
And you can't just derive the valid bits from the rbyd's canonical
cksum. This avoids erased-state poisoning, sure, but then nothing in the
new commit depends on the perturb bits! The catch-22 here is that we
need the valid bits to both depend on, and ignore, the erased-state
poisoned perturb bits.
As far as I can tell, the only way around this is to make the rybd's
canonical cksum not include the parity bits. Which is annoying, masking
out bits is not great for bulk cksum calculation...
But this does solve our problem:
.---+---+---+---. \ . . . . . .---+---+---+---. \ \ \ \
|v| tag | | |v| tag | | | o o
+---+---+---+---+ | +---+---+---+---+ | | | |
| commit | | | commit | | | | |
| | | | | | | | |
+---+---+---+---+ +-----. +---+---+---+---+ +-. | | |
|v|p| tag | | | |v|p| tag | | | | . .
+---+---+---+---+ / | +---+---+---+---+ / | | . .
| cksum | | | cksum | | . . .
+---+---+---+---+ | +---+---+---+---+ | . . .
| padding | | | padding | | . . .
| | | | | | . . .
+---+---+---+---+ . . . | . . +---+---+---+---+ | | | |
|v---------------- != --' |v------------------' | o o
| erased | +---+---+---+---+ | | |
. . | commit | | | +- rbyd
. . | | | | | cksum
+---+---+---+---+ +-. +-. /
|v|p| tag | | | o |
+---+---+---+---+ / | / |
| cksum ----------------'
+---+---+---+---+ |
| padding | |
| | |
+---+---+---+---+ |
|v---------------- != --'
| erased |
. .
. .
Note that because each commit's cksum derives from the canonical cksum,
the valid bits and commit cksums no longer contain the same data, so our
parity(m) = parity(crc32c(m)) trick no longer works.
However our crc32c still does tell us a bit about each tag's parity, so
with a couple well-placed xors we can at least avoid needing two
parallel calculations:
cksum' = crc32c(cksum, m)
valid' = parity(cksum' xor cksum) xor valid
This also means our commit cksums don't include any information about
the valid bits, since we mask these out before cksum calculation. Which
is a bit concerning, but as far as I can tell not a real problem.
---
An alternative design would be to just keep track of two cksums: A
commit cksum and a canonical cksum.
This would be much simpler, but would also require storing two cksums in
RAM in our lfsr_rbyd_t struct. A bit annoying for our 4-byte crc32cs,
and a bit more than a bit annoying for hypothetical 32-byte sha256s.
It's also not entirely clear how you would update both crc32cs
efficiently. There is a way to xor out the initial state before each
tag, but I think it would still require O(n) cycles of crc32c
calculation...
As it is, the extra bit needed to keep track of commit parity is easy
enough to sneak into some unused sign bits in our lfsr_rbyd_t struct.
---
I've also gone ahead and mixed in the current commit parity into our
cksum's perturb bits, so the commit cksum at least contains _some_
information about the previous parity.
But it's not entirely clear this actually adds anything. Our perturb
bits aren't _required_ to reflect the commit parity, so a very unlucky
power-loss could in theory still make a cksum valid for the wrong
parity.
At least this situation will be caught by later valid bits...
I've also carved out a tag encoding, LFSR_TAG_PERTURB, solely for adding
more perturb bits to commit cksums:
LFSR_TAG_CKSUM 0x3cpp v-11 cccc -ppp pppp
LFSR_TAG_CKSUM 0x30pp v-11 ---- -ppp pppp
LFSR_TAG_PERTURB 0x3100 v-11 ---1 ---- ----
LFSR_TAG_ECKSUM 0x3200 v-11 --1- ---- ----
LFSR_TAG_GCKSUMDELTA+ 0x3300 v-11 --11 ---- ----
+ Planned
This allows for more than 7 perturb bits, and could even mix in the
entire previous commit cksum, if we ever think that is worth the RAM
tradeoff.
LFSR_TAG_PERTURB also has the advantage that it is validated by the
cksum tag's valid bit before being included in the commit cksum, which
indirectly includes the current commit parity. We may eventually want to
use this instead of the cksum tag's perturb bits for this reason, but
right now I'm not sure this tiny bit of extra safety is worth the
minimum 5-byte per commit overhead...
Note if you want perturb bits that are also included in the rbyd's
canonical cksum, you can just use an LFSR_TAG_SHRUBDATA tag. Or any
unreferenced shrub tag really.
---
All of these changes required a decent amount of code, I think mostly
just to keep track of the parity bit. But the isolation of rbyd cksums
from erased-state is necessary for several future-planned features:
code stack
before: 33564 2816
after: 33916 (+1.0%) 2824 (+0.3%)
This plays better with prettyasserts.py, which prints the err value on
failure.
We _could_ extend prettyasserts.py to print the contents of !err
patterns, but this risks making the error message more confusing when
the target is an actual boolean expression. Keep in mind
prettyasserts.py is purely syntactical and doesn't really know the
expression's type.
I spoke too soon and made a mistake when reenabling color preservation
during range removals.
I assumed, that thanks to replacing the diverging alt with a new black
alt for stitching together diverging trunks, we would avoid the issue
where a deleted diverging alt violates our rbyd's tail-recursive
recoloring invariant.
Unfortunately, this is not the case. All the stitching alt did was make
this violation more difficult to reach, but still reachable. Arguable a
worse situation.
Now, for this violation to happen, in addition to all of the other
requirements, we need the lower-diverging trunk to become empty.
This is the only case where we have no stitching alt, because we don't
need to stitch an empty trunk. Which means if the upper-diverging trunk
has yellow nodes both before and after the diverging alt, our
tail-recursive recoloring invariant can break.
Here's an example:
.-------------r-------------.
.-o-. .---+---y----. .-o-.
.o. .o. .o. .o. .o. .-y-+-. .o. .o.
a a a a a a a a c c c e e e e e e e
'--+--'
remove
Again, this doesn't capture the alt-layout, which _is_ important, so
here's the dbgrbyd.py view:
.-> aa .-> aa
.-b-> a .-b-> a
| .-> a | .-> a
.-----------b-b-> a .-------b-b-> a
| .-> a | .-> a
| .---------b-> a | .-b-> a
| | .-> a | | .-> a
| | .-------b-> a | .-b-b-> a
r-b-y-r-b-----b-> cc -. => y-y-r-b-----> ee <- two yellows!
| | '-> c + rm | '-----b-> e different dirs!
| | .-> c -' | '-> e should not happen!
| '-y-r-b-> ee | .-> e
| | '---> e | .-b-> e
| '-----> e | | .-> e
| .-> e '-----b-b-> e
| .-b-> e
| | .-> e
'---------b-b-> e
And the steps in our appendattr algorithm that led to this state, which
is insightful:
read <r => [<r]
read >b => [<r >b]
read <r => [<r >b <r]
read <r => [<r >b <r <r]
^--^------ red + red implies yellow
ysplit => [<r >r <b]
reorder => [<r <r >b]
^--^------------- yellow-same-dir invariant held
read >b => [<r <r >b >b]
diverge => [<r <r >b]
read >r => [<r <r >b >r]
read >r => <r [<r >b >r >r]
^-----------^-- our 4-alt fifo for flips/coloring
ysplit => <r [<r >r >b]
reorder => <r [>r >r <b]
^--^---------- yellow-same-dir invariant held
^---^------------- yellow-same-dir invariant NOT held
though 2 yellows is also a problem
The previous commit fixing this bug for the one-pass algorithm may also
be useful.
This tree is now tested in test_rbyd_delete_range_rydye and
test_rbyd_delete_range_rydye_backwards, though only
test_rbyd_delete_range_rydye_backwards reveals the bug, since the bug
requires _specifically_ the lower-diverging trunk to become empty (both
rydy and rydye now have in-order and backwards tests in case of other
chirality issues).
---
Taking a step back, and looking at this bug from a higher-level, the
core of the issue is that we are somewhat arbitrarily deleting nodes
after splitting nodes. This can break our tail-recursive recoloring
invariant.
What the heck is our tail-recursive recoloring invariant?
This is a property of 2-3-4 and greater B-trees, and transitively
red-black and red-black-yellow trees, that allows for tail-recursive,
self-balancing node insertion.
Basically, if you eagerly split any 4-nodes you encounter as you descend
down the tree, you will always be guaranteed to have an open slot in
your parent, so pushing up split nodes (or recoloring) only ever
propagates up a single level:
.-----. .-------. .-------.
|.a.h.| |.a.c.h.| |.a.c.h.|
'|-|-|' '|-|-|-|' '|-|-|-|'
| .-' '-. .-' '--.
v v v v v
.-------. .---. .---. .---. .-----.
|.b.c.g.| => |.b.| |.g.| => |.b.| |.e.g.|
'|-|-|-|' '|-|' '|-|' '|-|' '|-|-|'
| | .-' '-.
v v v v
.-------. .-------. .---. .---.
|.d.e.f.| |.d.e.f.| |.d.| |.f.|
'|-|-|-|' '|-|-|-|' '|-|' '|-|'
If you lazily split, you aren't guaranteed an open slot in your parent,
so you need recursion to solve splits. This is why 2-3 trees, though
self-balancing, are not tail-recursive:
.-----. .-----.
|.a.h.| |.a.h.|
'|-|-|' '|-|-|'
| |
v v
.-------. .'''''''''.
|.b.c.g.| => >.b.c.e.g.< 5!?
'|-|-|-|' '|.|.|.|.|'
| .-' '-.
v v v
.-------. .---. .---.
|.d.e.f.| |.d.| |.f.|
'|-|-|-|' '|-|' '|-|'
But if you are eagerly splitting while also deleting nodes:
.-----. .-------. .-------. .'''''''''.
|.a.h.| |.a.c.h.| |.a.c.h.| 5!? >.a.c.e.h.<
'|-|-|' '|-|-|-|' '|-|-|-|' '|.|.|.|.|'
| .-' '-. .-' '---. .---' | '---.
v v v v v v v v
.-------. .---. .---. .---. .-------. .---. .---. .---.
|.b.c.g.| => |.b.| |.g.| => |.b.| |.d.e.f.| => |.b.| |.d.| |.g.|
'|-|-|-|' '|-|' '|-|' '|-|' '|-|-|-|' '|-|' '|-|' '|-|'
| x | x
v v
.-------. .-------.
|.d.e.f.| |.d.e.f.|
'|-|-|-|' '|-|-|-|'
Suddenly, recursion. This is a problem.
The workaround implemented here is to check during pruning if our parent
may risk recursion, and if so, recolor the last alt so nothing will
break.
This ends up equivalent to the following transformation:
.-----. .-------. .-----. .-----.
|.a.h.| |.a.c.h.| |.a.c.| |.a.c.|
'|-|-|' '|-|-|-|' '|-|-|' '|-|-|'
| .-' '-. .-' '-. .-' '--.
v v v v v v v
.-------. .---. .---. .---. .---. .---. .-----.
|.b.c.g.| => |.b.| |.g.| => |.b.| |.h.| => |.b.| |.e.h.|
'|-|-|-|' '|-|' '|-|' '|-|' '|-|' '|-|' '|-|-|'
| x | x | .-' '-.
v v v v v
.-------. .-------. .-------. .---. .---.
|.d.e.f.| |.d.e.f.| |.d.e.f.| |.d.| |.f.|
'|-|-|-|' '|-|-|-|' '|-|-|-|' '|-|' '|-|'
You may notice this isn't exactly optimal. The >h branch ends up one
level lower, making the balance of the tree off by one. But it at least
ends up with a functional tree.
I may try to find a better solution...
---
The test_rbyd_delete_range_rydy/rydye tests should cover the cases where
a diverging alt is deleted.
I also tried to write tests for the cases where an alt is pruned, the
closest I got is in test_rbyd_delete_range_dryy_backwards, but I
couldn't actually come up with a sequence that would break our rbyds.
In theory it's possible, but it would need this substructure:
.-------> c y-r-b-------> c
y-r-b-y-r-b-> c or | | '-y-r-b-> c
| | | | | | | |
Which, as far as I can tell, can't actually be created with our current
algorithm...
Note the inverse structure:
.---------> c
| .-y-r-b-> c
y-r- | |
Will be pruned before it has a chance to split. So there is no invariant
concerns there. We only have issues when it's the tail alts that get
pruned, because we decide to split before we know if we are pruning or
not. I don't think this can be avoided without additional read-ahead.
Also, even if we could create the above substructure, because we are on
a diverged trunk, and by definition all alts point the same direction,
we would never end up violating our same-dir yellow invariant/assert...
Code changes:
code stack
before: 33880 2880
after: 33912 (+0.1%) 2880 (+0.0%)
This was a nasty bug. I was initially concerned that this slipped
through our rbyd tests until I realized how excruciatingly rare it is.
If, during a range remove:
1. There is a pending yellow split immediately after the diverging alt
2. There is a pending yellow split immediately before the diverging alt
3. The diverging alt takes a black alt in the yellow split
4. There is a red node before the pending split before the diverging alt
5. The two alts in the red node point in different directions
We can end up violating our yellow node both-alts-point-same-direction
invariant.
The tree looks like this:
.-------------r-------------.
.-o-. .----y---+---. .-o-.
.o. .o. .-+-y-. .o. .o. .o. .o. .o.
a a a a a a a c e e e e e e e e e e
'+'
remove
Though this diagram doesn't capture the actual alt-layout, which does
matter here, so the dbgrbyd.py rendering may be more useful:
.-> aa .-> aa
.-b-> a .-b-> a
| .-> a | .-> a
.-----------b-b-> a .-----b-b-> a
| .-----> a | .-> a
| | .---> a | .-----b-> a
| .-y-r-b-> a | | .---> a
| | '-> cc <- rm | | |
r-b-y-r-b-----b-> ee => y-y-r-b-r-b-> ee <- two yellows!
| | | '-> e | | '-> e different dirs!
| | '-------b-> e | '-b-b-> e should not happen!
| | '-> e | | '-> e
| '---------b-> e | '-b-> e
| '-> e | '-> e
| .-> e | .-> e
| .-b-> e | .-b-> e
| | .-> e | | .-> e
'---------b-b-> e '-------+-b-> e
If all of these conditions are met, and we are preserving coloring, we
can end up with two yellow splits without an intermediate black alt,
implying recursion. But we're of course not recursive, so things just
break.
If we look at the trunk that is being built during our range removal:
read <r => [<r]
read >b => [<r >b]
read >r => [<r >b >r]
read >r => [<r >b >r >r]
^--^------ red+red implies yellow
ysplit => [<r >r >b]
reorder => [>r >r <b]
^--^------------- yellow-same-dir invariant held
read <b => [>r >r <b <b]
diverge => [>r >r <b]
read <r => [>r >r <b <r]
read <r => >r [>r <b <r <r]
^-----------^-- our 4-alt fifo for flips/coloring
ysplit => >r [>r <r <b]
reorder => >r [<r <r >b]
^--^---------- yellow-same-dir invariant held
^---^------------- yellow-same-dir invariant NOT held
though 2 yellows is also a problem
The important thing to note is that the diverging alt is effectively
deleted in both search paths. If the diverging alt is between two yellow
splits, that's not good.
If you think about the mapping to the underlying 2-3-4 tree, append is
only guaranteed to be tail-recursive because we eagerly split 4-nodes
into 2 2-nodes, ensuring that our parent always has a slot available for
a split (this is why 2-3 trees are not tail-recursive). But if we delete
one of the 2-nodes, and find another 4-node, the parent's slot has
already been taken. This is basically the problem we are running into
here.
A hypothetical 2-3-4-5 tree however...
Probably-isomorphic to a 2-3-4-5 tree, there are a couple of possible
solutions to this:
1. Increase the fifo to 5(?) alts and recursively propagate recolorings
up 2 nodes.
Note this would still be bounded and tail-recursive. Our current
implementation is basically an isomorphism of recursively propagating
recolorings up 1 node after all, if you want to think about it in
about the most complicated way possible...
Downsides: The increased fifo size means more RAM cost. And the
implementation would be complicated as hell. Not to mention error
prone. Imagine ~2x the current 15K lines of rbyd tests. It would be
bad.
2. Discard split recolorings after a diverged alt.
This would be quite a bit simpler, though would still require some
annoying state to know if the previous alt diverged.
If this state isn't perfect, the above checklist of conditions would
just be incremented by 1, making this bug even harder to track down.
I'm starting to think that preserving color during range removals is a
bit complicated for its own good.
Considering that color-preserving range removals aren't even rigorous
and don't guarantee a balanced tree, I think this all just needs to be
scrapped until a more rigorous solution is found.
---
So this commit drops color-preserving range removals, and moves to a
simpler paint it black + stitch together alternating red alt strategy
when encountering a diverging range removal.
Thanks to the red-stitching, the resulting search path is at least
tried to be kept as small as possible.
This results in the following, not-broken tree:
.-> aa .-> aa
.-b-> a .-b-> a
| .-> a | .-> a
.-----------b-b-> a .-----b-b-> a
| .-----> a | .-------> a
| | .---> a | | .---> a
| .-y-r-b-> a | | | .-> a
| | '-> cc <- rm | | | |
r-b-y-r-b-----b-> ee => y-r-b-r-b-r-b-> ee
| | | '-> e | | '-----> e
| | '-------b-> e | '-------b-b-> e
| | '-> e | | '-> e
| '---------b-> e | '-b-> e
| '-> e | '-> e
| .-> e | .-> e
| .-b-> e | .-b-> e
| | .-> e | | .-> e
'---------b-b-> e '---------b-b-> e
It's interesting to note that this bug is so rare that it was only
caught by test_dirs_mv_fuzz after 2180 heuristic powerlosses. But it
was caught, so that's a good sign.
But it would have been better if this was caught in the rbyd tests. I've
gone ahead and added a specialized test, test_rbyd_delete_range_rry (and
a few other), to prevent a regression, which is very likely. It's more
likely than not we'll revisit range removals in the future.
On the plus side, since recoloring is simpler than color-preservation,
this means less code:
code stack
before: 34072 2880
after: 33992 (-0.2%) 2880 (+0.0%)
Topologically, this isn't really much of a change. We just moved the
flcksum -> lfs.pcksum and made the internal API a bit better.
But hey, a better internal API at ~no cost is always a good thing:
code stack lfs_t
before: 33868 2880 212
after: 33856 (-0.0%) 2880 (+0.0%) 216 (+1.9%)
There wasn't really a collision with this, and I think it's clear what
these flags are doing.
Also fixed a missed renamed of lfsr_tag_issup/subwide ->
lfsr_tag_issup/sub
Thanks to poor compound literal optimization, it's actually cheaper to
pass lfsr_data_t by value everywhere, than to make all LFSR_DATA_*
macros lvalues:
before: 34340 2896
after: 34292 (-0.1%) 2896 (+0.0%)
Why are these two design choices linked? If lfsr_data_t is
pass-by-address, the rvalue/lvalue disinction is important because we
need to take the address of LFSR_DATA_* macros. If lfsr_data_t is
pass-by-value, rvalue/lvalue doesn't really matter because we, well,
pass by value.
To be honest, this is a bit of an excuse for better lfsr_data_t
ergonomics. It _is_ generally worse code-size wise to pass lfsr_data_t
by value, because most ABI optimizations stop at 2 words and
lfsr_data_t requires 3 words. But always passing lfsr_data_t by value
even if it is suboptimal makes for more consistent internal interfaces.
This also helps side-step a mistake I made earlier where I though
cat/fromimm/fromleb128 were the only LFSR_DATA_* macros that needed to
be lvalues to be consistent. THERE ARE MANY MORE LFSR_DATA_* macros,
every LFSR_DATA_FROMBLAH macro to be specific, and the resulting code
cost would be MUCH WORSE.
---
This also add lfsr_sprout_t to complement lfsr_bptr_t/lfsr_shrub_t/etc.
Unlike lfsr_data_t, lfsr_sprout_t _is_ pass-by-address
Actually that's the only difference, haha. lfsr_sprout_t is a typedef.
Though to be fair, by being pass-by-addres, lfsr_sprout_t keeps the
internal sprout/shrub/bptr/btree inferfaces consistent, and saves a bit
of code.
Before:
LFSR_ATTR(RM(SUBMASK(REG)), 0, BUF("hi", 2))
Now:
LFSR_ATTR(
LFSR_TAG_RM | LFSR_TAG_SUBMASK | LFSR_TAG_REG, 0,
LFSR_DATA_BUF("hi", 2))
Yes, it's more verbose now.
But there were a couple reasons for dropping the idea:
- The implicit prefixing is a bit magical, and not really all that
common in C code. It would likely confuse new users on first read.
- The implicitly prefixing macros did not play will with macro expansion
rules.
In particular, because the nested not-yet-prefixed macros aren't
really macros, they aren't expanded as a part of argument prescan.
This led to surprising compile-time errors, and prevented recursive
attr-lists (which may be useful for shrubs).
- Implicit prefixes is not very C-like, and in particular it gets in the
way of sed/grep operations on source files.
- RM(SUBMASK(REG)) for combining tags is (IMO) ugly, compared to
LFSR_TAG_RM | LFSR_TAG_SUBMASK | LFSR_TAG_REG, even if the latter
requires more typing.
- Sometimes you need runtime-dependent TAG/DATA values, which implicit
prefixing gets in the way of. The LFSR_TAG_TAG(tag)/
LFSR_DATA_DATA(tag) backdoors worked around this, but they are even
more magical, and added noise to a not-actually-all-that-uncommon use
case.
And it's really not _that_ much extra effort to write out the prefixes
everywhere.
lfs.c:
lines bytes
before: 16894 537171
after: 16907 (+0.1%) 538340 (+0.2%)
tests/*.toml:
lines bytes
before: 53306 1811035
after: 54517 (+2.3%) 1851006 (+2.2%)
qadte came in quite handy again for refactoring the tests without
completely losing my sanity.
We really had ~2 duplicate bd layers for a bit there.
This also involved a sort of rewrite of these low-level functions to see
if there were simplifications that could be made.
A couple tweaks:
- Added small low-level lfsr_bd_read/prog/erase/sync_ functions to
only wrap the bd callbacks and apply any relevant asserts.
These should be the only place we call the bd callbacks to make it
easy to read/audit/insert hooks in the future.
- Changed pcache flush lazily, rather than eagerly flushing when full.
This isn't for any real performance reason, it just makes the code
simpler. It's not like we can shove more data into the pcache once
full.
It's _probably_ a good idea to flush eagerly, to avoid delay more work
until sync, but I couldn't figure out how to make this work cleanly
without code duplication...
- Deduplicated read pcache overwrites via lfsr_bd_read__.
This logic is a bit annoying, but we need the pcache to take priority
whenever we read from disk, which happens when we both fill our
rcache, and bypass our rcache. Since these code paths go different
places, another internal function was the only way I could think to
deduplicate this.
It may appear that our pcache/rcache prioritization loop will make
this happen naturally, as it does in lfs_file_read for example, but
this doesn't quite work as read-alignment requirements may force us to
read past the pcache... Keep in mind read_size may be > prog_size.
- Dropped LFS_BLOCK_NULL, now using cache.size=0 to indicate a cache is
unused.
This avoids a special lfs_block_t value.
- Dropped lfsr_bd_readcksum, we never used this.
We can always add it back if necessary.
In total, the caching bd prog/read functions now look quite a bit more
like our file read/write functions, so hopefully that's a good thing.
By the virtue of not have ~2 duplicate bd layers, this saves a bit of
code:
code stack
before: 33700 2800
after: 33560 (-0.4%) 2808 (+0.3%)
These shims, originally intended to remap the tests to new internal
APIs without a significant rewrite, are a long-outstanding piece of
technical debt. Now that the internal API is more stable, it's time for
that rewrite.
Reasons for not keeping the internal shims:
- They add more complexity to the test suites.
- They come with (out-of-date) constraints that limit what we can test.
- It's more difficult to debug test failures, with 2 layers and all.
I ended up writing a small tree editor out of tree to do most of this
rewrite.
Did it save time? Probably not. But it was quite a bit more fun than
manaully rewriting ~21K lines of code.
Changing insert tags to append seems to have broken insertion into named
btrees in a subtle way.
Consider what happens when we insert immediately before a bid that
splits the btree:
1. namelookup returns the right rbyd, with rid=-1
2. converting this into a bid gives us the left rbyd, with rid=weight
3. the commit to insert the bid ends up inserting into the left rbyd
This doesn't initially seem like an issue, both entries are effectively
the same right? Well, not when you have names. The split name tells you
what _follows_, so this unintentional flipping causes the new name to
get placed in the wrong bucket.
It's not clear if it's possible to fix this, at least not without
inverting the split names to indicate what precedes, but that's a step
too far.
This was not detected earlier because I disabled the low-level
rbyd/btree/mtree tests temporarily due to high porting cost. Guess that
goes to show there's a cost to deferring test ports for too long.
---
This issue, along with being inconsistencies between rids/bids and mids,
and being a relatively unintuitive pattern, is the final nail in the
coffin for insert tags inserting after.
Now, insert tags insert before, like in most other systems, and insert
tags in attr-list just have an implicit +1 before them to allow splits
in attr-lists to work.
This is not a pure revert, as some of the changes with all the code
moving around revealed some better detail-level ideas.
And yes, rbyd/btree tests are up to date now. Unfortunately the mtree
tests require a bit more work.
---
One thing definitely worth noting, btree merges were broken! A mistake
in the has-parent condition meant we were never attempting to merge
btrees!
This hid some bugs in the actual btree merge code caused by mixing the
implicit swap of child rbyds to deduplicate code paths with btree commit
now needing to track bid/rid separately from the attr-list.
This should be fixed now. Interesting to note this bug has been in
lfsr_btree_commit_ for a while now! I think ever since we switched to
using trunks for the has-parent check. We just haven't been merging
btree nodes at all. But since not-merging isn't technically an error,
it's difficult to test for.
Code changes:
code stack
before: 33808 2896
after: 33964 (+0.5%) 2896 (+0.0%)
Found a bug, and maybe a fundamental issue:
- The lfs_btree_lookupnext_ in lfsr_btree_commit_ no longer needs the
min32, since we never commit with bid pointing past the end of the
btree anymore.
This was mixing the unsigned min32 with our now-signed bid type,
causing the wrong btree leaf to be fetched when inserting at bid=-1 in
a non-empty btree.
Easy fix.
- lfsr_btree_commit_ with bid!=-1, rid=-1 (inserting at the beginning of
not-the-first rbyd) now actually appends to the leaf to the left of
the rbyd instead of inserting into the expected rbyd because of how
lfs_btree_lookup_ works.
Initially, this doesn't seem like it would be an issue, these should
be more-or-less equivalent, but this doesn't match
lfsr_btree_namelookup! This is a big problem!
This wasn't noticed because it's rare for the high-level tests to
trigger that many btree splits with names. Named btrees are only used
for the mtree, and we need mdirs to split before the mtree even splits
once.
Not an easy fix.
On the upside, these low-level tests continue to prove themselves
valuable, if tedious to maintain...
Unreachable tag holes, null tags that _should_ be unreachable but
actually are reachable, are an unfortunate quirk to our alt tag
encoding. Because we only have an altgt, not altge, our "unreachable"
tag ends up encoded with an altgt 0, an alt, which you may notice, does
not guarantee unreachability.
Fortunately, tag 0, the null tag, should intentionally be unused. So as
long as we never lookup tag 0, nothing should break.
If you do lookup tag 0, you end up with spurious null tags, which can
complicate things.
The solution here is a tag_ = max(tag, 1) in lfsr_rbyd_lookupnext.
---
One interesting thing to note, as I was writing these tests I discovered
that setting tag=max(tag,1) in lfsr_rbyd_appendattr had no effect.
appendattr needs zip the rbyd tree to keep everything connected during
range removals, so tag=0/tag=1 both end up with the same tree.
So might as well drop the tag=max(tag,1) in lfsr_rbyd_appendattr.
A side effect of this, both before and after this commit, is that any
null tag holes created during range removals sort of stick around until
the next compaction.
---
Why altgt and not altge? altgt is the inverse of altle, requiring only
a single bit flip to flip between the two. And trust me, it would be
much more costly to make altle/altgt flips more complicated than a bit
flip.
---
Why altgt/altle and not altge/altlt? This is because our rbyds are
right-leaning, that is, lookups always find the requested rid+tag, or
the next smallest rid+tag.
Consider a simple tree:
<5
.----'|
>=2 |
|'-. |
1 2 5
What should lookup(3) return? If we are right-leaning, the answer
_should_ be 5. But we need to take the <5 branch to determine if there
is a hidden 3 or 4 in that subtree.
altgt/altle does not have that problem:
<=2
.----'|
>1 |
|'-. |
1 2 5
It might seem like you can workaround this by conservatively using the
neighbor +1 as the alt target, but this runs into tag overflow problems.
UATTR(0xff)+1 (0x057f+1) becomes UATTR(0x100) (0x0580) which is not
allowed due to reserving bit 7 for future subtype extensions.
Maybe you can workaround this workaround by using (tag+0x81)&~0x80
anywhere you need to increment (including lookupnext/iteration calls!),
but this becomes a bit of a mess. And there are still concerns about
overflows at the 0x77f boundary and 0xf7f boundary.
Like SUBWIDE, SUPWIDE allows for "mask-like" operation during rbyd
commits, where you replace an entire subrange of tags with a single tag.
- SUBWIDE - Replace all subtypes of the given suptype - Useful for
changing the subtype of an attr, for example replacing a BTREE with a
BSHRUB.
- SUPWIDE - Replace all suptypes of the given rid - Useful for changing
the suptype of an attr, for example replacing a REG file with an
ORPHAN file.
These are effectively the same modifier, just with different ranges.
One benefit is this simplifies mid-level operations a bit, rename,
remove, etc, and decreases the stack cost of the related attr lists.
Though this isn't on the hot-path, so not measurable:
code stack
before: 33956 2912
after: 33928 (-0.1%) 2912 (+0.0%)
But the real motivation for this change is to remove cases where
lfsr_mdir_commit needs to operate on multiple mids. There may be an API
simplification here.
So:
x = (cond) ? yes : no;
Where there are always parentheses around the condition, even if not
required for disambiguity. Additional parentheses are always allowed,
but the parenthesized condition helps signal that a ternary operator is
coming earlier in the expression.
This style has grown on me as I think it helps code readability. It
reminds me of the required parentheses for if/while statements.
Might as well adopt codebase-wide.
Checksumming unaligned data during block compaction is surprisingly
tricky. We don't know if our data will be aligned until after
a potentially unbounded number lookups, we need to write data into our
pcache as we go to avoid unnecessary lookups, but if we end up unaligned
we need to revert our checksum to the checksum of the aligned data.
The way I see it there are 4 options:
1. Calculate the checksum after writing data into the block.
This is the most expensive option, requiring a full second read of
the data to calculate the checksum. It is simple though.
2. Do a pass over the btree to figure out alignment before writing.
This at least only reads metadata twice, so is more efficient than
the 1st option.
3. Keep track of the aligned checksum on each flush, falling back to the
last flushed checksum if we need to correct alignment.
This solution is flexible though requires some extra state to track
multiple checksums.
4. Leverage the math behind CRCs to run the CRC backwards when we
truncate for alignment.
This works, though a bit inefficiently, but is strictly tied to
CRC-related checksums.
By inefficient I mean that we would likely be limited to a bit-level
"uncrc32c". It's possible to create nibble/byte tables for uncrc32c,
but this adds significant code cost for a relatively uncritical
function.
I was hopeful that we could leverage the existing tables in both
functions, but unfortunately it doesn't work out like that. You could
scan the crc32c table to find the constant to reverse, but this
requires ~16*2 or ~256 operations vs "naive" ~8 operations per byte.
This commit implements both 3 and 4, defaulting to 4 unless
LFS_NO_UNCRC32C is defined.
The current lfs_uncrc32c implementation is a simple bit-level
implementation, but does allow for crc32c truncation without any extra
state.
code stack
before: 32044 2880
uncrc32c: 32108 (+0.2%) 2880 (+0.0%)
flcksum: 32132 (+0.3%) 2880 (+0.0%)
This trades a runtime check for a different function call. Enforcing
some minor semantics in the function's type/asserts.
This also makes it so there are no special tag bits used during rbyds
lookup, only rbyd commits.
In theory this saves a bit of code, we don't have a runtime check, but
in practice the extra function apparently outweighs the cost of the
runtime check:
code stack
before: 31956 2880
after: 32024 (+0.2%) 2880 (+0.0%)
The idea here is to revert moving redund blocks into lfsr_rbyd_t, and
instead just keep a redundant copy of the rbyd blocks in the redund
blocks in lfsr_mdir_t.
Surprisingly, extra overhead in lfsr_mdir_t ended up with worse stack
usage than extra overhead in lfsr_rbyd_t. I guess we end up allocated
more mdirs than rbyds, which makes a bit of sense given how complicated
lfsr_mdir_commit is:
code stack structs
redund union: 30976 2496 1072
redund in rbyd: 30948 (-0.1%) 2528 (+1.3%) 1100 (+2.6%)
redund in mdir: 31000 (+0.1%) 2536 (+1.6%) 1092 (+1.8%)
The mdir option does seem to improve struct overhead, but this hasn't
been a reliable measurement since it doesn't take into account how many
of each struct is allocated.
Given that the mdir option is inferior in both code and stack cost, and
requires more care to keep the rbyd/redund blocks in sync, I think I'm
going to revert this for now but keep the commit in the commit history
since it's an interesting comparison.
This simplifies dependent structs with redundancy, mainly lfsr_mdir_t,
at a significant RAM cost:
code stack structs
before: 30976 2496 1072
after: 30948 (-0.1%) 2528 (+1.3%) 1100 (+2.6%)
Which, to be honest, is not as bad as I thought it would be. Though it
is still pretty bad for no new features.
The motivation for this change:
1. The organization of the previous lfsr_mdir_t struct was a bit hacky
and relied on exact padding so the redund block array and rbyd block
lined up at the right offset.
2. The previous organization prevented theoretical "read-only rbyd
structs" that could omit write-related fields, e.g. eoff and cksum.
This idea is currently unused.
3. The current mdir=level-1, btree/data=level-0 redund design makes this
RAM tradeoff pretty bad, but in theory higher btree redund levels
would need the extra redund blocks in the rbyd struct anyways.
Still, the RAM impact to the current default configuration means this
should probably be reverted...
We don't strictly need this for the mtree, but its impact is pretty
minimal, and it's useful for some future plans. It also makes low-level
benchmarks a bit easier to write.
The main change involves subtleties around vestigial names in leaf
rbyds (the bottom most layer of btree inner nodes). Since the mtree
terminates in mdirs, the left-most mdir in each leaf rbyd in the mtree
never actually needs a name. But in a hypothetical strict key->value
tree, every entry in the leaf rbyds need a name, and this name needs to
be respected during btree operations (mainly merges).
As a side-effect, our named btrees now require vestigial names for every
inner btree node, with the exception of the left-most inner nodes since
those can't be merged left with anything. On the bright side, being able
to assume a vestigial name on every mergable node does simplify merge
operations a bit.
It's worth noting that despite these changes, we still update vestigial
names on inner btree nodes lazily. It isn't super clear that this should
work, but it turns out that even though a leaf nodes may diverge from
the vestigial name in it's parent, it must still following the bounds of
the parent's vestigial name because of how btree lookups work. And this
property propagates up though each layer in the btree:
.---------------.
|a: |h: |-> |
'--|---|--------'
.---' '----------.
v v
.---------------. .---------------.
|a: |c: | | |i: |m: |-> |
'--|---|--------' '--|---|--------'
...--' | | '--------...
v v
.---------------. .---------------.
|d:0|e:1|f:2|-> | |j:3|k:4|l:5|-> |
'---------------' '---------------'
The exception are the left-most inner nodes, but these can never merge
left, so it doesn't really matter. The vestigial names on the left-most
inner nodes are truly vestigial:
.---------------.
|c: |e: |-> |
'--|---|--------'
.---' '--------...
v
.---------------.
|b: |d: | |
'--|---|--------'
.---' '-------...
v
.---------------.
|a:0|b:1|c:2|-> |
'---------------'
An alternative implementation may prefer to update these names eagerly,
but this would increase the amount of data written to each inner node
during btree commits. mdir updates are lazy by necessity, so even if you
adopted eager updates, the names of deleted files would still stick
around.
Our rbyds support changing the weight of a tag without knowing the
actual tag. This is useful for btrees, which always make weight changes
without knowing if the leading tag is a name or a branch (it depends on
the type of btree).
But to make this work, it needs the rm-bit to be set. This is because
internally the rm-bit indicates we don't want to write-out a tag. Which
we don't for grow tags, because, well, they're not real tags.
Previously this was done by putting LFSR_TAG_GROW(RM) everywhere a
generic grow as needed, but since this is so common we might as well
just set the rm-bit in LFSR_TAG_GROW.
Note that LFSR_TAG_GROW(tag) (the macro) does not set the rm-bit.
This makes the code a bit more readable at the risk of an unintuitive
relationship between LFSR_TAG_GROW and LFSR_TAG_GROW(tag).
With the introduction of lfsr_data_t, these stopped being useful
functions for littlefs internally.
Maybe these tests should be rewritten to use the *_lookup functions
directly? Unfortunately with the quantity of tests we have now this adds
non-trivial amount of work with questionable benefit.
Composable parsing functions always feel a bit weird to me in C. I don't
know if this is because of something C lacks, such as multiple return
values, or if composable parsers are just inherently awkward to describe
in procedural languages because of the different levels of state.
But I think the API here is pretty ok. The main idea is that data
parsers can be added as functions in the lfsr_data_* namespace that take
lfsr_data_t as a mutable reference, updating the lfsr_data_t's internal
state as data is parsed.
In practice you only need a couple of primitives, bytes, le32s, leb128s,
that touch the internals of lfsr_data_t, and the other parsers can be
built using these.
This leverages the pointer-like abstraction of lfsr_data_t, and avoids
needing to keep track of offsets. And thanks to lfsr_data_t being
relatively cheap to make copies, this API is relatively flexible.
Some other tweaks:
- Signed leb128 overflow detection is moved up into lfs_fromleb128.
littlefs now assumes _all_ leb128s are 31-bits, which is useful for
leveraging the sign bit internally.
This also fixes the an issue in overflow detection in lfs_fromleb128
which wouldn't catch overflows in the last byte of a >32-bit leb128.
- Most lfsr_data_t functions now take a pointer. This offered a small
bit of code savings and feels more natural in C. Though most functions
that accept lfsr_data_t still take a copy. Most of these functions
would need to make a copy anyways now that the parsers are consuming,
and these copies avoid concerns about shared state.
At 3-words, lfsr_data_t is right at that boundary of questionable
reasonableness for copying, but copying is a very useful feature of
this struct.
This ends up with some decent code/stack savings:
code stack
before: 22118 2048
after: 21722 (-1.8%) 1992 (-2.7%)
Generally the more creative you get with C macros, the more
unmaintainable your codebase becomes, but in this case I think a small
bit of macro sugar for the attribute lists in littlefs goes a long way
for making the internals flexible and readable.
Attribute lists generally look like this:
LFSR_ATTRS(
LFSR_ATTR(id, TAG, delta, DATA(data)),
LFSR_ATTR(id, TAG, delta, DATA(data)),
...
LFSR_ATTR(id, TAG, delta, DATA(data)))
Which more-or-less gets expanded to this:
((const lfsr_attr_t[]){
((lfsr_attr_t){id, LFSR_TAG_TAG, delta, LFSR_DATA_DATA(data)}),
((lfsr_attr_t){id, LFSR_TAG_TAG, delta, LFSR_DATA_DATA(data)}),
...
((lfsr_attr_t){id, LFSR_TAG_TAG, delta, LFSR_DATA_DATA(data)}),}),
attr_count
Note the use of preprocessor concatenation to put the TAG and DATA
identifiers in their respective namespaces. These can end up invoking
other macros, which allows attrs to be rather extensible.
Previously there were also LFSR_ATTR_ (note the trailing underscore)
macros to allow passing of variable tags/datas. This is replaced with
redundant macros which sort of "unwrap" themselves as a part of macro
expansion. This avoids a bunch of duplicate macro definitions.
#define LFSR_TAG_TAG(tag) (tag)
#define LFSR_DATA_DATA(data) (data)
So:
LFSR_ATTR(id, TAG(tag), delta, DATA(data))
Becomes:
((lfsr_attr_t){id, LFSR_TAG_TAG(tag), delta, LFSR_DATA_DATA(data)})
Becomes:
((lfsr_attr_t){id, tag, delta, data})
This checksum is used to keep track of if we have erased, and not yet
touched, the unused bytes trailing our current commit in the rbyd.
The working theory is that if any prog attempt is made, it will, most
likely, change the checksum of the contents, allowing littlefs to
determine if trailing erased-state is safe to use, even under powerloss.
littlefs can also perturb future data by a single bit, to force this
checksum to always be invalidated during normal operation.
The original name, "forward erased-state checksums (fcksum)", came from the
idea that the checksum "looks forward" into the next commit.
But after using them for a bit, I think the name is unnecessarily
confusing. It, uh, also looks a lot like a swear word. I think
shortening the name to just "erased-state checksums (ecksum)", even
though the previous name is already in use in a release, is reasonable.
---
It's probably hard to believe but the name change from fcrc -> ecrc
really was unrelated to the crc -> cksum change. But boy is it
convenient for avoiding an awkward name. A lot of these name changes
involved sed scripts, so I didn't notice how awkward fcksum would be to
use until writing this commit message.
The reason for this is to move away from the idea that littlefs is
strictly bound to CRCs and make the code more welcoming to other
checksum types, such as SHA256, etc.
Of course, changing the name doesn't really do anything. littlefs
actually _is_ strictly bound to CRCs in a couple ways that other
filesystems aren't. These would need to have workarounds for other
checksum types:
- We leverage the parity-preserving nature of (some) CRCs to not have
to also calculate the parity of metadata in rbyd commits.
- We leverage the linearity of CRCs to retroactively flip the
perturb bit in the cksum tag without needing to recalculate the
checksum. Though the fact we need to do this is because of how we
use parity above, so this may just not be needed for non-CRC
checksums.
- The plans for global-CRCs (not yet implemented) rely heavily on the
mathematical properties of CRC polynomials. This doesn't mean
global-CRCs can't work with other checksums, you would just need to
find a different type of polynomial.
Originally it made sense to name the rbyd ids, well, ids, at least in
the internals of the rbyd functions. But this doesn't work well outside
of the rbyd code, where littlefs has to juggle several different id
types with different purposes:
- rid => rbyd-id, 31-bit index into an rbyd
- bid => btree-id, 31-bit index into a btree
- mid => mdir-id, 15-bit+15-bit index into the mtree
- did => directory-id, 31-bit unique identifier for directories
Even though context makes it clear which id the id refers to in the rbyd
internals, updating the name to rid makes it clearer that these are the
same type of id when looking at code both inside and outside the rbyd
functions.
The previous system of relying on test name prefixes for ordering was
simple, but organizing tests by dependencies and topologically sorting
during compilation is 1. more flexible and 2. simplifies test names,
which get typed a lot.
Note these are not "hard" dependencies, each test suite should work fine
in isolation. These "after" dependencies just hint an ordering when all
tests are ran.
As such, it's worth noting the tests should NOT error of a dependency is
missing. This unfortunately makes it a bit hard to catch typos, but
allows faster compilation of a subset of tests.
---
To make this work the way tests are linked has changed from using custom
linker section (fun linker magic!) to a weakly linked array appended to
every source file (also fun linker magic!).
At least with this method test.py has strict control over the test
ordering, and doesn't depend on 1. the order in which the linker merges
sections, and 2. the order tests are passed to test.py. I didn't realize
the previous system was so fragile.
It doesn't make sense to test more complex logic, such as t2_btree.toml,
when the logic it is built on, t1_rbyd.toml, does not past testing. The
test runner already guarantees a consistent lexicographic order, so all
we need to do is renamed these from test_* -> tn_*.
Note, if we every have more than 10 tests, we will need to bump up the
number of digits for all tests, so t1_rbyd.toml -> t01_rbyd.toml. This
is the main downside of lexicographic ordering. But we'll cross that
bridge when we get to it.
- Since both trunks emit altle tags now, reworked the trunk merging to
reuse more code.
- Changed lfsr_mdir_fetch to rely on trunk=0 to detect the no-commit
state. This is purely for consistency.
This actually broke some tests that committed nothing, resulting in
trunk-less rbyds, which is a bit concerning, but I don't think
trunk-less rbyds will ever be valid in our system?
- Simplified lfsr_rbyd_estimate calculation, merged lfsr_rbyd_bisect
since this is almost always needed after an estimated failure, and
that way dependent function have to call fewer things to implement
rbyd splitting.
- Dropped vestigial names for now, though need to revisit this later.
After these changes the code size difference between rebalancing and
appending is a bit smaller at ~392 bytes: 16208 -> 16600 (+2.4%). It's
interesting to note this is mostly because the conservative overhead
calculation is easier with rebalancing.
In theory this also saves some stack usage, but since I'm measuring
maximum stack usage it doesn't show up since it's not on the deepest
path.
Now we read the revision count on-demand, trading off some extra reads
for a smaller lfsr_rbyd_t struct.
I believe this is worth it because:
1. We're created a lot of lfsr_rbyd_t structs as a part of the relatively
complicated mdir/btree commit logic in order to safely fallback on errors.
2. We don't really need the revision count for our Cow btrees, so we
only need to read the revision count on mdir fetch (which we were
already reading too many times), on mdir compact, and on rbyd fetch
as a part of checksum calculation.
This really only adds a O(1) cost when we are compacting, which is rather
small.
Current measurements:
code: 8980 -> 9036 (+0.6%)
stack: 1024 -> 1000 (-2.3%)
Though note this is currently without any mdir/btree commit code being
dragged in.
Wide tags are a happy accident that fell out of the realization that we
can view all subtypes of a given tag suptype as a range in our rbyd.
Combining this with how natural it is to operate on ranges in an rbyd
allows us to perform operations on an entire range of subtypes as though
it were a single tag.
- lookup wide tag => find the smallest tag with this tag's suptype, O(log(n))
- remove wide tag => remove all tags with this tag's suptype, O(log(n))
- append wide tag => remove all tags with this tag's suptype, and then
append our tag, O(log(n))
This is very useful for littlefs, where we've already been using tag's
subtypes to hold extra type info, and have had to rely on awkward
alternatives such as deleting existing subtypes before writing our new
subtype.
For example, when committing file metadata (not yet implemented), we can
append a wide struct tag to update the metadata while also clearing out any
lingering struct tags from previous commits, all in one rbyd append
operation.
This uses another mode bit in-device to change the behavior of
lfsr_rbyd_commit, of which we have a couple:
vwgrtttt 0TTTTTTT
^^^^---^--------^- valid bit (currently unused, maybe errors?)
'||---|--------|- wide bit, ignores subtype (in-device)
'|---|--------|- grow bit, don't create new id (in-device)
'---|--------|- rm bit, remove this tag (in-device)
'--------|- 4-bit suptype
'- leb128 subtype
This only affects the in-device tags, not the on-disk tags.
The mk variant of tags was seeing much more use than the grow variant,
since the grow variant is really only used by the btree internals. But
since the default encoding of tags cleared the mk-bit, this led to a
bunch of extra lfsr_tag_setmk calls just to reserialize things correctly
during compact, split, etc.
Flipping the logic so the bit needs to be set to grow tags simplified
things quite a bit.
Note that mk tags do nothing when their delta is zero, so zero-delta
tags are the same in both mk/grow mode.
Originally I thought doing a linear search during fetch was going to be
the best route for name lookups, since we already needed a O(b) fetch,
which set a hard ceiling for name lookup performance.
But it turns out we don't need to fetch during btree name lookups
unless we're also validating! Now that validation and lookups are
disentangled, we can do a binary search over the rbyd to drop our
name lookup down to O(log(b)^2).
Two other motivations for this change:
1. This removes the name search from lfsr_rbyd_fetch, which has been
surprisingly tricky to get right.
2. Now that lfsr_rbyd_fetch doesn't need to follow the create/delete
history to find and reconstruct the state of names, we only need to
know the create/delete state of tags post-fetch, freeing up the rbyd
encoding to be more flexible.
The next thing I plan to do is drop on-disk remove tags, for example.
This drops the total btree namelookup cost from O(b log_b(n)) to
O(log(b)^2 log_b(n)).
The exact behavior of lfsr_rbyd_lookup is a bit unusual, and has already
resulted in a few mistakes. To make this more clear at a glance, names
have been changed and a few more helper functions added.
The new names and expected behavior:
- *_lookupnext - lookup the smallest id/tag less than or equal to the
requested id/tag, returns LFS_ERR_NOENT if id/tag is greater than all
ids/tags in the data structure.
- *_lookup - lookup the exact id/tag, returns LFS_ERR_NOENT if id/tag
is not in the data structure.
These have been adopted in all current data structures: rbyd/btree/mdir
- lfsr_rbyd_lookup => lfsr_rbyd_lookupnext
- lfsr_btree_lookup => lfsr_btree_lookupnext
- lfsr_btree_namelookup => lfsr_btree_namelookupnext
- lfsr_mdir_lookup => lfsr_mdir_lookupnext
Note no lfsr_btree_namelookup is added, this is a more complicated
than lfsr_btree_lookup (we need to cmp the name on-disk for equality)
and also probably not needed.
Christopher Haster