Github/littlefs
1
Fork 0
Code Actions Activity

Preserve coloring during range removals

Browse Source 
This is the real kicker of our new-and-improved range removal algorithm.
We can actually preserve the existing tree coloring, and the underlying
rbyd invariants.

Well, sort of. We preserve the red-follows-yellow and black-follows-red
rules, but we don't (can't?) preserve the same-height for all black
edges property.

But note! The new range removal algorithm never creates _new_ black
edges. It can only delete black edges, and otherwise preserves the
structure of the underlying 2-3-4 tree.

This means that while the resulting tree may not be perfectly balanced
with h=2*log2n', where n' is the _new_ number of tags, the resulting tree
_is_ limited to h=2*log2(n) where n is the _old_ number of tags.

When applied to our bounded rbyd, with eventual compaction and
rebalancing, we end up with the guarantee that the rbyd's height will
never exceed h=2*log2(b) where b is the block size, even with arbitrary
range removals.

This is a great result!

---

Note that this algorithm does not suffer from the yellow-diverge-yellow
corner case that was an issue for preserving coloring in the previous
one-pass stitching algorithm. This is because the one-pass algorithm
effectively deleted the diverging alt, breaking the tail-recursive
invariant of the underlying 2-3-4 tree. With the new two-pass algorithm,
we _replace_ the diverging alt with a black stitching alt to stitch
together the diverging trunks, so no tail-recursive invariant breaking.

(Also note even if we could preserve coloring in the one-pass algorithm,
it would still be breaking invariants by introducing new black edges
when it stitches together diverging trunks. Worst case, resulting in ~2x
the height, even when stitching with red alts (The red alt stitching
brings this cost down from ~4x to ~2x worst case due to blanket
recoloring. With yellow alt stitching this could probably be brought
down to ~1.3x, but this would still mean every range removal could be
increasing the height of the tree, which is not great.).)

---

Pruning has to be a bit more complicated now, since we need to be able
to recolor skipped red alts. But other than pruning the cost of
recoloring vs not recoloring is pretty small:

                               code          stack
  one-pass, blanket recolor:  33852           2880
  two-pass, blanket recolor:  33860 (+0.0%)   2880 (+0.0%)
  two-pass, color preserving: 33880 (+0.1%)   2880 (+0.0%)

The non-rigorous random-file-write benchmark I've been using as a litmus
test did not really show any improvements, but in hindsight it might
have been a bit silly to use a uniform distribution of writes to test
for rbyd balancing issues... Building a tree from a uniform distribution
already results in a balanced tree without doing anything!
This commit is contained in:
Christopher Haster
2024-03-12 01:55:24 -05:00
parent 39413e7d78
commit 4d90be94f9
1 changed files with 22 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lfs.c
+22 -36
View File
@@ -2884,10 +2884,7 @@ again:;
if (d_state == LFSR_D_DIVERGEDUPPER && d_tag) {
err = lfsr_rbyd_p_push(lfs, rbyd,
p_alts, p_weights, p_jumps,
LFSR_TAG_ALT(
LFSR_TAG_LE,
LFSR_TAG_B,
d_tag),
LFSR_TAG_ALT(LFSR_TAG_LE, LFSR_TAG_B, d_tag),
d_rid - lower_rid,
d_branch);
if (err) {
@@ -2897,13 +2894,23 @@ again:;
}
}
// TODO rm me
if (lfsr_d_isdiverged(d_state)) {
alt &= ~LFSR_TAG_R;
}
// prune diverged?
if (d_state == LFSR_D_DIVERGINGLOWER
// prune?
// <b >b
// .-'| .-'|
// <y | | |
// .-------'| | | |
// | <r | => | <b
// | .----' | .-----------|-'|
// | | <b | <b |
// | | .----'| | .----'| |
// 1 2 3 4 4 1 2 3 4 4 2
if (lfsr_tag_prune2(
alt, weight,
p_alts[0], p_weights[0],
lower_rid, upper_rid,
lower_tag, upper_tag)
// prune because of diverged paths?
|| d_state == LFSR_D_DIVERGINGLOWER
|| (lfsr_d_isdiverged(d_state)
&& (d_state == LFSR_D_DIVERGEDUPPER)
^ lfsr_tag_isgt(alt)
@@ -2912,6 +2919,8 @@ again:;
p_alts[0], p_weights[0],
lower_rid, upper_rid,
a_rid, a_tag))) {
// note, yellow prunes always follow and have no weight, it's
// only diverged pruning that needs all these special cases
if (lfsr_tag_follow2(
alt, weight,
p_alts[0], p_weights[0],
@@ -2928,35 +2937,14 @@ again:;
&lower_rid, &upper_rid,
&lower_tag, &upper_tag);
y_branch = branch;
branch = branch_;
continue;
}
// prune?
// <b >b
// .-'| .-'|
// <y | | |
// .-------'| | | |
// | <r | => | <b
// | .----' | .-----------|-'|
// | | <b | <b |
// | | .----'| | .----'| |
// 1 2 3 4 4 1 2 3 4 4 2
if (lfsr_tag_prune2(
alt, weight,
p_alts[0], p_weights[0],
lower_rid, upper_rid,
lower_tag, upper_tag)) {
if (lfsr_tag_isred(p_alts[0])) {
alt = p_alts[0] & ~LFSR_TAG_R;
weight = p_weights[0];
branch_ = jump;
jump = p_jumps[0];
lfsr_rbyd_p_pop(p_alts, p_weights, p_jumps);
} else {
y_branch = branch;
branch = jump;
y_branch= branch;
branch = branch_;
continue;
}
}
@@ -3172,7 +3160,6 @@ again:;
// split less than
alt = LFSR_TAG_ALT(
LFSR_TAG_LE,
// TODO should this always be red?
(!lfsr_d_isdiverged(d_state))
? LFSR_TAG_R
: LFSR_TAG_B,
@@ -3199,7 +3186,6 @@ again:;
// split greater than
alt = LFSR_TAG_ALT(
LFSR_TAG_GT,
// TODO should this always be red?
(!lfsr_d_isdiverged(d_state))
? LFSR_TAG_R
: LFSR_TAG_B,