From 4d90be94f9f42b1f9559d218e83bf64ef36bf3d0 Mon Sep 17 00:00:00 2001 From: Christopher Haster Date: Tue, 12 Mar 2024 01:55:24 -0500 Subject: [PATCH] Preserve coloring during range removals This is the real kicker of our new-and-improved range removal algorithm. We can actually preserve the existing tree coloring, and the underlying rbyd invariants. Well, sort of. We preserve the red-follows-yellow and black-follows-red rules, but we don't (can't?) preserve the same-height for all black edges property. But note! The new range removal algorithm never creates _new_ black edges. It can only delete black edges, and otherwise preserves the structure of the underlying 2-3-4 tree. This means that while the resulting tree may not be perfectly balanced with h=2*log2n', where n' is the _new_ number of tags, the resulting tree _is_ limited to h=2*log2(n) where n is the _old_ number of tags. When applied to our bounded rbyd, with eventual compaction and rebalancing, we end up with the guarantee that the rbyd's height will never exceed h=2*log2(b) where b is the block size, even with arbitrary range removals. This is a great result! --- Note that this algorithm does not suffer from the yellow-diverge-yellow corner case that was an issue for preserving coloring in the previous one-pass stitching algorithm. This is because the one-pass algorithm effectively deleted the diverging alt, breaking the tail-recursive invariant of the underlying 2-3-4 tree. With the new two-pass algorithm, we _replace_ the diverging alt with a black stitching alt to stitch together the diverging trunks, so no tail-recursive invariant breaking. (Also note even if we could preserve coloring in the one-pass algorithm, it would still be breaking invariants by introducing new black edges when it stitches together diverging trunks. Worst case, resulting in ~2x the height, even when stitching with red alts (The red alt stitching brings this cost down from ~4x to ~2x worst case due to blanket recoloring. With yellow alt stitching this could probably be brought down to ~1.3x, but this would still mean every range removal could be increasing the height of the tree, which is not great.).) --- Pruning has to be a bit more complicated now, since we need to be able to recolor skipped red alts. But other than pruning the cost of recoloring vs not recoloring is pretty small: code stack one-pass, blanket recolor: 33852 2880 two-pass, blanket recolor: 33860 (+0.0%) 2880 (+0.0%) two-pass, color preserving: 33880 (+0.1%) 2880 (+0.0%) The non-rigorous random-file-write benchmark I've been using as a litmus test did not really show any improvements, but in hindsight it might have been a bit silly to use a uniform distribution of writes to test for rbyd balancing issues... Building a tree from a uniform distribution already results in a balanced tree without doing anything! --- lfs.c | 58 ++++++++++++++++++++++------------------------------------ 1 file changed, 22 insertions(+), 36 deletions(-) diff --git a/lfs.c b/lfs.c index 59f814dd..b3483ede 100644 --- a/lfs.c +++ b/lfs.c @@ -2884,10 +2884,7 @@ again:; if (d_state == LFSR_D_DIVERGEDUPPER && d_tag) { err = lfsr_rbyd_p_push(lfs, rbyd, p_alts, p_weights, p_jumps, - LFSR_TAG_ALT( - LFSR_TAG_LE, - LFSR_TAG_B, - d_tag), + LFSR_TAG_ALT(LFSR_TAG_LE, LFSR_TAG_B, d_tag), d_rid - lower_rid, d_branch); if (err) { @@ -2897,13 +2894,23 @@ again:; } } - // TODO rm me - if (lfsr_d_isdiverged(d_state)) { - alt &= ~LFSR_TAG_R; - } - - // prune diverged? - if (d_state == LFSR_D_DIVERGINGLOWER + // prune? + // b + // .-'| .-'| + // | b - // .-'| .-'| - // |