Github/littlefs
1
Fork 0
Code Actions Activity

rbyd-rr: Trying another approach, 2-trunk diverging

Browse Source 
This is a good checkpoint and is mostly working, though we're back to
recoloring the diverging alt black again. So no balance improvements.
But this already feels much better complexity-wise.

The fact that things could get back to a working state so quickly is a
good sign, or maybe just a sign I've been steeped in this algorithm for
too long...

---

The idea here is instead of a relatively complex 4-step state machine:

            diverged?                       diverged
  skip common -+-> write lower -> write common -> write upper -> done
               '-> write common -> done

We just write two trunks: one for the lower bound, one for the upper
bound.

We _do_ need to keep track of where we diverge so we can prune
correctly, so this is _technically_ still 4-steps, but it is at least
conceptually, and in code, much simpler:

             diverged?                       diverged
  write common -+-> write lower -> write common -> write upper -> done
                '-> done

Note that if we discover no tags in our range, we can terminate after
writing the lower/common trunk, which is nice. Previously we needed a
second pass.

The obvious downside is that we write the common trunk twice now. Which
is a bit of a downside, those alts will never really be used, but as a
tradeoff it really isn't that much of a waste. It's already possible for
range operations to need to write the full trunk twice, even for small
ranges:

         .-------o-------.
     .---o---.       .---o---.
   .-o-.   .-o-.   .-o-.   .-o-.
  .o. .o. .o. .o. .o. .o. .o. .o.
  a b c d e f g h i j k l m n o p
               '-+-'
               remove

The original motivation for trying yet-another-range-removal-algorithm
comes from attempting to solve issues with the color-balance of the
diverging alt.

The core conundrum being the case of two pending yellow splits. In our
previous algorithm, we only have one common trunk, so trying to
propagate two red edges violates tail recursion:

          .->                .->            .-> h=4 -.
    .-----b->              .-b->          .-b->      |
    |     .->              | .->          | .->      |
    | .---b->            .-y-b->        .-y-b->      |
    | |   .->            |   .->        |   .->      |
    | | .-b->            | .-b->        | .-b->      |
    | | | .->            b-b-b->      .-b-b-b->      |
  .-y-r-b-b-> rm me  =>           =>  |              +- unbalanced :(
  |       .->                         r-b-b-b->      |
  | .-----b->                           | | '->      |
  | |     .->                           | | .->      |
  | | .---b->                           | '-b->      |
  | | |   .->                           |   .->      |
  | | | .-b->                           | .-b->      |
  | | | | .->                           | | .->      |
  b-y-r-b-b->                           '-b-b-> h=3 -'
  ^                      ^
  diverging              lost color propagation

But if we have two trunks? Even only temporarily? This allows both
yellow splits/red edge propagation to settle tail recursively:

          .->                    .->              .-> h=3 -.
    .-----b->                  .-b->            .-b->      |
    |     .->                  | .->            | .->      |
    | .---b->                .-y-b->      .-----y-b->      |
    | |   .->                |   .->      |       .->      |
    | | .-b->                | .-b->      |     .-b->      |
    | | | .->            r---b-b-b->      | .---b-b->      |
  .-y-r-b-b-> rm me  =>  |            =>  | |              +- balanced :)
  |       .->            |       .->      y-r-b-b-b->      |
  | .-----b->            | .-----b->          | | '->      |
  | |     .->            | |     .->          | | .->      |
  | | .---b->            | | .---b->          | '-b->      |
  | | |   .->            | | |   .->          |   .->      |
  | | | .-b->            | | | .-b->          | .-b->      |
  | | | | .->            | | | | .->          | | .->      |
  b-y-r-b-b->            '-y-r-b-b->          '-b-b-> h=3 -'
  ^
  diverging

It's interesting to note while we _are_ currently recoloring the
diverging alt black (intentionally simplifying to algorithm to get
things moving), we are already allowing yellow splits/red edge
propagation by just writing out both trunks normally.

And that's what makes this yet-another-range-removal-algorithm appealing
and worth yet another iteration, the diverging trunks are no longer such
special cases. Not only will this make a better diverging color-balance
possible, it will hopefully make the whole diverging algorithm simpler,
easier, and cheaper. And it is already showing good signs so far.
This commit is contained in:
Christopher Haster
2024-04-04 12:49:35 -05:00
parent 233fc2c212
commit c73749039e
1 changed files with 266 additions and 85 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lfs.c
+266 -85
View File
@@ -2774,6 +2774,13 @@ enum {
LFSR_D_DIVERGEDUPPER = 4,
};
//enum {
// LFSR_D_NOTDIVERGEDLOWER = 0,
// LFSR_D_NOTDIVERGEDUPPER = 1,
// LFSR_D_DIVERGEDLOWER = 2,
// LFSR_D_DIVERGEDUPPER = 3,
//};
static inline bool lfsr_d_isdiverged(uint8_t d_state) {
return d_state >= LFSR_D_DIVERGEDLOWER;
}
@@ -2864,6 +2871,9 @@ static int lfsr_rbyd_appendattr(lfs_t *lfs, lfsr_rbyd_t *rbyd,
a_tag = lfs_max16(a_tag, 0x1);
b_tag = lfs_max16(b_tag, 0x1);
// follow the current trunk
lfs_size_t branch = lfsr_rbyd_trunk(rbyd);
// keep track of diverged state
//
// this is only used if we operate on a range of tags, in which case
@@ -2877,23 +2887,26 @@ static int lfsr_rbyd_appendattr(lfs_t *lfs, lfsr_rbyd_t *rbyd,
// we may also end up not diverging, in which case we fallback to a
// normal append
//
uint8_t d_state = (a_rid != b_rid || a_tag != b_tag)
? LFSR_D_DIVERGINGLOWER
: LFSR_D_NOTDIVERGING;
lfs_size_t d_branch = rbyd->eoff;
lfsr_tag_t d_tag = 0;
// uint8_t d_state = (a_rid != b_rid || a_tag != b_tag)
// ? LFSR_D_DIVERGINGLOWER
// : LFSR_D_NOTDIVERGING;
// uint8_t diverged = LFSR_D_NOTDIVERGED;
bool diverged = false;
bool d_will_diverge = false;
bool d_upper = false;
lfsr_srid_t d_rid = 0;
lfsr_tag_t d_tag = 0;
// lfs_size_t d_branch = rbyd->eoff;
again:;
// the new trunk starts here
lfs_size_t trunk = (rbyd->trunk & LFSR_RBYD_ISSHRUB) | rbyd->eoff;
lfs_size_t trunk_ = rbyd->eoff;
// keep track of bounds as we descend down the tree
//
// this gets a bit confusing as we also may need to keep
// track of both the lower and upper bounds of diverging paths
// in the case of range deletions
lfs_size_t branch = lfsr_rbyd_trunk(rbyd);
lfsr_srid_t lower_rid = 0;
lfsr_srid_t upper_rid = rbyd->weight;
lfsr_tag_t lower_tag = 0x000;
@@ -2948,7 +2961,7 @@ again:;
lfs_size_t branch_ = branch + d;
// do bounds want to take different paths? begin diverging
if (!lfsr_d_isdiverged(d_state)
if (!diverged
&& lfsr_tag_follow2(alt, weight,
p_alts[0], p_weights[0],
lower_rid, upper_rid,
@@ -2957,51 +2970,30 @@ again:;
p_alts[0], p_weights[0],
lower_rid, upper_rid,
b_rid, b_tag)) {
printf("%04x->%04x: %cdiverge 0x%x w%d 0x%x w%d\n",
printf("%04x->%04x: diverge 0x%x w%d 0x%x w%d\n",
branch,
rbyd->eoff,
lfsr_tag_isred(alt) ? 'r' : 'b',
alt,
weight,
p_alts[0],
p_weights[0]);
LFS_ASSERT(d_state != LFSR_D_NOTDIVERGING);
if (d_upper) {
alt &= ~LFSR_TAG_R;
diverged = true;
// transition to the diverged state
//
// note that if red alt diverged if would have been
// caught on the previous pass
d_state = lfsr_d_diverge(d_state);
// TODO trim or something?
if (d_state != LFSR_D_DIVERGEDUPPER) {
if (lfsr_tag_follow2(
alt, weight,
p_alts[0], p_weights[0],
lower_rid, upper_rid,
a_rid, a_tag)) {
lfsr_tag_flip2(
&alt, &weight,
p_alts[0], p_weights[0],
lower_rid, upper_rid);
lfs_swap32(&jump, &branch_);
}
lfsr_tag_trim2(
alt, weight,
p_alts[0], p_weights[0],
&lower_rid, &upper_rid,
&lower_tag, &upper_tag);
}
// stitch together diverged branches
if (d_state == LFSR_D_DIVERGEDUPPER) {
// alt = LFSR_TAG_ALT(
// alt & LFSR_TAG_GT,
// alt & LFSR_TAG_R,
// d_tag);
// if (lfsr_tag_isle(alt)) {
// weight = d_rid - lower_rid + weight;
// } else {
// weight = (upper_rid - lower_rid) - weight;
// }
if (lfsr_tag_isle(alt)) {
alt = LFSR_TAG_ALT(
LFSR_TAG_LE,
(!lfsr_tag_isred(p_alts[0]))
? (LFSR_TAG_R & alt)
: LFSR_TAG_B,
alt & LFSR_TAG_R,
d_tag);
printf("%04x->%04x: dle 0x%x %d w%d (%d %d)\n",
branch,
@@ -3018,7 +3010,7 @@ again:;
weight -= p_weights[0];
}
lower_rid -= weight;
jump = d_branch;
// jump = d_branch;
} else {
lfsr_tag_flip2(
&alt, &weight,
@@ -3026,9 +3018,7 @@ again:;
lower_rid, upper_rid);
alt = LFSR_TAG_ALT(
LFSR_TAG_LE,
(false) // (!lfsr_tag_isred(p_alts[0]))
? (LFSR_TAG_R & alt)
: LFSR_TAG_B,
alt & LFSR_TAG_R,
d_tag);
printf("%04x->%04x: dgt 0x%x %d w%d (%d %d)\n",
branch,
@@ -3049,7 +3039,7 @@ again:;
&alt, &weight,
p_alts[0], p_weights[0],
lower_rid, upper_rid);
branch_ = d_branch;
// branch_ = d_branch;
}
printf("%04x->%04x: dtag 0x%x w%d (%d %d)\n",
@@ -3059,27 +3049,21 @@ again:;
weight,
lower_rid,
upper_rid);
// TODO doc
y_branch = d_branch;
goto dont_trim_me;
} else {
alt &= ~LFSR_TAG_R;
//d_will_diverge = true;
diverged = true;
}
branch = branch_;
continue;
}
// trim unreachable diverged alts so they end up pruned
if ((lfsr_d_isdiverged(d_state)
&& (d_state == LFSR_D_DIVERGEDUPPER)
} else if (diverged
&& (d_upper
^ lfsr_tag_isgt(alt)
^ lfsr_tag_follow2(
alt, weight,
p_alts[0], p_weights[0],
lower_rid, upper_rid,
a_rid, a_tag))
// this includes all diverging-lower alts
|| d_state == LFSR_D_DIVERGINGLOWER) {
a_rid, a_tag))) {
if (lfsr_tag_follow2(
alt, weight,
p_alts[0], p_weights[0],
@@ -3098,7 +3082,157 @@ again:;
weight = 0;
}
dont_trim_me:;
// if (!lfsr_d_isdiverged(d_state)
// && lfsr_tag_follow2(alt, weight,
// p_alts[0], p_weights[0],
// lower_rid, upper_rid,
// a_rid, a_tag)
// ^ lfsr_tag_follow2(alt, weight,
// p_alts[0], p_weights[0],
// lower_rid, upper_rid,
// b_rid, b_tag)) {
// printf("%04x->%04x: %cdiverge 0x%x w%d 0x%x w%d\n",
// branch,
// rbyd->eoff,
// lfsr_tag_isred(alt) ? 'r' : 'b',
// alt,
// weight,
// p_alts[0],
// p_weights[0]);
// LFS_ASSERT(d_state != LFSR_D_NOTDIVERGING);
//
// // transition to the diverged state
// //
// // note that if red alt diverged if would have been
// // caught on the previous pass
// d_state = lfsr_d_diverge(d_state);
//
// // TODO trim or something?
// if (d_state != LFSR_D_DIVERGEDUPPER) {
// if (lfsr_tag_follow2(
// alt, weight,
// p_alts[0], p_weights[0],
// lower_rid, upper_rid,
// a_rid, a_tag)) {
// lfsr_tag_flip2(
// &alt, &weight,
// p_alts[0], p_weights[0],
// lower_rid, upper_rid);
// lfs_swap32(&jump, &branch_);
// }
//
// lfsr_tag_trim2(
// alt, weight,
// p_alts[0], p_weights[0],
// &lower_rid, &upper_rid,
// &lower_tag, &upper_tag);
// }
//
// // stitch together diverged branches
// if (d_state == LFSR_D_DIVERGEDUPPER) {
// if (lfsr_tag_isle(alt)) {
// alt = LFSR_TAG_ALT(
// LFSR_TAG_LE,
// (!lfsr_tag_isred(p_alts[0]))
// ? (LFSR_TAG_R & alt)
// : LFSR_TAG_B,
// d_tag);
// printf("%04x->%04x: dle 0x%x %d w%d (%d %d)\n",
// branch,
// rbyd->eoff,
// d_tag,
// d_rid,
// weight,
// lower_rid,
// upper_rid);
// lower_rid += weight;
// weight = d_rid - lower_rid + weight;
// if (lfsr_tag_isred(p_alts[0])
// && lfsr_tag_isle(p_alts[0])) {
// weight -= p_weights[0];
// }
// lower_rid -= weight;
// jump = d_branch;
// } else {
// lfsr_tag_flip2(
// &alt, &weight,
// p_alts[0], p_weights[0],
// lower_rid, upper_rid);
// alt = LFSR_TAG_ALT(
// LFSR_TAG_LE,
// (false) // (!lfsr_tag_isred(p_alts[0]))
// ? (LFSR_TAG_R & alt)
// : LFSR_TAG_B,
// d_tag);
// printf("%04x->%04x: dgt 0x%x %d w%d (%d %d)\n",
// branch,
// rbyd->eoff,
// d_tag,
// d_rid,
// weight,
// lower_rid,
// upper_rid);
// lower_rid += weight;
// weight = d_rid - lower_rid + weight;
// if (lfsr_tag_isred(p_alts[0])
// && lfsr_tag_isle(p_alts[0])) {
// weight -= p_weights[0];
// }
// lower_rid -= weight;
// lfsr_tag_flip2(
// &alt, &weight,
// p_alts[0], p_weights[0],
// lower_rid, upper_rid);
// branch_ = d_branch;
// }
//
// printf("%04x->%04x: dtag 0x%x w%d (%d %d)\n",
// branch,
// rbyd->eoff,
// alt,
// weight,
// lower_rid,
// upper_rid);
//
// // TODO doc
// y_branch = d_branch;
// goto dont_trim_me;
// }
//
// branch = branch_;
// continue;
// }
// // trim unreachable diverged alts so they end up pruned
// if ((lfsr_d_isdiverged(d_state)
// && (d_state == LFSR_D_DIVERGEDUPPER)
// ^ lfsr_tag_isgt(alt)
// ^ lfsr_tag_follow2(
// alt, weight,
// p_alts[0], p_weights[0],
// lower_rid, upper_rid,
// a_rid, a_tag))
// // this includes all diverging-lower alts
// || d_state == LFSR_D_DIVERGINGLOWER) {
// if (lfsr_tag_follow2(
// alt, weight,
// p_alts[0], p_weights[0],
// lower_rid, upper_rid,
// a_rid, a_tag)) {
// lfsr_tag_flip2(
// &alt, &weight,
// p_alts[0], p_weights[0],
// lower_rid, upper_rid);
// lfs_swap32(&jump, &branch_);
// }
// lfsr_tag_trim(
// alt, weight,
// &lower_rid, &upper_rid,
// &lower_tag, &upper_tag);
// weight = 0;
// }
// dont_trim_me:;
// prune?
// <b >b
// .-'| .-'|
@@ -3141,10 +3275,8 @@ again:;
jump = p_jumps[0];
lfsr_rbyd_p_pop(p_alts, p_weights, p_jumps);
// prune unreachable red alts
} else if (lfsr_tag_isred(alt)
// prune unreachable black alts if root
|| (!p_alts[0] && !lfsr_d_isdiverged(d_state))) {
// prune unreachable root alts and red alts
} else if (!p_alts[0] || lfsr_tag_isred(alt)) {
printf("%04x->%04x: rprune 0x%x w%d\n",
branch,
rbyd->eoff,
@@ -3295,6 +3427,11 @@ again:;
p_alts[0], p_weights[0],
&lower_rid, &upper_rid,
&lower_tag, &upper_tag);
if (d_will_diverge) {
diverged = true;
d_will_diverge = false;
}
}
// push alt onto our queue
@@ -3320,23 +3457,17 @@ again:;
// the last alt should always end up black
LFS_ASSERT(lfsr_tag_isblack(p_alts[0]));
// no divergence? guess we only need one trunk then, actually write
// it out this time
if (d_state == LFSR_D_DIVERGINGLOWER) {
printf("%04x->%04x: not diverging\n",
// diverged lower trunk? move on to upper trunk
if (diverged && !d_upper) {
printf("%04x->%04x: diverging again %d 0x%x\n",
branch,
rbyd->eoff);
d_state = LFSR_D_NOTDIVERGING;
goto again;
rbyd->eoff,
lower_rid,
lower_tag);
// diverged lower trunk? we need an upper trunk too
} else if (d_state == LFSR_D_DIVERGEDLOWER) {
printf("%04x->%04x: diverging again\n",
branch,
rbyd->eoff);
// keep track of last alt on diverged trunk to stitch the trunks
// together with
d_state = LFSR_D_DIVERGINGUPPER;
diverged = false;
d_upper = true;
// keep track of the lower diverged bound
d_rid = lower_rid;
d_tag = lower_tag;
@@ -3357,17 +3488,67 @@ again:;
return err;
}
// swap tag/rid and write out the upper trunk
// swap tag/rid and move on to upper trunk
// TODO handle trunk differently? do we need to bother to stage this?
branch = trunk_;
lfs_swap16(&a_tag, &b_tag);
lfs_sswap32(&a_rid, &b_rid);
goto again;
// diverged upper trunk? done diverging
} else if (d_state == LFSR_D_DIVERGEDUPPER) {
} else if (diverged && d_upper) {
// use the lower diverged bound for leaf weight calculation
lower_rid = d_rid;
lower_tag = d_tag;
}
// // no divergence? guess we only need one trunk then, actually write
// // it out this time
// if (d_state == LFSR_D_DIVERGINGLOWER) {
// printf("%04x->%04x: not diverging\n",
// branch,
// rbyd->eoff);
// d_state = LFSR_D_NOTDIVERGING;
// goto again;
//
// // diverged lower trunk? we need an upper trunk too
// } else if (d_state == LFSR_D_DIVERGEDLOWER) {
// printf("%04x->%04x: diverging again\n",
// branch,
// rbyd->eoff);
// // keep track of last alt on diverged trunk to stitch the trunks
// // together with
// d_state = LFSR_D_DIVERGINGUPPER;
// d_rid = lower_rid;
// d_tag = lower_tag;
//
// // flush any pending alts
// err = lfsr_rbyd_p_flush(lfs, rbyd,
// p_alts, p_weights, p_jumps, 3);
// if (err) {
// return err;
// }
//
// // terminate diverged trunk with an unreachable tag
// err = lfsr_rbyd_appendattr_(lfs, rbyd,
// (lfsr_rbyd_isshrub(rbyd) ? LFSR_TAG_SHRUB : 0)
// | LFSR_TAG_NULL,
// 0,
// LFSR_DATA_NULL());
// if (err) {
// return err;
// }
//
// // swap tag/rid and write out the upper trunk
// lfs_swap16(&a_tag, &b_tag);
// lfs_sswap32(&a_rid, &b_rid);
// goto again;
//
// // diverged upper trunk? done diverging
// } else if (d_state == LFSR_D_DIVERGEDUPPER) {
// // use the lower diverged bound for leaf weight calculation
// lower_rid = d_rid;
// lower_tag = d_tag;
// }
// split leaf nodes?
//
@@ -3467,7 +3648,7 @@ leaf:;
}
// update the trunk and weight
rbyd->trunk = trunk;
rbyd->trunk = (rbyd->trunk & LFSR_RBYD_ISSHRUB) | trunk_;
rbyd->weight += delta;
return 0;
}