Github/littlefs
1
Fork 0
Code Actions Activity

Rerouted all btree mutation through attr-list parser

Browse Source 
The idea here: Instead of having unique functionality for each
individual btree operation (push/set/pop/split), we treat btrees sort of
like rbyds, with a single commit entry point that operates on attr-lists.

This adds code cost, due to needing to parse the attr-list for properties
that can affect inlined btrees (tag changes mostly), but, in theory, comes
with some advantages:

1. A single btree commit entry point with all of the inlined/uninlining
   logic should offer better chances for code deduplication, vs
   spreading this logic out in each btree operation.

2. Higher-levels should know what the current weight of the branch is,
   so we may be able to avoid the implicit math needed to calculate
   deltas.

3. Higher-levels have more knowledge about the state of the btree in
   general, so there may be other shortcuts. The mtree, for example,
   only operates on weight=1 entries, which greatly simplifies a lot of
   the related math.

Note that btrees still have strict limits in what's possible in an
attr-list. Btree operations can't cross leaf-rbyd boundaries for
example.

---

A notable omission in this change is the loss of reinlining btrees.

This wase dropped for a couple reasons. It may be worth adding back at a
later time, maybe after we actually have files implemented, but for now
does not seem worth it:

1. Reinlining adds code cost. Reinlining is more complex than you might
   expect because we only reinline on compaction. And because we compact
   before playing out our attr-list, we need to know if a commit makes
   the btree inlinable before committing to the btree.

   This is still doable with our attr-lists. We already derive the
   change in tags, since we need this to know when to uninline. But it
   adds a kind of complex bailing out of btree commits.

2. The benefits of reinlining may not be that great. In most systems, a
   tree that is uninlined once is likely to be uninlined again. It's
   only if there is a bigger state change in a system that it makes
   sense to reinline.

   Though, to be fair, waiting for compaction to reinline handled this
   quite well. Only reinlining when all erased storage is used up...

3. Thanks to our roots did entry, our mtree can never reinline.

   It would be nice to change this, but this would require explicit
   handling in lfsr_mdir_commit. Future work?

4. Files are another can of worms, with more complex interactions with
   inlinability thanks to (at least on paper right now) always having
   inlined data even when uninlined.

   If reinlining is valuable for files this can change during that work.

5. Even if files never support reinlinability, truncating files (via
   either lfsr_file_truncate or LFSR_O_TRUNC) should give the file a
   blank slate, effectively reinlining the file in that case.

---

The current implementation also changes the attr-list to be mutable so
we can adjust attr-list based on the current btree node. This is a
temporary hack! We should add the appropriate functionality to our rbyd
utilities to revert this eventually.
This commit is contained in:
Christopher Haster
2023-08-13 12:40:34 -05:00
parent 3dbc986752
commit 9b2f3cd5bb
4 changed files with 840 additions and 607 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lfs.c
+520 -297
View File
@@ -1317,9 +1317,10 @@ typedef struct lfsr_attr {
#define LFSR_ATTR_NOOP LFSR_ATTR(-1, RM, 0, NULL)
// TODO make this const again eventually
#define LFSR_ATTRS(...) \
(const lfsr_attr_t[]){__VA_ARGS__}, \
sizeof((const lfsr_attr_t[]){__VA_ARGS__}) / sizeof(lfsr_attr_t)
(lfsr_attr_t[]){__VA_ARGS__}, \
sizeof((lfsr_attr_t[]){__VA_ARGS__}) / sizeof(lfsr_attr_t)
//struct lfsr_attr_from {
// const lfsr_rbyd_t *rbyd;
@@ -3362,38 +3363,39 @@ static lfs_ssize_t lfsr_rbyd_estimateall(lfs_t *lfs, const lfsr_rbyd_t *rbyd,
#endif
}
// determine if there are fewer than "cutoff" unique ids in the rbyd,
// this is used to determine if the underlying rbyd is degenerate and can
// be reverted to an inlined btree
// TODO
//// determine if there are fewer than "cutoff" unique ids in the rbyd,
//// this is used to determine if the underlying rbyd is degenerate and can
//// be reverted to an inlined btree
////
//// note cutoff is expected to be quite small, <= 2, so we should make sure
//// to exit our traverse early
//static int lfsr_rbyd_isdegenerate(lfs_t *lfs, const lfsr_rbyd_t *rbyd,
// lfs_ssize_t cutoff) {
// // cutoff=-1 => no cutoff
// if (cutoff < 0) {
// return false;
// }
//
// note cutoff is expected to be quite small, <= 2, so we should make sure
// to exit our traverse early
static int lfsr_rbyd_isdegenerate(lfs_t *lfs, const lfsr_rbyd_t *rbyd,
lfs_ssize_t cutoff) {
// cutoff=-1 => no cutoff
if (cutoff < 0) {
return false;
}
// count ids until we exceed our cutoff
lfs_ssize_t rid = -1;
lfs_size_t count = 0;
while (true) {
int err = lfsr_rbyd_lookupnext(lfs, rbyd, rid+1, 0,
&rid, NULL, NULL, NULL);
if (err && err != LFS_ERR_NOENT) {
return err;
}
if (err == LFS_ERR_NOENT) {
return true;
}
count += 1;
if (count > (lfs_size_t)cutoff) {
return false;
}
}
}
// // count ids until we exceed our cutoff
// lfs_ssize_t rid = -1;
// lfs_size_t count = 0;
// while (true) {
// int err = lfsr_rbyd_lookupnext(lfs, rbyd, rid+1, 0,
// &rid, NULL, NULL, NULL);
// if (err && err != LFS_ERR_NOENT) {
// return err;
// }
// if (err == LFS_ERR_NOENT) {
// return true;
// }
//
// count += 1;
// if (count > (lfs_size_t)cutoff) {
// return false;
// }
// }
//}
// some low-level name things
@@ -3607,7 +3609,7 @@ static int lfsr_data_readbtreeinlined(lfs_t *lfs, lfsr_data_t *data,
btree->u.i.weight = lfsr_btree_setinlined(weight);
btree->u.i.tag = tag;
lfs_ssize_t size = lfsr_data_read(lfs, data,
btree->u.i.buffer, LFSR_BTREE_INLINESIZE);
btree->u.i.buf, LFSR_BTREE_INLINESIZE);
if (size < 0) {
return size;
}
@@ -3660,7 +3662,7 @@ static int lfsr_btree_lookupnext_(lfs_t *lfs,
*weight_ = lfsr_btree_weight(btree);
}
if (data_) {
*data_ = LFSR_DATA(btree->u.i.buffer, btree->u.i.size);
*data_ = LFSR_DATA(btree->u.i.buf, btree->u.i.size);
}
return 0;
}
@@ -3847,7 +3849,7 @@ static int lfsr_btree_parent(lfs_t *lfs,
// core btree algorithm
static int lfsr_btree_commit(lfs_t *lfs,
lfsr_btree_t *btree, lfs_size_t bid, lfs_ssize_t cutoff,
lfsr_btree_t *btree, lfs_size_t bid, // lfs_ssize_t cutoff,
lfsr_rbyd_t *rbyd,
const lfsr_attr_t *attrs, lfs_size_t attr_count) {
// other layers should check for inlined btrees before this
@@ -3926,7 +3928,7 @@ static int lfsr_btree_commit(lfs_t *lfs,
}
*rbyd = parent;
cutoff = -1;
// cutoff = -1;
continue;
compact:;
@@ -3934,23 +3936,24 @@ static int lfsr_btree_commit(lfs_t *lfs,
lfsr_rbyd_t rbyd_;
lfs_size_t split_rid;
// first check if we are a degenerate root and can be reverted to
// an inlined btree
//
// This gets a bit weird since we're defering our pending
// attributes to after the compaction. When we can/can't be inlined
// depends on those attributes, but trying to evaluate attributes
// is complicated and expensive.
//
// Instead we just let the upper layers indicate a cutoff for when
// an rbyd can be inlined, and leave the inlining work up to the
// upper layers.
if (pid == -1) {
int degenerate = lfsr_rbyd_isdegenerate(lfs, rbyd, cutoff);
if (degenerate) {
return degenerate;
}
}
// TODO
// // first check if we are a degenerate root and can be reverted to
// // an inlined btree
// //
// // This gets a bit weird since we're defering our pending
// // attributes to after the compaction. When we can/can't be inlined
// // depends on those attributes, but trying to evaluate attributes
// // is complicated and expensive.
// //
// // Instead we just let the upper layers indicate a cutoff for when
// // an rbyd can be inlined, and leave the inlining work up to the
// // upper layers.
// if (pid == -1) {
// int degenerate = lfsr_rbyd_isdegenerate(lfs, rbyd, cutoff);
// if (degenerate) {
// return degenerate;
// }
// }
// check if we're within our compaction threshold, otherwise we
// need to split
@@ -4037,7 +4040,7 @@ static int lfsr_btree_commit(lfs_t *lfs,
}
*rbyd = parent;
cutoff = -1;
// cutoff = -1;
continue;
split:;
@@ -4181,7 +4184,7 @@ static int lfsr_btree_commit(lfs_t *lfs,
}
*rbyd = parent;
cutoff = -1;
// cutoff = -1;
continue;
merge:;
@@ -4386,7 +4389,7 @@ static int lfsr_btree_commit(lfs_t *lfs,
}
*rbyd = parent;
cutoff = -1;
// cutoff = -1;
continue;
}
@@ -4395,255 +4398,441 @@ static int lfsr_btree_commit(lfs_t *lfs,
return false;
}
static int lfsr_btree_push(lfs_t *lfs, lfsr_btree_t *btree,
lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) {
// TODO avoid mutable attrs? merge this things into rbyd_appendall?
static void lfsr_btree_adjust(lfs_ssize_t bid,
lfsr_attr_t *attrs, lfs_size_t attr_count) {
for (lfs_size_t i = 0; i < attr_count; i++) {
if (attrs[i].rid != -1) {
attrs[i].rid -= bid;
}
}
}
static int lfsr_btree_commit__(lfs_t *lfs, lfsr_btree_t *btree,
lfsr_attr_t *attrs, lfs_size_t attr_count) {
// first find the effective bid and any changes to the number of tags
lfs_size_t bid = -1;
lfs_ssize_t tag_delta = 0;
for (lfs_size_t i = 0; i < attr_count; i++) {
// note unsigned min here chooses non-negative bids
bid = lfs_min32(bid, attrs[i].rid);
// non-grow tags with deltas change the number of tags
if (!lfsr_tag_isgrow(attrs[i].tag)) {
tag_delta += lfs_sclamp32(attrs[i].delta, -1, +1);
}
}
LFS_ASSERT(bid <= lfsr_btree_weight(btree));
// null btree?
if (lfsr_btree_isinlined(btree) && lfsr_btree_weight(btree) == 0) {
LFS_ASSERT(bid == 0);
// inlined btree? staying inlined?
if (lfsr_btree_isinlined(btree)) {
// staying inlined?
if (lfs_clamp32(lfsr_btree_weight(btree), 0, 1) + tag_delta <= 1) {
for (lfs_size_t i = 0; i < attr_count; i++) {
// update our inlined tag, make sure to strip wide/grow bits
if (lfsr_tag_suptype(lfsr_tag_key(attrs[i].tag))
== LFSR_TAG_STRUCT) {
btree->u.i.tag = lfsr_tag_key(attrs[i].tag);
// mark as inlined
btree->u.i.weight = lfsr_btree_setinlined(weight);
btree->u.i.tag = tag;
lfs_ssize_t d = lfsr_data_read(lfs, &data,
btree->u.i.buffer, LFSR_BTREE_INLINESIZE);
lfsr_data_t data_ = attrs[i].data;
lfs_ssize_t d = lfsr_data_read(lfs, &data_,
btree->u.i.buf, LFSR_BTREE_INLINESIZE);
if (d < 0) {
return d;
}
LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
btree->u.i.size = d;
}
// update our btree weight
LFS_ASSERT((lfs_ssize_t)lfsr_btree_weight(btree)
+ attrs[i].delta >= 0);
btree->u.i.weight += attrs[i].delta;
}
return 0;
// inlined btree, need to expand into an rbyd
} else if (lfsr_btree_isinlined(btree)) {
// uninlining?
} else {
// allocate a root rbyd
lfsr_rbyd_t rbyd;
int err = lfsr_rbyd_alloc(lfs, &rbyd);
if (err) {
return err;
}
// commit our entries
err = lfsr_rbyd_commit(lfs, &rbyd, LFSR_ATTRS(
LFSR_ATTR(0, TAG(btree->u.i.tag), +lfsr_btree_weight(btree),
BUF(btree->u.i.buffer, btree->u.i.size)),
LFSR_ATTR(bid, TAG(tag), +weight, DATA(data))));
// prepend inlined tag if we have one
if (lfsr_btree_weight(btree) > 0) {
err = lfsr_rbyd_append(lfs, &rbyd,
0, btree->u.i.tag, +lfsr_btree_weight(btree),
LFSR_DATA_BUF(btree->u.i.buf, btree->u.i.size));
if (err) {
return err;
}
}
// commit our attrs
err = lfsr_rbyd_appendall(lfs, &rbyd, -1, -1,
attrs, attr_count);
if (err) {
return err;
}
err = lfsr_rbyd_appendcksum(lfs, &rbyd);
if (err) {
return err;
}
// save our new root
btree->u.r.rbyd = rbyd;
return 0;
}
}
// a normal btree
} else {
// lookup in which leaf our rid resides
// lookup in which leaf our bids reside
//
// for lfsr_btree_commit operations to work out, we need to
// limit our bid to an rid in the tree, which is what this min
// is doing
//
// note it is possible for our btree to have a weight of zero here,
// since we defer inlining until compaction time
lfs_size_t bid_ = lfs_min32(bid,
lfs_smax32(lfsr_btree_weight(btree)-1, 0));
// note it's entirely possible for our btree to have a weight of
// zero here
lfsr_rbyd_t rbyd = btree->u.r.rbyd;
lfs_ssize_t rid = -1;
lfs_size_t rweight = 0;
int err = lfsr_btree_lookupnext_(lfs, btree, bid_,
NULL, &rbyd, &rid, NULL, &rweight, NULL);
if (err && err != LFS_ERR_NOENT) {
if (lfsr_btree_weight(btree) > 0) {
lfs_ssize_t rid;
int err = lfsr_btree_lookupnext_(lfs, btree,
lfs_min32(bid, lfsr_btree_weight(btree)-1),
&bid, &rbyd, &rid, NULL, NULL, NULL);
if (err) {
LFS_ASSERT(err != LFS_ERR_NOENT);
return err;
}
// adjust rid for push
if (bid >= lfsr_btree_weight(btree)) {
rid += 1;
} else {
rid -= rweight-1;
// adjust bid to indicate the zero-most rid
bid -= rid;
}
// TODO do this in rbyd_appendall?
// adjust our attrs
lfsr_btree_adjust(bid, attrs, attr_count);
// commit our rid into the tree, letting lfsr_btree_commit take care
// of the rest
int degenerate = lfsr_btree_commit(lfs, btree, bid_, 0, &rbyd,
LFSR_ATTRS(
LFSR_ATTR(rid, TAG(tag), +weight, DATA(data))));
if (degenerate < 0) {
return degenerate;
}
// revert to an inlined btree
if (degenerate) {
// mark as inlined
btree->u.i.weight = lfsr_btree_setinlined(weight);
btree->u.i.tag = tag;
lfs_ssize_t d = lfsr_data_read(lfs, &data,
btree->u.i.buffer, LFSR_BTREE_INLINESIZE);
if (d < 0) {
return d;
}
LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
btree->u.i.size = d;
int err = lfsr_btree_commit(lfs, btree, bid, &rbyd,
attrs, attr_count);
if (err) {
return err;
}
return 0;
}
}
// TODO
static int lfsr_btree_push(lfs_t *lfs, lfsr_btree_t *btree,
lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) {
LFS_ASSERT(bid <= lfsr_btree_weight(btree));
return lfsr_btree_commit__(lfs, btree, LFSR_ATTRS(
LFSR_ATTR(bid, TAG(tag), +weight, DATA(data))));
}
// TODO
//static int lfsr_btree_push_(lfs_t *lfs, lfsr_btree_t *btree,
// lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) {
// LFS_ASSERT(bid <= lfsr_btree_weight(btree));
//
// // null btree?
// if (lfsr_btree_isinlined(btree) && lfsr_btree_weight(btree) == 0) {
// LFS_ASSERT(bid == 0);
//
// // mark as inlined
// btree->u.i.weight = lfsr_btree_setinlined(weight);
// btree->u.i.tag = tag;
//
// lfs_ssize_t d = lfsr_data_read(lfs, &data,
// btree->u.i.buf, LFSR_BTREE_INLINESIZE);
// if (d < 0) {
// return d;
// }
// LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
// btree->u.i.size = d;
// return 0;
//
// // inlined btree, need to expand into an rbyd
// } else if (lfsr_btree_isinlined(btree)) {
// lfsr_rbyd_t rbyd;
// int err = lfsr_rbyd_alloc(lfs, &rbyd);
// if (err) {
// return err;
// }
//
// // commit our entries
// err = lfsr_rbyd_commit(lfs, &rbyd, LFSR_ATTRS(
// LFSR_ATTR(0, TAG(btree->u.i.tag), +lfsr_btree_weight(btree),
// BUF(btree->u.i.buf, btree->u.i.size)),
// LFSR_ATTR(bid, TAG(tag), +weight, DATA(data))));
// if (err) {
// return err;
// }
//
// btree->u.r.rbyd = rbyd;
// return 0;
//
// // a normal btree
// } else {
// // lookup in which leaf our rid resides
// //
// // for lfsr_btree_commit operations to work out, we need to
// // limit our bid to an rid in the tree, which is what this min
// // is doing
// //
// // note it is possible for our btree to have a weight of zero here,
// // since we defer inlining until compaction time
// lfs_size_t bid_ = lfs_min32(bid,
// lfs_smax32(lfsr_btree_weight(btree)-1, 0));
// lfsr_rbyd_t rbyd = btree->u.r.rbyd;
// lfs_ssize_t rid = -1;
// lfs_size_t rweight = 0;
// int err = lfsr_btree_lookupnext_(lfs, btree, bid_,
// NULL, &rbyd, &rid, NULL, &rweight, NULL);
// if (err && err != LFS_ERR_NOENT) {
// return err;
// }
//
// // adjust rid for push
// if (bid >= lfsr_btree_weight(btree)) {
// rid += 1;
// } else {
// rid -= rweight-1;
// }
//
// // commit our rid into the tree, letting lfsr_btree_commit take care
// // of the rest
// int degenerate = lfsr_btree_commit(lfs, btree, bid_, 0, &rbyd,
// LFSR_ATTRS(
// LFSR_ATTR(rid, TAG(tag), +weight, DATA(data))));
// if (degenerate < 0) {
// return degenerate;
// }
//
// // revert to an inlined btree
// if (degenerate) {
// // mark as inlined
// btree->u.i.weight = lfsr_btree_setinlined(weight);
// btree->u.i.tag = tag;
//
// lfs_ssize_t d = lfsr_data_read(lfs, &data,
// btree->u.i.buf, LFSR_BTREE_INLINESIZE);
// if (d < 0) {
// return d;
// }
// LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
// btree->u.i.size = d;
// }
//
// return 0;
// }
//}
// TODO
static int lfsr_btree_set(lfs_t *lfs, lfsr_btree_t *btree,
lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) {
LFS_ASSERT(bid < lfsr_btree_weight(btree));
LFS_ASSERT(lfsr_btree_weight(btree) > 0);
// inlined btree?
if (lfsr_btree_isinlined(btree)) {
LFS_ASSERT(bid == lfsr_btree_weight(btree)-1);
// mark as inlined
btree->u.i.weight = lfsr_btree_setinlined(weight);
btree->u.i.tag = tag;
lfs_ssize_t d = lfsr_data_read(lfs, &data,
btree->u.i.buffer, LFSR_BTREE_INLINESIZE);
if (d < 0) {
return d;
}
LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
btree->u.i.size = d;
return 0;
// a normal btree
} else {
// lookup in which leaf our rid resides
lfsr_rbyd_t rbyd;
lfsr_tag_t rtag;
lfs_ssize_t rid;
lfs_size_t rweight;
int err = lfsr_btree_lookupnext_(lfs, btree, bid,
NULL, &rbyd, &rid, &rtag, &rweight, NULL);
// TODO yes this is completely redundant and eventually should be removed
//
// We're looking up the bid here to find it's weight so we can compute the
// correct delta. We could move this into lfsr_btree_commit__ hackily, but
// the currrent theory is we don't need this at all and upper layers can
// calculate the delta instead of the absolute weight when needed.
lfs_size_t weight_;
int err = lfsr_btree_lookupnext(lfs, btree, bid,
NULL, NULL, &weight_, NULL);
if (err) {
return err;
}
// commit our rid into the tree, letting lfsr_btree_commit take care
// of the rest
int degenerate = lfsr_btree_commit(lfs, btree, bid, 1, &rbyd,
LFSR_ATTRS(
LFSR_ATTR(rid, WIDE(TAG(tag)), 0, DATA(data)),
LFSR_ATTR(rid, GROW(RM), +weight-rweight, NULL)));
if (degenerate < 0) {
return degenerate;
}
// revert to an inlined btree
if (degenerate) {
// mark as inlined
btree->u.i.weight = lfsr_btree_setinlined(weight);
btree->u.i.tag = tag;
lfs_ssize_t d = lfsr_data_read(lfs, &data,
btree->u.i.buffer, LFSR_BTREE_INLINESIZE);
if (d < 0) {
return d;
}
LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
btree->u.i.size = d;
}
return 0;
}
// note we need a second tag here in case our entry has a
// name attributes, the name attribute holds the weight not
// the struct tag
return lfsr_btree_commit__(lfs, btree, LFSR_ATTRS(
LFSR_ATTR(bid, WIDE(TAG(tag)), 0, DATA(data)),
LFSR_ATTR(bid, GROW(RM), weight - weight_, NULL)));
}
// TODO
//static int lfsr_btree_set(lfs_t *lfs, lfsr_btree_t *btree,
// lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) {
// LFS_ASSERT(bid < lfsr_btree_weight(btree));
// LFS_ASSERT(lfsr_btree_weight(btree) > 0);
//
// // inlined btree?
// if (lfsr_btree_isinlined(btree)) {
// LFS_ASSERT(bid == lfsr_btree_weight(btree)-1);
//
// // mark as inlined
// btree->u.i.weight = lfsr_btree_setinlined(weight);
// btree->u.i.tag = tag;
//
// lfs_ssize_t d = lfsr_data_read(lfs, &data,
// btree->u.i.buf, LFSR_BTREE_INLINESIZE);
// if (d < 0) {
// return d;
// }
// LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
// btree->u.i.size = d;
// return 0;
//
// // a normal btree
// } else {
// // lookup in which leaf our rid resides
// lfsr_rbyd_t rbyd;
// lfsr_tag_t rtag;
// lfs_ssize_t rid;
// lfs_size_t rweight;
// int err = lfsr_btree_lookupnext_(lfs, btree, bid,
// NULL, &rbyd, &rid, &rtag, &rweight, NULL);
// if (err) {
// return err;
// }
//
// // commit our rid into the tree, letting lfsr_btree_commit take care
// // of the rest
// int degenerate = lfsr_btree_commit(lfs, btree, bid, 1, &rbyd,
// LFSR_ATTRS(
// LFSR_ATTR(rid, WIDE(TAG(tag)), 0, DATA(data)),
// LFSR_ATTR(rid, GROW(RM), +weight-rweight, NULL)));
// if (degenerate < 0) {
// return degenerate;
// }
//
// // revert to an inlined btree
// if (degenerate) {
// // mark as inlined
// btree->u.i.weight = lfsr_btree_setinlined(weight);
// btree->u.i.tag = tag;
//
// lfs_ssize_t d = lfsr_data_read(lfs, &data,
// btree->u.i.buf, LFSR_BTREE_INLINESIZE);
// if (d < 0) {
// return d;
// }
// LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
// btree->u.i.size = d;
// }
//
// return 0;
// }
//}
// TODO
static int lfsr_btree_pop(lfs_t *lfs, lfsr_btree_t *btree, lfs_size_t bid) {
LFS_ASSERT(bid < lfsr_btree_weight(btree));
LFS_ASSERT(lfsr_btree_weight(btree) > 0);
// inlined btree?
if (lfsr_btree_isinlined(btree)) {
LFS_ASSERT(bid == lfsr_btree_weight(btree)-1);
*btree = LFSR_BTREE_NULL;
return 0;
// a normal btree
} else {
// lookup in which leaf our rid resides
lfsr_rbyd_t rbyd;
lfsr_tag_t rtag;
lfs_ssize_t rid;
lfs_size_t rweight;
int err = lfsr_btree_lookupnext_(lfs, btree, bid,
NULL, &rbyd, &rid, &rtag, &rweight, NULL);
if (err) {
return err;
}
// remove our rid, letting lfsr_btree_commit take care
// of the rest
// TODO yes this is completely redundant and eventually should be removed
//
// note we use a cutoff of 2 here, if we have 2 entries before
// the commit, we should have 1 entry after the commit and can
// revert to an inlined btree
int degenerate = lfsr_btree_commit(lfs, btree, bid, 2, &rbyd,
LFSR_ATTRS(
LFSR_ATTR(rid, RM, -rweight, NULL)));
if (degenerate < 0) {
return degenerate;
}
// revert to a null btree
if (degenerate && rweight >= rbyd.weight) {
*btree = LFSR_BTREE_NULL;
// revert to an inlined btree
} else if (degenerate) {
lfs_ssize_t sid;
// left sibling
if ((lfs_size_t)rid == rbyd.weight-1) {
sid = rid-rweight;
// right sibling
} else {
sid = rid+1;
}
lfsr_tag_t stag;
lfs_size_t sweight;
lfsr_data_t sdata;
int err = lfsr_rbyd_lookupnext(lfs, &rbyd, sid, LFSR_TAG_NAME,
&sid, &stag, &sweight, &sdata);
if (err) {
LFS_ASSERT(err == LFS_ERR_NOENT);
return err;
}
if (lfsr_tag_suptype(stag) == LFSR_TAG_NAME) {
err = lfsr_rbyd_lookup(lfs, &rbyd, sid, LFSR_TAG_WIDE(STRUCT),
&stag, &sdata);
if (err) {
LFS_ASSERT(err == LFS_ERR_NOENT);
return err;
}
}
LFS_ASSERT(sweight+rweight == rbyd.weight);
// mark as inlined
btree->u.i.weight = lfsr_btree_setinlined(sweight);
btree->u.i.tag = stag;
LFS_ASSERT(lfsr_data_size(&sdata) <= LFSR_BTREE_INLINESIZE);
err = lfsr_bd_read(lfs, sdata.u.d.block, sdata.u.d.off, 0,
btree->u.i.buffer, lfsr_data_size(&sdata));
// We're looking up the bid here to find it's weight so we can compute the
// correct delta. We could move this into lfsr_btree_commit__ hackily, but
// the currrent theory is we don't need this at all and upper layers can
// calculate the delta instead of the absolute weight when needed.
lfs_size_t weight_;
int err = lfsr_btree_lookupnext(lfs, btree, bid,
NULL, NULL, &weight_, NULL);
if (err) {
return err;
}
btree->u.i.size = lfsr_data_size(&sdata);
}
return 0;
}
return lfsr_btree_commit__(lfs, btree, LFSR_ATTRS(
LFSR_ATTR(bid, RM, -weight_, NULL)));
}
//static int lfsr_btree_pop(lfs_t *lfs, lfsr_btree_t *btree, lfs_size_t bid) {
// LFS_ASSERT(bid < lfsr_btree_weight(btree));
// LFS_ASSERT(lfsr_btree_weight(btree) > 0);
//
// // inlined btree?
// if (lfsr_btree_isinlined(btree)) {
// LFS_ASSERT(bid == lfsr_btree_weight(btree)-1);
// *btree = LFSR_BTREE_NULL;
// return 0;
//
// // a normal btree
// } else {
// // lookup in which leaf our rid resides
// lfsr_rbyd_t rbyd;
// lfsr_tag_t rtag;
// lfs_ssize_t rid;
// lfs_size_t rweight;
// int err = lfsr_btree_lookupnext_(lfs, btree, bid,
// NULL, &rbyd, &rid, &rtag, &rweight, NULL);
// if (err) {
// return err;
// }
//
// // remove our rid, letting lfsr_btree_commit take care
// // of the rest
// //
// // note we use a cutoff of 2 here, if we have 2 entries before
// // the commit, we should have 1 entry after the commit and can
// // revert to an inlined btree
// int degenerate = lfsr_btree_commit(lfs, btree, bid, 2, &rbyd,
// LFSR_ATTRS(
// LFSR_ATTR(rid, RM, -rweight, NULL)));
// if (degenerate < 0) {
// return degenerate;
// }
//
// // revert to a null btree
// if (degenerate && rweight >= rbyd.weight) {
// *btree = LFSR_BTREE_NULL;
//
// // revert to an inlined btree
// } else if (degenerate) {
// lfs_ssize_t sid;
// // left sibling
// if ((lfs_size_t)rid == rbyd.weight-1) {
// sid = rid-rweight;
// // right sibling
// } else {
// sid = rid+1;
// }
//
// lfsr_tag_t stag;
// lfs_size_t sweight;
// lfsr_data_t sdata;
// int err = lfsr_rbyd_lookupnext(lfs, &rbyd, sid, LFSR_TAG_NAME,
// &sid, &stag, &sweight, &sdata);
// if (err) {
// LFS_ASSERT(err == LFS_ERR_NOENT);
// return err;
// }
//
// if (lfsr_tag_suptype(stag) == LFSR_TAG_NAME) {
// err = lfsr_rbyd_lookup(lfs, &rbyd, sid, LFSR_TAG_WIDE(STRUCT),
// &stag, &sdata);
// if (err) {
// LFS_ASSERT(err == LFS_ERR_NOENT);
// return err;
// }
// }
//
// LFS_ASSERT(sweight+rweight == rbyd.weight);
// // mark as inlined
// btree->u.i.weight = lfsr_btree_setinlined(sweight);
// btree->u.i.tag = stag;
//
// LFS_ASSERT(lfsr_data_size(&sdata) <= LFSR_BTREE_INLINESIZE);
// err = lfsr_bd_read(lfs, sdata.u.d.block, sdata.u.d.off, 0,
// btree->u.i.buf, lfsr_data_size(&sdata));
// if (err) {
// return err;
// }
// btree->u.i.size = lfsr_data_size(&sdata);
// }
//
// return 0;
// }
//}
// lfsr_btree_split can be done with a update+push, but this function
// does all this in one commit, which is much more efficient
//
@@ -4657,68 +4846,102 @@ static int lfsr_btree_split(lfs_t *lfs, lfsr_btree_t *btree,
LFS_ASSERT(bid < lfsr_btree_weight(btree));
LFS_ASSERT(lfsr_btree_weight(btree) > 0);
// inlined btree, need to expand into an rbyd
if (lfsr_btree_isinlined(btree)) {
lfsr_rbyd_t rbyd;
int err = lfsr_rbyd_alloc(lfs, &rbyd);
// TODO yes this is completely redundant and eventually should be removed
//
// We're looking up the bid here to find it's weight so we can compute the
// correct delta. We could move this into lfsr_btree_commit__ hackily, but
// the currrent theory is we don't need this at all and upper layers can
// calculate the delta instead of the absolute weight when needed.
lfs_size_t weight_;
int err = lfsr_btree_lookupnext(lfs, btree, bid,
NULL, NULL, &weight_, NULL);
if (err) {
return err;
}
// commit our entries
err = lfsr_rbyd_commit(lfs, &rbyd, LFSR_ATTRS(
LFSR_ATTR(0, TAG(tag1), +weight1, DATA(data1)),
return lfsr_btree_commit__(lfs, btree, LFSR_ATTRS(
LFSR_ATTR(bid, GROW(RM), +weight1-weight_, NULL),
LFSR_ATTR(bid-(weight_-1)+weight1-1, TAG(tag1), 0, DATA(data1)),
(lfsr_data_size(&name) > 0
? LFSR_ATTR(weight1, BNAME, +weight2, DATA(name))
: LFSR_ATTR_NOOP),
(lfsr_data_size(&name) > 0
? LFSR_ATTR(weight1+weight2-1, TAG(tag2), 0, DATA(data2))
: LFSR_ATTR(weight1, TAG(tag2), +weight2, DATA(data2)))));
if (err) {
return err;
}
btree->u.r.rbyd = rbyd;
return 0;
// a normal btree
} else {
// lookup in which leaf our bid resides
lfsr_rbyd_t rbyd;
lfs_ssize_t rid;
lfs_size_t rweight;
int err = lfsr_btree_lookupnext_(lfs, btree, bid,
NULL, &rbyd, &rid, NULL, &rweight, NULL);
if (err) {
return err;
}
// commit our bid into the tree, letting lfsr_btree_commit take care
// of the rest
int degenerate = lfsr_btree_commit(lfs, btree, bid, -1, &rbyd,
LFSR_ATTRS(
LFSR_ATTR(rid, GROW(RM), +weight1-rweight, NULL),
LFSR_ATTR(rid-(rweight-1)+weight1-1, TAG(tag1), 0,
DATA(data1)),
(lfsr_data_size(&name) > 0
? LFSR_ATTR(rid-(rweight-1)+weight1,
? LFSR_ATTR(bid-(weight_-1)+weight1,
BNAME, +weight2, DATA(name))
: LFSR_ATTR_NOOP),
(lfsr_data_size(&name) > 0
? LFSR_ATTR(rid-(rweight-1)+weight1+weight2-1,
? LFSR_ATTR(bid-(weight_-1)+weight1+weight2-1,
TAG(tag2), 0, DATA(data2))
: LFSR_ATTR(rid-(rweight-1)+weight1,
: LFSR_ATTR(bid-(weight_-1)+weight1,
TAG(tag2), +weight2, DATA(data2)))));
if (degenerate < 0) {
return degenerate;
}
// this should never happen
LFS_ASSERT(!degenerate);
return 0;
}
}
//static int lfsr_btree_split(lfs_t *lfs, lfsr_btree_t *btree,
// lfs_size_t bid, lfsr_data_t name,
// lfsr_tag_t tag1, lfs_size_t weight1, lfsr_data_t data1,
// lfsr_tag_t tag2, lfs_size_t weight2, lfsr_data_t data2) {
// LFS_ASSERT(bid < lfsr_btree_weight(btree));
// LFS_ASSERT(lfsr_btree_weight(btree) > 0);
//
// // inlined btree, need to expand into an rbyd
// if (lfsr_btree_isinlined(btree)) {
// lfsr_rbyd_t rbyd;
// int err = lfsr_rbyd_alloc(lfs, &rbyd);
// if (err) {
// return err;
// }
//
// // commit our entries
// err = lfsr_rbyd_commit(lfs, &rbyd, LFSR_ATTRS(
// LFSR_ATTR(0, TAG(tag1), +weight1, DATA(data1)),
// (lfsr_data_size(&name) > 0
// ? LFSR_ATTR(weight1, BNAME, +weight2, DATA(name))
// : LFSR_ATTR_NOOP),
// (lfsr_data_size(&name) > 0
// ? LFSR_ATTR(weight1+weight2-1, TAG(tag2), 0, DATA(data2))
// : LFSR_ATTR(weight1, TAG(tag2), +weight2, DATA(data2)))));
// if (err) {
// return err;
// }
//
// btree->u.r.rbyd = rbyd;
// return 0;
//
// // a normal btree
// } else {
// // lookup in which leaf our bid resides
// lfsr_rbyd_t rbyd;
// lfs_ssize_t rid;
// lfs_size_t rweight;
// int err = lfsr_btree_lookupnext_(lfs, btree, bid,
// NULL, &rbyd, &rid, NULL, &rweight, NULL);
// if (err) {
// return err;
// }
//
// // commit our bid into the tree, letting lfsr_btree_commit take care
// // of the rest
// int degenerate = lfsr_btree_commit(lfs, btree, bid, &rbyd,
// LFSR_ATTRS(
// LFSR_ATTR(rid, GROW(RM), +weight1-rweight, NULL),
// LFSR_ATTR(rid-(rweight-1)+weight1-1, TAG(tag1), 0,
// DATA(data1)),
// (lfsr_data_size(&name) > 0
// ? LFSR_ATTR(rid-(rweight-1)+weight1,
// BNAME, +weight2, DATA(name))
// : LFSR_ATTR_NOOP),
// (lfsr_data_size(&name) > 0
// ? LFSR_ATTR(rid-(rweight-1)+weight1+weight2-1,
// TAG(tag2), 0, DATA(data2))
// : LFSR_ATTR(rid-(rweight-1)+weight1,
// TAG(tag2), +weight2, DATA(data2)))));
// if (degenerate < 0) {
// return degenerate;
// }
//
// // this should never happen
// LFS_ASSERT(!degenerate);
// return 0;
// }
//}
// lookup in a btree by name
static int lfsr_btree_namelookup(lfs_t *lfs, const lfsr_btree_t *btree,
lfs_size_t did, const char *name, lfs_size_t name_size,
@@ -4742,7 +4965,7 @@ static int lfsr_btree_namelookup(lfs_t *lfs, const lfsr_btree_t *btree,
*weight_ = lfsr_btree_weight(btree);
}
if (data_) {
*data_ = LFSR_DATA(btree->u.i.buffer, btree->u.i.size);
*data_ = LFSR_DATA(btree->u.i.buf, btree->u.i.size);
}
return 0;
}
@@ -4845,7 +5068,7 @@ static int lfsr_btree_traversal_next(lfs_t *lfs,
*weight_ = lfsr_btree_weight(btree);
}
if (data_) {
*data_ = LFSR_DATA(btree->u.i.buffer, btree->u.i.size);
*data_ = LFSR_DATA(btree->u.i.buf, btree->u.i.size);
}
return 0;
}
@@ -5884,7 +6107,7 @@ static int lfsr_mdir_commit(lfs_t *lfs, lfsr_mdir_t *mdir,
} else if (lfsr_btree_isinlined(&mtree_)) {
LFS_ASSERT(mtree_.u.i.tag == LFSR_TAG_MDIR);
mtree_tag = LFSR_TAG_WIDE(MDIR);
memcpy(mtree_buf, mtree_.u.i.buffer, mtree_.u.i.size);
memcpy(mtree_buf, mtree_.u.i.buf, mtree_.u.i.size);
mtree_dsize = mtree_.u.i.size;
} else {
mtree_tag = LFSR_TAG_WIDE(MTREE);
lfs.h
+1 -1
View File
@@ -371,7 +371,7 @@ typedef struct lfsr_btree {
lfs_size_t weight;
lfsr_tag_t tag;
uint16_t size;
uint8_t buffer[LFSR_BTREE_INLINESIZE];
uint8_t buf[LFSR_BTREE_INLINESIZE];
} i;
struct {
lfsr_rbyd_t rbyd;
lfs_util.h
+10
View File
@@ -163,6 +163,7 @@ static inline int32_t lfs_smin32(int32_t a, int32_t b) {
return (a < b) ? a : b;
}
// TODO other 16-bit ops?
static inline uint16_t lfs_max16(uint16_t a, uint16_t b) {
return (a > b) ? a : b;
}
@@ -171,6 +172,15 @@ static inline uint16_t lfs_min16(uint16_t a, uint16_t b) {
return (a < b) ? a : b;
}
// Clamp is useful as the logic for min/max when clamping can become confusing
static inline uint32_t lfs_clamp32(uint32_t a, uint32_t min, uint32_t max) {
return lfs_min32(lfs_max32(a, min), max);
}
static inline int32_t lfs_sclamp32(int32_t a, int32_t min, int32_t max) {
return lfs_smin32(lfs_smax32(a, min), max);
}
// Absolute value of signed numbers
static inline int32_t lfs_abs32(int32_t a) {
return a < 0 ? -a : a;
tests/test_btree.toml
+257 -257
View File
@@ -2424,263 +2424,263 @@ code = '''
free(sim);
'''
# test we reinline (go from uninlined to inlined) correctly, this is a bit
# tricky since our btrees lazily reinline
[cases.test_btree_reinline_pop_set]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.start = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count);
lfs.lookahead.next = 0;
lfs_alloc_ack(&lfs);
// create an uninlined tree
lfsr_btree_t btree = LFSR_BTREE_NULL;
lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("a", 1)) => 0;
lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1,
LFSR_DATA("b", 1)) => 0;
assert(lfsr_btree_weight(&btree) == 2);
assert(!lfsr_btree_isinlined(&btree));
// pop! our btree should now be reinlinable
lfsr_btree_pop(&lfs, &btree, 0) => 0;
// but thanks to lazy reinlining, our btree won't reinline until
// it is compacted, so we need to add commits until it is compacted
lfs_block_t before_block = btree.u.r.rbyd.block;
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i < BLOCK_SIZE);
// commit to btree
lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("b", 1)) => 0;
assert(lfsr_btree_weight(&btree) == 1);
// try looking up tag to hopefully catch if something breaks
uint8_t buffer[4];
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_btree_get(&lfs, &btree, 0,
&tag_, &weight_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_INLINED);
assert(weight_ == 1);
assert(memcmp(buffer, "b", 1) == 0);
// inlined? consider this a success
if (lfsr_btree_isinlined(&btree)) {
break;
}
// assert if a compaction occurred that wasn't inlined
assert(btree.u.r.rbyd.block == before_block);
}
printf("btree: w%d 0x%x.%x\n",
btree.u.r.rbyd.weight,
btree.u.r.rbyd.block,
btree.u.r.rbyd.trunk);
'''
[cases.test_btree_reinline_pop_pop_push]
in = 'lfs.c'
defines.SHIFT = 'range(5)'
code = '''
lfs_t lfs;
lfs_init(&lfs, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.start = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count);
lfs.lookahead.next = 0;
lfs_alloc_ack(&lfs);
// create an uninlined tree
lfsr_btree_t btree = LFSR_BTREE_NULL;
lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("a", 1)) => 0;
lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1,
LFSR_DATA("b", 1)) => 0;
assert(lfsr_btree_weight(&btree) == 2);
assert(!lfsr_btree_isinlined(&btree));
// pop! our btree should now be reinlinable
lfsr_btree_pop(&lfs, &btree, 0) => 0;
// It's difficult to test reinlining during push or pop, since we can't just
// repeat the action until compaction occurs.
//
// What we do here is alternate between 0 and 1 entries, eventually we
// will compact during one of either a push or pop. To try to cover both,
// test with some number of extra commits to hopefully adjust where the
// compaction ends up.
for (lfs_size_t i = 0; i < SHIFT; i++) {
lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("b", 1)) => 0;
}
// alternate between push/pop until compaction occurs
lfs_block_t before_block = btree.u.r.rbyd.block;
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i < BLOCK_SIZE);
// pop!
lfsr_btree_pop(&lfs, &btree, 0) => 0;
assert(lfsr_btree_weight(&btree) == 0);
// inlined? consider this a success
if (lfsr_btree_isinlined(&btree)) {
break;
}
// assert if a compaction occurred that wasn't inlined
assert(btree.u.r.rbyd.block == before_block);
// push!
lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("c", 1)) => 0;
// try looking up tag to hopefully catch if something breaks
uint8_t buffer[4];
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_btree_get(&lfs, &btree, 0,
&tag_, &weight_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_INLINED);
assert(weight_ == 1);
assert(memcmp(buffer, "c", 1) == 0);
// inlined? consider this a success
if (lfsr_btree_isinlined(&btree)) {
break;
}
// assert if a compaction occurred that wasn't inlined
assert(btree.u.r.rbyd.block == before_block);
}
printf("btree: w%d 0x%x.%x\n",
btree.u.r.rbyd.weight,
btree.u.r.rbyd.block,
btree.u.r.rbyd.trunk);
'''
[cases.test_btree_reinline_pop_push]
in = 'lfs.c'
defines.SIBLING = [0, 1]
defines.SHIFT = 'range(5)'
code = '''
lfs_t lfs;
lfs_init(&lfs, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.start = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count);
lfs.lookahead.next = 0;
lfs_alloc_ack(&lfs);
// create an uninlined tree
lfsr_btree_t btree = LFSR_BTREE_NULL;
lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("a", 1)) => 0;
lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1,
LFSR_DATA("b", 1)) => 0;
assert(lfsr_btree_weight(&btree) == 2);
assert(!lfsr_btree_isinlined(&btree));
// It's difficult to test reinlining during push or pop, since we can't just
// repeat the action until compaction occurs.
//
// Here we alternate between 1 and 2 entries, with the hope that compaction
// occurs on the pop. We try this with some number of extra commits to make
// it more likely pop is tested.
//
// It's possible our commits line up so compaction always occurs on a push!
// For this reason, we end the test if an non-reinlining compaction occurs.
for (lfs_size_t i = 0; i < SHIFT; i++) {
lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("a", 1)) => 0;
}
// alternate between push/pop until compaction occurs
lfs_block_t before_block = btree.u.r.rbyd.block;
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i < BLOCK_SIZE);
// pop!
lfsr_btree_pop(&lfs, &btree, SIBLING) => 0;
assert(lfsr_btree_weight(&btree) == 1);
// try looking up tag to hopefully catch if something breaks
uint8_t buffer[4];
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_btree_get(&lfs, &btree, 0,
&tag_, &weight_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_INLINED);
assert(weight_ == 1);
assert(memcmp(buffer, (SIBLING == 1 ? "a" : "b"), 1) == 0);
// inlined? consider this a success
if (lfsr_btree_isinlined(&btree)) {
break;
}
// abort if a compaction occurs
if (btree.u.r.rbyd.block != before_block) {
break;
}
// push!
lfsr_btree_push(&lfs, &btree, SIBLING, LFSR_TAG_INLINED, 1,
LFSR_DATA("c", 1)) => 0;
// try looking up tag to hopefully catch if something breaks
lfsr_btree_get(&lfs, &btree, 0,
&tag_, &weight_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_INLINED);
assert(weight_ == 1);
assert(memcmp(buffer, (SIBLING == 1 ? "a" : "c"), 1) == 0);
lfsr_btree_get(&lfs, &btree, 1,
&tag_, &weight_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_INLINED);
assert(weight_ == 1);
assert(memcmp(buffer, (SIBLING == 1 ? "c" : "b"), 1) == 0);
// inlined? consider this a success
if (lfsr_btree_isinlined(&btree)) {
break;
}
// abort if a compaction occurs
if (btree.u.r.rbyd.block != before_block) {
break;
}
}
printf("btree: w%d 0x%x.%x\n",
btree.u.r.rbyd.weight,
btree.u.r.rbyd.block,
btree.u.r.rbyd.trunk);
'''
# TODO
## test we reinline (go from uninlined to inlined) correctly, this is a bit
## tricky since our btrees lazily reinline
#[cases.test_btree_reinline_pop_set]
#in = 'lfs.c'
#code = '''
# lfs_t lfs;
# lfs_init(&lfs, CFG) => 0;
# // create free lookahead
# memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
# lfs.lookahead.start = 0;
# lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
# CFG->block_count);
# lfs.lookahead.next = 0;
# lfs_alloc_ack(&lfs);
#
# // create an uninlined tree
# lfsr_btree_t btree = LFSR_BTREE_NULL;
# lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
# LFSR_DATA("a", 1)) => 0;
# lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1,
# LFSR_DATA("b", 1)) => 0;
# assert(lfsr_btree_weight(&btree) == 2);
# assert(!lfsr_btree_isinlined(&btree));
#
# // pop! our btree should now be reinlinable
# lfsr_btree_pop(&lfs, &btree, 0) => 0;
#
# // but thanks to lazy reinlining, our btree won't reinline until
# // it is compacted, so we need to add commits until it is compacted
# lfs_block_t before_block = btree.u.r.rbyd.block;
# for (lfs_block_t i = 0;; i++) {
# // a bit hacky, but this catches infinite loops
# assert(i < BLOCK_SIZE);
#
# // commit to btree
# lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
# LFSR_DATA("b", 1)) => 0;
#
# assert(lfsr_btree_weight(&btree) == 1);
#
# // try looking up tag to hopefully catch if something breaks
# uint8_t buffer[4];
# lfsr_tag_t tag_;
# lfs_size_t weight_;
#
# lfsr_btree_get(&lfs, &btree, 0,
# &tag_, &weight_, buffer, 4) => 1;
# assert(tag_ == LFSR_TAG_INLINED);
# assert(weight_ == 1);
# assert(memcmp(buffer, "b", 1) == 0);
#
# // inlined? consider this a success
# if (lfsr_btree_isinlined(&btree)) {
# break;
# }
#
# // assert if a compaction occurred that wasn't inlined
# assert(btree.u.r.rbyd.block == before_block);
# }
#
# printf("btree: w%d 0x%x.%x\n",
# btree.u.r.rbyd.weight,
# btree.u.r.rbyd.block,
# btree.u.r.rbyd.trunk);
#'''
#
#[cases.test_btree_reinline_pop_pop_push]
#in = 'lfs.c'
#defines.SHIFT = 'range(5)'
#code = '''
# lfs_t lfs;
# lfs_init(&lfs, CFG) => 0;
# // create free lookahead
# memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
# lfs.lookahead.start = 0;
# lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
# CFG->block_count);
# lfs.lookahead.next = 0;
# lfs_alloc_ack(&lfs);
#
# // create an uninlined tree
# lfsr_btree_t btree = LFSR_BTREE_NULL;
# lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
# LFSR_DATA("a", 1)) => 0;
# lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1,
# LFSR_DATA("b", 1)) => 0;
# assert(lfsr_btree_weight(&btree) == 2);
# assert(!lfsr_btree_isinlined(&btree));
#
# // pop! our btree should now be reinlinable
# lfsr_btree_pop(&lfs, &btree, 0) => 0;
#
# // It's difficult to test reinlining during push or pop, since we can't just
# // repeat the action until compaction occurs.
# //
# // What we do here is alternate between 0 and 1 entries, eventually we
# // will compact during one of either a push or pop. To try to cover both,
# // test with some number of extra commits to hopefully adjust where the
# // compaction ends up.
# for (lfs_size_t i = 0; i < SHIFT; i++) {
# lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
# LFSR_DATA("b", 1)) => 0;
# }
#
# // alternate between push/pop until compaction occurs
# lfs_block_t before_block = btree.u.r.rbyd.block;
# for (lfs_block_t i = 0;; i++) {
# // a bit hacky, but this catches infinite loops
# assert(i < BLOCK_SIZE);
#
# // pop!
# lfsr_btree_pop(&lfs, &btree, 0) => 0;
#
# assert(lfsr_btree_weight(&btree) == 0);
#
# // inlined? consider this a success
# if (lfsr_btree_isinlined(&btree)) {
# break;
# }
#
# // assert if a compaction occurred that wasn't inlined
# assert(btree.u.r.rbyd.block == before_block);
#
# // push!
# lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
# LFSR_DATA("c", 1)) => 0;
#
# // try looking up tag to hopefully catch if something breaks
# uint8_t buffer[4];
# lfsr_tag_t tag_;
# lfs_size_t weight_;
#
# lfsr_btree_get(&lfs, &btree, 0,
# &tag_, &weight_, buffer, 4) => 1;
# assert(tag_ == LFSR_TAG_INLINED);
# assert(weight_ == 1);
# assert(memcmp(buffer, "c", 1) == 0);
#
# // inlined? consider this a success
# if (lfsr_btree_isinlined(&btree)) {
# break;
# }
#
# // assert if a compaction occurred that wasn't inlined
# assert(btree.u.r.rbyd.block == before_block);
# }
#
# printf("btree: w%d 0x%x.%x\n",
# btree.u.r.rbyd.weight,
# btree.u.r.rbyd.block,
# btree.u.r.rbyd.trunk);
#'''
#
#[cases.test_btree_reinline_pop_push]
#in = 'lfs.c'
#defines.SIBLING = [0, 1]
#defines.SHIFT = 'range(5)'
#code = '''
# lfs_t lfs;
# lfs_init(&lfs, CFG) => 0;
# // create free lookahead
# memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
# lfs.lookahead.start = 0;
# lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
# CFG->block_count);
# lfs.lookahead.next = 0;
# lfs_alloc_ack(&lfs);
#
# // create an uninlined tree
# lfsr_btree_t btree = LFSR_BTREE_NULL;
# lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
# LFSR_DATA("a", 1)) => 0;
# lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1,
# LFSR_DATA("b", 1)) => 0;
# assert(lfsr_btree_weight(&btree) == 2);
# assert(!lfsr_btree_isinlined(&btree));
#
# // It's difficult to test reinlining during push or pop, since we can't just
# // repeat the action until compaction occurs.
# //
# // Here we alternate between 1 and 2 entries, with the hope that compaction
# // occurs on the pop. We try this with some number of extra commits to make
# // it more likely pop is tested.
# //
# // It's possible our commits line up so compaction always occurs on a push!
# // For this reason, we end the test if an non-reinlining compaction occurs.
# for (lfs_size_t i = 0; i < SHIFT; i++) {
# lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
# LFSR_DATA("a", 1)) => 0;
# }
#
# // alternate between push/pop until compaction occurs
# lfs_block_t before_block = btree.u.r.rbyd.block;
# for (lfs_block_t i = 0;; i++) {
# // a bit hacky, but this catches infinite loops
# assert(i < BLOCK_SIZE);
#
# // pop!
# lfsr_btree_pop(&lfs, &btree, SIBLING) => 0;
#
# assert(lfsr_btree_weight(&btree) == 1);
#
# // try looking up tag to hopefully catch if something breaks
# uint8_t buffer[4];
# lfsr_tag_t tag_;
# lfs_size_t weight_;
#
# lfsr_btree_get(&lfs, &btree, 0,
# &tag_, &weight_, buffer, 4) => 1;
# assert(tag_ == LFSR_TAG_INLINED);
# assert(weight_ == 1);
# assert(memcmp(buffer, (SIBLING == 1 ? "a" : "b"), 1) == 0);
#
# // inlined? consider this a success
# if (lfsr_btree_isinlined(&btree)) {
# break;
# }
#
# // abort if a compaction occurs
# if (btree.u.r.rbyd.block != before_block) {
# break;
# }
#
# // push!
# lfsr_btree_push(&lfs, &btree, SIBLING, LFSR_TAG_INLINED, 1,
# LFSR_DATA("c", 1)) => 0;
#
# // try looking up tag to hopefully catch if something breaks
# lfsr_btree_get(&lfs, &btree, 0,
# &tag_, &weight_, buffer, 4) => 1;
# assert(tag_ == LFSR_TAG_INLINED);
# assert(weight_ == 1);
# assert(memcmp(buffer, (SIBLING == 1 ? "a" : "c"), 1) == 0);
#
# lfsr_btree_get(&lfs, &btree, 1,
# &tag_, &weight_, buffer, 4) => 1;
# assert(tag_ == LFSR_TAG_INLINED);
# assert(weight_ == 1);
# assert(memcmp(buffer, (SIBLING == 1 ? "c" : "b"), 1) == 0);
#
# // inlined? consider this a success
# if (lfsr_btree_isinlined(&btree)) {
# break;
# }
#
# // abort if a compaction occurs
# if (btree.u.r.rbyd.block != before_block) {
# break;
# }
# }
#
# printf("btree: w%d 0x%x.%x\n",
# btree.u.r.rbyd.weight,
# btree.u.r.rbyd.block,
# btree.u.r.rbyd.trunk);
#'''
# Some more general fuzz testing