Github/littlefs
1
Fork 0
Code Actions Activity

Rerouted all btree mutation through attr-list parser

Browse Source 
The idea here: Instead of having unique functionality for each
individual btree operation (push/set/pop/split), we treat btrees sort of
like rbyds, with a single commit entry point that operates on attr-lists.

This adds code cost, due to needing to parse the attr-list for properties
that can affect inlined btrees (tag changes mostly), but, in theory, comes
with some advantages:

1. A single btree commit entry point with all of the inlined/uninlining
   logic should offer better chances for code deduplication, vs
   spreading this logic out in each btree operation.

2. Higher-levels should know what the current weight of the branch is,
   so we may be able to avoid the implicit math needed to calculate
   deltas.

3. Higher-levels have more knowledge about the state of the btree in
   general, so there may be other shortcuts. The mtree, for example,
   only operates on weight=1 entries, which greatly simplifies a lot of
   the related math.

Note that btrees still have strict limits in what's possible in an
attr-list. Btree operations can't cross leaf-rbyd boundaries for
example.

---

A notable omission in this change is the loss of reinlining btrees.

This wase dropped for a couple reasons. It may be worth adding back at a
later time, maybe after we actually have files implemented, but for now
does not seem worth it:

1. Reinlining adds code cost. Reinlining is more complex than you might
   expect because we only reinline on compaction. And because we compact
   before playing out our attr-list, we need to know if a commit makes
   the btree inlinable before committing to the btree.

   This is still doable with our attr-lists. We already derive the
   change in tags, since we need this to know when to uninline. But it
   adds a kind of complex bailing out of btree commits.

2. The benefits of reinlining may not be that great. In most systems, a
   tree that is uninlined once is likely to be uninlined again. It's
   only if there is a bigger state change in a system that it makes
   sense to reinline.

   Though, to be fair, waiting for compaction to reinline handled this
   quite well. Only reinlining when all erased storage is used up...

3. Thanks to our roots did entry, our mtree can never reinline.

   It would be nice to change this, but this would require explicit
   handling in lfsr_mdir_commit. Future work?

4. Files are another can of worms, with more complex interactions with
   inlinability thanks to (at least on paper right now) always having
   inlined data even when uninlined.

   If reinlining is valuable for files this can change during that work.

5. Even if files never support reinlinability, truncating files (via
   either lfsr_file_truncate or LFSR_O_TRUNC) should give the file a
   blank slate, effectively reinlining the file in that case.

---

The current implementation also changes the attr-list to be mutable so
we can adjust attr-list based on the current btree node. This is a
temporary hack! We should add the appropriate functionality to our rbyd
utilities to revert this eventually.
This commit is contained in:
Christopher Haster
2023-08-13 12:40:34 -05:00
parent 3dbc986752
commit 9b2f3cd5bb
4 changed files with 840 additions and 607 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lfs.c
+520 -297
View File
@@ -1317,9 +1317,10 @@ typedef struct lfsr_attr {
#define LFSR_ATTR_NOOP LFSR_ATTR(-1, RM, 0, NULL) #define LFSR_ATTR_NOOP LFSR_ATTR(-1, RM, 0, NULL)
// TODO make this const again eventually
#define LFSR_ATTRS(...) \ #define LFSR_ATTRS(...) \
(const lfsr_attr_t[]){__VA_ARGS__}, \ (lfsr_attr_t[]){__VA_ARGS__}, \
sizeof((const lfsr_attr_t[]){__VA_ARGS__}) / sizeof(lfsr_attr_t) sizeof((lfsr_attr_t[]){__VA_ARGS__}) / sizeof(lfsr_attr_t)
//struct lfsr_attr_from { //struct lfsr_attr_from {
// const lfsr_rbyd_t *rbyd; // const lfsr_rbyd_t *rbyd;
@@ -3362,38 +3363,39 @@ static lfs_ssize_t lfsr_rbyd_estimateall(lfs_t *lfs, const lfsr_rbyd_t *rbyd,
#endif #endif
} }
// determine if there are fewer than "cutoff" unique ids in the rbyd, // TODO
// this is used to determine if the underlying rbyd is degenerate and can //// determine if there are fewer than "cutoff" unique ids in the rbyd,
// be reverted to an inlined btree //// this is used to determine if the underlying rbyd is degenerate and can
//// be reverted to an inlined btree
////
//// note cutoff is expected to be quite small, <= 2, so we should make sure
//// to exit our traverse early
//static int lfsr_rbyd_isdegenerate(lfs_t *lfs, const lfsr_rbyd_t *rbyd,
// lfs_ssize_t cutoff) {
// // cutoff=-1 => no cutoff
// if (cutoff < 0) {
// return false;
// }
// //
// note cutoff is expected to be quite small, <= 2, so we should make sure // // count ids until we exceed our cutoff
// to exit our traverse early // lfs_ssize_t rid = -1;
static int lfsr_rbyd_isdegenerate(lfs_t *lfs, const lfsr_rbyd_t *rbyd, // lfs_size_t count = 0;
lfs_ssize_t cutoff) { // while (true) {
// cutoff=-1 => no cutoff // int err = lfsr_rbyd_lookupnext(lfs, rbyd, rid+1, 0,
if (cutoff < 0) { // &rid, NULL, NULL, NULL);
return false; // if (err && err != LFS_ERR_NOENT) {
} // return err;
// }
// count ids until we exceed our cutoff // if (err == LFS_ERR_NOENT) {
lfs_ssize_t rid = -1; // return true;
lfs_size_t count = 0; // }
while (true) { //
int err = lfsr_rbyd_lookupnext(lfs, rbyd, rid+1, 0, // count += 1;
&rid, NULL, NULL, NULL); // if (count > (lfs_size_t)cutoff) {
if (err && err != LFS_ERR_NOENT) { // return false;
return err; // }
} // }
if (err == LFS_ERR_NOENT) { //}
return true;
}
count += 1;
if (count > (lfs_size_t)cutoff) {
return false;
}
}
}
// some low-level name things // some low-level name things
@@ -3607,7 +3609,7 @@ static int lfsr_data_readbtreeinlined(lfs_t *lfs, lfsr_data_t *data,
btree->u.i.weight = lfsr_btree_setinlined(weight); btree->u.i.weight = lfsr_btree_setinlined(weight);
btree->u.i.tag = tag; btree->u.i.tag = tag;
lfs_ssize_t size = lfsr_data_read(lfs, data, lfs_ssize_t size = lfsr_data_read(lfs, data,
btree->u.i.buffer, LFSR_BTREE_INLINESIZE); btree->u.i.buf, LFSR_BTREE_INLINESIZE);
if (size < 0) { if (size < 0) {
return size; return size;
} }
@@ -3660,7 +3662,7 @@ static int lfsr_btree_lookupnext_(lfs_t *lfs,
*weight_ = lfsr_btree_weight(btree); *weight_ = lfsr_btree_weight(btree);
} }
if (data_) { if (data_) {
*data_ = LFSR_DATA(btree->u.i.buffer, btree->u.i.size); *data_ = LFSR_DATA(btree->u.i.buf, btree->u.i.size);
} }
return 0; return 0;
} }
@@ -3847,7 +3849,7 @@ static int lfsr_btree_parent(lfs_t *lfs,
// core btree algorithm // core btree algorithm
static int lfsr_btree_commit(lfs_t *lfs, static int lfsr_btree_commit(lfs_t *lfs,
lfsr_btree_t *btree, lfs_size_t bid, lfs_ssize_t cutoff, lfsr_btree_t *btree, lfs_size_t bid, // lfs_ssize_t cutoff,
lfsr_rbyd_t *rbyd, lfsr_rbyd_t *rbyd,
const lfsr_attr_t *attrs, lfs_size_t attr_count) { const lfsr_attr_t *attrs, lfs_size_t attr_count) {
// other layers should check for inlined btrees before this // other layers should check for inlined btrees before this
@@ -3926,7 +3928,7 @@ static int lfsr_btree_commit(lfs_t *lfs,
} }
*rbyd = parent; *rbyd = parent;
cutoff = -1; // cutoff = -1;
continue; continue;
compact:; compact:;
@@ -3934,23 +3936,24 @@ static int lfsr_btree_commit(lfs_t *lfs,
lfsr_rbyd_t rbyd_; lfsr_rbyd_t rbyd_;
lfs_size_t split_rid; lfs_size_t split_rid;
// first check if we are a degenerate root and can be reverted to // TODO
// an inlined btree // // first check if we are a degenerate root and can be reverted to
// // // an inlined btree
// This gets a bit weird since we're defering our pending // //
// attributes to after the compaction. When we can/can't be inlined // // This gets a bit weird since we're defering our pending
// depends on those attributes, but trying to evaluate attributes // // attributes to after the compaction. When we can/can't be inlined
// is complicated and expensive. // // depends on those attributes, but trying to evaluate attributes
// // // is complicated and expensive.
// Instead we just let the upper layers indicate a cutoff for when // //
// an rbyd can be inlined, and leave the inlining work up to the // // Instead we just let the upper layers indicate a cutoff for when
// upper layers. // // an rbyd can be inlined, and leave the inlining work up to the
if (pid == -1) { // // upper layers.
int degenerate = lfsr_rbyd_isdegenerate(lfs, rbyd, cutoff); // if (pid == -1) {
if (degenerate) { // int degenerate = lfsr_rbyd_isdegenerate(lfs, rbyd, cutoff);
return degenerate; // if (degenerate) {
} // return degenerate;
} // }
// }
// check if we're within our compaction threshold, otherwise we // check if we're within our compaction threshold, otherwise we
// need to split // need to split
@@ -4037,7 +4040,7 @@ static int lfsr_btree_commit(lfs_t *lfs,
} }
*rbyd = parent; *rbyd = parent;
cutoff = -1; // cutoff = -1;
continue; continue;
split:; split:;
@@ -4181,7 +4184,7 @@ static int lfsr_btree_commit(lfs_t *lfs,
} }
*rbyd = parent; *rbyd = parent;
cutoff = -1; // cutoff = -1;
continue; continue;
merge:; merge:;
@@ -4386,7 +4389,7 @@ static int lfsr_btree_commit(lfs_t *lfs,
} }
*rbyd = parent; *rbyd = parent;
cutoff = -1; // cutoff = -1;
continue; continue;
} }
@@ -4395,255 +4398,441 @@ static int lfsr_btree_commit(lfs_t *lfs,
return false; return false;
} }
static int lfsr_btree_push(lfs_t *lfs, lfsr_btree_t *btree, // TODO avoid mutable attrs? merge this things into rbyd_appendall?
lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) { static void lfsr_btree_adjust(lfs_ssize_t bid,
lfsr_attr_t *attrs, lfs_size_t attr_count) {
for (lfs_size_t i = 0; i < attr_count; i++) {
if (attrs[i].rid != -1) {
attrs[i].rid -= bid;
}
}
}
static int lfsr_btree_commit__(lfs_t *lfs, lfsr_btree_t *btree,
lfsr_attr_t *attrs, lfs_size_t attr_count) {
// first find the effective bid and any changes to the number of tags
lfs_size_t bid = -1;
lfs_ssize_t tag_delta = 0;
for (lfs_size_t i = 0; i < attr_count; i++) {
// note unsigned min here chooses non-negative bids
bid = lfs_min32(bid, attrs[i].rid);
// non-grow tags with deltas change the number of tags
if (!lfsr_tag_isgrow(attrs[i].tag)) {
tag_delta += lfs_sclamp32(attrs[i].delta, -1, +1);
}
}
LFS_ASSERT(bid <= lfsr_btree_weight(btree)); LFS_ASSERT(bid <= lfsr_btree_weight(btree));
// null btree? // inlined btree? staying inlined?
if (lfsr_btree_isinlined(btree) && lfsr_btree_weight(btree) == 0) { if (lfsr_btree_isinlined(btree)) {
LFS_ASSERT(bid == 0); // staying inlined?
if (lfs_clamp32(lfsr_btree_weight(btree), 0, 1) + tag_delta <= 1) {
for (lfs_size_t i = 0; i < attr_count; i++) {
// update our inlined tag, make sure to strip wide/grow bits
if (lfsr_tag_suptype(lfsr_tag_key(attrs[i].tag))
== LFSR_TAG_STRUCT) {
btree->u.i.tag = lfsr_tag_key(attrs[i].tag);
// mark as inlined lfsr_data_t data_ = attrs[i].data;
btree->u.i.weight = lfsr_btree_setinlined(weight); lfs_ssize_t d = lfsr_data_read(lfs, &data_,
btree->u.i.tag = tag; btree->u.i.buf, LFSR_BTREE_INLINESIZE);
lfs_ssize_t d = lfsr_data_read(lfs, &data,
btree->u.i.buffer, LFSR_BTREE_INLINESIZE);
if (d < 0) { if (d < 0) {
return d; return d;
} }
LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
btree->u.i.size = d; btree->u.i.size = d;
}
// update our btree weight
LFS_ASSERT((lfs_ssize_t)lfsr_btree_weight(btree)
+ attrs[i].delta >= 0);
btree->u.i.weight += attrs[i].delta;
}
return 0; return 0;
// inlined btree, need to expand into an rbyd // uninlining?
} else if (lfsr_btree_isinlined(btree)) { } else {
// allocate a root rbyd
lfsr_rbyd_t rbyd; lfsr_rbyd_t rbyd;
int err = lfsr_rbyd_alloc(lfs, &rbyd); int err = lfsr_rbyd_alloc(lfs, &rbyd);
if (err) { if (err) {
return err; return err;
} }
// commit our entries // prepend inlined tag if we have one
err = lfsr_rbyd_commit(lfs, &rbyd, LFSR_ATTRS( if (lfsr_btree_weight(btree) > 0) {
LFSR_ATTR(0, TAG(btree->u.i.tag), +lfsr_btree_weight(btree), err = lfsr_rbyd_append(lfs, &rbyd,
BUF(btree->u.i.buffer, btree->u.i.size)), 0, btree->u.i.tag, +lfsr_btree_weight(btree),
LFSR_ATTR(bid, TAG(tag), +weight, DATA(data)))); LFSR_DATA_BUF(btree->u.i.buf, btree->u.i.size));
if (err) {
return err;
}
}
// commit our attrs
err = lfsr_rbyd_appendall(lfs, &rbyd, -1, -1,
attrs, attr_count);
if (err) { if (err) {
return err; return err;
} }
err = lfsr_rbyd_appendcksum(lfs, &rbyd);
if (err) {
return err;
}
// save our new root
btree->u.r.rbyd = rbyd; btree->u.r.rbyd = rbyd;
return 0; return 0;
}
}
// a normal btree // lookup in which leaf our bids reside
} else {
// lookup in which leaf our rid resides
// //
// for lfsr_btree_commit operations to work out, we need to // for lfsr_btree_commit operations to work out, we need to
// limit our bid to an rid in the tree, which is what this min // limit our bid to an rid in the tree, which is what this min
// is doing // is doing
// //
// note it is possible for our btree to have a weight of zero here, // note it's entirely possible for our btree to have a weight of
// since we defer inlining until compaction time // zero here
lfs_size_t bid_ = lfs_min32(bid,
lfs_smax32(lfsr_btree_weight(btree)-1, 0));
lfsr_rbyd_t rbyd = btree->u.r.rbyd; lfsr_rbyd_t rbyd = btree->u.r.rbyd;
lfs_ssize_t rid = -1; if (lfsr_btree_weight(btree) > 0) {
lfs_size_t rweight = 0; lfs_ssize_t rid;
int err = lfsr_btree_lookupnext_(lfs, btree, bid_, int err = lfsr_btree_lookupnext_(lfs, btree,
NULL, &rbyd, &rid, NULL, &rweight, NULL); lfs_min32(bid, lfsr_btree_weight(btree)-1),
if (err && err != LFS_ERR_NOENT) { &bid, &rbyd, &rid, NULL, NULL, NULL);
if (err) {
LFS_ASSERT(err != LFS_ERR_NOENT);
return err; return err;
} }
// adjust rid for push // adjust bid to indicate the zero-most rid
if (bid >= lfsr_btree_weight(btree)) { bid -= rid;
rid += 1;
} else {
rid -= rweight-1;
} }
// TODO do this in rbyd_appendall?
// adjust our attrs
lfsr_btree_adjust(bid, attrs, attr_count);
// commit our rid into the tree, letting lfsr_btree_commit take care // commit our rid into the tree, letting lfsr_btree_commit take care
// of the rest // of the rest
int degenerate = lfsr_btree_commit(lfs, btree, bid_, 0, &rbyd, int err = lfsr_btree_commit(lfs, btree, bid, &rbyd,
LFSR_ATTRS( attrs, attr_count);
LFSR_ATTR(rid, TAG(tag), +weight, DATA(data)))); if (err) {
if (degenerate < 0) { return err;
return degenerate;
}
// revert to an inlined btree
if (degenerate) {
// mark as inlined
btree->u.i.weight = lfsr_btree_setinlined(weight);
btree->u.i.tag = tag;
lfs_ssize_t d = lfsr_data_read(lfs, &data,
btree->u.i.buffer, LFSR_BTREE_INLINESIZE);
if (d < 0) {
return d;
}
LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
btree->u.i.size = d;
} }
return 0; return 0;
}
} }
// TODO
static int lfsr_btree_push(lfs_t *lfs, lfsr_btree_t *btree,
lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) {
LFS_ASSERT(bid <= lfsr_btree_weight(btree));
return lfsr_btree_commit__(lfs, btree, LFSR_ATTRS(
LFSR_ATTR(bid, TAG(tag), +weight, DATA(data))));
}
// TODO
//static int lfsr_btree_push_(lfs_t *lfs, lfsr_btree_t *btree,
// lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) {
// LFS_ASSERT(bid <= lfsr_btree_weight(btree));
//
// // null btree?
// if (lfsr_btree_isinlined(btree) && lfsr_btree_weight(btree) == 0) {
// LFS_ASSERT(bid == 0);
//
// // mark as inlined
// btree->u.i.weight = lfsr_btree_setinlined(weight);
// btree->u.i.tag = tag;
//
// lfs_ssize_t d = lfsr_data_read(lfs, &data,
// btree->u.i.buf, LFSR_BTREE_INLINESIZE);
// if (d < 0) {
// return d;
// }
// LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
// btree->u.i.size = d;
// return 0;
//
// // inlined btree, need to expand into an rbyd
// } else if (lfsr_btree_isinlined(btree)) {
// lfsr_rbyd_t rbyd;
// int err = lfsr_rbyd_alloc(lfs, &rbyd);
// if (err) {
// return err;
// }
//
// // commit our entries
// err = lfsr_rbyd_commit(lfs, &rbyd, LFSR_ATTRS(
// LFSR_ATTR(0, TAG(btree->u.i.tag), +lfsr_btree_weight(btree),
// BUF(btree->u.i.buf, btree->u.i.size)),
// LFSR_ATTR(bid, TAG(tag), +weight, DATA(data))));
// if (err) {
// return err;
// }
//
// btree->u.r.rbyd = rbyd;
// return 0;
//
// // a normal btree
// } else {
// // lookup in which leaf our rid resides
// //
// // for lfsr_btree_commit operations to work out, we need to
// // limit our bid to an rid in the tree, which is what this min
// // is doing
// //
// // note it is possible for our btree to have a weight of zero here,
// // since we defer inlining until compaction time
// lfs_size_t bid_ = lfs_min32(bid,
// lfs_smax32(lfsr_btree_weight(btree)-1, 0));
// lfsr_rbyd_t rbyd = btree->u.r.rbyd;
// lfs_ssize_t rid = -1;
// lfs_size_t rweight = 0;
// int err = lfsr_btree_lookupnext_(lfs, btree, bid_,
// NULL, &rbyd, &rid, NULL, &rweight, NULL);
// if (err && err != LFS_ERR_NOENT) {
// return err;
// }
//
// // adjust rid for push
// if (bid >= lfsr_btree_weight(btree)) {
// rid += 1;
// } else {
// rid -= rweight-1;
// }
//
// // commit our rid into the tree, letting lfsr_btree_commit take care
// // of the rest
// int degenerate = lfsr_btree_commit(lfs, btree, bid_, 0, &rbyd,
// LFSR_ATTRS(
// LFSR_ATTR(rid, TAG(tag), +weight, DATA(data))));
// if (degenerate < 0) {
// return degenerate;
// }
//
// // revert to an inlined btree
// if (degenerate) {
// // mark as inlined
// btree->u.i.weight = lfsr_btree_setinlined(weight);
// btree->u.i.tag = tag;
//
// lfs_ssize_t d = lfsr_data_read(lfs, &data,
// btree->u.i.buf, LFSR_BTREE_INLINESIZE);
// if (d < 0) {
// return d;
// }
// LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
// btree->u.i.size = d;
// }
//
// return 0;
// }
//}
// TODO
static int lfsr_btree_set(lfs_t *lfs, lfsr_btree_t *btree, static int lfsr_btree_set(lfs_t *lfs, lfsr_btree_t *btree,
lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) { lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) {
LFS_ASSERT(bid < lfsr_btree_weight(btree)); LFS_ASSERT(bid < lfsr_btree_weight(btree));
LFS_ASSERT(lfsr_btree_weight(btree) > 0); LFS_ASSERT(lfsr_btree_weight(btree) > 0);
// inlined btree? // TODO yes this is completely redundant and eventually should be removed
if (lfsr_btree_isinlined(btree)) { //
LFS_ASSERT(bid == lfsr_btree_weight(btree)-1); // We're looking up the bid here to find it's weight so we can compute the
// correct delta. We could move this into lfsr_btree_commit__ hackily, but
// mark as inlined // the currrent theory is we don't need this at all and upper layers can
btree->u.i.weight = lfsr_btree_setinlined(weight); // calculate the delta instead of the absolute weight when needed.
btree->u.i.tag = tag; lfs_size_t weight_;
int err = lfsr_btree_lookupnext(lfs, btree, bid,
lfs_ssize_t d = lfsr_data_read(lfs, &data, NULL, NULL, &weight_, NULL);
btree->u.i.buffer, LFSR_BTREE_INLINESIZE);
if (d < 0) {
return d;
}
LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
btree->u.i.size = d;
return 0;
// a normal btree
} else {
// lookup in which leaf our rid resides
lfsr_rbyd_t rbyd;
lfsr_tag_t rtag;
lfs_ssize_t rid;
lfs_size_t rweight;
int err = lfsr_btree_lookupnext_(lfs, btree, bid,
NULL, &rbyd, &rid, &rtag, &rweight, NULL);
if (err) { if (err) {
return err; return err;
} }
// commit our rid into the tree, letting lfsr_btree_commit take care // note we need a second tag here in case our entry has a
// of the rest // name attributes, the name attribute holds the weight not
int degenerate = lfsr_btree_commit(lfs, btree, bid, 1, &rbyd, // the struct tag
LFSR_ATTRS( return lfsr_btree_commit__(lfs, btree, LFSR_ATTRS(
LFSR_ATTR(rid, WIDE(TAG(tag)), 0, DATA(data)), LFSR_ATTR(bid, WIDE(TAG(tag)), 0, DATA(data)),
LFSR_ATTR(rid, GROW(RM), +weight-rweight, NULL))); LFSR_ATTR(bid, GROW(RM), weight - weight_, NULL)));
if (degenerate < 0) {
return degenerate;
}
// revert to an inlined btree
if (degenerate) {
// mark as inlined
btree->u.i.weight = lfsr_btree_setinlined(weight);
btree->u.i.tag = tag;
lfs_ssize_t d = lfsr_data_read(lfs, &data,
btree->u.i.buffer, LFSR_BTREE_INLINESIZE);
if (d < 0) {
return d;
}
LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
btree->u.i.size = d;
}
return 0;
}
} }
// TODO
//static int lfsr_btree_set(lfs_t *lfs, lfsr_btree_t *btree,
// lfs_size_t bid, lfsr_tag_t tag, lfs_size_t weight, lfsr_data_t data) {
// LFS_ASSERT(bid < lfsr_btree_weight(btree));
// LFS_ASSERT(lfsr_btree_weight(btree) > 0);
//
// // inlined btree?
// if (lfsr_btree_isinlined(btree)) {
// LFS_ASSERT(bid == lfsr_btree_weight(btree)-1);
//
// // mark as inlined
// btree->u.i.weight = lfsr_btree_setinlined(weight);
// btree->u.i.tag = tag;
//
// lfs_ssize_t d = lfsr_data_read(lfs, &data,
// btree->u.i.buf, LFSR_BTREE_INLINESIZE);
// if (d < 0) {
// return d;
// }
// LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
// btree->u.i.size = d;
// return 0;
//
// // a normal btree
// } else {
// // lookup in which leaf our rid resides
// lfsr_rbyd_t rbyd;
// lfsr_tag_t rtag;
// lfs_ssize_t rid;
// lfs_size_t rweight;
// int err = lfsr_btree_lookupnext_(lfs, btree, bid,
// NULL, &rbyd, &rid, &rtag, &rweight, NULL);
// if (err) {
// return err;
// }
//
// // commit our rid into the tree, letting lfsr_btree_commit take care
// // of the rest
// int degenerate = lfsr_btree_commit(lfs, btree, bid, 1, &rbyd,
// LFSR_ATTRS(
// LFSR_ATTR(rid, WIDE(TAG(tag)), 0, DATA(data)),
// LFSR_ATTR(rid, GROW(RM), +weight-rweight, NULL)));
// if (degenerate < 0) {
// return degenerate;
// }
//
// // revert to an inlined btree
// if (degenerate) {
// // mark as inlined
// btree->u.i.weight = lfsr_btree_setinlined(weight);
// btree->u.i.tag = tag;
//
// lfs_ssize_t d = lfsr_data_read(lfs, &data,
// btree->u.i.buf, LFSR_BTREE_INLINESIZE);
// if (d < 0) {
// return d;
// }
// LFS_ASSERT(d <= LFSR_BTREE_INLINESIZE);
// btree->u.i.size = d;
// }
//
// return 0;
// }
//}
// TODO
static int lfsr_btree_pop(lfs_t *lfs, lfsr_btree_t *btree, lfs_size_t bid) { static int lfsr_btree_pop(lfs_t *lfs, lfsr_btree_t *btree, lfs_size_t bid) {
LFS_ASSERT(bid < lfsr_btree_weight(btree)); LFS_ASSERT(bid < lfsr_btree_weight(btree));
LFS_ASSERT(lfsr_btree_weight(btree) > 0); LFS_ASSERT(lfsr_btree_weight(btree) > 0);
// inlined btree? // TODO yes this is completely redundant and eventually should be removed
if (lfsr_btree_isinlined(btree)) {
LFS_ASSERT(bid == lfsr_btree_weight(btree)-1);
*btree = LFSR_BTREE_NULL;
return 0;
// a normal btree
} else {
// lookup in which leaf our rid resides
lfsr_rbyd_t rbyd;
lfsr_tag_t rtag;
lfs_ssize_t rid;
lfs_size_t rweight;
int err = lfsr_btree_lookupnext_(lfs, btree, bid,
NULL, &rbyd, &rid, &rtag, &rweight, NULL);
if (err) {
return err;
}
// remove our rid, letting lfsr_btree_commit take care
// of the rest
// //
// note we use a cutoff of 2 here, if we have 2 entries before // We're looking up the bid here to find it's weight so we can compute the
// the commit, we should have 1 entry after the commit and can // correct delta. We could move this into lfsr_btree_commit__ hackily, but
// revert to an inlined btree // the currrent theory is we don't need this at all and upper layers can
int degenerate = lfsr_btree_commit(lfs, btree, bid, 2, &rbyd, // calculate the delta instead of the absolute weight when needed.
LFSR_ATTRS( lfs_size_t weight_;
LFSR_ATTR(rid, RM, -rweight, NULL))); int err = lfsr_btree_lookupnext(lfs, btree, bid,
if (degenerate < 0) { NULL, NULL, &weight_, NULL);
return degenerate;
}
// revert to a null btree
if (degenerate && rweight >= rbyd.weight) {
*btree = LFSR_BTREE_NULL;
// revert to an inlined btree
} else if (degenerate) {
lfs_ssize_t sid;
// left sibling
if ((lfs_size_t)rid == rbyd.weight-1) {
sid = rid-rweight;
// right sibling
} else {
sid = rid+1;
}
lfsr_tag_t stag;
lfs_size_t sweight;
lfsr_data_t sdata;
int err = lfsr_rbyd_lookupnext(lfs, &rbyd, sid, LFSR_TAG_NAME,
&sid, &stag, &sweight, &sdata);
if (err) {
LFS_ASSERT(err == LFS_ERR_NOENT);
return err;
}
if (lfsr_tag_suptype(stag) == LFSR_TAG_NAME) {
err = lfsr_rbyd_lookup(lfs, &rbyd, sid, LFSR_TAG_WIDE(STRUCT),
&stag, &sdata);
if (err) {
LFS_ASSERT(err == LFS_ERR_NOENT);
return err;
}
}
LFS_ASSERT(sweight+rweight == rbyd.weight);
// mark as inlined
btree->u.i.weight = lfsr_btree_setinlined(sweight);
btree->u.i.tag = stag;
LFS_ASSERT(lfsr_data_size(&sdata) <= LFSR_BTREE_INLINESIZE);
err = lfsr_bd_read(lfs, sdata.u.d.block, sdata.u.d.off, 0,
btree->u.i.buffer, lfsr_data_size(&sdata));
if (err) { if (err) {
return err; return err;
} }
btree->u.i.size = lfsr_data_size(&sdata);
}
return 0; return lfsr_btree_commit__(lfs, btree, LFSR_ATTRS(
} LFSR_ATTR(bid, RM, -weight_, NULL)));
} }
//static int lfsr_btree_pop(lfs_t *lfs, lfsr_btree_t *btree, lfs_size_t bid) {
// LFS_ASSERT(bid < lfsr_btree_weight(btree));
// LFS_ASSERT(lfsr_btree_weight(btree) > 0);
//
// // inlined btree?
// if (lfsr_btree_isinlined(btree)) {
// LFS_ASSERT(bid == lfsr_btree_weight(btree)-1);
// *btree = LFSR_BTREE_NULL;
// return 0;
//
// // a normal btree
// } else {
// // lookup in which leaf our rid resides
// lfsr_rbyd_t rbyd;
// lfsr_tag_t rtag;
// lfs_ssize_t rid;
// lfs_size_t rweight;
// int err = lfsr_btree_lookupnext_(lfs, btree, bid,
// NULL, &rbyd, &rid, &rtag, &rweight, NULL);
// if (err) {
// return err;
// }
//
// // remove our rid, letting lfsr_btree_commit take care
// // of the rest
// //
// // note we use a cutoff of 2 here, if we have 2 entries before
// // the commit, we should have 1 entry after the commit and can
// // revert to an inlined btree
// int degenerate = lfsr_btree_commit(lfs, btree, bid, 2, &rbyd,
// LFSR_ATTRS(
// LFSR_ATTR(rid, RM, -rweight, NULL)));
// if (degenerate < 0) {
// return degenerate;
// }
//
// // revert to a null btree
// if (degenerate && rweight >= rbyd.weight) {
// *btree = LFSR_BTREE_NULL;
//
// // revert to an inlined btree
// } else if (degenerate) {
// lfs_ssize_t sid;
// // left sibling
// if ((lfs_size_t)rid == rbyd.weight-1) {
// sid = rid-rweight;
// // right sibling
// } else {
// sid = rid+1;
// }
//
// lfsr_tag_t stag;
// lfs_size_t sweight;
// lfsr_data_t sdata;
// int err = lfsr_rbyd_lookupnext(lfs, &rbyd, sid, LFSR_TAG_NAME,
// &sid, &stag, &sweight, &sdata);
// if (err) {
// LFS_ASSERT(err == LFS_ERR_NOENT);
// return err;
// }
//
// if (lfsr_tag_suptype(stag) == LFSR_TAG_NAME) {
// err = lfsr_rbyd_lookup(lfs, &rbyd, sid, LFSR_TAG_WIDE(STRUCT),
// &stag, &sdata);
// if (err) {
// LFS_ASSERT(err == LFS_ERR_NOENT);
// return err;
// }
// }
//
// LFS_ASSERT(sweight+rweight == rbyd.weight);
// // mark as inlined
// btree->u.i.weight = lfsr_btree_setinlined(sweight);
// btree->u.i.tag = stag;
//
// LFS_ASSERT(lfsr_data_size(&sdata) <= LFSR_BTREE_INLINESIZE);
// err = lfsr_bd_read(lfs, sdata.u.d.block, sdata.u.d.off, 0,
// btree->u.i.buf, lfsr_data_size(&sdata));
// if (err) {
// return err;
// }
// btree->u.i.size = lfsr_data_size(&sdata);
// }
//
// return 0;
// }
//}
// lfsr_btree_split can be done with a update+push, but this function // lfsr_btree_split can be done with a update+push, but this function
// does all this in one commit, which is much more efficient // does all this in one commit, which is much more efficient
// //
@@ -4657,68 +4846,102 @@ static int lfsr_btree_split(lfs_t *lfs, lfsr_btree_t *btree,
LFS_ASSERT(bid < lfsr_btree_weight(btree)); LFS_ASSERT(bid < lfsr_btree_weight(btree));
LFS_ASSERT(lfsr_btree_weight(btree) > 0); LFS_ASSERT(lfsr_btree_weight(btree) > 0);
// inlined btree, need to expand into an rbyd // TODO yes this is completely redundant and eventually should be removed
if (lfsr_btree_isinlined(btree)) { //
lfsr_rbyd_t rbyd; // We're looking up the bid here to find it's weight so we can compute the
int err = lfsr_rbyd_alloc(lfs, &rbyd); // correct delta. We could move this into lfsr_btree_commit__ hackily, but
// the currrent theory is we don't need this at all and upper layers can
// calculate the delta instead of the absolute weight when needed.
lfs_size_t weight_;
int err = lfsr_btree_lookupnext(lfs, btree, bid,
NULL, NULL, &weight_, NULL);
if (err) { if (err) {
return err; return err;
} }
// commit our entries return lfsr_btree_commit__(lfs, btree, LFSR_ATTRS(
err = lfsr_rbyd_commit(lfs, &rbyd, LFSR_ATTRS( LFSR_ATTR(bid, GROW(RM), +weight1-weight_, NULL),
LFSR_ATTR(0, TAG(tag1), +weight1, DATA(data1)), LFSR_ATTR(bid-(weight_-1)+weight1-1, TAG(tag1), 0, DATA(data1)),
(lfsr_data_size(&name) > 0 (lfsr_data_size(&name) > 0
? LFSR_ATTR(weight1, BNAME, +weight2, DATA(name)) ? LFSR_ATTR(bid-(weight_-1)+weight1,
: LFSR_ATTR_NOOP),
(lfsr_data_size(&name) > 0
? LFSR_ATTR(weight1+weight2-1, TAG(tag2), 0, DATA(data2))
: LFSR_ATTR(weight1, TAG(tag2), +weight2, DATA(data2)))));
if (err) {
return err;
}
btree->u.r.rbyd = rbyd;
return 0;
// a normal btree
} else {
// lookup in which leaf our bid resides
lfsr_rbyd_t rbyd;
lfs_ssize_t rid;
lfs_size_t rweight;
int err = lfsr_btree_lookupnext_(lfs, btree, bid,
NULL, &rbyd, &rid, NULL, &rweight, NULL);
if (err) {
return err;
}
// commit our bid into the tree, letting lfsr_btree_commit take care
// of the rest
int degenerate = lfsr_btree_commit(lfs, btree, bid, -1, &rbyd,
LFSR_ATTRS(
LFSR_ATTR(rid, GROW(RM), +weight1-rweight, NULL),
LFSR_ATTR(rid-(rweight-1)+weight1-1, TAG(tag1), 0,
DATA(data1)),
(lfsr_data_size(&name) > 0
? LFSR_ATTR(rid-(rweight-1)+weight1,
BNAME, +weight2, DATA(name)) BNAME, +weight2, DATA(name))
: LFSR_ATTR_NOOP), : LFSR_ATTR_NOOP),
(lfsr_data_size(&name) > 0 (lfsr_data_size(&name) > 0
? LFSR_ATTR(rid-(rweight-1)+weight1+weight2-1, ? LFSR_ATTR(bid-(weight_-1)+weight1+weight2-1,
TAG(tag2), 0, DATA(data2)) TAG(tag2), 0, DATA(data2))
: LFSR_ATTR(rid-(rweight-1)+weight1, : LFSR_ATTR(bid-(weight_-1)+weight1,
TAG(tag2), +weight2, DATA(data2))))); TAG(tag2), +weight2, DATA(data2)))));
if (degenerate < 0) {
return degenerate;
}
// this should never happen
LFS_ASSERT(!degenerate);
return 0;
}
} }
//static int lfsr_btree_split(lfs_t *lfs, lfsr_btree_t *btree,
// lfs_size_t bid, lfsr_data_t name,
// lfsr_tag_t tag1, lfs_size_t weight1, lfsr_data_t data1,
// lfsr_tag_t tag2, lfs_size_t weight2, lfsr_data_t data2) {
// LFS_ASSERT(bid < lfsr_btree_weight(btree));
// LFS_ASSERT(lfsr_btree_weight(btree) > 0);
//
// // inlined btree, need to expand into an rbyd
// if (lfsr_btree_isinlined(btree)) {
// lfsr_rbyd_t rbyd;
// int err = lfsr_rbyd_alloc(lfs, &rbyd);
// if (err) {
// return err;
// }
//
// // commit our entries
// err = lfsr_rbyd_commit(lfs, &rbyd, LFSR_ATTRS(
// LFSR_ATTR(0, TAG(tag1), +weight1, DATA(data1)),
// (lfsr_data_size(&name) > 0
// ? LFSR_ATTR(weight1, BNAME, +weight2, DATA(name))
// : LFSR_ATTR_NOOP),
// (lfsr_data_size(&name) > 0
// ? LFSR_ATTR(weight1+weight2-1, TAG(tag2), 0, DATA(data2))
// : LFSR_ATTR(weight1, TAG(tag2), +weight2, DATA(data2)))));
// if (err) {
// return err;
// }
//
// btree->u.r.rbyd = rbyd;
// return 0;
//
// // a normal btree
// } else {
// // lookup in which leaf our bid resides
// lfsr_rbyd_t rbyd;
// lfs_ssize_t rid;
// lfs_size_t rweight;
// int err = lfsr_btree_lookupnext_(lfs, btree, bid,
// NULL, &rbyd, &rid, NULL, &rweight, NULL);
// if (err) {
// return err;
// }
//
// // commit our bid into the tree, letting lfsr_btree_commit take care
// // of the rest
// int degenerate = lfsr_btree_commit(lfs, btree, bid, &rbyd,
// LFSR_ATTRS(
// LFSR_ATTR(rid, GROW(RM), +weight1-rweight, NULL),
// LFSR_ATTR(rid-(rweight-1)+weight1-1, TAG(tag1), 0,
// DATA(data1)),
// (lfsr_data_size(&name) > 0
// ? LFSR_ATTR(rid-(rweight-1)+weight1,
// BNAME, +weight2, DATA(name))
// : LFSR_ATTR_NOOP),
// (lfsr_data_size(&name) > 0
// ? LFSR_ATTR(rid-(rweight-1)+weight1+weight2-1,
// TAG(tag2), 0, DATA(data2))
// : LFSR_ATTR(rid-(rweight-1)+weight1,
// TAG(tag2), +weight2, DATA(data2)))));
// if (degenerate < 0) {
// return degenerate;
// }
//
// // this should never happen
// LFS_ASSERT(!degenerate);
// return 0;
// }
//}
// lookup in a btree by name // lookup in a btree by name
static int lfsr_btree_namelookup(lfs_t *lfs, const lfsr_btree_t *btree, static int lfsr_btree_namelookup(lfs_t *lfs, const lfsr_btree_t *btree,
lfs_size_t did, const char *name, lfs_size_t name_size, lfs_size_t did, const char *name, lfs_size_t name_size,
@@ -4742,7 +4965,7 @@ static int lfsr_btree_namelookup(lfs_t *lfs, const lfsr_btree_t *btree,
*weight_ = lfsr_btree_weight(btree); *weight_ = lfsr_btree_weight(btree);
} }
if (data_) { if (data_) {
*data_ = LFSR_DATA(btree->u.i.buffer, btree->u.i.size); *data_ = LFSR_DATA(btree->u.i.buf, btree->u.i.size);
} }
return 0; return 0;
} }
@@ -4845,7 +5068,7 @@ static int lfsr_btree_traversal_next(lfs_t *lfs,
*weight_ = lfsr_btree_weight(btree); *weight_ = lfsr_btree_weight(btree);
} }
if (data_) { if (data_) {
*data_ = LFSR_DATA(btree->u.i.buffer, btree->u.i.size); *data_ = LFSR_DATA(btree->u.i.buf, btree->u.i.size);
} }
return 0; return 0;
} }
@@ -5884,7 +6107,7 @@ static int lfsr_mdir_commit(lfs_t *lfs, lfsr_mdir_t *mdir,
} else if (lfsr_btree_isinlined(&mtree_)) { } else if (lfsr_btree_isinlined(&mtree_)) {
LFS_ASSERT(mtree_.u.i.tag == LFSR_TAG_MDIR); LFS_ASSERT(mtree_.u.i.tag == LFSR_TAG_MDIR);
mtree_tag = LFSR_TAG_WIDE(MDIR); mtree_tag = LFSR_TAG_WIDE(MDIR);
memcpy(mtree_buf, mtree_.u.i.buffer, mtree_.u.i.size); memcpy(mtree_buf, mtree_.u.i.buf, mtree_.u.i.size);
mtree_dsize = mtree_.u.i.size; mtree_dsize = mtree_.u.i.size;
} else { } else {
mtree_tag = LFSR_TAG_WIDE(MTREE); mtree_tag = LFSR_TAG_WIDE(MTREE);
lfs.h
+1 -1
View File
@@ -371,7 +371,7 @@ typedef struct lfsr_btree {
lfs_size_t weight; lfs_size_t weight;
lfsr_tag_t tag; lfsr_tag_t tag;
uint16_t size; uint16_t size;
uint8_t buffer[LFSR_BTREE_INLINESIZE]; uint8_t buf[LFSR_BTREE_INLINESIZE];
} i; } i;
struct { struct {
lfsr_rbyd_t rbyd; lfsr_rbyd_t rbyd;
lfs_util.h
+10
View File
@@ -163,6 +163,7 @@ static inline int32_t lfs_smin32(int32_t a, int32_t b) {
return (a < b) ? a : b; return (a < b) ? a : b;
} }
// TODO other 16-bit ops?
static inline uint16_t lfs_max16(uint16_t a, uint16_t b) { static inline uint16_t lfs_max16(uint16_t a, uint16_t b) {
return (a > b) ? a : b; return (a > b) ? a : b;
} }
@@ -171,6 +172,15 @@ static inline uint16_t lfs_min16(uint16_t a, uint16_t b) {
return (a < b) ? a : b; return (a < b) ? a : b;
} }
// Clamp is useful as the logic for min/max when clamping can become confusing
static inline uint32_t lfs_clamp32(uint32_t a, uint32_t min, uint32_t max) {
return lfs_min32(lfs_max32(a, min), max);
}
static inline int32_t lfs_sclamp32(int32_t a, int32_t min, int32_t max) {
return lfs_smin32(lfs_smax32(a, min), max);
}
// Absolute value of signed numbers // Absolute value of signed numbers
static inline int32_t lfs_abs32(int32_t a) { static inline int32_t lfs_abs32(int32_t a) {
return a < 0 ? -a : a; return a < 0 ? -a : a;
tests/test_btree.toml
+257 -257
View File
@@ -2424,263 +2424,263 @@ code = '''
free(sim); free(sim);
''' '''
# TODO
# test we reinline (go from uninlined to inlined) correctly, this is a bit ## test we reinline (go from uninlined to inlined) correctly, this is a bit
# tricky since our btrees lazily reinline ## tricky since our btrees lazily reinline
[cases.test_btree_reinline_pop_set] #[cases.test_btree_reinline_pop_set]
in = 'lfs.c' #in = 'lfs.c'
code = ''' #code = '''
lfs_t lfs; # lfs_t lfs;
lfs_init(&lfs, CFG) => 0; # lfs_init(&lfs, CFG) => 0;
// create free lookahead # // create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size); # memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.start = 0; # lfs.lookahead.start = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size, # lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count); # CFG->block_count);
lfs.lookahead.next = 0; # lfs.lookahead.next = 0;
lfs_alloc_ack(&lfs); # lfs_alloc_ack(&lfs);
#
// create an uninlined tree # // create an uninlined tree
lfsr_btree_t btree = LFSR_BTREE_NULL; # lfsr_btree_t btree = LFSR_BTREE_NULL;
lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1, # lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("a", 1)) => 0; # LFSR_DATA("a", 1)) => 0;
lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1, # lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1,
LFSR_DATA("b", 1)) => 0; # LFSR_DATA("b", 1)) => 0;
assert(lfsr_btree_weight(&btree) == 2); # assert(lfsr_btree_weight(&btree) == 2);
assert(!lfsr_btree_isinlined(&btree)); # assert(!lfsr_btree_isinlined(&btree));
#
// pop! our btree should now be reinlinable # // pop! our btree should now be reinlinable
lfsr_btree_pop(&lfs, &btree, 0) => 0; # lfsr_btree_pop(&lfs, &btree, 0) => 0;
#
// but thanks to lazy reinlining, our btree won't reinline until # // but thanks to lazy reinlining, our btree won't reinline until
// it is compacted, so we need to add commits until it is compacted # // it is compacted, so we need to add commits until it is compacted
lfs_block_t before_block = btree.u.r.rbyd.block; # lfs_block_t before_block = btree.u.r.rbyd.block;
for (lfs_block_t i = 0;; i++) { # for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops # // a bit hacky, but this catches infinite loops
assert(i < BLOCK_SIZE); # assert(i < BLOCK_SIZE);
#
// commit to btree # // commit to btree
lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1, # lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("b", 1)) => 0; # LFSR_DATA("b", 1)) => 0;
#
assert(lfsr_btree_weight(&btree) == 1); # assert(lfsr_btree_weight(&btree) == 1);
#
// try looking up tag to hopefully catch if something breaks # // try looking up tag to hopefully catch if something breaks
uint8_t buffer[4]; # uint8_t buffer[4];
lfsr_tag_t tag_; # lfsr_tag_t tag_;
lfs_size_t weight_; # lfs_size_t weight_;
#
lfsr_btree_get(&lfs, &btree, 0, # lfsr_btree_get(&lfs, &btree, 0,
&tag_, &weight_, buffer, 4) => 1; # &tag_, &weight_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_INLINED); # assert(tag_ == LFSR_TAG_INLINED);
assert(weight_ == 1); # assert(weight_ == 1);
assert(memcmp(buffer, "b", 1) == 0); # assert(memcmp(buffer, "b", 1) == 0);
#
// inlined? consider this a success # // inlined? consider this a success
if (lfsr_btree_isinlined(&btree)) { # if (lfsr_btree_isinlined(&btree)) {
break; # break;
} # }
#
// assert if a compaction occurred that wasn't inlined # // assert if a compaction occurred that wasn't inlined
assert(btree.u.r.rbyd.block == before_block); # assert(btree.u.r.rbyd.block == before_block);
} # }
#
printf("btree: w%d 0x%x.%x\n", # printf("btree: w%d 0x%x.%x\n",
btree.u.r.rbyd.weight, # btree.u.r.rbyd.weight,
btree.u.r.rbyd.block, # btree.u.r.rbyd.block,
btree.u.r.rbyd.trunk); # btree.u.r.rbyd.trunk);
''' #'''
#
[cases.test_btree_reinline_pop_pop_push] #[cases.test_btree_reinline_pop_pop_push]
in = 'lfs.c' #in = 'lfs.c'
defines.SHIFT = 'range(5)' #defines.SHIFT = 'range(5)'
code = ''' #code = '''
lfs_t lfs; # lfs_t lfs;
lfs_init(&lfs, CFG) => 0; # lfs_init(&lfs, CFG) => 0;
// create free lookahead # // create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size); # memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.start = 0; # lfs.lookahead.start = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size, # lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count); # CFG->block_count);
lfs.lookahead.next = 0; # lfs.lookahead.next = 0;
lfs_alloc_ack(&lfs); # lfs_alloc_ack(&lfs);
#
// create an uninlined tree # // create an uninlined tree
lfsr_btree_t btree = LFSR_BTREE_NULL; # lfsr_btree_t btree = LFSR_BTREE_NULL;
lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1, # lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("a", 1)) => 0; # LFSR_DATA("a", 1)) => 0;
lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1, # lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1,
LFSR_DATA("b", 1)) => 0; # LFSR_DATA("b", 1)) => 0;
assert(lfsr_btree_weight(&btree) == 2); # assert(lfsr_btree_weight(&btree) == 2);
assert(!lfsr_btree_isinlined(&btree)); # assert(!lfsr_btree_isinlined(&btree));
#
// pop! our btree should now be reinlinable # // pop! our btree should now be reinlinable
lfsr_btree_pop(&lfs, &btree, 0) => 0; # lfsr_btree_pop(&lfs, &btree, 0) => 0;
#
// It's difficult to test reinlining during push or pop, since we can't just # // It's difficult to test reinlining during push or pop, since we can't just
// repeat the action until compaction occurs. # // repeat the action until compaction occurs.
// # //
// What we do here is alternate between 0 and 1 entries, eventually we # // What we do here is alternate between 0 and 1 entries, eventually we
// will compact during one of either a push or pop. To try to cover both, # // will compact during one of either a push or pop. To try to cover both,
// test with some number of extra commits to hopefully adjust where the # // test with some number of extra commits to hopefully adjust where the
// compaction ends up. # // compaction ends up.
for (lfs_size_t i = 0; i < SHIFT; i++) { # for (lfs_size_t i = 0; i < SHIFT; i++) {
lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1, # lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("b", 1)) => 0; # LFSR_DATA("b", 1)) => 0;
} # }
#
// alternate between push/pop until compaction occurs # // alternate between push/pop until compaction occurs
lfs_block_t before_block = btree.u.r.rbyd.block; # lfs_block_t before_block = btree.u.r.rbyd.block;
for (lfs_block_t i = 0;; i++) { # for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops # // a bit hacky, but this catches infinite loops
assert(i < BLOCK_SIZE); # assert(i < BLOCK_SIZE);
#
// pop! # // pop!
lfsr_btree_pop(&lfs, &btree, 0) => 0; # lfsr_btree_pop(&lfs, &btree, 0) => 0;
#
assert(lfsr_btree_weight(&btree) == 0); # assert(lfsr_btree_weight(&btree) == 0);
#
// inlined? consider this a success # // inlined? consider this a success
if (lfsr_btree_isinlined(&btree)) { # if (lfsr_btree_isinlined(&btree)) {
break; # break;
} # }
#
// assert if a compaction occurred that wasn't inlined # // assert if a compaction occurred that wasn't inlined
assert(btree.u.r.rbyd.block == before_block); # assert(btree.u.r.rbyd.block == before_block);
#
// push! # // push!
lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1, # lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("c", 1)) => 0; # LFSR_DATA("c", 1)) => 0;
#
// try looking up tag to hopefully catch if something breaks # // try looking up tag to hopefully catch if something breaks
uint8_t buffer[4]; # uint8_t buffer[4];
lfsr_tag_t tag_; # lfsr_tag_t tag_;
lfs_size_t weight_; # lfs_size_t weight_;
#
lfsr_btree_get(&lfs, &btree, 0, # lfsr_btree_get(&lfs, &btree, 0,
&tag_, &weight_, buffer, 4) => 1; # &tag_, &weight_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_INLINED); # assert(tag_ == LFSR_TAG_INLINED);
assert(weight_ == 1); # assert(weight_ == 1);
assert(memcmp(buffer, "c", 1) == 0); # assert(memcmp(buffer, "c", 1) == 0);
#
// inlined? consider this a success # // inlined? consider this a success
if (lfsr_btree_isinlined(&btree)) { # if (lfsr_btree_isinlined(&btree)) {
break; # break;
} # }
#
// assert if a compaction occurred that wasn't inlined # // assert if a compaction occurred that wasn't inlined
assert(btree.u.r.rbyd.block == before_block); # assert(btree.u.r.rbyd.block == before_block);
} # }
#
printf("btree: w%d 0x%x.%x\n", # printf("btree: w%d 0x%x.%x\n",
btree.u.r.rbyd.weight, # btree.u.r.rbyd.weight,
btree.u.r.rbyd.block, # btree.u.r.rbyd.block,
btree.u.r.rbyd.trunk); # btree.u.r.rbyd.trunk);
''' #'''
#
[cases.test_btree_reinline_pop_push] #[cases.test_btree_reinline_pop_push]
in = 'lfs.c' #in = 'lfs.c'
defines.SIBLING = [0, 1] #defines.SIBLING = [0, 1]
defines.SHIFT = 'range(5)' #defines.SHIFT = 'range(5)'
code = ''' #code = '''
lfs_t lfs; # lfs_t lfs;
lfs_init(&lfs, CFG) => 0; # lfs_init(&lfs, CFG) => 0;
// create free lookahead # // create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size); # memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.start = 0; # lfs.lookahead.start = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size, # lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count); # CFG->block_count);
lfs.lookahead.next = 0; # lfs.lookahead.next = 0;
lfs_alloc_ack(&lfs); # lfs_alloc_ack(&lfs);
#
// create an uninlined tree # // create an uninlined tree
lfsr_btree_t btree = LFSR_BTREE_NULL; # lfsr_btree_t btree = LFSR_BTREE_NULL;
lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1, # lfsr_btree_push(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("a", 1)) => 0; # LFSR_DATA("a", 1)) => 0;
lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1, # lfsr_btree_push(&lfs, &btree, 1, LFSR_TAG_INLINED, 1,
LFSR_DATA("b", 1)) => 0; # LFSR_DATA("b", 1)) => 0;
assert(lfsr_btree_weight(&btree) == 2); # assert(lfsr_btree_weight(&btree) == 2);
assert(!lfsr_btree_isinlined(&btree)); # assert(!lfsr_btree_isinlined(&btree));
#
// It's difficult to test reinlining during push or pop, since we can't just # // It's difficult to test reinlining during push or pop, since we can't just
// repeat the action until compaction occurs. # // repeat the action until compaction occurs.
// # //
// Here we alternate between 1 and 2 entries, with the hope that compaction # // Here we alternate between 1 and 2 entries, with the hope that compaction
// occurs on the pop. We try this with some number of extra commits to make # // occurs on the pop. We try this with some number of extra commits to make
// it more likely pop is tested. # // it more likely pop is tested.
// # //
// It's possible our commits line up so compaction always occurs on a push! # // It's possible our commits line up so compaction always occurs on a push!
// For this reason, we end the test if an non-reinlining compaction occurs. # // For this reason, we end the test if an non-reinlining compaction occurs.
for (lfs_size_t i = 0; i < SHIFT; i++) { # for (lfs_size_t i = 0; i < SHIFT; i++) {
lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1, # lfsr_btree_set(&lfs, &btree, 0, LFSR_TAG_INLINED, 1,
LFSR_DATA("a", 1)) => 0; # LFSR_DATA("a", 1)) => 0;
} # }
#
// alternate between push/pop until compaction occurs # // alternate between push/pop until compaction occurs
lfs_block_t before_block = btree.u.r.rbyd.block; # lfs_block_t before_block = btree.u.r.rbyd.block;
for (lfs_block_t i = 0;; i++) { # for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops # // a bit hacky, but this catches infinite loops
assert(i < BLOCK_SIZE); # assert(i < BLOCK_SIZE);
#
// pop! # // pop!
lfsr_btree_pop(&lfs, &btree, SIBLING) => 0; # lfsr_btree_pop(&lfs, &btree, SIBLING) => 0;
#
assert(lfsr_btree_weight(&btree) == 1); # assert(lfsr_btree_weight(&btree) == 1);
#
// try looking up tag to hopefully catch if something breaks # // try looking up tag to hopefully catch if something breaks
uint8_t buffer[4]; # uint8_t buffer[4];
lfsr_tag_t tag_; # lfsr_tag_t tag_;
lfs_size_t weight_; # lfs_size_t weight_;
#
lfsr_btree_get(&lfs, &btree, 0, # lfsr_btree_get(&lfs, &btree, 0,
&tag_, &weight_, buffer, 4) => 1; # &tag_, &weight_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_INLINED); # assert(tag_ == LFSR_TAG_INLINED);
assert(weight_ == 1); # assert(weight_ == 1);
assert(memcmp(buffer, (SIBLING == 1 ? "a" : "b"), 1) == 0); # assert(memcmp(buffer, (SIBLING == 1 ? "a" : "b"), 1) == 0);
#
// inlined? consider this a success # // inlined? consider this a success
if (lfsr_btree_isinlined(&btree)) { # if (lfsr_btree_isinlined(&btree)) {
break; # break;
} # }
#
// abort if a compaction occurs # // abort if a compaction occurs
if (btree.u.r.rbyd.block != before_block) { # if (btree.u.r.rbyd.block != before_block) {
break; # break;
} # }
#
// push! # // push!
lfsr_btree_push(&lfs, &btree, SIBLING, LFSR_TAG_INLINED, 1, # lfsr_btree_push(&lfs, &btree, SIBLING, LFSR_TAG_INLINED, 1,
LFSR_DATA("c", 1)) => 0; # LFSR_DATA("c", 1)) => 0;
#
// try looking up tag to hopefully catch if something breaks # // try looking up tag to hopefully catch if something breaks
lfsr_btree_get(&lfs, &btree, 0, # lfsr_btree_get(&lfs, &btree, 0,
&tag_, &weight_, buffer, 4) => 1; # &tag_, &weight_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_INLINED); # assert(tag_ == LFSR_TAG_INLINED);
assert(weight_ == 1); # assert(weight_ == 1);
assert(memcmp(buffer, (SIBLING == 1 ? "a" : "c"), 1) == 0); # assert(memcmp(buffer, (SIBLING == 1 ? "a" : "c"), 1) == 0);
#
lfsr_btree_get(&lfs, &btree, 1, # lfsr_btree_get(&lfs, &btree, 1,
&tag_, &weight_, buffer, 4) => 1; # &tag_, &weight_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_INLINED); # assert(tag_ == LFSR_TAG_INLINED);
assert(weight_ == 1); # assert(weight_ == 1);
assert(memcmp(buffer, (SIBLING == 1 ? "c" : "b"), 1) == 0); # assert(memcmp(buffer, (SIBLING == 1 ? "c" : "b"), 1) == 0);
#
// inlined? consider this a success # // inlined? consider this a success
if (lfsr_btree_isinlined(&btree)) { # if (lfsr_btree_isinlined(&btree)) {
break; # break;
} # }
#
// abort if a compaction occurs # // abort if a compaction occurs
if (btree.u.r.rbyd.block != before_block) { # if (btree.u.r.rbyd.block != before_block) {
break; # break;
} # }
} # }
#
printf("btree: w%d 0x%x.%x\n", # printf("btree: w%d 0x%x.%x\n",
btree.u.r.rbyd.weight, # btree.u.r.rbyd.weight,
btree.u.r.rbyd.block, # btree.u.r.rbyd.block,
btree.u.r.rbyd.trunk); # btree.u.r.rbyd.trunk);
''' #'''
# Some more general fuzz testing # Some more general fuzz testing