Github/littlefs
1
Fork 0
Code Actions Activity
Files
a49e13b992f7f2be9aab11b2eab739355b1739b8
littlefs/tests/test_btree.toml
T
Christopher Haster a49e13b992 Attempted to implement per-btree leaf caches 
The idea here, is we give each lfsr_btree_t an optional leaf rbyd, in
addition to the root rbyd. This leaf rbyd acts as a cache for the most
recent leaf, allowing nearby btree lookups to skip the full btree walk.

Unfortunately, this failed on pretty much every measurable metric...

---

The motivation for this is that we often do a bunch of nearby btree
lookups:

- Btree iteration via lfsr_btree_lookupnext is a bit naive, walking from
  the root every step.

- Our crystallization algorithm requires a bunch of nearby lookups to
  figure out our crystallization heuristic. Currently at most 4, when
  you need to lookup both crystal neighbors and then _also_ both
  fragment neighbors for coalescing.

- Checksum collision resolution for dids and (FUTURE) ddkeys can require
  an unbounded number of sequential lookups.

  Though to be fair, this is an exceptional case if our checksum is any
  good.

- Bids with multiple rattrs require nearby lookups to resolve.

  Though currently this can be explicitly avoided via
  lfsr_btree_lookupleaf + lfsr_rbyd_lookup.

The theory was that cases like these could explicitly keep track of the
leaf rbyd to avoid full btree walks, but in practice this never really
worked out. Tracking if we're still in the relevant leaf rbyd just adds
too much logic/code cost.

But if this leaf tracking logic was implemented once in the btree
layer...

The other theoretical benefit was being able to move more rbyds off the
stack. Sure our btrees take up more RAM, but if that results in stack
savings, that may be a win.

Oh, and this would let our btree API and rbyd API converge without
performance concerns. Internal users could in theory call
lfsr_btree_lookupnext + lfsr_btree_lookup with the same performance as
explicitly tracking the rbyd.

---

But this was a complete failure!

First the good news: There was a modest speedup of around ~2x to linear
reads.

And that's the good news.

Now the bad news:

1. There was no noticeable performance gain in any other benchmarks.

   To be fair, we're at the early stages of benchmarking, so the
   benchmarks may not be the most thorough, but thinking about it, there
   are some explanations:

   - In any benchmark that writes, fetch + erase + prog dominates. Being
     able to skip fetches during lookups makes our btree lookups
     surprisingly cheap!

   - Any random read heavy benchmark is likely thrashing this cache,
     which is to be expected.

   - For small 1-block btrees, the leaf cache is useless because the
     entire btree is cache in the root rbyd.

     And keep in mind, our blocks are BIG. "Small" here could be on
     the order of ~128KiB-1MiB for NAND flash.

   - For the mtree, fetched mdirs actually already act as a sort of leaf
     cache.

     The extra btree leaf cache isn't doing _nothing_, but each layer of
     the mtree has diminishing returns due to btree's ridiculous
     branching factor.

   - For file btrees, we're explicitly caching the leaf fragments/
     blocks, so the extra btree leaf cache has diminishing returns for
     the same reason.

2. Code cost was bad, stack cost was worse:

              code          stack          ctx
     before: 37172           2288          636
     after:  38068 (+2.4%)   2416 (+5.6%)  664 (+4.4%)

   Tracking the leaf required more code, that's expected. And, to be
   fair, the current code has had a lot more time to congeal.

   What wasn't expected was the stack cost.

   Unfortunately these caches didn't really take any rbyds off the stack
   hot-path:

   - We _can_ get rid of the rbyd in lfsr_btree_lookup/namelookup, but
     we were already hacking our way around the critical one in
     lfsr_mtree_lookup/namelookup by reusing the mdir's rbyd!

   - We can't even abuse the leaf rbyd in the commit logic, since the
     target btree can end up iterated/traversed by lfs_alloc.

     That was a fun bug.

   And the addition of a second rbyd to lfsr_btree_t increases both ctx
   and stack anywhere btrees are allocated.

Maybe this will make more sense when we add the auxiliary btrees, or
after more benchmarking, but for now the theoretical performance
improvements just aren't worth it.

Will probably revert this, but I wanted to commit it in case the idea is
worth resurrecting in the future, if in the future nearby btree lookups
are a bigger penalty than they are now.
2025-05-24 18:37:37 -05:00

4495 lines
144 KiB
TOML
Raw Blame History

# Test the mid-level B-trees
after = 'test_rbyd'
# maximize lookahead buffer, we don't actually gc so we only get one pass
# of the disk for these tests
defines.LOOKAHEAD_SIZE = '(BLOCK_COUNT+8-1) / 8'
# test an empty tree
[cases.test_btree_zero]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create an empty tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 0);
// try looking up tags
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
# test an inlined tree
[cases.test_btree_one]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a single-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 1);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(bid_ == 0);
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "a", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
# test a single-rbyd tree
[cases.test_btree_two]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a two-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "b", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 2);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "a", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "b", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 2,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_two_backwards]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a two-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "b", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 2);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "a", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "b", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 2,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
# still a single-rbyd tree, just making sure it works
[cases.test_btree_three]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a two-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "b", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 2, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "c", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 3);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "a", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "b", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 2,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "c", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 3,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_three_backwards]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a two-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "c", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "b", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 3);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "a", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "b", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 2,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "c", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 3,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
# try larger trees, when exactly a tree splits depends on the disk geometry, so
# we don't really have a better way of testing multi-rbyd trees
[cases.test_btree_push]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfs_size_t n = 0;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
n += 1;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == n);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < n; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, n,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_push_backwards]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfs_size_t n = 0;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+((N-1-i) % 26)}, 1))) => 0;
n += 1;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == n);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < n; i++) {
lfsr_btree_lookupnext(&lfs, &btree, n-1-i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &(uint8_t){'a'+((N-1-i) % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, n,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_push_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.SEED = 'range(20)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
lfs_size_t sim_size = 0;
memset(sim, 0, N);
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % (sim_size+1);
// add to btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
// add to sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
sim[bid] = 'a'+(i % 26);
sim_size += 1;
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
if (!first) {
printf(", ");
}
first = false;
printf("%c", sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == sim_size);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, sim_size,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
lfs_deinit(&lfs) => 0;
'''
[cases.test_btree_push_sparse]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.W = 5
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfs_size_t n = 0;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i*W, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +W,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
n += 1;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == n*W);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < n; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i*W+W-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == W);
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, n*W,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// also test that we can traverse the tree without prior knowledge
bid_ = -1;
for (lfs_size_t i = 0; i < n; i++) {
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => 0;
assert(bid_ == i*W+W-1);
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == W);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_push_sparse_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.W = 5
defines.SEED = 'range(20)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
lfs_size_t *sim_weights = malloc(N*sizeof(lfs_size_t));
lfs_size_t sim_size = 0;
memset(sim, 0, N);
memset(sim_weights, 0, N*sizeof(lfs_size_t));
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % (sim_size+1);
// choose a pseudo-random weight
lfs_size_t weight = 1 + (TEST_PRNG(&prng) % W);
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < bid; j++) {
weighted_bid += sim_weights[j];
}
// add to btree
lfsr_btree_commit(&lfs, &btree, weighted_bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +weight,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
// add to sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
memmove(&sim_weights[bid+1], &sim_weights[bid],
(sim_size-bid)*sizeof(lfs_size_t));
sim[bid] = 'a'+(i % 26);
sim_weights[bid] = weight;
sim_size += 1;
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
if (!first) {
printf(", ");
}
first = false;
printf("%dw%d=%c", weighted_bid+sim_weights[i]-1,
sim_weights[i], sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
lfs_size_t total_weight = 0;
for (lfs_size_t j = 0; j < sim_size; j++) {
total_weight += sim_weights[j];
}
assert(lfsr_btree_weight(&btree) == total_weight);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, weighted_bid+sim_weights[i]-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, total_weight,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// also test that we can traverse the tree without prior knowledge
bid_ = -1;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => 0;
assert(bid_ == weighted_bid+sim_weights[i]-1);
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &sim[i], 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
'''
# test btree updates
# try some small trees for easy corner cases first
[cases.test_btree_update_one]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a single-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
// update the tree
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
"A", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 1);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "A", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_update_two]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a two-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "b", 1))) => 0;
// update the tree
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
"A", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
"B", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 2);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "A", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "B", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 2,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_update_three]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a two-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "b", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 2, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "c", 1))) => 0;
// update the tree
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
"A", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
"B", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 2, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
"C", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 3);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "A", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "B", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 2,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "C", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 3,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_update]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
}
// update the tree
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
&(uint8_t){'A'+(i % 26)}, 1))) => 0;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == N);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &(uint8_t){'A'+(i % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, N,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_update_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.SEED = 'range(20)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
}
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
for (lfs_size_t i = 0; i < N; i++) {
sim[i] = 'a'+(i % 26);
}
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % N;
// update btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
&(uint8_t){'A'+(i % 26)}, 1))) => 0;
// update sim
sim[bid] = 'A'+(i % 26);
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < N; i++) {
if (!first) {
printf(", ");
}
first = false;
printf("%c", sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == N);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, N,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
'''
[cases.test_btree_update_sparse]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.W = 5
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i*W, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +W,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
}
// update the tree
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i*W+W-1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
&(uint8_t){'A'+(i % 26)}, 1))) => 0;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == N*W);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i*W+W-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == W);
assert(memcmp(buffer, &(uint8_t){'A'+(i % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, N*W,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// also test that we can traverse the tree without prior knowledge
bid_ = -1;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => 0;
assert(bid_ == i*W+W-1);
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == W);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &(uint8_t){'A'+(i % 26)}, 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_update_sparse_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.W = 5
defines.SEED = 'range(20)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i*W, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +W,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
}
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
lfs_size_t *sim_weights = malloc(N*sizeof(lfs_size_t));
for (lfs_size_t i = 0; i < N; i++) {
sim[i] = 'a'+(i % 26);
sim_weights[i] = W;
}
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % N;
// choose a pseudo-random weight
lfs_size_t weight = 1 + (TEST_PRNG(&prng) % W);
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < bid; j++) {
weighted_bid += sim_weights[j];
}
// update btree
lfsr_btree_commit(&lfs, &btree,
weighted_bid+sim_weights[bid]-1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
&(uint8_t){'A'+(i % 26)}, 1),
LFSR_RATTR(
LFSR_TAG_GROW, +weight-sim_weights[bid]))) => 0;
// update sim
sim[bid] = 'A'+(i % 26);
sim_weights[bid] = weight;
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < N; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
if (!first) {
printf(", ");
}
first = false;
printf("%dw%d=%c", weighted_bid+sim_weights[i]-1,
sim_weights[i], sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
lfs_size_t total_weight = 0;
for (lfs_size_t j = 0; j < N; j++) {
total_weight += sim_weights[j];
}
assert(lfsr_btree_weight(&btree) == total_weight);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < N; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, weighted_bid+sim_weights[i]-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, total_weight,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// also test that we can traverse the tree without prior knowledge
bid_ = -1;
for (lfs_size_t i = 0; i < N; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => 0;
assert(bid_ == weighted_bid+sim_weights[i]-1);
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &sim[i], 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
'''
# test btree pops
# try some corner cases first, these are actually pretty tricky since we
# need to recognize when to collapse back into an inlined tree
[cases.test_btree_pop_one]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a single-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
// pop!
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 0);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// try to putting it back to see if things still work
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "A", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 1);
// try looking up tags
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "A", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_pop_two]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a single-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "b", 1))) => 0;
// pop!
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 1);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "a", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// try to putting it back to see if things still work
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "B", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 2);
// try looking up tags
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "a", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "B", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 2,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_pop_two_other]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a single-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "b", 1))) => 0;
// pop!
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 1);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "b", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// try to putting it back to see if things still work
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "A", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 2);
// try looking up tags
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "A", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "b", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 2,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_pop_three]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a single-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "a", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "b", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 2, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "c", 1))) => 0;
// pop!
lfsr_btree_commit(&lfs, &btree, 2, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 2);
// try looking up tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "a", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "b", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 2,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// try to putting it back to see if things still work
lfsr_btree_commit(&lfs, &btree, 2, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "C", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 3);
// try looking up tags
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "a", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "b", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 2,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "C", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, 3,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_pop]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.REMAINING = [64, 2, 1, 0]
if = 'N > REMAINING'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
}
// drain the tree
for (lfs_size_t i = 0; i < N-REMAINING; i++) {
lfsr_btree_commit(&lfs, &btree, N-1-i, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == REMAINING);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < REMAINING; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, REMAINING,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// try recovering
lfsr_btree_commit(&lfs, &btree, REMAINING, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "R", 1))) => 0;
for (lfs_size_t i = 0; i < REMAINING; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, REMAINING,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "R", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, REMAINING+1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_pop_backwards]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.REMAINING = [64, 2, 1, 0]
if = 'N > REMAINING'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
}
// drain the tree
for (lfs_size_t i = 0; i < N-REMAINING; i++) {
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == REMAINING);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < REMAINING; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer,
&(uint8_t){'a'+((i+(N-REMAINING)) % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, REMAINING,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// try recovering
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "R", 1))) => 0;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, "R", 1) == 0);
for (lfs_size_t i = 0; i < REMAINING; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i+1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer,
&(uint8_t){'a'+((i+(N-REMAINING)) % 26)}, 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, REMAINING+1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_pop_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.REMAINING = [64, 2, 1, 0]
defines.SEED = 'range(20)'
fuzz = 'SEED'
if = 'N > REMAINING'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
}
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
lfs_size_t sim_size = N;
for (lfs_size_t i = 0; i < N; i++) {
sim[i] = 'a'+(i % 26);
}
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < (N-REMAINING); i++) {
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % sim_size;
// remove from btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
// remove from sim
memmove(&sim[bid], &sim[bid+1], sim_size-(bid+1));
sim_size -= 1;
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
if (!first) {
printf(", ");
}
first = false;
printf("%c", sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == sim_size);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, sim_size,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
'''
[cases.test_btree_pop_sparse]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.W = 5
defines.REMAINING = [64, 2, 1, 0]
if = 'N > REMAINING'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i*W, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +W,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
}
// drain the tree
for (lfs_size_t i = 0; i < N-REMAINING; i++) {
lfsr_btree_commit(&lfs, &btree, (N-1-i)*W+W-1, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -W))) => 0;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == REMAINING*W);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < REMAINING; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i*W+W-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == W);
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, REMAINING*W+W-1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// try recovering
lfsr_btree_commit(&lfs, &btree, REMAINING*W, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +W, "R", 1))) => 0;
for (lfs_size_t i = 0; i < REMAINING; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i*W+W-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == W);
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, REMAINING*W+W-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == W);
assert(memcmp(buffer, "R", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, (REMAINING+1)*W+W-1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// also test that we can traverse the tree without prior knowledge
bid_ = -1;
for (lfs_size_t i = 0; i < REMAINING; i++) {
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => 0;
assert(bid_ == i*W+W-1);
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == W);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => 0;
assert(bid_ == REMAINING*W+W-1);
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == W);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "R", 1) == 0);
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_pop_sparse_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.W = 5
defines.REMAINING = [64, 2, 1, 0]
defines.SEED = 'range(20)'
fuzz = 'SEED'
if = 'N > REMAINING'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
lfs_size_t *sim_weights = malloc(N*sizeof(lfs_size_t));
lfs_size_t sim_size = 0;
// set up simulation and btree with pseudo-random weights
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random weight
lfs_size_t weight = 1 + (TEST_PRNG(&prng) % W);
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_commit(&lfs, &btree, weighted_bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +weight,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
sim[i] = 'a'+(i % 26);
sim_weights[i] = weight;
sim_size += 1;
}
for (lfs_size_t i = 0; i < (N-REMAINING); i++) {
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % sim_size;
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < bid; j++) {
weighted_bid += sim_weights[j];
}
// remove from btree
lfsr_btree_commit(&lfs, &btree,
weighted_bid+sim_weights[bid]-1, LFSR_RATTRS(
LFSR_RATTR(
LFSR_TAG_RM, -sim_weights[bid]))) => 0;
// remove from sim
memmove(&sim[bid], &sim[bid+1], sim_size-(bid+1));
memmove(&sim_weights[bid], &sim_weights[bid+1],
(sim_size-(bid+1))*sizeof(lfs_size_t));
sim_size -= 1;
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
if (!first) {
printf(", ");
}
first = false;
printf("%dw%d=%c", weighted_bid+sim_weights[i]-1,
sim_weights[i], sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
lfs_size_t total_weight = 0;
for (lfs_size_t j = 0; j < sim_size; j++) {
total_weight += sim_weights[j];
}
assert(lfsr_btree_weight(&btree) == total_weight);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, weighted_bid+sim_weights[i]-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, total_weight,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// also test that we can traverse the tree without prior knowledge
bid_ = -1;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => 0;
assert(bid_ == weighted_bid+sim_weights[i]-1);
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &sim[i], 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
'''
# test btree splits
[cases.test_btree_split]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(0 % 26)}, 1))) => 0;
lfs_size_t n = 1;
for (lfs_size_t i = 1; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i-1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'a'+((i-1) % 26)}, 1),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+((i-0) % 26)}, 1))) => 0;
n += 1;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == n);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < n; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, n,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_split_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.SEED = 'range(20)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "_", 1))) => 0;
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
lfs_size_t sim_size = 1;
memset(sim, 0, N);
sim[0] = '_';
uint32_t prng = SEED;
for (lfs_size_t i = 1; i < N; i++) {
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % sim_size;
// split btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'a'+(i % 26)}, 1),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'A'+(i % 26)}, 1))) => 0;
// split sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
sim[bid+0] = 'a'+(i % 26);
sim[bid+1] = 'A'+(i % 26);
sim_size += 1;
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
if (!first) {
printf(", ");
}
first = false;
printf("%c", sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == sim_size);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, sim_size,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
lfs_deinit(&lfs) => 0;
'''
[cases.test_btree_split_sparse]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.W = 5
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +W,
&(uint8_t){'a'+(0 % 26)}, 1))) => 0;
lfs_size_t n = 1;
for (lfs_size_t i = 1; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, (i-1)*W+W-1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'a'+((i-1) % 26)}, 1),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +W,
&(uint8_t){'a'+((i-0) % 26)}, 1))) => 0;
n += 1;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == n*W);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < n; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i*W+W-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == W);
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, n*W,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_split_sparse_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.W = 5
defines.SEED = 'range(20)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +W, "_", 1))) => 0;
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
lfs_size_t *sim_weights = malloc(N*sizeof(lfs_size_t));
lfs_size_t sim_size = 1;
memset(sim, 0, N);
memset(sim_weights, 0, N*sizeof(lfs_size_t));
sim[0] = '_';
sim_weights[0] = W;
uint32_t prng = SEED;
for (lfs_size_t i = 1; i < N; i++) {
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % sim_size;
// choose pseudo-random weights
lfs_size_t weight1 = 1 + (TEST_PRNG(&prng) % W);
lfs_size_t weight2 = 1 + (TEST_PRNG(&prng) % W);
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < bid; j++) {
weighted_bid += sim_weights[j];
}
// split btree
lfsr_btree_commit(&lfs, &btree,
weighted_bid+sim_weights[bid]-1, LFSR_RATTRS(
LFSR_RATTR(
LFSR_TAG_GROW, +weight1-sim_weights[bid]),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'a'+(i % 26)}, 1),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +weight2,
&(uint8_t){'A'+(i % 26)}, 1))) => 0;
// add to sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
memmove(&sim_weights[bid+1], &sim_weights[bid],
(sim_size-bid)*sizeof(lfs_size_t));
sim[bid+0] = 'a'+(i % 26);
sim[bid+1] = 'A'+(i % 26);
sim_weights[bid+0] = weight1;
sim_weights[bid+1] = weight2;
sim_size += 1;
// TODO rm
lfs_size_t total_weight = 0;
for (lfs_size_t j = 0; j < sim_size; j++) {
total_weight += sim_weights[j];
}
assert(lfsr_btree_weight(&btree) == total_weight);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, weighted_bid+sim_weights[i]-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
if (!first) {
printf(", ");
}
first = false;
printf("%dw%d=%c", weighted_bid+sim_weights[i]-1,
sim_weights[i], sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
lfs_size_t total_weight = 0;
for (lfs_size_t j = 0; j < sim_size; j++) {
total_weight += sim_weights[j];
}
assert(lfsr_btree_weight(&btree) == total_weight);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, weighted_bid+sim_weights[i]-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, total_weight,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// also test that we can traverse the tree without prior knowledge
bid_ = -1;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => 0;
assert(bid_ == weighted_bid+sim_weights[i]-1);
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &sim[i], 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
'''
# Some specific corner cases
[cases.test_btree_drop]
# this should large enough so only one entry can fit in a block
defines.SIZE = 'BLOCK_SIZE / 4'
defines.SIBLING = [0, 1]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
// force it to split
// the extra push here avoids trying to inline the big entry
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "_", 1))) => 0;
uint8_t buf1[SIZE];
memset(buf1, 'a', SIZE);
uint8_t buf2[SIZE];
memset(buf2, 'b', SIZE);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, buf1, SIZE),
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, buf2, SIZE))) => 0;
// force compaction
lfsr_btree_claim(&btree);
memset(buf2, 'b', SIZE);
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
buf2, SIZE))) => 0;
assert(lfsr_btree_weight(&btree) == 2);
// now remove one entry, since this brings the rbyd down to zero,
// this should force one of the blocks to drop
lfsr_btree_commit(&lfs, &btree, SIBLING, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 1);
// check that our other entry is fine
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buf1, SIZE) => SIZE;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buf1, ((SIBLING) ? "a" : "b"), 1) == 0);
// and check that our pop worked
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_drop_compact]
# this should large enough so only one entry can fit in a block
defines.SIZE = 'BLOCK_SIZE / 4'
defines.SIBLING = [0, 1]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
// force it to split
// the extra push here avoids trying to inline the big entry
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "_", 1))) => 0;
uint8_t buf1[SIZE];
memset(buf1, 'a', SIZE);
uint8_t buf2[SIZE];
memset(buf2, 'b', SIZE);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, buf1, SIZE),
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, buf2, SIZE))) => 0;
// force compaction
lfsr_btree_claim(&btree);
memset(buf2, 'b', SIZE);
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
buf2, SIZE))) => 0;
assert(lfsr_btree_weight(&btree) == 2);
// now remove one entry, since this brings the rbyd down this zero,
// this should force one of the blocks to drop
//
// do this while forcing a compaction
lfsr_btree_claim(&btree);
lfsr_btree_commit(&lfs, &btree, SIBLING, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 1);
// check that our other entry is fine
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buf1, SIZE) => SIZE;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buf1, ((SIBLING) ? "a" : "b"), 1) == 0);
// and check that our pop worked
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_drop_split]
# this should large enough so only one entry can fit in a block
defines.SIZE = 'BLOCK_SIZE / 4'
defines.SIBLING = [0, 1]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
// force it to split
// the extra push here avoids trying to inline the big entry
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "_", 1))) => 0;
uint8_t buf1[SIZE];
memset(buf1, 'a', SIZE);
uint8_t buf2[SIZE];
memset(buf2, 'b', SIZE);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, buf1, SIZE),
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, buf2, SIZE))) => 0;
// force compaction, causing a split, but while we're splitting,
// also remove an entry, bringing the split rbyd down to zero mid split
//
// messy, isn't it? this is why we need an explicit test
//
lfsr_btree_claim(&btree);
lfsr_btree_commit(&lfs, &btree, SIBLING, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 1);
// check that our other entry is fine
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buf1, SIZE) => SIZE;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buf1, ((SIBLING) ? "a" : "b"), 1) == 0);
// and check that our pop worked
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_drop_merge]
# this should large enough so only one entry can fit in a block
defines.SIZE = 'BLOCK_SIZE / 4'
defines.SIBLING = [0, 1]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
// force it to split
// the extra push here avoids trying to inline the big entry
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, "_", 1))) => 0;
uint8_t buf1[SIZE];
memset(buf1, 'a', SIZE);
uint8_t buf2[SIZE];
memset(buf2, 'b', SIZE);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, buf1, SIZE),
LFSR_RATTR_BUF(LFSR_TAG_DATA, +1, buf2, SIZE))) => 0;
// force compaction
lfsr_btree_claim(&btree);
memset(buf2, 'b', SIZE);
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
buf2, SIZE))) => 0;
assert(lfsr_btree_weight(&btree) == 2);
// now make both entries small so they should be merged if either compacts
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
"a", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
"b", 1))) => 0;
// force compaction, while removing one entry, this drops the rbyd
// down to zero while also triggering a merge
lfsr_btree_claim(&btree);
lfsr_btree_commit(&lfs, &btree, SIBLING, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 1);
// check that our other entry is fine
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_lookupnext(&lfs, &btree, 0,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buf1, SIZE) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buf1, ((SIBLING) ? "a" : "b"), 1) == 0);
// and check that our pop worked
lfsr_btree_lookupnext(&lfs, &btree, 1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
# Some more general fuzz testing
[cases.test_btree_general_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.SEED = 'range(100)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
lfs_size_t sim_size = 0;
memset(sim, 0, N);
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random op
uint8_t op = TEST_PRNG(&prng) % 3;
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % (sim_size+1);
if (op == 0 || bid == sim_size) {
// push to btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
// push to sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
sim[bid] = 'a'+(i % 26);
sim_size += 1;
} else if (op == 1) {
// update btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
// update sim
sim[bid] = 'a'+(i % 26);
} else {
// pop from btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
// pop from sim
memmove(&sim[bid], &sim[bid+1], sim_size-(bid+1));
sim_size -= 1;
}
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
if (!first) {
printf(", ");
}
first = false;
printf("%c", sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == sim_size);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, sim_size,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
'''
[cases.test_btree_general_sparse_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.W = 5
defines.SEED = 'range(100)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
lfs_size_t *sim_weights = malloc(N*sizeof(lfs_size_t));
lfs_size_t sim_size = 0;
memset(sim, 0, N);
memset(sim_weights, 0, N*sizeof(lfs_size_t));
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random op
uint8_t op = TEST_PRNG(&prng) % 3;
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % (sim_size+1);
// choose a pseudo-random weight
lfs_size_t weight = 1 + (TEST_PRNG(&prng) % W);
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < bid; j++) {
weighted_bid += sim_weights[j];
}
if (op == 0 || bid == sim_size) {
// push to btree
lfsr_btree_commit(&lfs, &btree, weighted_bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +weight,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
// push to sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
memmove(&sim_weights[bid+1], &sim_weights[bid],
(sim_size-bid)*sizeof(lfs_size_t));
sim[bid] = 'a'+(i % 26);
sim_weights[bid] = weight;
sim_size += 1;
} else if (op == 1) {
// update btree
lfsr_btree_commit(&lfs, &btree,
weighted_bid+sim_weights[bid]-1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
&(uint8_t){'a'+(i % 26)}, 1),
LFSR_RATTR(
LFSR_TAG_GROW, +weight-sim_weights[bid]))) => 0;
// update sim
sim[bid] = 'a'+(i % 26);
sim_weights[bid] = weight;
} else {
// remove from btree
lfsr_btree_commit(&lfs, &btree,
weighted_bid+sim_weights[bid]-1, LFSR_RATTRS(
LFSR_RATTR(
LFSR_TAG_RM, -sim_weights[bid]))) => 0;
// remove from sim
memmove(&sim[bid], &sim[bid+1], sim_size-(bid+1));
memmove(&sim_weights[bid], &sim_weights[bid+1],
(sim_size-(bid+1))*sizeof(lfs_size_t));
sim_size -= 1;
}
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
if (!first) {
printf(", ");
}
first = false;
printf("%dw%d=%c", weighted_bid+sim_weights[i]-1,
sim_weights[i], sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
lfs_size_t total_weight = 0;
for (lfs_size_t j = 0; j < sim_size; j++) {
total_weight += sim_weights[j];
}
assert(lfsr_btree_weight(&btree) == total_weight);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, weighted_bid+sim_weights[i]-1,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, total_weight,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// also test that we can traverse the tree without prior knowledge
bid_ = -1;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => 0;
assert(bid_ == weighted_bid+sim_weights[i]-1);
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == sim_weights[i]);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &sim[i], 1) == 0);
}
lfsr_btree_lookupnext(&lfs, &btree, bid_+1,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
'''
# test key-value btrees
[cases.test_btree_find_zero]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a zero-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 0);
// try to find tags
lfsr_bid_t bid_;
lfs_size_t weight_;
lfsr_btree_namelookup(&lfs, &btree, 0, "aaa", 3,
&bid_, NULL, &weight_, NULL) => LFS_ERR_NOENT;
'''
[cases.test_btree_find_one]
# true or false for if we should use dids vs names
defines.DID = [false, true]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a single-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, "aaa", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "0", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 1);
// try to find tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_namelookup(&lfs, &btree, 0*DID, "aaa", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 0);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "0", 1) == 0);
lfsr_btree_namelookup(&lfs, &btree, 1*DID, "aab", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_LT;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 0);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "0", 1) == 0);
'''
[cases.test_btree_find_two]
# true or false for if we should use dids vs names
defines.DID = [false, true]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a two-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, "aaa", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "0", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "0", 1),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, "aab", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "1", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 2);
// try to find tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_namelookup(&lfs, &btree, 0, "aaa", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 0);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "0", 1) == 0);
lfsr_btree_namelookup(&lfs, &btree, 0, "aab", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 1);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "1", 1) == 0);
lfsr_btree_namelookup(&lfs, &btree, 0, "aac", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_LT;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 1);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "1", 1) == 0);
'''
[cases.test_btree_find_three]
in = 'lfs.c'
# true or false for if we should use dids vs names
defines.DID = [false, true]
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a two-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, "aaa", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "0", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "0", 1),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
1*DID, "aab", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "1", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 1, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "1", 1),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
2*DID, "aac", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "2", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 3);
// try to find tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_namelookup(&lfs, &btree, 0*DID, "aaa", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 0);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "0", 1) == 0);
lfsr_btree_namelookup(&lfs, &btree, 1*DID, "aab", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 1);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "1", 1) == 0);
lfsr_btree_namelookup(&lfs, &btree, 2*DID, "aac", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 2);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "2", 1) == 0);
lfsr_btree_namelookup(&lfs, &btree, 3*DID, "aad", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_LT;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 2);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "2", 1) == 0);
'''
[cases.test_btree_find_three_backwards]
# true or false for if we should use dids vs names
defines.DID = [false, true]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a two-entry tree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, "aaa", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "0", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "1", 1),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
2*DID, "aac", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "2", 1))) => 0;
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "0", 1),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
1*DID, "aab", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "1", 1))) => 0;
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == 3);
// try to find tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
lfsr_btree_namelookup(&lfs, &btree, 0*DID, "aaa", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 0);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "0", 1) == 0);
lfsr_btree_namelookup(&lfs, &btree, 1*DID, "aab", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 1);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "1", 1) == 0);
lfsr_btree_namelookup(&lfs, &btree, 2*DID, "aac", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 2);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "2", 1) == 0);
lfsr_btree_namelookup(&lfs, &btree, 3*DID, "aad", 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_LT;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == 2);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, "2", 1) == 0);
'''
[cases.test_btree_find]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
# true or false for if we should use dids vs names
defines.DID = [false, true]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
char name[3] = {
'a'+((0/26/26) % 26), 'a'+((0/26) % 26), 'a'+(0 % 26)
};
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, name, 3),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(0 % 10)}, 1))) => 0;
lfs_size_t n = 1;
for (lfs_size_t i = 1; i < N; i++) {
char name[3] = {
'a'+((i/26/26) % 26), 'a'+((i/26) % 26), 'a'+(i % 26)
};
lfsr_btree_commit(&lfs, &btree, i-1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+((i-1) % 10)}, 1),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
i*DID, name, 3),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+((i-0) % 10)}, 1))) => 0;
n += 1;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == n);
// try to find tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < n; i++) {
char name[3] = {
'a'+((i/26/26) % 26), 'a'+((i/26) % 26), 'a'+(i % 26)
};
lfsr_btree_namelookup(&lfs, &btree, i*DID, name, 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == i);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &(uint8_t){'0'+(i % 10)}, 1) == 0);
}
'''
[cases.test_btree_find_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.SEED = 'range(20)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, "___", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "_", 1))) => 0;
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
char (*sim_names)[3] = malloc(N*3);
lfs_size_t sim_size = 1;
memset(sim, 0, N);
memset(sim_names, 0, N*3);
sim[0] = '_';
memcpy(&sim_names[0], "___", 3);
uint32_t prng = SEED;
for (lfs_size_t i = 1; i < N; i++) {
// choose a pseudo-random name
lfs_size_t x = TEST_PRNG(&prng) % (26*26*26);
char name[3] = {
'a'+((x/26/26) % 26), 'a'+((x/26) % 26), 'a'+(x % 26)
};
// find where to split
lfs_size_t bid = 0;
while (bid+1 < sim_size && memcmp(sim_names[bid+1], name, 3) <= 0) {
bid += 1;
}
// just skip exact matches for now
if (memcmp(sim_names[bid], name, 3) == 0) {
continue;
}
// split btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(i % 10)}, 1),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, name, 3),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(i % 10)}, 1))) => 0;
// split sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
memmove(&sim_names[bid+1], &sim_names[bid], (sim_size-bid)*3);
sim[bid+0] = '0'+(i % 10);
sim[bid+1] = '0'+(i % 10);
memcpy(&sim_names[bid+1], name, 3);
sim_size += 1;
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
if (!first) {
printf(", ");
}
first = false;
printf("%.3s=%c", sim_names[i], sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == sim_size);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
lfsr_btree_namelookup(&lfs, &btree, 0, sim_names[i], 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == i);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// clean up sim
free(sim);
free(sim_names);
lfs_deinit(&lfs) => 0;
'''
[cases.test_btree_find_sparse]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.W = 5
# true or false for if we should use dids vs names
defines.DID = [false, true]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
char name[3] = {
'a'+((0/26/26) % 26), 'a'+((0/26) % 26), 'a'+(0 % 26)
};
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +W,
0, name, 3),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(0 % 10)}, 1))) => 0;
lfs_size_t n = 1;
for (lfs_size_t i = 1; i < N; i++) {
char name[3] = {
'a'+((i/26/26) % 26), 'a'+((i/26) % 26), 'a'+(i % 26)
};
lfsr_btree_commit(&lfs, &btree, (i-1)*W+W-1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+((i-1) % 10)}, 1),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +W,
i*DID, name, 3),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+((i-0) % 10)}, 1))) => 0;
n += 1;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == n*W);
// try to find tags
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < n; i++) {
char name[3] = {
'a'+((i/26/26) % 26), 'a'+((i/26) % 26), 'a'+(i % 26)
};
lfsr_btree_namelookup(&lfs, &btree, i*DID, name, 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == i*W+W-1);
assert(weight_ == W);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &(uint8_t){'0'+(i % 10)}, 1) == 0);
}
'''
[cases.test_btree_find_sparse_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.W = 5
defines.SEED = 'range(20)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +W,
0, "___", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "_", 1))) => 0;
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
char (*sim_names)[3] = malloc(N*3);
lfs_size_t *sim_weights = malloc(N*sizeof(lfs_size_t));
lfs_size_t sim_size = 1;
memset(sim, 0, N);
memset(sim_names, 0, N*3);
memset(sim_weights, 0, N*sizeof(lfs_size_t));
sim[0] = '_';
memcpy(&sim_names[0], "___", 3);
sim_weights[0] = W;
uint32_t prng = SEED;
for (lfs_size_t i = 1; i < N; i++) {
// choose a pseudo-random name
lfs_size_t x = TEST_PRNG(&prng) % (26*26*26);
char name[3] = {
'a'+((x/26/26) % 26), 'a'+((x/26) % 26), 'a'+(x % 26)
};
// choose pseudo-random weights
lfs_size_t weight1 = 1 + (TEST_PRNG(&prng) % W);
lfs_size_t weight2 = 1 + (TEST_PRNG(&prng) % W);
// find where to split
lfs_size_t bid = 0;
while (bid+1 < sim_size && memcmp(sim_names[bid+1], name, 3) <= 0) {
bid += 1;
}
// just skip exact matches for now
if (memcmp(sim_names[bid], name, 3) == 0) {
continue;
}
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < bid; j++) {
weighted_bid += sim_weights[j];
}
// split btree
lfsr_btree_commit(&lfs, &btree,
weighted_bid+sim_weights[bid]-1, LFSR_RATTRS(
LFSR_RATTR(
LFSR_TAG_GROW, +weight1-sim_weights[bid]),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(i % 10)}, 1),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +weight2,
0, name, 3),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(i % 10)}, 1))) => 0;
// split sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
memmove(&sim_names[bid+1], &sim_names[bid], (sim_size-bid)*3);
memmove(&sim_weights[bid+1], &sim_weights[bid],
(sim_size-bid)*sizeof(lfs_size_t));
sim[bid+0] = '0'+(i % 10);
sim[bid+1] = '0'+(i % 10);
memcpy(&sim_names[bid+1], name, 3);
sim_weights[bid+0] = weight1;
sim_weights[bid+1] = weight2;
sim_size += 1;
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
if (!first) {
printf(", ");
}
first = false;
printf("%.3sid%dw%d=%c",
sim_names[i],
weighted_bid+sim_weights[i]-1,
sim_weights[i],
sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
lfs_size_t total_weight = 0;
for (lfs_size_t j = 0; j < sim_size; j++) {
total_weight += sim_weights[j];
}
assert(lfsr_btree_weight(&btree) == total_weight);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_namelookup(&lfs, &btree, 0, sim_names[i], 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == weighted_bid+sim_weights[i]-1);
assert(weight_ == sim_weights[i]);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// clean up sim
free(sim);
free(sim_names);
free(sim_weights);
lfs_deinit(&lfs) => 0;
'''
# make sure we test finds with other operations
[cases.test_btree_find_general_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.SEED = 'range(100)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, "___", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "_", 1))) => 0;
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
char (*sim_names)[3] = malloc(N*3);
lfs_size_t sim_size = 1;
memset(sim, 0, N);
memset(sim_names, 0, N*3);
sim[0] = '_';
memcpy(&sim_names[0], "___", 3);
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random op
uint8_t op = TEST_PRNG(&prng) % 3;
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % ((sim_size == 0) ? 1 : sim_size);
// choose a pseudo-random name
lfs_size_t x = TEST_PRNG(&prng) % (26*26*26);
char name[3] = {
'a'+((x/26/26) % 26), 'a'+((x/26) % 26), 'a'+(x % 26)
};
// don't let sim drop below one element
if (op == 0 || sim_size <= 1) {
// find where to split
lfs_size_t bid = 0;
while (bid < sim_size && memcmp(name, sim_names[bid], 3) > 0) {
bid += 1;
}
// just skip exact matches for now
if (memcmp(name, sim_names[bid], 3) == 0) {
continue;
}
// split btree
//
// note! all name updates _must_ be via splits (except for
// the first one)
//
// This is because our btrees contain vestigial names, i.e.
// our inner nodes may contain names no longer in the tree.
// This simplifies lfsr_btree_commit_, but means
// insert-before-bid+1 is _not_ the same as insert-after-bid
// when named btrees are involved. If you try this it _will
// not_ work and if try to make it work you _will_ cry:
//
// .-----f-----. insert-after-d .-------f-----.
// .-b--. .--j-. => .-b---. .--j-.
// | .-. .-. | | .---. .-. |
// a c d h i k a c d e h i k
// ^
// insert-before-h
// => .-----f-------.
// .-b--. .---j-.
// | .-. .---. |
// a c d g h i k
// ^
lfsr_bid_t split_bid;
lfs_scmp_t cmp = lfsr_btree_namelookup(&lfs, &btree,
0, name, 3,
&split_bid, NULL, NULL, NULL);
assert(cmp >= 0);
assert(cmp != LFS_CMP_EQ);
if (cmp > LFS_CMP_EQ) {
lfsr_btree_commit(&lfs, &btree, split_bid, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, name, 3),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(i % 10)}, 1))) => 0;
} else {
lfsr_btree_commit(&lfs, &btree, split_bid, LFSR_RATTRS(
// yes, we need this noop, see above
LFSR_RATTR_NOOP(),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +1,
0, name, 3),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(i % 10)}, 1))) => 0;
}
// split sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
memmove(&sim_names[bid+1], &sim_names[bid], (sim_size-bid)*3);
sim[bid] = '0'+(i % 10);
memcpy(&sim_names[bid], name, 3);
sim_size += 1;
} else if (op == 1) {
// update btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(i % 10)}, 1))) => 0;
// update sim
sim[bid] = '0'+(i % 10);
} else {
// pop from btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1))) => 0;
// pop from sim
memmove(&sim[bid], &sim[bid+1], sim_size-(bid+1));
memmove(&sim_names[bid], &sim_names[bid+1], (sim_size-(bid+1))*3);
sim_size -= 1;
}
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
if (!first) {
printf(", ");
}
first = false;
printf("%.3s=%c", sim_names[i], sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == sim_size);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
lfsr_btree_namelookup(&lfs, &btree, 0, sim_names[i], 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == i);
assert(weight_ == 1);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// clean up sim
free(sim);
free(sim_names);
'''
[cases.test_btree_find_general_sparse_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
defines.W = 5
defines.SEED = 'range(100)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfsr_btree_commit(&lfs, &btree, 0, LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +W,
0, "___", 3),
LFSR_RATTR_BUF(LFSR_TAG_DATA, 0, "_", 1))) => 0;
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
char (*sim_names)[3] = malloc(N*3);
lfs_size_t *sim_weights = malloc(N*sizeof(lfs_size_t));
lfs_size_t sim_size = 1;
memset(sim, 0, N);
memset(sim_names, 0, N*3);
memset(sim_weights, 0, N*sizeof(lfs_size_t));
sim[0] = '_';
memcpy(&sim_names[0], "___", 3);
sim_weights[0] = W;
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random op
uint8_t op = TEST_PRNG(&prng) % 3;
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % ((sim_size == 0) ? 1 : sim_size);
// choose a pseudo-random name
lfs_size_t x = TEST_PRNG(&prng) % (26*26*26);
char name[3] = {
'a'+((x/26/26) % 26), 'a'+((x/26) % 26), 'a'+(x % 26)
};
// choose a pseudo-random weight
lfs_size_t weight = 1 + (TEST_PRNG(&prng) % W);
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < bid; j++) {
weighted_bid += sim_weights[j];
}
// don't let sim drop below one element
if (op == 0 || sim_size <= 1) {
// find where to split
lfs_size_t bid = 0;
while (bid < sim_size && memcmp(name, sim_names[bid], 3) > 0) {
bid += 1;
}
// just skip exact matches for now
if (memcmp(name, sim_names[bid], 3) == 0) {
continue;
}
// split btree
//
// note! all name updates _must_ be via splits (except for
// the first one)
//
// This is because our btrees contain vestigial names, i.e.
// our inner nodes may contain names no longer in the tree.
// This simplifies lfsr_btree_commit_, but means
// insert-before-bid+1 is _not_ the same as insert-after-bid
// when named btrees are involved. If you try this it _will
// not_ work and if try to make it work you _will_ cry:
//
//
// .-----f-----. insert-after-d .-------f-----.
// .-b--. .--j-. => .-b---. .--j-.
// | .-. .-. | | .---. .-. |
// a c d h i k a c d e h i k
// ^
// insert-before-h
// => .-----f-------.
// .-b--. .---j-.
// | .-. .---. |
// a c d g h i k
// ^
lfsr_bid_t split_bid;
lfsr_bid_t split_weight;
lfs_scmp_t cmp = lfsr_btree_namelookup(&lfs, &btree,
0, name, 3,
&split_bid, NULL, &split_weight, NULL);
assert(cmp >= 0);
assert(cmp != LFS_CMP_EQ);
if (cmp > LFS_CMP_EQ) {
lfsr_btree_commit(&lfs, &btree,
split_bid-(split_weight-1), LFSR_RATTRS(
LFSR_RATTR_NAME(
LFSR_TAG_REG, +weight,
0, name, 3),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(i % 10)}, 1))) => 0;
} else {
lfsr_btree_commit(&lfs, &btree, split_bid, LFSR_RATTRS(
// yes, we need this noop, see above
LFSR_RATTR_NOOP(),
LFSR_RATTR_NAME(
LFSR_TAG_REG, +weight,
0, name, 3),
LFSR_RATTR_BUF(
LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(i % 10)}, 1))) => 0;
}
// split sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
memmove(&sim_names[bid+1], &sim_names[bid], (sim_size-bid)*3);
memmove(&sim_weights[bid+1], &sim_weights[bid],
(sim_size-bid)*sizeof(lfs_size_t));
sim[bid] = '0'+(i % 10);
memcpy(&sim_names[bid], name, 3);
sim_weights[bid] = weight;
sim_size += 1;
} else if (op == 1) {
// update btree
lfsr_btree_commit(&lfs, &btree,
weighted_bid+sim_weights[bid]-1, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_MASK8 | LFSR_TAG_DATA, 0,
&(uint8_t){'0'+(i % 10)}, 1),
LFSR_RATTR(
LFSR_TAG_GROW, +weight-sim_weights[bid]))) => 0;
// update sim
sim[bid] = '0'+(i % 10);
sim_weights[bid] = weight;
} else {
// pop from btree
lfsr_btree_commit(&lfs, &btree,
weighted_bid+sim_weights[bid]-1, LFSR_RATTRS(
LFSR_RATTR(
LFSR_TAG_RM, -sim_weights[bid]))) => 0;
// pop from sim
memmove(&sim[bid], &sim[bid+1], sim_size-(bid+1));
memmove(&sim_names[bid], &sim_names[bid+1], (sim_size-(bid+1))*3);
memmove(&sim_weights[bid], &sim_weights[bid+1],
(sim_size-(bid+1))*sizeof(lfs_size_t));
sim_size -= 1;
}
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
if (!first) {
printf(", ");
}
first = false;
printf("%.3sid%dw%d=%c",
sim_names[i],
weighted_bid+sim_weights[i]-1,
sim_weights[i],
sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
lfs_size_t total_weight = 0;
for (lfs_size_t j = 0; j < sim_size; j++) {
total_weight += sim_weights[j];
}
assert(lfsr_btree_weight(&btree) == total_weight);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
// calculate actual bid in btree space
lfs_size_t weighted_bid = 0;
for (lfs_size_t j = 0; j < i; j++) {
weighted_bid += sim_weights[j];
}
lfsr_btree_namelookup(&lfs, &btree, 0, sim_names[i], 3,
&bid_, NULL, &weight_, NULL) => LFS_CMP_EQ;
lfsr_btree_lookup(&lfs, &btree, bid_, LFSR_TAG_MASK8 | LFSR_TAG_STRUCT,
&tag_, &data_) => 0;
assert(tag_ == LFSR_TAG_DATA);
assert(bid_ == weighted_bid+sim_weights[i]-1);
assert(weight_ == sim_weights[i]);
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// clean up sim
free(sim);
free(sim_names);
free(sim_weights);
'''
## B-tree traversal tests ##
# some simple btree traversals
[cases.test_btree_traversal]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a tree with N elements
lfsr_btree_t btree;
lfsr_btree_init(&btree);
lfs_size_t n = 0;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_btree_commit(&lfs, &btree, i, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
n += 1;
}
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == n);
// check that the elements are in the tree
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < n; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, n,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// test that we can traverse the tree, keeping track of all blocks we see
uint8_t *seen = malloc((BLOCK_COUNT+7)/8);
memset(seen, 0, (BLOCK_COUNT+7)/8);
lfsr_btraversal_t bt;
lfsr_btraversal_init(&bt);
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i <= 2*N);
lfsr_bid_t bid;
lfsr_tag_t tag;
lfsr_data_t data;
int err = lfsr_btree_traverse(&lfs, &btree, &bt,
&bid, &tag, &data);
assert(!err || err == LFS_ERR_NOENT);
if (err == LFS_ERR_NOENT) {
break;
}
if (tag == LFSR_TAG_BRANCH) {
lfsr_rbyd_t *rbyd = (lfsr_rbyd_t*)data.u.buffer;
printf("traversal: %d 0x%x btree 0x%x.%x\n",
bid,
tag,
rbyd->blocks[0], rbyd->trunk);
// keep track of seen blocks
seen[rbyd->blocks[0] / 8] |= 1 << (rbyd->blocks[0] % 8);
} else if (tag == LFSR_TAG_DATA) {
printf("traversal: %d 0x%x data %d\n",
bid,
tag,
lfsr_data_size(data));
} else {
// well this shouldn't happen
printf("traversal: %d 0x%x\n",
bid,
tag);
assert(false);
}
}
// if traversal worked, we should be able to clobber all other blocks
uint8_t clobber_buf[BLOCK_SIZE];
memset(clobber_buf, 0xcc, BLOCK_SIZE);
for (lfs_block_t block = 0; block < BLOCK_COUNT; block++) {
if (!(seen[block / 8] & (1 << (block % 8)))) {
CFG->erase(CFG, block) => 0;
CFG->prog(CFG, block, 0, clobber_buf, BLOCK_SIZE) => 0;
}
}
free(seen);
// and the tree should still work
// check that the elements are in the tree
for (lfs_size_t i = 0; i < n; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &(uint8_t){'a'+(i % 26)}, 1) == 0);
}
// and check that we can't lookup elements that aren't in the tree
lfsr_btree_lookupnext(&lfs, &btree, n,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
'''
[cases.test_btree_traversal_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
defines.SEED = 'range(20)'
fuzz = 'SEED'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfs_init(&lfs, LFS_M_RDWR, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.window = 2;
lfs.lookahead.off = 0;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count-2);
lfs_alloc_ckpoint(&lfs);
// create a btree
lfsr_btree_t btree;
lfsr_btree_init(&btree);
// set up a simulation to compare against
//
// fun fact this is slower than our actual tree! unfun fact this is
// starting to be a problem...
char *sim = malloc(N);
lfs_size_t sim_size = 0;
memset(sim, 0, N);
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random bid
lfs_size_t bid = TEST_PRNG(&prng) % (sim_size+1);
// add to btree
lfsr_btree_commit(&lfs, &btree, bid, LFSR_RATTRS(
LFSR_RATTR_BUF(
LFSR_TAG_DATA, +1,
&(uint8_t){'a'+(i % 26)}, 1))) => 0;
// add to sim
memmove(&sim[bid+1], &sim[bid], sim_size-bid);
sim[bid] = 'a'+(i % 26);
sim_size += 1;
}
// check that btree matches sim
printf("expd: [");
bool first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
if (!first) {
printf(", ");
}
first = false;
printf("%c", sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == sim_size);
uint8_t buffer[4];
lfsr_bid_t bid_;
lfsr_tag_t tag_;
lfs_size_t weight_;
lfsr_data_t data_;
for (lfs_size_t i = 0; i < sim_size; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, sim_size,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// test that we can traverse the tree, keeping track of all blocks
// we see
uint8_t *seen = malloc((BLOCK_COUNT+7)/8);
memset(seen, 0, (BLOCK_COUNT+7)/8);
lfsr_btraversal_t bt;
lfsr_btraversal_init(&bt);
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i <= 2*N);
lfsr_bid_t bid;
lfsr_tag_t tag;
lfsr_data_t data;
int err = lfsr_btree_traverse(&lfs, &btree, &bt,
&bid, &tag, &data);
assert(!err || err == LFS_ERR_NOENT);
if (err == LFS_ERR_NOENT) {
break;
}
if (tag == LFSR_TAG_BRANCH) {
lfsr_rbyd_t *rbyd = (lfsr_rbyd_t*)data.u.buffer;
printf("traversal: %d 0x%x btree 0x%x.%x\n",
bid,
tag,
rbyd->blocks[0], rbyd->trunk);
// keep track of seen blocks
seen[rbyd->blocks[0] / 8] |= 1 << (rbyd->blocks[0] % 8);
} else if (tag == LFSR_TAG_DATA) {
printf("traversal: %d 0x%x data %d\n",
bid,
tag,
lfsr_data_size(data));
} else {
// well this shouldn't happen
printf("traversal: %d 0x%x\n",
bid,
tag);
assert(false);
}
}
// if traversal worked, we should be able to clobber all other blocks
uint8_t clobber_buf[BLOCK_SIZE];
memset(clobber_buf, 0xcc, BLOCK_SIZE);
for (lfs_block_t block = 0; block < BLOCK_COUNT; block++) {
if (!(seen[block / 8] & (1 << (block % 8)))) {
CFG->erase(CFG, block) => 0;
CFG->prog(CFG, block, 0, clobber_buf, BLOCK_SIZE) => 0;
}
}
free(seen);
// and the tree should still work
// check that btree matches sim
printf("expd: [");
first = true;
for (lfs_size_t i = 0; i < sim_size; i++) {
if (!first) {
printf(", ");
}
first = false;
printf("%c", sim[i]);
}
printf("]\n");
printf("btree: w%d 0x%x.%x\n",
lfsr_btree_weight(&btree),
lfsr_btree_block(&btree),
lfsr_btree_trunk(&btree));
assert(lfsr_btree_weight(&btree) == sim_size);
for (lfs_size_t i = 0; i < sim_size; i++) {
lfsr_btree_lookupnext(&lfs, &btree, i,
&bid_, &tag_, &weight_, &data_) => 0;
lfsr_data_read(&lfs, &data_, buffer, 4) => 1;
assert(tag_ == LFSR_TAG_DATA);
assert(weight_ == 1);
assert(memcmp(buffer, &sim[i], 1) == 0);
}
// and no extra elements
lfsr_btree_lookupnext(&lfs, &btree, sim_size,
&bid_, &tag_, &weight_, &data_) => LFS_ERR_NOENT;
// clean up sim
free(sim);
lfs_deinit(&lfs) => 0;
'''
View Git Blame Copy Permalink