Github/littlefs
1
Fork 0
Code Actions Activity
Files
eb6c361dfa6ee2179679c8cf9098fb6dc74c930d
littlefs/tests/test_mtree.toml
T
Christopher Haster eb6c361dfa Adopted lazy orphaned mdir drops 
This ended up being much less of a simplification than I hoped it would.

It's still easier/more efficient to revert to a relocation in most cases
when dropping in an mdir split, and the small gain from simplifying how
drops/commits interact is overshadowed by the code duplication necessary
to separate lfsr_mdir_drop out from lfsr_mdir_commit:

            code          stack
  before:  30952           2528
  after:   31280 (+1.1%)   2648 (+4.7%)

Still, this does at least simplify the logical corner cases (we don't
need to abort commits when droppable anymore), and lfsr_mdir_drop is
ultimately necessary for supporting lazy file creation.

Also having a fix-orphans step during mount allows other littlefs
implementations the option to create orphanned mdirs without compat
issues. So this ends up the more flexible approach.

It _might_ be worth having both eager mdir drops and an explicit
lfsr_mdir_drop for lazy file creation in the future, but I doubt this
will end up worth the code duplication...

---

Oh right, I forgot to actually describe this change.

This trades eager mdir drops:

1. Drop mdirs from the mtree immediately as soon as their weight goes
   to zero.

For lazy mdir drops:

1. Drop mdirs from the mtree in a second commit.
2. Scan and drop orphaned mdirs on the first write after mount.

This sounds very similar to the previous "deorphan" scan, which risked
an extreme performance cost during mount, but it should be noted this
orphan scan only needs to touch every mdir once. This makes it no worse
than the overhead of actually mounting the filesystem.

We can also keep an eye out for orphaned mdirs when we mount, so no
extra scan is needed unless there was an unlucky powerloss.

Eager mdir dropping sounds simpler, but thanks to deferred commits
introduces some subtle complexity around aborting commits that would
drop an mdir to zero. Remember commits are viewable on-disk as soon as a
commit completes.

In _theory_, lazy mdir drops simplify the logic around committing to
mdirs.

Though the real kicker is that lazy mdir drops are required for lazy file
creation.

The current idea for lazy file creation involves tracking mid-less
opened-but-not-yet-created files. These files can have bshrubs, so they
need space on an mdir somewhere. But they aren't actually created yet,
so they don't have an mid.

This is fine (though it's probably going to be tricky) as long as we
allocate an mid on file sync, but there is always a risk of losing power
with mdirs that contain only RAM-backed files. Fortunately, no-mids
means no orphaned files, but it does mean orphaned mdirs with no synced
contents.

Long story short, lazy mdir drops are currently a necessary evil, and
logical simplification, that unfortunately comes with some cost.
2023-12-06 22:23:28 -06:00

4516 lines
145 KiB
TOML
Raw Blame History

# Test the high-level metadata tree in the core of littlefs
after = ['test_rbyd', 'test_btree']
# maximize lookahead buffer, we don't actually gc so we only get one pass
# of the disk for these tests
defines.LOOKAHEAD_SIZE = 'BLOCK_COUNT / 8'
# some helper functions
in = 'lfs.c'
code = '''
static lfs_ssize_t lfsr_mdir_get(lfs_t *lfs, const lfsr_mdir_t *mdir,
lfs_ssize_t rid, lfsr_tag_t tag, void *buffer, lfs_size_t size) {
lfsr_data_t data;
int err = lfsr_mdir_lookup(lfs, mdir, rid, tag, NULL, &data);
if (err) {
return err;
}
return lfsr_data_read(lfs, &data, buffer, size);
}
'''
# test a single mroot
[cases.test_mtree_mroot]
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfsr_unmount(&lfs) => 0;
'''
# test a single mroot with attributes
[cases.test_mtree_mroot_attrs]
defines.N = [1, 3]
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
for (lfs_size_t i = 0; i < N; i++) {
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(i), 0, BUF(&alphas[i % 26], 1)))) => 0;
}
for (lfs_size_t i = 0; i < N; i++) {
uint8_t buffer[1];
lfsr_mdir_get(&lfs, &lfs.mroot,
-1, LFSR_TAG_UATTR(i), buffer, 1) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
}
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
for (lfs_size_t i = 0; i < N; i++) {
uint8_t buffer[1];
lfsr_mdir_get(&lfs, &lfs.mroot,
-1, LFSR_TAG_UATTR(i), buffer, 1) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
}
lfsr_unmount(&lfs) => 0;
'''
# test a single mroot with forced compaction
[cases.test_mtree_mroot_compact]
defines.N = [1, 3]
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
for (lfs_size_t i = 0; i < N; i++) {
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(i), 0, BUF(&alphas[i % 26], 1)))) => 0;
}
for (lfs_size_t i = 0; i < N; i++) {
uint8_t buffer[1];
lfsr_mdir_get(&lfs, &lfs.mroot,
-1, LFSR_TAG_UATTR(i), buffer, 1) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
}
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
for (lfs_size_t i = 0; i < N; i++) {
uint8_t buffer[1];
lfsr_mdir_get(&lfs, &lfs.mroot,
-1, LFSR_TAG_UATTR(i), buffer, 1) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
}
lfsr_unmount(&lfs) => 0;
'''
# test a single mroot with many commits
[cases.test_mtree_mroot_many_commits]
defines.N = [5, 5000]
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
for (lfs_size_t i = 0; i < N; i++) {
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(&alphas[i % 26], 1)))) => 0;
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &lfs.mroot,
-1, LFSR_TAG_UATTR(1), buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
}
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &lfs.mroot,
-1, LFSR_TAG_UATTR(1), buffer, 4) => 1;
assert(memcmp(buffer, &alphas[(N-1) % 26], 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
lfsr_mdir_get(&lfs, &lfs.mroot,
-1, LFSR_TAG_UATTR(1), buffer, 4) => 1;
assert(memcmp(buffer, &alphas[(N-1) % 26], 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
## Splitting operations ##
# specific split corner cases
[cases.test_mtree_uninline]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// prepare mroot with a large attr so the next entry can not fit
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// create a large entry that needs to be uninlined (but not split!)
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_uninline_split]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create 2 large entries that needs to be uninlined and split
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our entries are still in the mtree
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_split]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// now add another large entry to the mdir, forcing a split
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mdir to compact
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
// assert mdir was split correctly
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was split correctly
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
# try creating a range of entries that may or may not split our mtree
[cases.test_mtree_split_many]
defines.N = [5, 10, 20, 40, 80, 160, 320]
defines.FORCE_COMPACTION = [false, true]
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create entries
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs,
lfs_smax32(
lfsr_mtree_weight(&lfs) - lfsr_mweight(&lfs),
0),
&mdir) => 0;
for (lfs_size_t i = 0; i < N; i++) {
// force a compaction?
if (FORCE_COMPACTION) {
mdir.rbyd.eoff = -1;
lfs.mroot.rbyd.eoff = -1;
}
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1,
BUF(&alphas[i % 26], 1)))) => 0;
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
mdir.mid += 1;
}
// try looking up each entry
lfs_size_t i = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
i += 1;
}
}
assert(i == N);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// try looking up each entry
i = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
i += 1;
}
}
assert(i == N);
lfsr_unmount(&lfs) => 0;
'''
# create random entries
[cases.test_mtree_split_fuzz]
defines.N = [5, 10, 20, 40, 80, 160]
defines.FORCE_COMPACTION = [false, true]
defines.SEED = 'range(100)'
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// at least keep track of the number of entries we expect
lfs_size_t count = 0;
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random mid
lfs_ssize_t mid = (lfs_ssize_t)(
TEST_PRNG(&prng) % lfs_max32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs)));
// fetch mdir
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
// limit our mid to our mdir's weight
mdir.mid = lfsr_mdir_bid(&lfs, &mdir)-(lfsr_mweight(&lfs)-1)
+ (mdir.mid % (mdir.rbyd.weight+1));
// force a compaction?
if (FORCE_COMPACTION) {
mdir.rbyd.eoff = -1;
lfs.mroot.rbyd.eoff = -1;
}
// add to rbyd, potentially splitting the mdir
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1,
BUF(&alphas[i % 26], 1)))) => 0;
// make sure we can look up the new entry
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
count += 1;
}
// try looking up each entry
lfs_size_t count_ = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
count_ += 1;
}
}
// the mtree is a bit difficult to simulate, but we can at least test
// we ended up with the right number of entries
assert(count_ == count);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// try looking up each entry
count_ = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
count_ += 1;
}
}
// the mtree is a bit difficult to simulate, but we can at least test
// we ended up with the right number of entries
assert(count_ == count);
lfsr_unmount(&lfs) => 0;
'''
## Dropping operations ##
# specific drop corner cases
[cases.test_mtree_drop]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// remove the entry, forcing the mdir to be dropped
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_drop_compact]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// remove the entry, forcing the mdir to be dropped
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
// force mdir to compact while we're removing
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_drop_uninline]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
// remove the entry as we compact, forcing the mdir to be dropped
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_drop_uninline_split_l]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an mdir that needs to be split
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
// remove the left entry as we compact, forcing the left
// mdir to be dropped
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that one entry is still in the mtree
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that one entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_drop_uninline_split_r]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an mdir that needs to be split
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
// remove the right entry as we compact, forcing the right mdir
// to be dropped
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, RM, -1, NULL()))) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that one entry is still in the mtree
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that one entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_drop_uninline_split_both]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an mdir that needs to be split
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
// remove both entries as we compact, forcing both mdirs to be dropped
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()),
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_drop_split_l]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// now add another large entry to the mdir, forcing a split
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mdir to compact
mdir.rbyd.eoff = -1;
// remove the left entry as we compact, forcing the left
// mdir to be dropped
mdir.mid = 0*lfsr_mweight(&lfs)+0;
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, RM, -1, NULL()))) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that one entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that one entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_drop_split_r]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// now add another large entry to the mdir, forcing a split
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mdir to compact
mdir.rbyd.eoff = -1;
// remove the right entry as we compact, forcing the right
// mdir to be dropped
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(1, RM, -1, NULL()))) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that one entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that one entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_drop_split_both]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// now add another large entry to the mdir, forcing a split
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mdir to compact
mdir.rbyd.eoff = -1;
// remove both entries as we compact, forcing both mdirs to be dropped
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()),
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
# try creating an mtree and then dropping mdirs
[cases.test_mtree_drop_many]
defines.N = [5, 10, 20, 40, 80, 160, 320]
# note we never drop all the way to zero, our mtree does not support this
defines.REMAINING = [20, 5, 1]
if = 'N > REMAINING'
defines.FORCE_COMPACTION = [false, true]
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create entries
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs,
lfs_smax32(
lfsr_mtree_weight(&lfs) - lfsr_mweight(&lfs),
0),
&mdir) => 0;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1,
BUF(&alphas[i % 26], 1)))) => 0;
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
mdir.mid += 1;
}
// remove entries
for (lfs_size_t i = 0; i < N - REMAINING; i++) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs,
0*lfsr_mweight(&lfs)+0, &mdir) => 0;
// drop should make sure we never have empty mdirs
assert(lfsr_mtree_ismptr(&lfs) || mdir.rbyd.weight > 0);
// force a compaction?
if (FORCE_COMPACTION) {
mdir.rbyd.eoff = -1;
lfs.mroot.rbyd.eoff = -1;
}
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
}
// try looking up each entry
lfs_size_t i = N - REMAINING;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
i += 1;
}
}
assert(i == N);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// try looking up each entry
i = N - REMAINING;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
i += 1;
}
}
assert(i == N);
lfsr_unmount(&lfs) => 0;
'''
# this one has some pretty nasty corner cases
[cases.test_mtree_repeated_drop]
defines.N = [5, 10, 20, 40]
# note we never drop all the way to zero, our mtree does not support this
defines.REMAINING = 1
defines.FORCE_COMPACTION = [false, true]
defines.CYCLES = 10
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
for (lfs_size_t cycle = 0; cycle < CYCLES; cycle++) {
// create entries, note we may have leftovers from the previous cycle
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs,
lfs_smax32(
lfsr_mtree_weight(&lfs) - lfsr_mweight(&lfs),
0),
&mdir) => 0;
for (lfs_size_t i = 0; i < (cycle == 0 ? N : N-1); i++) {
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1,
BUF(&alphas[i % 26], 1)))) => 0;
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
mdir.mid += 1;
}
// try looking up each entry
lfs_size_t i = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
i += 1;
}
}
assert(i == N);
// remove entries
for (lfs_size_t i = 0; i < N-1; i++) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs,
0*lfsr_mweight(&lfs)+0, &mdir) => 0;
// drop should make sure we never have empty mdirs
assert(lfsr_mtree_ismptr(&lfs) || mdir.rbyd.weight > 0);
// force a compaction?
if (FORCE_COMPACTION) {
mdir.rbyd.eoff = -1;
lfs.mroot.rbyd.eoff = -1;
}
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
}
// try looking up each entry
i = N - REMAINING;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
i += 1;
}
}
assert(i == N);
}
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// try looking up each entry
lfs_size_t i = N - REMAINING;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
i += 1;
}
}
assert(i == N);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_drop_fuzz]
defines.N = [5, 10, 20, 40, 80, 160]
defines.FORCE_COMPACTION = [false, true]
defines.SEED = 'range(100)'
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// at least keep track of the number of entries we expect
lfs_size_t count = 0;
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random mid
lfs_ssize_t mid = (lfs_ssize_t)(
TEST_PRNG(&prng) % lfs_max32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs)));
// fetch mdir
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
// limit our mid to our mdir's weight
mdir.mid = lfsr_mdir_bid(&lfs, &mdir)-(lfsr_mweight(&lfs)-1)
+ (mdir.mid % (mdir.rbyd.weight+1));
// choose to create or delete, note we make sure to never delete to zero
uint8_t op = (lfsr_mdir_rid(&lfs, &mdir) == mdir.rbyd.weight
|| (lfsr_mdir_rid(&lfs, &mdir) == mdir.rbyd.weight-1
&& lfsr_mtree_weight(&lfs) == lfsr_mweight(&lfs))
? 0
: TEST_PRNG(&prng) % 2);
// force a compaction?
if (FORCE_COMPACTION) {
mdir.rbyd.eoff = -1;
lfs.mroot.rbyd.eoff = -1;
}
// create
if (op == 0) {
// add to rbyd, potentially splitting the mdir
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1,
BUF(&alphas[i % 26], 1)))) => 0;
// make sure we can look up the new entry
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
count += 1;
// delete
} else {
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, RM, -1, NULL()))) => 0;
count -= 1;
}
}
// try looking up each entry
lfs_size_t count_ = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
// drop should make sure we never have empty mdirs
assert(lfsr_mtree_ismptr(&lfs) || mdir.rbyd.weight > 0);
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
count_ += 1;
}
}
// the mtree is a bit difficult to simulate, but we can at least test
// we ended up with the right number of entries
assert(count_ == count);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// try looking up each entry
count_ = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
// drop should make sure we never have empty mdirs
assert(lfsr_mtree_ismptr(&lfs) || mdir.rbyd.weight > 0);
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
count_ += 1;
}
}
// the mtree is a bit difficult to simulate, but we can at least test
// we ended up with the right number of entries
assert(count_ == count);
lfsr_unmount(&lfs) => 0;
'''
## Relocation operations ##
# specific relocation corner cases
[cases.test_mtree_relocate]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// prepare mroot with a large attr so the next entry can not fit
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// create a large entry that needs to be uninlined (but not split!)
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mtree has one mdir
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// force mdir to compact twice, this should relocate
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_t old_mdir = mdir;
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
mdir.rbyd.eoff = -1;
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mdir, &mdir) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mtree has one mdir
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated
assert(lfsr_mdir_cmp(&old_mdir, &mdir) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_relocate_sibling_l]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create 2 large entries that needs to be uninlined and split
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// force mdir to compact twice, this should relocate
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_t old_mdir = mdir;
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
mdir.rbyd.eoff = -1;
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mdir, &mdir) != 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated
assert(lfsr_mdir_cmp(&old_mdir, &mdir) != 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_relocate_sibling_r]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create 2 large entries that needs to be uninlined and split
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// force mdir to compact twice, this should relocate
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_t old_mdir = mdir;
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
mdir.rbyd.eoff = -1;
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mdir, &mdir) != 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated
assert(lfsr_mdir_cmp(&old_mdir, &mdir) != 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_extend]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// prepare mroot with an attr
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// force mroot to compact twice, this should extend the mroot
lfsr_mdir_t old_mroot = lfs.mroot;
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
lfs.mroot.rbyd.eoff = -1;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_extend_twice]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
# force our block to compact by setting prog_size=block_size, we don't have
# any way to indirectly force the intermediary mroots to compact otherwise
defines.PROG_SIZE = 'BLOCK_SIZE'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// prepare mroot with an attr
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// force mroot to compact 2x2 times, this should extend the mroot twice
lfsr_mdir_t old_mroot = lfs.mroot;
for (int i = 0; i < 4; i++) {
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
}
lfs.mroot.rbyd.eoff = -1;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_relocate_mroot]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// prepare mroot with an attr
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// force mroot to compact twice, this should extend the mroot
lfsr_mdir_t old_mroot = lfs.mroot;
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// force mroot to compact twice again, this should relocate the mroot
old_mroot = lfs.mroot;
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
lfs.mroot.rbyd.eoff = -1;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_relocate_extend]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// prepare mroot with a large attr so the next entry can not fit
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// create a large entry that needs to be uninlined (but not split!)
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mtree has one mdir
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// setup mroot to need to compact, this should trigger a relocation when
// we relocate the mdir below
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_t old_mroot = lfs.mroot;
// force mdir to compact twice, this should relocate
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_t old_mdir = mdir;
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
mdir.rbyd.eoff = -1;
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated our mdir
assert(lfsr_mdir_cmp(&old_mdir, &mdir) != 0);
// assert we relocated our mroot
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mtree has one mdir
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated our mdir
assert(lfsr_mdir_cmp(&old_mdir, &mdir) != 0);
// assert we relocated our mroot
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_split_extend]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// setup mroot to need to compact, this should trigger a relocation when
// we relocate the mdir below
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_t old_mroot = lfs.mroot;
// now add another large entry to the mdir, forcing a split
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mdir to compact
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
// assert mdir was split correctly
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated our mroot
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was split correctly
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated our mroot
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_drop_extend]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// setup mroot to need to compact, this should trigger a relocation when
// we relocate the mdir below
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_t old_mroot = lfs.mroot;
// remove the entry, forcing the mdir to be dropped
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, RM, -1, NULL()))) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated our mroot
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was dropped
assert(lfsr_mtree_weight(&lfs) == 0*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated our mroot
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_uninline_extend]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// force mroot to compact once, so the second compact below will trigger
// a relocation
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// prepare mroot with a large attr so the next entry can not fit
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// create a large entry that needs to be uninlined (but not split!)
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact, this should trigger a relocation
lfsr_mdir_t old_mroot = lfs.mroot;
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated our mroot
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated our mroot
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_uninline_split_extend]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// force mroot to compact once, so the second compact below will trigger
// a relocation
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// create 2 large entries that needs to be uninlined and split
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact, this should trigger a relocation
lfsr_mdir_t old_mroot = lfs.mroot;
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated our mroot
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our entries are still in the mtree
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert we relocated our mroot
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
# this fuzz covers a lot of configuratinos
[cases.test_mtree_relocating_fuzz]
defines.N = [5, 10, 20, 40]
defines.FORCE_COMPACTION = [false, true]
defines.BLOCK_CYCLES = [5, 2, 1]
defines.SEED = 'range(500)'
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// at least keep track of the number of entries we expect
lfs_size_t count = 0;
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random mid
lfs_ssize_t mid = (lfs_ssize_t)(
TEST_PRNG(&prng) % lfs_max32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs)));
// fetch mdir
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
// limit our mid to our mdir's weight
mdir.mid = lfsr_mdir_bid(&lfs, &mdir)-(lfsr_mweight(&lfs)-1)
+ (mdir.mid % (mdir.rbyd.weight+1));
// choose to create or delete, note we make sure to never delete to zero
uint8_t op = (lfsr_mdir_rid(&lfs, &mdir) == mdir.rbyd.weight
|| (lfsr_mdir_rid(&lfs, &mdir) == mdir.rbyd.weight-1
&& lfsr_mtree_weight(&lfs) == lfsr_mweight(&lfs))
? 0
: TEST_PRNG(&prng) % 3);
// force a compaction?
if (FORCE_COMPACTION) {
mdir.rbyd.eoff = -1;
lfs.mroot.rbyd.eoff = -1;
}
// create
if (op == 0) {
// add to rbyd
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1,
BUF(&alphas[i % 26], 1)))) => 0;
// make sure we can look up the new entry
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
count += 1;
// update
} else if (op == 1) {
// update rbyd
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, 0,
BUF(&alphas[i % 26], 1)))) => 0;
// make sure we can look up the new entry
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
// delete
} else {
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, RM, -1, NULL()))) => 0;
count -= 1;
}
}
// try looking up each entry
lfs_size_t count_ = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
// drop should make sure we never have empty mdirs
assert(lfsr_mtree_ismptr(&lfs) || mdir.rbyd.weight > 0);
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
count_ += 1;
}
}
// the mtree is a bit difficult to simulate, but we can at least test
// we ended up with the right number of entries
assert(count_ == count);
lfsr_unmount(&lfs) => 0;
// check things stay sane after remount
lfsr_mount(&lfs, CFG) => 0;
// try looking up each entry
count_ = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
// drop should make sure we never have empty mdirs
assert(lfsr_mtree_ismptr(&lfs) || mdir.rbyd.weight > 0);
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
count_ += 1;
}
}
// the mtree is a bit difficult to simulate, but we can at least test
// we ended up with the right number of entries
assert(count_ == count);
lfsr_unmount(&lfs) => 0;
'''
## Neighboring mdir updates ##
[cases.test_mtree_neighbor]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// setup our neighbors
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF("a", 1)),
LFSR_ATTR(1, REG, +1, BUF("b", 1)))) => 0;
// this test only works if these all fit in the mroot
assert(lfsr_mtree_ismptr(&lfs));
lfsr_openedmdir_t left_neighbor = {
.mdir={.mid=0, .rbyd=lfs.mroot.rbyd}};
lfsr_openedmdir_t right_neighbor = {
.mdir={.mid=1, .rbyd=lfs.mroot.rbyd}};
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
// insert a new entry, this should update our neighbors
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF("c", 1)))) => 0;
// assert that our entry is still in the mtree
assert(lfs.mroot.rbyd.weight == 3);
uint8_t buffer[1];
lfsr_mdir_get(&lfs, &lfs.mroot, 1, LFSR_TAG_REG,
buffer, 1) => 1;
assert(memcmp(buffer, "c", 1) == 0);
// assert that our neighbors were updated correctly
assert(left_neighbor.mdir.mid == 0);
assert(memcmp(&left_neighbor.mdir.rbyd, &lfs.mroot.rbyd,
sizeof(lfs.mroot.rbyd)) == 0);
assert(right_neighbor.mdir.mid == 2);
assert(memcmp(&right_neighbor.mdir.rbyd, &lfs.mroot.rbyd,
sizeof(lfs.mroot.rbyd)) == 0);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_neighbor_remove_l]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// setup our neighbors
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF("a", 1)),
LFSR_ATTR(1, REG, +1, BUF("b", 1)))) => 0;
// this test only works if these all fit in the mroot
assert(lfsr_mtree_ismptr(&lfs));
lfsr_openedmdir_t left_neighbor = {
.mdir={.mid=0, .rbyd=lfs.mroot.rbyd}};
lfsr_openedmdir_t right_neighbor = {
.mdir={.mid=1, .rbyd=lfs.mroot.rbyd}};
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
// try removing our left entry
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// assert that an entry was removed
assert(lfs.mroot.rbyd.weight == 1);
// assert that our neighbors were updated correctly
assert(left_neighbor.mdir.mid == -1);
assert(right_neighbor.mdir.mid == 0);
assert(memcmp(&right_neighbor.mdir.rbyd, &lfs.mroot.rbyd,
sizeof(lfs.mroot.rbyd)) == 0);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_neighbor_remove_r]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// setup our neighbors
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF("a", 1)),
LFSR_ATTR(1, REG, +1, BUF("b", 1)))) => 0;
// this test only works if these all fit in the mroot
assert(lfsr_mtree_ismptr(&lfs));
lfsr_openedmdir_t left_neighbor = {
.mdir={.mid=0, .rbyd=lfs.mroot.rbyd}};
lfsr_openedmdir_t right_neighbor = {
.mdir={.mid=1, .rbyd=lfs.mroot.rbyd}};
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
// try removing our left entry
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, RM, -1, NULL()))) => 0;
// assert that an entry was removed
assert(lfs.mroot.rbyd.weight == 1);
// assert that our neighbors were updated correctly
assert(left_neighbor.mdir.mid == 0);
assert(memcmp(&left_neighbor.mdir.rbyd, &lfs.mroot.rbyd,
sizeof(lfs.mroot.rbyd)) == 0);
assert(right_neighbor.mdir.mid == -1);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_neighbor_uninline]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// setup our neighbors
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF("a", 1)),
LFSR_ATTR(1, REG, +1, BUF("b", 1)))) => 0;
// this test only works if these all fit in the mroot
assert(lfsr_mtree_ismptr(&lfs));
lfsr_openedmdir_t left_neighbor = {
.mdir={.mid=0, .rbyd=lfs.mroot.rbyd}};
lfsr_openedmdir_t right_neighbor = {
.mdir={.mid=1, .rbyd=lfs.mroot.rbyd}};
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
// prepare mroot with a large attr so the next entry can not fit
uint8_t buffer[SIZE];
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// create a large entry that needs to be uninlined (but not split!)
memset(buffer, 'd', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 2);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "d", 1) == 0);
// note that our current implementation splits here, which is suboptimal
// but saves on code size
lfsr_mdir_t msibling;
lfsr_mtree_lookup(&lfs,
1*lfsr_mweight(&lfs)+0, &msibling) => 0;
assert(msibling.rbyd.weight == 1);
// assert that our neighbors were updated correctly
assert(left_neighbor.mdir.mid == 0*lfsr_mweight(&lfs)+0);
assert(memcmp(&left_neighbor.mdir.rbyd, &mdir.rbyd,
sizeof(mdir.rbyd)) == 0);
assert(right_neighbor.mdir.mid == 1*lfsr_mweight(&lfs)+0);
assert(memcmp(&right_neighbor.mdir.rbyd, &msibling.rbyd,
sizeof(msibling.rbyd)) == 0);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_neighbor_uninline_split]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// setup our neighbors
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF("a", 1)),
LFSR_ATTR(1, REG, +1, BUF("b", 1)))) => 0;
// this test only works if these all fit in the mroot
assert(lfsr_mtree_ismptr(&lfs));
lfsr_openedmdir_t left_neighbor = {
.mdir={.mid=0, .rbyd=lfs.mroot.rbyd}};
lfsr_openedmdir_t right_neighbor = {
.mdir={.mid=1, .rbyd=lfs.mroot.rbyd}};
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
// create 2 large entries that needs to be uninlined and split
uint8_t buffer[SIZE];
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'd', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(2, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our entries are still in the mtree
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 2);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
lfsr_mdir_t msibling;
lfsr_mtree_lookup(&lfs,
1*lfsr_mweight(&lfs)+0, &msibling) => 0;
assert(msibling.rbyd.weight == 2);
lfsr_mdir_get(&lfs, &msibling, msibling.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "d", 1) == 0);
// assert that our neighbors were updated correctly
assert(left_neighbor.mdir.mid == 0*lfsr_mweight(&lfs)+0);
assert(memcmp(&left_neighbor.mdir.rbyd, &mdir.rbyd,
sizeof(mdir.rbyd)) == 0);
assert(right_neighbor.mdir.mid == 1*lfsr_mweight(&lfs)+1);
assert(memcmp(&right_neighbor.mdir.rbyd, &msibling.rbyd,
sizeof(msibling.rbyd)) == 0);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_neighbor_split]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'd', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// setup our neighbors
//
// note we do this after uninlining! this is because uninlining may
// aggresively split the mtree if there are already neighbors in the mdir
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid+0, REG, +1, BUF("a", 1)),
LFSR_ATTR(mdir.mid+2, REG, +1, BUF("b", 1)))) => 0;
// this test only works if these all fit in the mdir
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
assert(mdir.rbyd.weight == 3);
lfsr_openedmdir_t left_neighbor = {
.mdir={.mid=mdir.mid+0, .rbyd=mdir.rbyd}};
lfsr_openedmdir_t right_neighbor = {
.mdir={.mid=mdir.mid+2, .rbyd=mdir.rbyd}};
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
// now add another large entry to the mdir, forcing a split
memset(buffer, 'e', SIZE);
mdir.mid = 0*lfsr_mweight(&lfs)+2;
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mdir to compact
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
// assert mdir was split correctly
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot still has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 2);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "d", 1) == 0);
lfsr_mdir_t msibling;
lfsr_mtree_lookup(&lfs,
1*lfsr_mweight(&lfs)+0, &msibling) => 0;
assert(msibling.rbyd.weight == 2);
lfsr_mdir_get(&lfs, &msibling, msibling.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "e", 1) == 0);
// assert that our neighbors were updated correctly
assert(left_neighbor.mdir.mid == 0*lfsr_mweight(&lfs)+0);
assert(memcmp(&left_neighbor.mdir.rbyd, &mdir.rbyd,
sizeof(mdir.rbyd)) == 0);
assert(right_neighbor.mdir.mid == 1*lfsr_mweight(&lfs)+1);
assert(memcmp(&right_neighbor.mdir.rbyd, &msibling.rbyd,
sizeof(msibling.rbyd)) == 0);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_neighbor_extend]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// setup our neighbors
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF("a", 1)),
LFSR_ATTR(1, REG, +1, BUF("b", 1)))) => 0;
// this test only works if these all fit in the mroot
assert(lfsr_mtree_ismptr(&lfs));
lfsr_openedmdir_t left_neighbor = {
.mdir={.mid=0, .rbyd=lfs.mroot.rbyd}};
lfsr_openedmdir_t right_neighbor = {
.mdir={.mid=1, .rbyd=lfs.mroot.rbyd}};
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
// prepare mroot with an attr
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// force mroot to compact twice, this should extend the mroot
lfsr_mdir_t old_mroot = lfs.mroot;
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
lfs.mroot.rbyd.eoff = -1;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
// assert that our neighbors were updated correctly
assert(left_neighbor.mdir.mid == 0);
assert(memcmp(&left_neighbor.mdir.rbyd, &lfs.mroot.rbyd,
sizeof(lfs.mroot.rbyd)) == 0);
assert(right_neighbor.mdir.mid == 1);
assert(memcmp(&right_neighbor.mdir.rbyd, &lfs.mroot.rbyd,
sizeof(lfs.mroot.rbyd)) == 0);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_neighbor_relocate]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create an uninlined mdir
uint8_t buffer[SIZE];
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'd', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// setup our neighbors
//
// note we do this after uninlining! this is because uninlining may
// aggresively split the mtree if there are already neighbors in the mdir
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid+0, REG, +1, BUF("a", 1)),
LFSR_ATTR(mdir.mid+2, REG, +1, BUF("b", 1)))) => 0;
// this test only works if these all fit in the mdir
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
assert(mdir.rbyd.weight == 3);
lfsr_openedmdir_t left_neighbor = {
.mdir={.mid=mdir.mid+0, .rbyd=mdir.rbyd}};
lfsr_openedmdir_t right_neighbor = {
.mdir={.mid=mdir.mid+2, .rbyd=mdir.rbyd}};
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
// force mdir to compact twice, this should relocate
lfsr_mdir_t old_mdir = mdir;
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
mdir.rbyd.eoff = -1;
memset(buffer, 'e', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(1, REG, 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mdir, &mdir) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "c", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 3);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "e", 1) == 0);
// assert that our neighbors were updated correctly
assert(left_neighbor.mdir.mid == 0*lfsr_mweight(&lfs)+0);
assert(memcmp(&left_neighbor.mdir.rbyd, &mdir.rbyd,
sizeof(mdir.rbyd)) == 0);
assert(right_neighbor.mdir.mid == 0*lfsr_mweight(&lfs)+2);
assert(memcmp(&right_neighbor.mdir.rbyd, &mdir.rbyd,
sizeof(mdir.rbyd)) == 0);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_neighbor_middle_split]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
//// create a situation where we have 3 mdirs in our tree
// first force mroot to uninlined+split
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// we should now have 2 mdirs
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// now force one of our siblings to split
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mdir to compact
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
// we should now have 3 mdirs
assert(lfsr_mtree_weight(&lfs) == 3*lfsr_mweight(&lfs));
//// Now test splitting updates mids correctly
// setup our neighbors
lfsr_openedmdir_t left_neighbor;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0,
&left_neighbor.mdir) => 0;
assert(left_neighbor.mdir.rbyd.weight == 1);
lfsr_openedmdir_t right_neighbor;
lfsr_mtree_lookup(&lfs, 2*lfsr_mweight(&lfs)+0,
&right_neighbor.mdir) => 0;
assert(right_neighbor.mdir.rbyd.weight == 1);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
// cause middle mdir to split
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
memset(buffer, 'd', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mdir to compact
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
// we should now have 4 mdirs
assert(lfsr_mtree_weight(&lfs) == 4*lfsr_mweight(&lfs));
// assert that our neighbors were updated correctly
assert(left_neighbor.mdir.mid == 0*lfsr_mweight(&lfs)+0);
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(memcmp(&left_neighbor.mdir.rbyd, &mdir.rbyd,
sizeof(mdir.rbyd)) == 0);
assert(right_neighbor.mdir.mid == 3*lfsr_mweight(&lfs)+0);
lfsr_mtree_lookup(&lfs, 3*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(memcmp(&right_neighbor.mdir.rbyd, &mdir.rbyd,
sizeof(mdir.rbyd)) == 0);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_neighbor_middle_drop]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
//// create a situation where we have 3 mdirs in our tree
// first force mroot to uninlined+split
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// we should now have 2 mdirs
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// now force one of our siblings to split
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+1, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
memset(buffer, 'c', SIZE);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mdir to compact
mdir.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &mdir, NULL, 0) => 0;
// we should now have 3 mdirs
assert(lfsr_mtree_weight(&lfs) == 3*lfsr_mweight(&lfs));
//// Now test dropping updates mids correctly
// setup our neighbors
lfsr_openedmdir_t left_neighbor;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0,
&left_neighbor.mdir) => 0;
assert(left_neighbor.mdir.rbyd.weight == 1);
lfsr_openedmdir_t right_neighbor;
lfsr_mtree_lookup(&lfs, 2*lfsr_mweight(&lfs)+0,
&right_neighbor.mdir) => 0;
assert(right_neighbor.mdir.rbyd.weight == 1);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_addopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
// cause middle mdir to drop
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, RM, -1, NULL()))) => 0;
// we should now have 2 mdirs
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert that our neighbors were updated correctly
assert(left_neighbor.mdir.mid == 0*lfsr_mweight(&lfs)+0);
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(memcmp(&left_neighbor.mdir.rbyd, &mdir.rbyd,
sizeof(mdir.rbyd)) == 0);
assert(right_neighbor.mdir.mid == 1*lfsr_mweight(&lfs)+0);
lfsr_mtree_lookup(&lfs, 1*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(memcmp(&right_neighbor.mdir.rbyd, &mdir.rbyd,
sizeof(mdir.rbyd)) == 0);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &left_neighbor);
lfsr_mdir_removeopened(&lfs, LFS_TYPE_INTERNAL, &right_neighbor);
lfsr_unmount(&lfs) => 0;
'''
## mtree traversal ##
# test specific corner cases
[cases.test_mtree_traversal]
defines.VALIDATE = [false, true]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// insert a new entry, this should update our neighbors
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF("a", 1)))) => 0;
// assert that our entry is still in the mtree
assert(lfs.mroot.rbyd.weight == 1);
uint8_t buffer[1];
lfsr_mdir_get(&lfs, &lfs.mroot, 0, LFSR_TAG_REG,
buffer, 1) => 1;
assert(memcmp(buffer, "a", 1) == 0);
// test that we can traverse the tree, keeping track of all blocks we see
uint8_t *seen = malloc((BLOCK_COUNT+7)/8);
memset(seen, 0, (BLOCK_COUNT+7)/8);
lfsr_traversal_t traversal = LFSR_TRAVERSAL(
VALIDATE ? LFSR_TRAVERSAL_VALIDATE : 0);
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i < 2*BLOCK_COUNT);
lfsr_tinfo_t tinfo;
int err = lfsr_traversal_read(&lfs, &traversal, &tinfo);
assert(!err || err == LFS_ERR_NOENT);
if (err == LFS_ERR_NOENT) {
break;
}
if (tinfo.tag == LFSR_TAG_MDIR) {
printf("traversal: 0x%x mdir 0x{%x,%x}\n",
tinfo.tag,
tinfo.u.mdir.rbyd.blocks[0],
tinfo.u.mdir.rbyd.blocks[1]);
// keep track of seen blocks
seen[tinfo.u.mdir.rbyd.blocks[1] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[1] % 8);
seen[tinfo.u.mdir.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[0] % 8);
} else if (tinfo.tag == LFSR_TAG_BRANCH) {
printf("traversal: 0x%x btree 0x%x.%x\n",
tinfo.tag,
tinfo.u.rbyd.blocks[0], tinfo.u.rbyd.trunk);
// keep track of seen blocks
seen[tinfo.u.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.rbyd.blocks[0] % 8);
} else {
// this shouldn't happen
printf("traversal: 0x%x\n", tinfo.tag);
assert(false);
}
}
// if traversal worked, we should be able to clobber all other blocks
uint8_t clobber_buf[BLOCK_SIZE];
memset(clobber_buf, 0xcc, BLOCK_SIZE);
for (lfs_block_t block = 0; block < BLOCK_COUNT; block++) {
if (!(seen[block / 8] & (1 << (block % 8)))) {
CFG->erase(CFG, block) => 0;
CFG->prog(CFG, block, 0, clobber_buf, BLOCK_SIZE) => 0;
}
}
free(seen);
// and the tree should still work
// assert that our entry is still in the mtree
assert(lfs.mroot.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &lfs.mroot, 0, LFSR_TAG_REG,
buffer, 1) => 1;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_traversal_uninline]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
defines.VALIDATE = [false, true]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// prepare mroot with a large attr so the next entry can not fit
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// create a large entry that needs to be uninlined (but not split!)
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
// test that we can traverse the tree, keeping track of all blocks we see
uint8_t *seen = malloc((BLOCK_COUNT+7)/8);
memset(seen, 0, (BLOCK_COUNT+7)/8);
lfsr_traversal_t traversal = LFSR_TRAVERSAL(
VALIDATE ? LFSR_TRAVERSAL_VALIDATE : 0);
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i < 2*BLOCK_COUNT);
lfsr_tinfo_t tinfo;
int err = lfsr_traversal_read(&lfs, &traversal, &tinfo);
assert(!err || err == LFS_ERR_NOENT);
if (err == LFS_ERR_NOENT) {
break;
}
if (tinfo.tag == LFSR_TAG_MDIR) {
printf("traversal: 0x%x mdir 0x{%x,%x}\n",
tinfo.tag,
tinfo.u.mdir.rbyd.blocks[0],
tinfo.u.mdir.rbyd.blocks[1]);
// keep track of seen blocks
seen[tinfo.u.mdir.rbyd.blocks[1] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[1] % 8);
seen[tinfo.u.mdir.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[0] % 8);
} else if (tinfo.tag == LFSR_TAG_BRANCH) {
printf("traversal: 0x%x btree 0x%x.%x\n",
tinfo.tag,
tinfo.u.rbyd.blocks[0], tinfo.u.rbyd.trunk);
// keep track of seen blocks
seen[tinfo.u.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.rbyd.blocks[0] % 8);
} else {
// this shouldn't happen
printf("traversal: 0x%x\n", tinfo.tag);
assert(false);
}
}
// if traversal worked, we should be able to clobber all other blocks
uint8_t clobber_buf[BLOCK_SIZE];
memset(clobber_buf, 0xcc, BLOCK_SIZE);
for (lfs_block_t block = 0; block < BLOCK_COUNT; block++) {
if (!(seen[block / 8] & (1 << (block % 8)))) {
CFG->erase(CFG, block) => 0;
CFG->prog(CFG, block, 0, clobber_buf, BLOCK_SIZE) => 0;
}
}
free(seen);
// and the tree should still work
// assert mdir was unininlined correctly
assert(lfsr_mtree_weight(&lfs) == 1*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
// assert that our entry is still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_traversal_split]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
defines.VALIDATE = [false, true]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create 2 large entries that needs to be uninlined and split
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF(buffer, SIZE)))) => 0;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(1, REG, +1, BUF(buffer, SIZE)))) => 0;
// force mroot to compact
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our entries are still in the mtree
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_mdir_t msibling;
lfsr_mtree_lookup(&lfs,
1*lfsr_mweight(&lfs)+0, &msibling) => 0;
assert(msibling.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &msibling, msibling.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
// test that we can traverse the tree, keeping track of all blocks we see
uint8_t *seen = malloc((BLOCK_COUNT+7)/8);
memset(seen, 0, (BLOCK_COUNT+7)/8);
lfsr_traversal_t traversal = LFSR_TRAVERSAL(
VALIDATE ? LFSR_TRAVERSAL_VALIDATE : 0);
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i < 2*BLOCK_COUNT);
lfsr_tinfo_t tinfo;
int err = lfsr_traversal_read(&lfs, &traversal, &tinfo);
assert(!err || err == LFS_ERR_NOENT);
if (err == LFS_ERR_NOENT) {
break;
}
if (tinfo.tag == LFSR_TAG_MDIR) {
printf("traversal: 0x%x mdir 0x{%x,%x}\n",
tinfo.tag,
tinfo.u.mdir.rbyd.blocks[0],
tinfo.u.mdir.rbyd.blocks[1]);
// keep track of seen blocks
seen[tinfo.u.mdir.rbyd.blocks[1] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[1] % 8);
seen[tinfo.u.mdir.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[0] % 8);
} else if (tinfo.tag == LFSR_TAG_BRANCH) {
printf("traversal: 0x%x btree 0x%x.%x\n",
tinfo.tag,
tinfo.u.rbyd.blocks[0], tinfo.u.rbyd.trunk);
// keep track of seen blocks
seen[tinfo.u.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.rbyd.blocks[0] % 8);
} else {
// this shouldn't happen
printf("traversal: 0x%x\n", tinfo.tag);
assert(false);
}
}
// if traversal worked, we should be able to clobber all other blocks
uint8_t clobber_buf[BLOCK_SIZE];
memset(clobber_buf, 0xcc, BLOCK_SIZE);
for (lfs_block_t block = 0; block < BLOCK_COUNT; block++) {
if (!(seen[block / 8] & (1 << (block % 8)))) {
CFG->erase(CFG, block) => 0;
CFG->prog(CFG, block, 0, clobber_buf, BLOCK_SIZE) => 0;
}
}
free(seen);
// and the tree should still work
// assert mdirs were unininlined and split
assert(lfsr_mtree_weight(&lfs) == 2*lfsr_mweight(&lfs));
// assert mroot now has no entries
assert(lfs.mroot.rbyd.weight == 0);
// assert that our entries are still in the mtree
lfsr_mtree_lookup(&lfs, 0*lfsr_mweight(&lfs)+0, &mdir) => 0;
assert(mdir.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "a", 1) == 0);
lfsr_mtree_lookup(&lfs,
1*lfsr_mweight(&lfs)+0, &msibling) => 0;
assert(msibling.rbyd.weight == 1);
lfsr_mdir_get(&lfs, &msibling, msibling.mid, LFSR_TAG_REG,
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_traversal_extend]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
defines.VALIDATE = [false, true]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// prepare mroot with an attr
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// force mroot to compact twice, this should extend the mroot
lfsr_mdir_t old_mroot = lfs.mroot;
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
lfs.mroot.rbyd.eoff = -1;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
// test that we can traverse the tree, keeping track of all blocks we see
uint8_t *seen = malloc((BLOCK_COUNT+7)/8);
memset(seen, 0, (BLOCK_COUNT+7)/8);
lfsr_traversal_t traversal = LFSR_TRAVERSAL(
VALIDATE ? LFSR_TRAVERSAL_VALIDATE : 0);
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i < 2*BLOCK_COUNT);
lfsr_tinfo_t tinfo;
int err = lfsr_traversal_read(&lfs, &traversal, &tinfo);
assert(!err || err == LFS_ERR_NOENT);
if (err == LFS_ERR_NOENT) {
break;
}
if (tinfo.tag == LFSR_TAG_MDIR) {
printf("traversal: 0x%x mdir 0x{%x,%x}\n",
tinfo.tag,
tinfo.u.mdir.rbyd.blocks[0],
tinfo.u.mdir.rbyd.blocks[1]);
// keep track of seen blocks
seen[tinfo.u.mdir.rbyd.blocks[1] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[1] % 8);
seen[tinfo.u.mdir.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[0] % 8);
} else if (tinfo.tag == LFSR_TAG_BRANCH) {
printf("traversal: 0x%x btree 0x%x.%x\n",
tinfo.tag,
tinfo.u.rbyd.blocks[0], tinfo.u.rbyd.trunk);
// keep track of seen blocks
seen[tinfo.u.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.rbyd.blocks[0] % 8);
} else {
// this shouldn't happen
printf("traversal: 0x%x\n", tinfo.tag);
assert(false);
}
}
// if traversal worked, we should be able to clobber all other blocks
uint8_t clobber_buf[BLOCK_SIZE];
memset(clobber_buf, 0xcc, BLOCK_SIZE);
for (lfs_block_t block = 0; block < BLOCK_COUNT; block++) {
if (!(seen[block / 8] & (1 << (block % 8)))) {
CFG->erase(CFG, block) => 0;
CFG->prog(CFG, block, 0, clobber_buf, BLOCK_SIZE) => 0;
}
}
free(seen);
// and the tree should still work
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
'''
# larger traversal tests
[cases.test_mtree_traversal_many]
defines.N = [5, 10, 20, 40, 80, 160, 320]
defines.VALIDATE = [false, true]
defines.FORCE_COMPACTION = [false, true]
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create entries
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs,
lfs_smax32(
lfsr_mtree_weight(&lfs) - lfsr_mweight(&lfs),
0),
&mdir) => 0;
for (lfs_size_t i = 0; i < N; i++) {
// force a compaction?
if (FORCE_COMPACTION) {
mdir.rbyd.eoff = -1;
lfs.mroot.rbyd.eoff = -1;
}
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1,
BUF(&alphas[i % 26], 1)))) => 0;
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
mdir.mid += 1;
}
// try looking up each entry
lfs_size_t i = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
i += 1;
}
}
assert(i == N);
// test that we can traverse the tree, keeping track of all blocks we see
uint8_t *seen = malloc((BLOCK_COUNT+7)/8);
memset(seen, 0, (BLOCK_COUNT+7)/8);
lfsr_traversal_t traversal = LFSR_TRAVERSAL(
VALIDATE ? LFSR_TRAVERSAL_VALIDATE : 0);
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i < 2*BLOCK_COUNT);
lfsr_tinfo_t tinfo;
int err = lfsr_traversal_read(&lfs, &traversal, &tinfo);
assert(!err || err == LFS_ERR_NOENT);
if (err == LFS_ERR_NOENT) {
break;
}
if (tinfo.tag == LFSR_TAG_MDIR) {
printf("traversal: 0x%x mdir 0x{%x,%x}\n",
tinfo.tag,
tinfo.u.mdir.rbyd.blocks[0],
tinfo.u.mdir.rbyd.blocks[1]);
// keep track of seen blocks
seen[tinfo.u.mdir.rbyd.blocks[1] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[1] % 8);
seen[tinfo.u.mdir.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[0] % 8);
} else if (tinfo.tag == LFSR_TAG_BRANCH) {
printf("traversal: 0x%x btree 0x%x.%x\n",
tinfo.tag,
tinfo.u.rbyd.blocks[0], tinfo.u.rbyd.trunk);
// keep track of seen blocks
seen[tinfo.u.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.rbyd.blocks[0] % 8);
} else {
// this shouldn't happen
printf("traversal: 0x%x\n", tinfo.tag);
assert(false);
}
}
// if traversal worked, we should be able to clobber all other blocks
uint8_t clobber_buf[BLOCK_SIZE];
memset(clobber_buf, 0xcc, BLOCK_SIZE);
for (lfs_block_t block = 0; block < BLOCK_COUNT; block++) {
if (!(seen[block / 8] & (1 << (block % 8)))) {
CFG->erase(CFG, block) => 0;
CFG->prog(CFG, block, 0, clobber_buf, BLOCK_SIZE) => 0;
}
}
free(seen);
// and the tree should still work
// try looking up each entry
i = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
i += 1;
}
}
assert(i == N);
lfsr_unmount(&lfs) => 0;
'''
[cases.test_mtree_traversal_fuzz]
defines.N = [5, 10, 20, 40, 80, 160]
defines.VALIDATE = [false, true]
defines.FORCE_COMPACTION = [false, true]
defines.SEED = 'range(100)'
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// at least keep track of the number of entries we expect
lfs_size_t count = 0;
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < N; i++) {
// choose a pseudo-random mid
lfs_ssize_t mid = (lfs_ssize_t)(
TEST_PRNG(&prng) % lfs_max32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs)));
// fetch mdir
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
// limit our mid to our mdir's weight
mdir.mid = lfsr_mdir_bid(&lfs, &mdir)-(lfsr_mweight(&lfs)-1)
+ (mdir.mid % (mdir.rbyd.weight+1));
// force a compaction?
if (FORCE_COMPACTION) {
mdir.rbyd.eoff = -1;
lfs.mroot.rbyd.eoff = -1;
}
// add to rbyd, potentially splitting the mdir
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1,
BUF(&alphas[i % 26], 1)))) => 0;
// make sure we can look up the new entry
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
count += 1;
}
// try looking up each entry
lfs_size_t count_ = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
count_ += 1;
}
}
// the mtree is a bit difficult to simulate, but we can at least test
// we ended up with the right number of entries
assert(count_ == count);
// test that we can traverse the tree, keeping track of all blocks
// we see
uint8_t *seen = malloc((BLOCK_COUNT+7)/8);
memset(seen, 0, (BLOCK_COUNT+7)/8);
lfsr_traversal_t traversal = LFSR_TRAVERSAL(
VALIDATE ? LFSR_TRAVERSAL_VALIDATE : 0);
for (lfs_block_t i = 0;; i++) {
// a bit hacky, but this catches infinite loops
assert(i < 2*BLOCK_COUNT);
lfsr_tinfo_t tinfo;
int err = lfsr_traversal_read(&lfs, &traversal, &tinfo);
assert(!err || err == LFS_ERR_NOENT);
if (err == LFS_ERR_NOENT) {
break;
}
if (tinfo.tag == LFSR_TAG_MDIR) {
printf("traversal: 0x%x mdir 0x{%x,%x}\n",
tinfo.tag,
tinfo.u.mdir.rbyd.blocks[0],
tinfo.u.mdir.rbyd.blocks[1]);
// keep track of seen blocks
seen[tinfo.u.mdir.rbyd.blocks[1] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[1] % 8);
seen[tinfo.u.mdir.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.mdir.rbyd.blocks[0] % 8);
} else if (tinfo.tag == LFSR_TAG_BRANCH) {
printf("traversal: 0x%x btree 0x%x.%x\n",
tinfo.tag,
tinfo.u.rbyd.blocks[0], tinfo.u.rbyd.trunk);
// keep track of seen blocks
seen[tinfo.u.rbyd.blocks[0] / 8]
|= 1 << (tinfo.u.rbyd.blocks[0] % 8);
} else {
// this shouldn't happen
printf("traversal: 0x%x\n", tinfo.tag);
assert(false);
}
}
// if traversal worked, we should be able to clobber all other blocks
uint8_t clobber_buf[BLOCK_SIZE];
memset(clobber_buf, 0xcc, BLOCK_SIZE);
for (lfs_block_t block = 0; block < BLOCK_COUNT; block++) {
if (!(seen[block / 8] & (1 << (block % 8)))) {
CFG->erase(CFG, block) => 0;
CFG->prog(CFG, block, 0, clobber_buf, BLOCK_SIZE) => 0;
}
}
free(seen);
// and the tree should still work
// try looking up each entry
count_ = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
count_ += 1;
}
}
// the mtree is a bit difficult to simulate, but we can at least test
// we ended up with the right number of entries
assert(count_ == count);
lfsr_unmount(&lfs) => 0;
'''
## Cycle detection? ##
# test that our cycle detector at least works in common cases
[cases.test_mtree_traversal_mroot_cycle]
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
uint8_t buf[LFSR_MPTR_DSIZE];
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1,
MROOT, 0, FROMMPTR(&LFSR_MPTR_MROOTANCHOR(), buf)))) => 0;
// technically, cycle detection only needs to work when we're validating
lfsr_traversal_t traversal = LFSR_TRAVERSAL(LFSR_TRAVERSAL_VALIDATE);
for (lfs_block_t i = 0;; i++) {
// assert that we detect the cycle in a reasonable number of iterations
assert(i < 2*BLOCK_COUNT);
lfsr_tinfo_t tinfo;
int err = lfsr_traversal_read(&lfs, &traversal, &tinfo);
assert(!err || err == LFS_ERR_CORRUPT);
if (err == LFS_ERR_CORRUPT) {
break;
}
if (tinfo.tag == LFSR_TAG_MDIR) {
printf("traversal: 0x%x mdir 0x{%x,%x}\n",
tinfo.tag,
tinfo.u.mdir.rbyd.blocks[0],
tinfo.u.mdir.rbyd.blocks[1]);
} else if (tinfo.tag == LFSR_TAG_BRANCH) {
printf("traversal: 0x%x btree 0x%x.%x\n",
tinfo.tag,
tinfo.u.rbyd.blocks[0], tinfo.u.rbyd.trunk);
} else {
// this shouldn't happen
printf("traversal: 0x%x\n", tinfo.tag);
assert(false);
}
}
lfsr_unmount(&lfs) => 0;
'''
## Magic consistency ##
# make sure our magic string ("littlefs") shows up in the same place (off=8)
[cases.test_mtree_magic]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
// check our magic string
//
// note if we lose power we may not have the magic string in both blocks!
// but we don't lose power in this test so we can assert the magic string
// is present in both
uint8_t magic[lfs_max(16, READ_SIZE)];
CFG->read(CFG, 0, 0, magic, lfs_max(16, READ_SIZE)) => 0;
assert(memcmp(&magic[8], "littlefs", 8) == 0);
CFG->read(CFG, 1, 0, magic, lfs_max(16, READ_SIZE)) => 0;
assert(memcmp(&magic[8], "littlefs", 8) == 0);
'''
[cases.test_mtree_magic_extend]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// prepare mroot with an attr
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// force mroot to compact twice, this should extend the mroot
lfsr_mdir_t old_mroot = lfs.mroot;
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
lfs.mroot.rbyd.eoff = -1;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check our magic string
//
// note if we lose power we may not have the magic string in both blocks!
// but we don't lose power in this test so we can assert the magic string
// is present in both
uint8_t magic[lfs_max(16, READ_SIZE)];
CFG->read(CFG, 0, 0, magic, lfs_max(16, READ_SIZE)) => 0;
assert(memcmp(&magic[8], "littlefs", 8) == 0);
CFG->read(CFG, 1, 0, magic, lfs_max(16, READ_SIZE)) => 0;
assert(memcmp(&magic[8], "littlefs", 8) == 0);
'''
[cases.test_mtree_magic_extend_twice]
# this should be set so only one entry can fit in a metadata block
defines.SIZE = 'BLOCK_SIZE / 4'
# make it so blocks relocate every two compacts
defines.BLOCK_CYCLES = 2
# force our block to compact by setting prog_size=block_size, we don't have
# any way to indirectly force the intermediary mroots to compact otherwise
defines.PROG_SIZE = 'BLOCK_SIZE'
in = 'lfs.c'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// prepare mroot with an attr
uint8_t buffer[SIZE];
memset(buffer, 'a', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// force mroot to compact 2x2 times, this should extend the mroot twice
lfsr_mdir_t old_mroot = lfs.mroot;
for (int i = 0; i < 4; i++) {
lfs.mroot.rbyd.eoff = -1;
lfsr_mdir_commit(&lfs, &lfs.mroot, NULL, 0) => 0;
}
lfs.mroot.rbyd.eoff = -1;
memset(buffer, 'b', SIZE);
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(-1, UATTR(1), 0, BUF(buffer, SIZE)))) => 0;
// assert we relocated
assert(lfsr_mdir_cmp(&old_mroot, &lfs.mroot) != 0);
// assert that our attr is still in the mroot
lfsr_mdir_get(&lfs, &lfs.mroot, -1, LFSR_TAG_UATTR(1),
buffer, SIZE) => SIZE;
assert(memcmp(buffer, "b", 1) == 0);
lfsr_unmount(&lfs) => 0;
// check our magic string
//
// note if we lose power we may not have the magic string in both blocks!
// but we don't lose power in this test so we can assert the magic string
// is present in both
uint8_t magic[lfs_max(16, READ_SIZE)];
CFG->read(CFG, 0, 0, magic, lfs_max(16, READ_SIZE)) => 0;
assert(memcmp(&magic[8], "littlefs", 8) == 0);
CFG->read(CFG, 1, 0, magic, lfs_max(16, READ_SIZE)) => 0;
assert(memcmp(&magic[8], "littlefs", 8) == 0);
'''
## Orphaned mdirs ##
# orphaned mdirs can happen if we lose power, test we can clean them up
[cases.test_mtree_orphans]
defines.N = 320
defines.ORPHANS = [1, 2, 3, 4]
defines.SEED = 42
in = 'lfs.c'
code = '''
const char *alphas = "abcdefghijklmnopqrstuvwxyz";
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
lfs_alloc_ack(&lfs);
// remove root bookmark for now
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_ATTRS(
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
// create entries
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs,
lfs_smax32(
lfsr_mtree_weight(&lfs) - lfsr_mweight(&lfs),
0),
&mdir) => 0;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_mdir_commit(&lfs, &mdir, LFSR_ATTRS(
LFSR_ATTR(mdir.mid, REG, +1,
BUF(&alphas[i % 26], 1)))) => 0;
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
mdir.mid += 1;
}
lfsr_mid_t old_weight = lfsr_mtree_weight(&lfs);
// this test only works with a full mtree
LFS_ASSERT(lfsr_mtree_isbtree(&lfs));
// bypass the mdir logic and create some orphans
uint32_t prng = SEED;
for (lfs_size_t i = 0; i < ORPHANS; i++) {
// note we should never have orphan.mid=0
lfsr_bid_t bid_ = ((TEST_PRNG(&prng)
% (lfsr_mtree_weight(&lfs)/lfsr_mweight(&lfs))) + 1)
* lfsr_mweight(&lfs);
// manually allocate/commit an empty mdir, otherwise
// lfsr_mdir_commit automatically cleans up empty mdirs
lfsr_mptr_t mptr;
for (lfs_size_t j = 0; j < 2; j++) {
lfsr_rbyd_t rbyd;
lfsr_rbyd_alloc(&lfs, &rbyd) => 0;
lfsr_rbyd_commit(&lfs, &rbyd, LFSR_ATTRS(
LFSR_ATTR(0, REG, +1, BUF("a", 1)),
LFSR_ATTR(0, RM, -1, NULL()))) => 0;
mptr.blocks[j] = rbyd.blocks[0];
}
// commit orphan to tree
uint8_t mptr_buf[LFSR_MPTR_DSIZE];
lfsr_mtree_commit(&lfs, LFSR_ATTRS(
LFSR_ATTR(bid_,
MDIR, +lfsr_mweight(&lfs),
FROMMPTR(&mptr, mptr_buf)))) => 0;
}
LFS_ASSERT(lfsr_mtree_weight(&lfs) > old_weight);
// trigger lfsr_fs_fixorphans
lfs.hasorphans = true;
lfsr_fs_preparemutation(&lfs) => 0;
// this should have removed all of our orphans
LFS_ASSERT(lfsr_mtree_weight(&lfs) == old_weight);
// try looking up each entry
lfs_size_t i = 0;
for (lfs_ssize_t mid = 0;
mid < lfs_smax32(
lfsr_mtree_weight(&lfs),
lfsr_mweight(&lfs));
mid += lfsr_mweight(&lfs)) {
lfsr_mdir_t mdir;
lfsr_mtree_lookup(&lfs, mid, &mdir) => 0;
for (; lfsr_mdir_rid(&lfs, &mdir) < mdir.rbyd.weight;
mdir.mid += 1) {
uint8_t buffer[4];
lfsr_mdir_get(&lfs, &mdir, mdir.mid, LFSR_TAG_REG,
buffer, 4) => 1;
assert(memcmp(buffer, &alphas[i % 26], 1) == 0);
i += 1;
}
}
assert(i == N);
lfsr_unmount(&lfs) => 0;
'''
View Git Blame Copy Permalink