(Re)implemented lfsr_fs_grow, variable block counts, etc

Well this turned into a never-ending can of worms...

I guess the good news is our newly added lfsr_grow_incr_* tests are
_very_ good at finding post-error-resume bugs.

Implementation-wise, this was fairly straightforward thanks to prior
work by BrianPugh, kaetemi, and myself:

1. Made block_count pseudo-optional by adding lfs.block_count so we can
   mutate it based on what we find on-disk.

   This was done a bit different from the previous implementation,
   instead of setting block_count=0 to read the block_count from disk,
   we allow any block_count <= the configured block_count.

   This matches how we handle name_limit/file_limit/etc, and allows
   users to mount a filesystem with unknown block_count while asserting
   an upper bound.

2. Added lfsr_fs_grow, which can grow the filesystem.

   The is basically the same as the previous implementation except we're
   a bit more careful with the lookahead buffer.

   I thought the previous impl might have been broken w.r.t. lookahead
   buffer, but fortunately it's only broken in a way that makes us think
   newly available blocks are temporarily in-use. Which is a bit funny.

   One interesting thing that came out with more aggressive tests is
   that it's possible to get locked-up in lfsr_fs_preparemutation trying
   to clean up grms/orphans before we change the filesystem size.

   Fortunately it turns out we don't _really_ need to call
   lfsr_fs_preparemutation here. This gets a bit delicate, but means we
   should always be able to grow a full filesystem.

To test this I've added both the simple grow/error tests from the
previous version, as well as a set of fuzz tests (a la test_relocations
and friends) that incrementally grow the filesystem when encountering
LFS_ERR_NOSPC. These have a surprising amount coverage, testing
lfsr_fs_grow, lfsr_fs_stat, lfsr_fs_size, and resuming operations after
encountering an error.

Which also means they found bugs:

- lfs_alloc_setinuse was not broken before, because lookahead.start was
  always a multiple of lookahead_size. But now with lfs_alloc_discard,
  this invariant may not be true.

  I've just changed all lookahead.start updates to mod block_count. This
  adds a bit of code, but is much easier to reason about.

  While fixing this, I also added an assert to never allocate blocks
  {0,1} in lfs_alloc. This is a good assert to have, but did require
  some tweaks to test_btree to avoid these blocks.

- We were incorrectly patching grms in lfsr_mdir_commit when mdelta=0.

  Funnily enough we also proceed to ignore the patched grm most of the
  time when mdelta=0, so this went unnoticed.

- It turns out we're completely ignoring rid=-1 attrs if we split the
  mroot. Not sure how this was missed. It's a bit important.

  Note this is still broken. Fixing this requires some rather invasive
  changes to lfsr_mdir_commit's internal logic that should probably be
  in another commit...

Note again fwrite_fuzz is omitted. Currently the state of data in opened
files is undefined after a failed write, so this wouldn't really be
testing anything interesting...

More features = more code, and all of this bug fixing meant several
things contributed to code/stack changes in this commit:

                           code          stack
  before:                 33654           2592
  +variable block_count:  33646 (-0.0%)   2584 (+0.0%)
  +lfsr_fs_grow:          33818 (+0.5%)   2584 (-0.3%)
  +lookahead-start-fix:   33842 (+0.6%)   2584 (-0.3%)
  +grm-patch-fix (after): 33850 (+0.6%)   2584 (-0.3%)

Wild that variable block_count actually saves code/stack. I guess the
indirect lfs->cfg->block_count load can get costly...
This commit is contained in:
Christopher Haster
2024-06-11 16:13:58 -05:00
parent ae0e3348fe
commit 9ad59dcfe6
5 changed files with 3383 additions and 189 deletions
+9 -9
View File
@@ -46,10 +46,10 @@ code = '''
lfs_init(&lfs, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.start = 0;
lfs.lookahead.start = 2;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count);
lfs.lookahead.next = 0;
CFG->block_count-2);
lfs.lookahead.next = 2;
lfs_alloc_ckpoint(&lfs);
// create a btree
@@ -1742,10 +1742,10 @@ code = '''
lfs_init(&lfs, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.start = 0;
lfs.lookahead.start = 2;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count);
lfs.lookahead.next = 0;
CFG->block_count-2);
lfs.lookahead.next = 2;
lfs_alloc_ckpoint(&lfs);
// create a btree
@@ -3438,10 +3438,10 @@ code = '''
lfs_init(&lfs, CFG) => 0;
// create free lookahead
memset(lfs.lookahead.buffer, 0, CFG->lookahead_size);
lfs.lookahead.start = 0;
lfs.lookahead.start = 2;
lfs.lookahead.size = lfs_min(8*CFG->lookahead_size,
CFG->block_count);
lfs.lookahead.next = 0;
CFG->block_count-2);
lfs.lookahead.next = 2;
lfs_alloc_ckpoint(&lfs);
// create a btree