Found a better heuristic for did truncation, dropped mlimit, found bugs

The idea here is to combine the current mtree size with the theoretical
upper bound on the number of directories in a single mdir, assuming our
block size, to give us a heuristic for did truncation that does not
require any extra state.

- Each directory needs 1 name tag, 1 did tag, and 1 dstart
- Each tag needs ~2 alts with our current compaction strategy
- Each tag/alt encodes to a minimum of 4 bytes
- We can also assume ~1/2 block utilization due to our split threshold

This gives us ~3*3*4*2 or ~72 bytes per directory at minimum, or
rounding down, ~block_size/32 directories per mdir.

This is a nice number because for common NOR flash geometry,
4096/32 = 128, so a filesystem with a single mdir encodes dids in a
single byte.

The biggest benefit though is being able to drop the mlimit state from
the lfs_t struct.

---

Unfortunately, this change revealed several bugs.

It turns out __builtin_clz in GCC is undefined at 0, which caused our
lfs_nlog2 function to return incorrect values at 1. This was causing
our dids to all collide when the mtree was inlined, which was resolved
by the linear scanning that resolves dids, but was severely limiting
what exactly our tests covered.

Now that this is fixed (with a simple if statement in lfs_nlog2,
lfs_nlog2 now always has defined behavior, even at 0), several bugs
needed fixing:

- We update the rid based on attrs in lfsr_mdir_commit before updating
  the mdir. If we have multiple attrs this causes the assert on
  rid-in-bounds to trigger incorrectly. Just removed that assert for now.

- We needed to adjust second grms if they are affected by the fixing
  of the first grm.

- Directory position updates are incorrectly updated if an unrelated
  weight change occurs before an opened directory, but is not a part of
  that opened directory.

  This is NOT fixed yet, the current implementation is just broken
  enough that I've just ripped it out for now (it was causing the
  read_with_rms test to fail because pos backed up into the "."/".."
  entries).

  This needs some thinking to fix.

Because of that last, unfixed bug, tests are not all passing at the
moment. To pass testing -DSEEK=0 is needed to disable the failing tests.
This commit is contained in:
Christopher Haster
2023-07-26 15:22:36 -05:00
parent d931c19dda
commit e0f416f6ca
3 changed files with 36 additions and 36 deletions
-1
View File
@@ -497,7 +497,6 @@ typedef struct lfs {
// begin lfsr things
lfsr_mdir_t mroot;
lfsr_btree_t mtree;
lfs_size_t mlimit;
// TODO do we really need separate decoded/encoded grms?
lfsr_grm_t grm_;