Fixed mdir drop during compaction breaking fixorphan loop

The core problem is that we weren't updating dropped mdirs with weight=0
if the mdir was compacted at the same time. This is hard to notice,
because most operations that can drop don't care about the mdir
afterwards, but in lfsr_fs_fixorphans this caused the fixorphan loop to
think it might still have orphans it could remove.

The implementation is very subtle here:

- In lfsr_mdir_commit_, if an error occurs during lfsr_mdir_compact__,
  we need to revert to the original mdir state to allow fallback to mdir
  split.

- In lfsr_mdir_commit_, if an error occurs during lfsr_mdir_commit__
  (even after a compact), we need to update the mdir in case a drop
  reduced the mdir weight to zero.

  We also need to update the mdir for things like erased state, but this
  doesn't come into play in the compaction route.

Fixed the bug by updating the mdir copy before lfsr_mdir_commit__.

Also added asserts to all insert/delete operations in test_mtree.toml.
We already had drop-during-compaction tests, but these didn't check that
the mdir was updated correctly. The new asserts catch this bug and
should prevent a regression.
This commit is contained in:
Christopher Haster
2024-03-03 13:25:46 -06:00
parent a6a5f43027
commit 34be5055b4
2 changed files with 96 additions and 6 deletions
+8 -6
View File
@@ -6300,18 +6300,20 @@ compact:;
return err;
}
// update mdir, we need to propagate mdir changes if commit fails
*mdir = mdir_;
// now try to commit again
//
// upper layers should make sure this can't fail by limiting the
// maximum commit size
err = lfsr_mdir_commit__(lfs, &mdir_, start_rid, end_rid,
err = lfsr_mdir_commit__(lfs, mdir, start_rid, end_rid,
mid, attrs, attr_count);
if (err) {
LFS_ASSERT(err != LFS_ERR_RANGE);
return err;
}
*mdir = mdir_;
return 0;
}
@@ -8498,7 +8500,7 @@ static int lfsr_fs_preparemutation(lfs_t *lfs) {
lfs_alloc_ckpoint(lfs);
// fix pending grms
bool pl = false;
bool inconsistent = false;
if (lfsr_grm_hasrm(&lfs->grm)) {
if (lfsr_grm_count(&lfs->grm) == 2) {
LFS_DEBUG("Fixing grm "
@@ -8512,7 +8514,7 @@ static int lfsr_fs_preparemutation(lfs_t *lfs) {
lfsr_mid_bid(lfs, lfs->grm.rms[0]) >> lfs->mleaf_bits,
lfsr_mid_rid(lfs, lfs->grm.rms[0]));
}
pl = true;
inconsistent = true;
int err = lfsr_fs_fixgrm(lfs);
if (err) {
@@ -8531,7 +8533,7 @@ static int lfsr_fs_preparemutation(lfs_t *lfs) {
//
if (lfs->hasorphans) {
LFS_DEBUG("Fixing orphans...");
pl = true;
inconsistent = true;
int err = lfsr_fs_fixorphans(lfs);
if (err) {
@@ -8543,7 +8545,7 @@ static int lfsr_fs_preparemutation(lfs_t *lfs) {
lfs_alloc_ckpoint(lfs);
}
if (pl) {
if (inconsistent) {
LFS_DEBUG("littlefs is now consistent");
}
return 0;