Fixed grafting allocator checkpoint hole

This was quite a deep bug.

We don't track the original bshrub when grafting, so it was possible to
realloc those blocks even when we need their contents to finish the
graft operation.

This was found while experimenting with eager leaf grafting, but can
also occur when grafting data fragments.

---

In theory, the block allocator's checkpoint mechanism protects against
this.

Before we alloc, we set a checkpoint with lfs3_alloc_ckpoint. This marks
the position of the block allocator before allocation, so if we loop
around the entire block device we don't double alloc any in-flight
blocks:

                     ckpoint      lookahead
                        v         .---'---.
  [mm---ddd-d---d-------|dd--d-ddd|--------d-----d-]
                         '---.---'
                    in-flight allocations

But this only protects _new_ blocks, _old_ blocks can be anywhere on
disk and are unprotected.

In theory again, old blocks are always tracked via copy-on-write
snapshots, but this is not the case for bshrubs while grafting!

Grafting is unfortunately a multi-commit operation (we may remove
multiple fragments that span different btree nodes), and each bshrub
commit discards the old snapshot. This creates a window where old blocks
can be double alloced _while grafting_, leading to corrupted data.

You may wonder why are we discarding the old snapshot? Why not keep
track of it until the grafting completes?

The problem there is that we need the intermediate snapshot in order for
shrubs to survive compactions. We really have 3 states:

  old -> mid-graft -> new

And the only one we don't need to fallback to is the old state.

---

A couple solutions:

1. Track all three states

   This would add complexity increase the cost of every lfs3_file_t.

2. Open a temporary file to track the old state

   This would add complexity and a big chunk of stack to what is already
   one of the critical functions on our stack hot-path.

3. Carefully make sure graft commits don't lose track of in-flight data
   until an atomic commit

   This doesn't work when you're trying to coalesce two data fragments
   in two different btree nodes. At least not without completely
   restructuring the btree commit logic.

4. Just explicitly track in-flight graft state out-of-band

This goes with option no 4., adding lfs3->graft and lfs3->graft_count to
track in-flight graft state when we're grafting. lfs3_mtree_traverse_
can include the relevant blocks during traversals, effectively masking
out graft state from the lookahead buffer.

This adds a bit of code/ctx, but is probably the cheapest option:

           code          stack          ctx
  before: 37936           2456          636
  after:  38092 (+0.4%)   2456 (+0.0%)  656 (+3.1%)
This commit is contained in:
Christopher Haster
2025-06-30 15:23:35 -05:00
parent 13fbd2f006
commit 1bf2a4b520
2 changed files with 89 additions and 23 deletions
+7
View File
@@ -793,6 +793,8 @@ typedef struct lfs3_traversal {
} mtortoise;
// btree traversal state
lfs3_btraversal_t bt;
// graft traversal state
lfs3_size_t gt;
} u;
// recalculate gcksum when traversing with ckmeta
@@ -872,6 +874,11 @@ typedef struct lfs3 {
} lookahead;
#endif
#if !defined(LFS3_RDONLY) && !defined(LFS3_2BONLY)
const lfs3_data_t *graft;
lfs3_ssize_t graft_count;
#endif
// global state
uint32_t gcksum;
#ifndef LFS3_RDONLY