mdirs behave a bit differently than btree nodes here. When an mdir's
weight drops to zero, we eagerly drop the mdir. Unfortunately this
introduce a large number of conditions into lfsr_mdir_commit. Maybe
there's some different way to structure to code to avoid this...
Also expanded mtree tests to cover more corner cases, these are
desperately for any confidence that mdir drops work.
This isn't the greatest coverage as we don't have a verifiable simulation.
Simulating the splitting-bucket-tree that is the mtree is tricky.
So right now this mostly just checks there's no internal assert failures and
if we have the expected number of entries afterwards.
Currently relying on lfsr_rbyd_append/appendattrs to inject extra
attributes during lfsr_mdir_commit, need to consider if this is really
the best solution. This probably results in more function calls than we
really need.
This became surprisingly tricky.
The main issue is knowing when to split mdirs, and how to determine
this without wasting erase cycles.
Unlike splitting btree nodes, we can't salvage failed compacts here. As
soon as the salvage commit is written to disk, the commit becomes immediately
visibile to the filesystem because it still exists in the mtree. This is
a problem if we lose power.
We're likely going to need to implement rbyd estimates. This is
something I hoped to avoid because it brings in quite a bit of
complexity and might lead to an annoying amount of storage waste since
our estimates will need to be conservative to avoid unrecoverable
situations.
---
Also changed the on-disk btree/branch struct to store a copy of the weight.
This was already required for the root of the btree, requiring the
weight to be stored in every btree pointer allows better code
deduplication at the cost of some redundancy on btree branches, where
the weight is already implied by the rbyd structure.
This weight is usually a single byte for most branches anyways.
This may be worth revisiting at some point to see if there's any other
unexpected tradeoffs.
Christopher Haster