Instead of reading eagerly and retreating with the hopes of terminating
early (which almost never happens when compacting, since we need to find
the split_id). lfsr_rbyd_estimate now works inward from the first and
last id to find both the dsize and split_id.
One thing that helps this is the addition of a separate per-id
lfsr_rbyd_estimate, which will be useful for checking if the quantity of
file attributes overflows our mdir limitations.
lfsr_rbyd_estimate also now ignores the -1 id for split_id calculation,
since -1 ids are always cleaned up during splitting, though it does
include it in the calculated dsize so that the condition to split is
determined correctly.
---
This also required rebalance changes. Fortunately, one improvement here
is that we can make a simplifying assumption tha the number of tags
can't exceed the maximum possible number of tags in the calculated
dsize. So worst case, if every tag is empty, the maximum possible dsize
becomes 4*(2*log2(dsize/4))+dsize.
Though it's still unclear if rebalance is worth keeping. Current
comparison:
code stack
rebalance: 22362 2120
no_rebalance: 21922 (-2.0%) 2120 (+0.0%)
Christopher Haster
9d0edea7e3