Changed mtree to be weighted by mdir upper bound

Browse Source

More on this when explaining compressed mids, but basically the idea is
instead of just storing all mdirs in our mtree as single element
entries, store each mdir in as a weighted entry, where the weight is a
known upper bound on the possible number of mid entries in a single
mdir.

With the current mid representation, this just complicates things
without much benefits. But with compressed mids it allows us to lookup
mdirs with the mid directly, and avoid decoding the bid from the mid in
some cases.

The mid-per-mdir upper bound is derived from the block size. We know:

1. Each tag needs <=2 alts+null with our current compaction strategy
2. Each tag/alt encodes to a minimum of 4 bytes

This gives us ~4*4 or ~16 bytes per mid at minimum. If we cram an mdir
with the smallest possible mids, this gives us at most ~block_size/16
mids in a single mdir before the mdir runs out of space.

Note we can't assume ~1/2 block utilization here, as an mdir may
temporarily fill with more mids before compaction occurs.

...

This commit is contained in:

Christopher Haster

2023-08-27 21:14:26 -05:00

parent af5f4bff21

commit 94941806c7

4 changed files with 231 additions and 198 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

tests/test_mtree.toml

+175 -175

File diff suppressed because it is too large Load Diff