Adopted a tighter, block-size dependendent attr-estimate
The concern right now is small-block filesystems, anything in the 512B
to <4KiB range. With such small blocks, and rbyd's relatively high
per-attr overhead, there's a real risk that littlefs may just not be
able to function without quickly running to metadata limits.
I realize these are pretty rare geometries for flash, but they are still
common for anything that 1. pretends to be a spinny disks, SD cards,
FTLs, eMMCs, etc, and 2. mapping into RAM, which is surprisingly common.
It is possible to require this sort of geometry to pretend to be a
larger logical block-size, but since this is a regression from the
previous version of littlefs, it would be nice to avoid this if
possible.
Anyways, what actually is this commit. Consider our tag encoding:
.---+---+---+- -+- -+- -+- -+---+- -+- -+- -. tag: 2 bytes
| tag | weight | size | weight: <=5 bytes
'---+---+---+- -+- -+- -+- -+---+- -+- -+- -' size: <=4 bytes
total: <=11 bytes
With our current 32-bit (really 31-bit) version of littlefs, the worst
case tag encoding is 11 bytes.
This doesn't sound that bad, but with our current compaction algorithm we
need ~2.5 tags for each attr:
5t 5*11
a_1 = -- + 2 = ---- = 30 bytes
2 2
Are there any additional assumptions we can make to push our attr
estimate lower?
- tag - Ignoring a complete redesign of our tag encoding (which has
already been heavily iterated over), this just needs 2 bytes, which is
not that bad.
- weight - This is the real painful one because, for the most part,
weight=0. But weight _can_ store a full size, in the case it is the
root of a file's btree. So this is pretty much stuck at an annoying
5 bytes.
I suppose this could be tied to our size-limit. I hadn't thought about
that until writing this commit message. Maybe that can be a future
improvement, though it won't really have a big effect on most systems.
- size/jump - Now this field is interesting. When expressing both the
size of tag payloads, and the relative jump offset for alt-pointers,
this field should never exceed a single block.
We've already pushed this down to 4 bytes at compile time, by assuming
at most 28-bit block-sizes, but if we know the block-size, we could in
theory push this even lower.
This is extra enticing, because the block-sizes where the size/jump
field can be shrunk, are _also_ the block-sizes where the metadata
density is so critical!
So that's what this commit does. For the purpose of compaction estimates
(not stack allocations!) we calculate attr estimate based on our
runtime-determined block_size.
Here are some cutoff points for our new attr estimate:
block-size tag-estimate attr-estimate
512B => 9 bytes 25 bytes
16KiB => 10 bytes 27 bytes
2MiB => 11 bytes 30 bytes
256MiB => 12 bytes 32 bytes
There is a question of when to actually do this calculation. We always
know our block-size, so we could recalculate the attr-estimate every
time we need to estimate a compaction. But for now I'm just
precalculating the attr estimate in lfs_init and storing in the lfs_t
struct. It's only a byte after all.
If I did my math correctly, we won't exceed a byte until we have a
block-size of 2^1750, at which point we may have other problems.
Code changes:
code stack lfs_t
before: 34068 2880 216
after: 34104 (-0.1%) 2880 (+0.0%) 220 (+1.9%)
The jump in lfs_t cost is probably just from a word alignment boundary.
In the future, if we have compile-time block-sizes, the entire
attr-estimate could even be compile-time.
This commit is contained in:
Reference in New Issue
Block a user