btree: Moved leaf caching behind LFS3_BLEAFCACHE ifdef
This is motivated by the observation that the O(n log_b n) btree
iteration really just hasn't been a bottleneck in our benchmarks.
Our write performance is mostly dominated by compaction costs, and while
filesystem _traversals_ are a concern, it's easy to explicitly track
rbyds in lfs3_btrv_t.
Additionally:
- We track mdirs during mtree iteration, which are the true mtree
leaves.
- We already cache file leaves, i.e. bptrs and read-fragments.
On top of this, leaf caching adds complexity, both in terms of
code/stack costs, but also in terms of reliability. It introducing the
need for cache invalidation, which is infamously one of the two hard
problems in computer science!
This is the second(?) time btree leaf traversals have been reverted, so
see previous commit messages for even more arguments against.
---
Eventually, we should probably just delete the btree leaf cache logic to
avoid the maintenance headache (cache invalidation + opt+in/less
testing = ouch). But I want to do a bit more benchmarking comparing the
two modes, so just moving this behind an ifdef for now.
Saves code, and of course RAM:
code stack ctx
before btrv: 37160 2352 688
before: 37088 (-0.2%) 2384 (+1.4%) 688 (+0.0%)
after: 36480 (-1.8%) 2304 (-2.0%) 660 (-4.1%)
But note while this keeps the performance implications of btree leaf
caching, it does not keep the code/stack optimizations that internally
reuse the leaf cache for things (btrv, lookupnext_ rbyd side-channel,
etc).
In _theory_ these could have been kept with enough ifdefs, but it would
have made the codebase quite a bit of a hell to maintain:
code stack ctx
always-bleafcache: 37160 2352 688
no-bleafcache: 36480 (-1.8%) 2304 (-2.0%) 660 (-4.1%)
yes-bleafcache: 37044 (-0.3%) 2384 (+1.4%) 688 (+0.0%)
Gbmap mode has even more savings due to how many gbmap copies we have
flying around:
code stack ctx
gbmap + always-bleafcache: 40132 2368 856
gbmap + no-bleafcache: 39464 (-1.7%) 2320 (-2.0%) 772 (-9.8%)
gbmap + yes-bleafcache: 40052 (-0.2%) 2400 (+1.4%) 856 (+0.0%)
---
In the future, _maybe_ we can revisit this. But I think a better design
would be to cache btree leaves globally, in lfs3_t, similarly to the
theoretical mdir cache. This would allow a user-configurable number of
cached btree nodes, and may make cache invalidation easier.
Note, however, that btree nodes don't need to be fetched (even for
commits now!), so the benefits would be much smaller than for the
theoretical mdir cache.
But hey, it would defend the lack of low-level rbyd tracking during
iteration/rattr queries!
This commit is contained in:
Reference in New Issue
Block a user