btree: Moved leaf caching behind LFS3_BLEAFCACHE ifdef

This is motivated by the observation that the O(n log_b n) btree
iteration really just hasn't been a bottleneck in our benchmarks.

Our write performance is mostly dominated by compaction costs, and while
filesystem _traversals_ are a concern, it's easy to explicitly track
rbyds in lfs3_btrv_t.

Additionally:

- We track mdirs during mtree iteration, which are the true mtree
  leaves.

- We already cache file leaves, i.e. bptrs and read-fragments.

On top of this, leaf caching adds complexity, both in terms of
code/stack costs, but also in terms of reliability. It introducing the
need for cache invalidation, which is infamously one of the two hard
problems in computer science!

This is the second(?) time btree leaf traversals have been reverted, so
see previous commit messages for even more arguments against.

---

Eventually, we should probably just delete the btree leaf cache logic to
avoid the maintenance headache (cache invalidation + opt+in/less
testing = ouch). But I want to do a bit more benchmarking comparing the
two modes, so just moving this behind an ifdef for now.

Saves code, and of course RAM:

                              code          stack          ctx
  before btrv:               37160           2352          688
  before:                    37088 (-0.2%)   2384 (+1.4%)  688 (+0.0%)
  after:                     36480 (-1.8%)   2304 (-2.0%)  660 (-4.1%)

But note while this keeps the performance implications of btree leaf
caching, it does not keep the code/stack optimizations that internally
reuse the leaf cache for things (btrv, lookupnext_ rbyd side-channel,
etc).

In _theory_ these could have been kept with enough ifdefs, but it would
have made the codebase quite a bit of a hell to maintain:

                              code          stack          ctx
  always-bleafcache:         37160           2352          688
  no-bleafcache:             36480 (-1.8%)   2304 (-2.0%)  660 (-4.1%)
  yes-bleafcache:            37044 (-0.3%)   2384 (+1.4%)  688 (+0.0%)

Gbmap mode has even more savings due to how many gbmap copies we have
flying around:

                              code          stack          ctx
  gbmap + always-bleafcache: 40132           2368          856
  gbmap + no-bleafcache:     39464 (-1.7%)   2320 (-2.0%)  772 (-9.8%)
  gbmap + yes-bleafcache:    40052 (-0.2%)   2400 (+1.4%)  856 (+0.0%)

---

In the future, _maybe_ we can revisit this. But I think a better design
would be to cache btree leaves globally, in lfs3_t, similarly to the
theoretical mdir cache. This would allow a user-configurable number of
cached btree nodes, and may make cache invalidation easier.

Note, however, that btree nodes don't need to be fetched (even for
commits now!), so the benefits would be much smaller than for the
theoretical mdir cache.

But hey, it would defend the lack of low-level rbyd tracking during
iteration/rattr queries!
This commit is contained in:
Christopher Haster
2025-10-26 13:39:33 -05:00
parent 39a265ce90
commit a01b1b73b2
3 changed files with 125 additions and 51 deletions
+2
View File
@@ -776,10 +776,12 @@ typedef struct lfs3_rbyd {
// performance
typedef struct lfs3_btree {
lfs3_rbyd_t r;
#ifdef LFS3_BLEAFCACHE
struct {
lfs3_bid_t bid;
lfs3_rbyd_t r;
} leaf;
#endif
} lfs3_btree_t;
// littlefs's atomic metadata log type