Started adopting lazy attr encoding

The idea here is to move as much attr encoding logic as possible into
lfsr_rbyd_appendrattr_, so we don't encode most attrs until the last
minute, right before we write the tag+data to disk.

This has some pretty big theoretical benefits:

- Deduplicates encoding logic, so most attrs will only have a single
  lfsr_data_from* call in the entire system.

  This saves code size used for function calls, stack allocations, etc.

- In theory, _significantly_ better stack usage.

  The main downside with eager encoding is that we need a buffer to
  hold the encoding, and this buffer needs to stay allocated while all
  of the commit machinery does its work.

  This ends up stacking when any low-level attr buffers in
  lfsr_btree_commit/lfsr_mdir_commit/etc, even though we don't _really_
  need all of these attrs encoded at the same time.

  Heck, we don't even need all of the attrs in the same _commit_ to be
  encoded at the same time.

  Lazily encoding avoids all of this.

- It's actually a nicer internal API, and means less risk we lose/
  misallocate one of the encoding buffers.

The main downside is this makes attr encodings less gc-able. However, so
far it seems like you need most tags the moment you try to write to the
filesystem, and unwanted code costs can be worked around by allowing
more code to be conditionally compiled-out (at a testing cost).

This also means we don't know the actual on-disk attr size until we're
writing attrs out to disk. Fortunately, we've ended up relying on attr
size less than I thought we would. We still need it for shrub estimates,
but we can use the worst-case encoding size (LFSR_BPTR_DSIZE) there.

---

To start, this adopts lazy attr encoding for most of the obvious/
less-involved attrs:

- LFSR_TAG_BSHRUB ---> lfsr_data_fromshrub
- LFSR_TAG_BTREE  -+-> lfsr_data_frombtree
- LFSR_TAG_MTREE  -'
- LFSR_TAG_MROOT  -+-> lfsr_data_frommptr
- LFSR_TAG_MDIR   -'
- LFSR_TAG_ECKSUM ---> lfsr_data_fromecksum

Of interesting note is LFSR_TAG_BSHRUB. These changes actually make
shrub trunk encoding less of a special case, which _must_ be lazily
encoded due to last minute shrub changes caused by mdir compactions,
relocations, etc. This lets us drop the unique LFSR_TAG_SHRUBTRUNK
handling.

Though it does risk bugs if a future refactor ever reverts to eager
encoding... I've tried to highlight this with comments around
LFSR_TAG_BSHRUB's encoding.

These changes also required moving a significant number of the
LFSR_*_DSIZE macros around so they are declared before
lfsr_rbyd_appendrattr_. This is unfortunate as it moves them farther
away from from the related lfsr_data_from* implementations, but as far
as I'm aware there's no way around this.

We also need to _not_ lazily encode when an attr is in the concatenated-
data form (count < 0), or else this breaks mdir compaction. This has the
interesting side-effect of still allowing eager encoding with
LFSR_DATA_BUF, which, while less efficient, is very useful for our
tests.

---

So far, code/stack changes look promising:

           code          stack          ctx
  before: 36280           2576          636
  after:  35848 (-1.2%)   2504 (-2.8%)  636 (+0.0%)
This commit is contained in:
Christopher Haster
2025-02-09 04:43:51 -06:00
parent 76e0f8f73c
commit 11c30929e9
3 changed files with 293 additions and 257 deletions
+5 -6
View File
@@ -1107,9 +1107,9 @@ code = '''
LFSR_RATTR(LFSR_TAG_RM, -1, LFSR_DATA_NULL())}),
1))) => 0;
lfsr_mdir_commit(&lfs, &file.b.o.mdir, LFSR_RATTRS(
LFSR_RATTR_SHRUBTRUNK(
LFSR_TAG_SUB | LFSR_TAG_SHRUBTRUNK, 0,
&file.b))) => 0;
LFSR_RATTR__(
LFSR_TAG_SUB | LFSR_TAG_BSHRUB, 0,
&file.b.shrub_, LFSR_SHRUB_DSIZE))) => 0;
lfsr_file_close(&lfs, &file) => 0;
@@ -1229,11 +1229,10 @@ code = '''
LFSR_RATTR(LFSR_TAG_DATA, +1, LFSR_DATA_BUF("?", 1)))) => 0;
lfsr_rbyd_commit(&lfs, &file.b.shrub, 0, LFSR_RATTRS(
LFSR_RATTR(LFSR_TAG_RM, -1, LFSR_DATA_NULL()))) => 0;
uint8_t buf[LFSR_BTREE_DSIZE];
lfsr_mdir_commit(&lfs, &file.b.o.mdir, LFSR_RATTRS(
LFSR_RATTR(
LFSR_RATTR__(
LFSR_TAG_SUB | LFSR_TAG_BTREE, 0,
LFSR_DATA_BTREE(&file.b.shrub, buf)))) => 0;
&file.b.shrub, LFSR_BTREE_DSIZE))) => 0;
lfsr_file_close(&lfs, &file) => 0;
+2 -5
View File
@@ -4341,13 +4341,10 @@ code = '''
lfsr_mount(&lfs, LFS_M_RDWR | M_FLAGS, CFG) => 0;
lfs_alloc_ckpoint(&lfs);
uint8_t mptr_buf[LFSR_MPTR_DSIZE];
lfsr_mdir_commit(&lfs, &lfs.mroot, LFSR_RATTRS(
LFSR_RATTR(
LFSR_RATTR__(
LFSR_TAG_MROOT, 0,
LFSR_DATA_MPTR(
LFSR_MPTR_MROOTANCHOR(),
mptr_buf)))) => 0;
LFSR_MPTR_MROOTANCHOR(), LFSR_MPTR_DSIZE))) => 0;
// technically, cycle detection only needs to work when we're validating
lfsr_traversal_t t;