Github/littlefs
1
Fork 0
Code Actions Activity

rattrs: Converted rattrs to full variable-length isa

Browse Source 
It's funny to see what originally started as a simple list of rbyd attrs
slowly morph into a full isa. But it makes sense. What we really want is
an abstract description of operations that can be played and replayed as
necessary to atomically update the mtree.

Using a fixed lfs3_rattr_t struct to represent this in C is easy, and
avoids strict-aliasing issues, but ultimately limited when it comes to
the wide-range of data we want to attach to attributes.

Unlike a computer's isa, we want to be able to include full 12-24 byte
branch pointers directly in the instruction!

---

So here's a full variable-length isa organized by words (max(uintptr_t,
uint32_t)).

The first 32-bit word extends the 16-bit tag with an extra 16-bits of
control information:

  wwll llff ffcc cccc tttt tttt tttt tttt
   ^'-.-''-.-''--.--' :                 :
   '--|----|-----|----:-----------------:-- compressed weight
  ::  '----|-----|----:-----------------:-- total len
  ::       '-----|----:-----------------:-- from encoder
  ::             '----:-----------------:-- optional count
  ::                  rgmm kkkk -kkk kkkk
  11 => w=-1          ^^ ^ '-.' '---.---'
  00 => w=0           '|-|---|------|------ rm bit
  01 => w=+1           '-|---|------|------ grow bit
  10 => w=attached       '---|------|------ mask bits
                             '------|------ tag suptype
                                    '------ tag subtype

The 4-bit length field always encodes the full length of the
instruction, including the instruction itself and optional weight. The
4-bit from + 6-bit count fields operate independently and tell
lfs3_rbyd_appendrattr_ how to actually encode the data related to the
instruction.

To work around strict-aliasing issues, complex structs are expected to
be broken down into words and reconstructed in lfs3_rbyd_appendrattr_.
Most of our structs are organized into words anyways. For example:

  // new child
  *r++ = LFS3_RATTR(5, LFS3_TAG_BRANCH, -2, LFS3_FROM_BRANCH);
  *r++ = LFS3_RATTR_WEIGHT(+child_->weight);
  *r++ = LFS3_RATTR_ARG(child_->blocks[0]);
  *r++ = LFS3_RATTR_ARG(child_->trunk);
  *r++ = LFS3_RATTR_ARG(child_->cksum);

This also changes rattr-lists to be null-terminated, which makes a bit
more sense in a variable-length isa:

  *r++ = LFS3_RATTR_NULL; // all zeros, including length

One concern with null-terminated rattr-lists is how easy it is to
forget the null-terminator, but an assert that all non-null rattrs have
non-zero length seemed to catch the many many mistakes during adoption.

Alternatively, separate LFS3_FROM_NULL/LFS3_FROM_NIL from fields could
be used if encoding space gets tight.

I'm also quite happy with the 2-bit weight feild, which allows omitting
the optional weight word for -1,0,+1 weights. These should cover at
least all mdir operations.

Note the exact encoding of the rattr fields is less of a concern than
the tag fields, as it doesn't reside on-disk can be changed on whim.

---

Saves a nice chunk of code and stack:

                 code          stack          ctx
  before:       35920           2280          660
  after:        35324 (-1.7%)   2176 (-4.6%)  660 (+0.0%)

                 code          stack          ctx
  gbmap before: 38812           2296          772
  gbmap after:  38156 (-1.7%)   2192 (-4.5%)  772 (+0.0%)

The stack savings are obvious, but the code savings a bit less so. A
variable length isa _is_ more complicated, but by limiting most encoding
decisions to compile-time (2-bit weights vs 32-bit weights for example),
the savings from fewer word manipulations on the stack wins.
This commit is contained in:
Christopher Haster
2025-11-29 19:15:54 -06:00
parent 20747cc4c6
commit dca915dd95
11 changed files with 4546 additions and 4153 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_files.toml
+29 -24
View File
@@ -978,8 +978,8 @@ code = '''
// delete any bshrub/btree
lfs3_mdir_commit(&lfs3, &file.b.h.mdir, LFS3_RATTRS(
LFS3_RATTR(
LFS3_tag_RM | LFS3_tag_MASK8 | LFS3_TAG_STRUCT, 0))) => 0;
LFS3_RATTR(1, LFS3_tag_RM | LFS3_tag_MASK8 | LFS3_TAG_STRUCT, 0),
LFS3_RATTR_NULL)) => 0;
lfs3_file_close(&lfs3, &file) => 0;
@@ -1094,25 +1094,27 @@ code = '''
// create an empty bshrub
lfs3_mdir_commit(&lfs3, &file.b.h.mdir, LFS3_RATTRS(
LFS3_RATTR_SHRUBCOMMIT(
(&(lfs3_shrubcommit_t){
.bshrub=&file.b,
.rid=0,
.rattrs=((lfs3_rattr_t[]){
LFS3_RATTR_BUF(LFS3_TAG_DATA, +1, "?", 1)}),
.rattr_count=1})))) => 0;
LFS3_RATTR(4, LFS3_tag_SHRUBCOMMIT, 0),
LFS3_RATTR_ARG(&file.b),
LFS3_RATTR_ARG(0),
LFS3_RATTR_ARG(LFS3_RATTRS(
LFS3_RATTR(2, LFS3_TAG_DATA, +1, LFS3_FROM_BUF, 1),
LFS3_RATTR_ARG("?"),
LFS3_RATTR_NULL)),
LFS3_RATTR_NULL)) => 0;
lfs3_mdir_commit(&lfs3, &file.b.h.mdir, LFS3_RATTRS(
LFS3_RATTR_SHRUBCOMMIT(
(&(lfs3_shrubcommit_t){
.bshrub=&file.b,
.rid=0,
.rattrs=((lfs3_rattr_t[]){
LFS3_RATTR(LFS3_tag_RM, -1)}),
.rattr_count=1})))) => 0;
LFS3_RATTR(4, LFS3_tag_SHRUBCOMMIT, 0),
LFS3_RATTR_ARG(&file.b),
LFS3_RATTR_ARG(0),
LFS3_RATTR_ARG(LFS3_RATTRS(
LFS3_RATTR(1, LFS3_tag_RM, -1),
LFS3_RATTR_NULL)),
LFS3_RATTR_NULL)) => 0;
lfs3_mdir_commit(&lfs3, &file.b.h.mdir, LFS3_RATTRS(
LFS3_RATTR_SHRUB(
LFS3_tag_MASK8 | LFS3_TAG_BSHRUB, 0,
&file.b.b_))) => 0;
LFS3_RATTR(2, LFS3_tag_MASK8 | LFS3_TAG_BSHRUB, 0,
LFS3_FROM_SHRUB),
LFS3_RATTR_ARG(&file.b.b_),
LFS3_RATTR_NULL)) => 0;
lfs3_file_close(&lfs3, &file) => 0;
@@ -1229,13 +1231,16 @@ code = '''
lfs3_alloc_ckpoint(&lfs3);
lfs3_rbyd_alloc(&lfs3, &file.b.b.r) => 0;
lfs3_rbyd_commit(&lfs3, &file.b.b.r, 0, LFS3_RATTRS(
LFS3_RATTR_BUF(LFS3_TAG_DATA, +1, "?", 1))) => 0;
LFS3_RATTR(2, LFS3_TAG_DATA, +1, LFS3_FROM_BUF, 1),
LFS3_RATTR_ARG("?"),
LFS3_RATTR_NULL)) => 0;
lfs3_rbyd_commit(&lfs3, &file.b.b.r, 0, LFS3_RATTRS(
LFS3_RATTR(LFS3_tag_RM, -1))) => 0;
LFS3_RATTR(1, LFS3_tag_RM, -1),
LFS3_RATTR_NULL)) => 0;
lfs3_mdir_commit(&lfs3, &file.b.h.mdir, LFS3_RATTRS(
LFS3_RATTR_BTREE(
LFS3_tag_MASK8 | LFS3_TAG_BTREE, 0,
&file.b.b))) => 0;
LFS3_RATTR(2, LFS3_tag_MASK8 | LFS3_TAG_BTREE, 0,
LFS3_FROM_BTREE),
LFS3_RATTR_ARG(&file.b.b))) => 0;
lfs3_file_close(&lfs3, &file) => 0;