Github/littlefs
1
Fork 0
Code Actions Activity

Added REVPERTURB, reworked how we handle revision counts

Browse Source 
The main change is adding LFS3_M_REVPERTURB, which will be necessary for
preerase allocations, but I got distracted and ended up giving the
revision count subsystem a bit of a refactor.

Main changes:

- Added LFS3_M_REVPERTURB, which ensures the leading bit in the
  revision count changes after each allocation/relocation/compaction.

  This is generally optional, but will be required for preerase
  allocations. Our ecksum system is only reliable if we ensure at least
  one bit changes, otherwise the chance of ecksum collision is very
  high.

  The downside of LFS3_M_REVPERTURB is that we need to read the contents
  of the new block to figure out what the bit should change to. Probably
  a minimal cost in the system, but still a good reason to make the
  behavior optional.

  Does LFS3_M_REVPERTURB have any use outside of preerased allocation?
  I'm not sure. Maybe it has some niche use reducing the chance of bd
  ECC collisions?

- Dropped LFS3_M_REVDBG, but adding low-effort debug bits that are
  always enabled.

  Making LFS3_M_REVDBG conditional was probably overkill. The flag
  checks probably cost more than the actual debug bits when enabled.

  Instead, replaced with a simpler, low-effort debug bit system, where
  we only set the debug bits during mdir allocation/relocation. These
  bits shouldn't change during normal compaction, but we _don't_
  introduce debug bits if mounting a filesystem from a driver without
  these debug bits.

- Restricted recycle counter to at most 20-bits to make space for
  things. This ensures perturb/debug bits don't get overwritten (though
  we really only care about perturb bits).

  2^20 (~1M) recycles is probably enough for any device littlefs will
  run on, especially considering the recycle_count should probably be
  several orders of magnitude smaller than the device's expected erase
  cycles.

  Worst case this can always be increased in the future without
  backwards incompatible changes. The only hard requirement for revision
  counts is that the full 32-bits are comparable.

- Simplified lfs3_rev_inc and friends, and moved most of the
  disk-dependent revision count stuff down into lfs3_rbyd appendrev.

  This deduplicates the messy revision count handling in
  lfs3_btree_commit_.

  Though note the implicit lfs3_rbyd_appendrev now defaults to writing
  the btree debug bits ('b'). A bit of a hack, but works for littlefs.

Here's the resulting encoding:

  vvvv---- -------- -------- -ddddddd
  vvvvrrrr rrrrrr-- -------- -ddddddd
  vvvvrrrr rrrrrrnn nnnnnnnn pddddddd
  '-.''----.----''----.----' ^'--.--'
    '------|----------|------|---|---- 4-bit relocation revision
           '----------|------|---|---- recycle-bits recycle counter
                      '------|---|---- pseudorandom noise (if revnoise)
                             '---|---- perturb bit (if revperturb)
                                 '---- low-effort debug bits
                              11-1---  - h = mroot anchor
                              11-11-1  - m = mdir
                              11---1-  - b = btree node

Note we store revision counts as le32s, so the perturb bit should end up
as the leading bit in the first byte.

Costs a bit more code (mostly because the debug bits are now
unconditional, even if low-effort), but simplifies the codebase:

                        code          stack          ctx
  before:              35124           2136          660
  after:               35144 (+0.1%)   2136 (+0.0%)  660 (+0.0%)
  after+yesrevperturb: 35192 (+0.2%)   2136 (+0.0%)  660 (+0.0%)

                        code          stack          ctx
  gbmap+np before:     38252           2144          776
  gbmap+np after:      38272 (+0.1%)   2144 (+0.0%)  776 (+0.0%)
  gbmap+np after+yrp:  38328 (+0.2%)   2144 (+0.0%)  776 (+0.0%)

                        code          stack          ctx
  gbmap+yp before:     38832           2168          796
  gbmap+yp after:      38852 (+0.1%)   2168 (+0.0%)  796 (+0.0%)
  gbmap+yp after+yrp:  38908 (+0.2%)   2168 (+0.0%)  796 (+0.0%)
This commit is contained in:
Christopher Haster
2025-12-29 16:50:04 -06:00
parent 2b1f2e3ca9
commit b3ab83d5b5
8 changed files with 220 additions and 320 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/dbgflags.py
+3 -3
View File
@@ -75,7 +75,7 @@ F_MODE = 1 # -m Format's access mode
F_RDWR = 0 # -^ Format the filesystem as read and write
F_GBMAP = 0x02000000 # y- Use the global on-disk block-map
F_REVDBG = 0x00000010 # y- Add debug info to revision counts
F_REVPERTURB = 0x00000010 # y- Perturb first bit in revision count
F_REVNOISE = 0x00000020 # y- Add noise to revision counts
F_CKPROGS = 0x00100000 # y- Check progs by reading back progged data
F_CKFETCHES = 0x00200000 # y- Check block checksums before first use
@@ -97,7 +97,7 @@ M_RDWR = 0 # -^ Mount the filesystem as read and write
M_RDONLY = 1 # -^ Mount the filesystem as read only
M_FLUSH = 0x00000040 # y- Open all files with LFS3_O_FLUSH
M_SYNC = 0x00000080 # y- Open all files with LFS3_O_SYNC
M_REVDBG = 0x00000010 # y- Add debug info to revision counts
M_REVPERTURB = 0x00000010 # y- Perturb first bit in revision count
M_REVNOISE = 0x00000020 # y- Add noise to revision counts
M_CKPROGS = 0x00100000 # y- Check progs by reading back progged data
M_CKFETCHES = 0x00200000 # y- Check block checksums before first use
@@ -139,7 +139,7 @@ I_GBMAP = 0x02000000 # -- Global on-disk block-map in use
I_FLUSH = 0x00000040 # -- Mounted with LFS3_M_FLUSH
I_SYNC = 0x00000080 # -- Mounted with LFS3_M_SYNC
I_REVDBG = 0x00000010 # -- Mounted with LFS3_M_REVDBG
I_REVPERTURB = 0x00000010 # -- Mounted with LFS3_M_REVPERTURB
I_REVNOISE = 0x00000020 # -- Mounted with LFS3_M_REVNOISE
I_CKPROGS = 0x00100000 # -- Mounted with LFS3_M_CKPROGS
I_CKFETCHES = 0x00200000 # -- Mounted with LFS3_M_CKFETCHES