Limited ckcksums to check data cksums

So... Long store short, checking metadata cksums is just intractably
slow.

But data cksums?

Yes checking data cksums is still O(b^2), but unlike metadata lookups,
which involve many small backwards reads, data reads are very easy to
cache. So instead of O(b^2), it's more like O(b^2/c), where c is your
rcache size.

Still O(b^2) when c << b, but I'm not sure that's avoidable without
adding more cksums.

At the very least, if you have enough RAM, c == b reduces this to O(b),
which is nice for "large" systems that want hardened reads without a
performance loss.

---

But why bother checking data cksums if we still have a read-hole with
metadata cksums?

Well, while considering the problem in the context of future features, I
noticed something _really interesting_:

- ckredund + metadata - reasonable ✓
- ckredund + data     - impractical ✗, parity fanout + O(f+r) is bad
- ckcksums + metadata - impractical ✗, small reads + O(b^2) is bad
- ckcksums + data     - reasonable ✓, assuming enough rcache

The current planned design for data redundancy makes it also intractably
slow to check every read, since it would require xoring all blocks that
contribute to the relevant parity block, but this isn't a problem for
metadata redundancy.

So while neither ckredund nor ckcksums can tractably close the read-hole
on their own, it looks like together they will be able to cover
everything without completely sacrificing performance. Neat!

Of course this isn't possible if ckcksums/ckredund imply checking both
metadata and data, so they need to be split apart.

And I don't really see a point in keeping the intractable variants
around in the codebase.

---

Dropping metadata ckcksums also means we can get rid of the ugly
lfsr_bd_ckrbydprefix and lfsr_bd_ckrbydsuffix functions, which were
basically duplicating all of lfsr_rbyd_fetch. That was quite a wart!

This saves a nice chunk of code when ckcksums is enabled:

                    code          stack          ctx
  default before:  38128           2624          752
  default after:   38128 (+0.0%)   2624 (+0.0%)  752 (+0.0%)

  ckparity before: 39724           3048          764
  ckparity after:  39700 (-0.1%)   3048 (+0.0%)  760 (-0.5%)

  ckcksums before: 40612           3184          772
  ckcksums after:  39396 (-3.0%)   3096 (-2.8%)  760 (-1.6%)
This commit is contained in:
Christopher Haster
2025-01-03 00:42:59 -06:00
parent 39a5b9578b
commit 7edb3b231f
3 changed files with 53 additions and 621 deletions
+5 -5
View File
@@ -169,7 +169,7 @@ enum lfs_type {
#define LFS_F_CKPARITY 0x00400000 // Check tag parity bits on reads
#endif
#ifdef LFS_CKCKSUMS
#define LFS_F_CKCKSUMS 0x00800000 // Check checksums on reads (expensive!)
#define LFS_F_CKCKSUMS 0x00800000 // Check data checksums on reads
#endif
#define LFS_F_MTREEONLY 0x00000800 // Only traverse the mtree
@@ -192,7 +192,7 @@ enum lfs_type {
#define LFS_M_CKPARITY 0x00400000 // Check tag parity bits on reads
#endif
#ifdef LFS_CKCKSUMS
#define LFS_M_CKCKSUMS 0x00800000 // Check checksums on reads (expensive!)
#define LFS_M_CKCKSUMS 0x00800000 // Check data checksums on reads
#endif
#define LFS_M_MTREEONLY 0x00000800 // Only traverse the mtree
@@ -808,11 +808,11 @@ typedef struct lfsr_grm {
lfsr_smid_t mids[2];
} lfsr_grm_t;
#if defined(LFS_CKPARITY) || defined(LFS_CKCKSUMS)
#ifdef LFS_CKPARITY
typedef struct lfsr_tailck {
lfs_block_t ckblock;
// sign(ckoff) => tail parity
lfs_size_t ckoff;
uint32_t cksum;
} lfsr_tailck_t;
#endif
@@ -848,7 +848,7 @@ typedef struct lfs {
uint8_t *buffer;
} pcache;
#if defined(LFS_CKPARITY) || defined(LFS_CKCKSUMS)
#ifdef LFS_CKPARITY
lfsr_tailck_t tailck;
#endif