From 7c8c7e662ac6b5e94cd0ff70db5fdabb64f499ee Mon Sep 17 00:00:00 2001 From: Christopher Haster Date: Thu, 15 May 2025 18:16:35 -0500 Subject: [PATCH] Tightened mdir compaction estimate, added mattr_estimate This adds mattr_estimate, which is basically the same as rattr_estimate, but assumes weight <= 1: rattr tag: .---+---+---+- -+- -+- -+- -+---+- -+- -+- -. worst case: <=11 bytes | tag | weight | size | rattr est: <=3t + 4 '---+---+---+- -+- -+- -+- -+---+- -+- -+- -' <=37 bytes mattr tag: .---+---+---+---+- -+- -+- -. worst case: <=7 bytes | tag | w | size | mattr est: <=3t + 4 '---+---+---+---+- -+- -+- -' <=25 bytes This may seem like only a minor improvement, but with 3 tags for every attr, this really adds up. And with our compaction estimate overheads we need every byte of shaving we can get. --- This ended up necessary to get littlefs running with 512 byte blocks again. Now that our compaction overheads are so high, littlefs is having a hard time fitting even just the filesystem config in a single block: mroot estimate 512B before: 246/256 mroot estimate 512B after: 162/256 (-34.1%) Whether or not it makes sense to run littlefs with 512 byte blocks is still an open question, even after this tweak. Note that even if 512 byte blocks ends up intractable, this doesn't mean littlefs won't be able to run on SD/eMMC! The configured block_size can always be a multiple, >=, of the physical block_size, and choosing a larger block_size completely side-steps this problem. The new design of littlefs is primarily focused on devices with very large block sizes, so you may want to use larger block sizes on SD/eMMC for performance reasons anyways. --- Code changes were pretty minimal. This does add an additional field to lfs_t, but it's just a byte and fits into padding with the other small precomputed constants: code stack ctx before: 35824 2368 636 after: 35836 (+0.0%) 2368 (+0.0%) 636 (+0.0%) --- lfs.c | 13 ++++++++++++- lfs.h | 3 ++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/lfs.c b/lfs.c index bbb9c60b..f32faa07 100644 --- a/lfs.c +++ b/lfs.c @@ -8182,7 +8182,7 @@ static lfs_ssize_t lfsr_mdir_estimate__(lfs_t *lfs, const lfsr_mdir_t *mdir, } else { // include the cost of this tag - dsize_ += lfs->rattr_estimate + lfsr_data_size(data); + dsize_ += lfs->mattr_estimate + lfsr_data_size(data); } } @@ -13453,6 +13453,17 @@ static int lfs_init(lfs_t *lfs, uint32_t flags, LFS_ASSERT(tag_estimate <= LFSR_TAG_DSIZE); lfs->rattr_estimate = 3*tag_estimate + 4; + // calculate the upper-bound cost of a single mdir attr after compaction + // + // This is the same as rattr_estimate, except we can assume a weight<=1. + // + tag_estimate + = 2 + + 1 + + (lfs_nlog2(lfs->cfg->block_size)+7-1)/7; + LFS_ASSERT(tag_estimate <= LFSR_TAG_DSIZE); + lfs->mattr_estimate = 3*tag_estimate + 4; + // calculate the number of bits we need to reserve for mdir rids // // Worst case (or best case?) each metadata entry is a single tag. In diff --git a/lfs.h b/lfs.h index c77ada5f..1fdc43c5 100644 --- a/lfs.h +++ b/lfs.h @@ -833,8 +833,9 @@ typedef struct lfs { lfs_off_t file_limit; int8_t recycle_bits; - uint8_t rattr_estimate; uint8_t mbits; + uint8_t rattr_estimate; + uint8_t mattr_estimate; // linked-list of opened mdirs lfsr_omdir_t *omdirs;