From 09ebf70bd93d7908d8b57c17f8f3375234dec7ba Mon Sep 17 00:00:00 2001 From: Christopher Haster Date: Mon, 26 Feb 2024 02:23:56 -0600 Subject: [PATCH] Updated did truncation comment based on the new compaction algorithm Note no code actually changed. The new compaction algorithm _does_ bring the directory estimate down from ~96 -> ~72 bytes, but because we want a power-of-two for cheap division, we floor both of these to ~64 bytes. It's interesting to note that a perfect compaction algorithm _could_ bring the directory estimate down across the power-of-two boundary: ~72 -> ~48 bytes. But fortunately we're not perfect so we don't have care about that. --- lfs.c | 46 ++++++++++++++++++++++++++++++++-------------- 1 file changed, 32 insertions(+), 14 deletions(-) diff --git a/lfs.c b/lfs.c index 6ef05c0c..9f5a7fbf 100644 --- a/lfs.c +++ b/lfs.c @@ -8634,25 +8634,43 @@ int lfsr_mkdir(lfs_t *lfs, const char *path) { return LFS_ERR_NAMETOOLONG; } - // Our directory needs an arbitrary directory-rid. To find one with - // hopefully few collisions, we use a hash of the full path using our CRC, - // since we have it handy. + // Our directory needs an arbitrary directory-id. To find one with + // hopefully few collisions, we checksum our full path, but this is + // arbitrary. // // We also truncate to make better use of our leb128 encoding. This is - // relatively arbitrary, but if we truncate too much we risk increasing + // somewhat arbitrary, but if we truncate too much we risk increasing // the number of collisions, so we want to aim for ~2x the number dids - // in the system. We don't actually know the number of dids in the system, - // but we can use a heuristic based on the maximum possible number of - // directories in the current mtree assuming our block size. + // in the system: // - // - Each directory needs 1 name tag, 1 did tag, and 1 bookmark - // - Each tag needs ~2 alts+null with our current compaction strategy - // - Each tag/alt encodes to a minimum of 4 bytes - // - We can also assume ~1/2 block utilization due to our split threshold + // dmask = 2*dids // - // This gives us ~3*4*4*2 or ~96 bytes per directory at minimum. - // Multiplying by 2 and rounding down to the nearest power of 2 for cheaper - // division gives us a heuristic of ~block_size/32 directories per mdir. + // But we don't actually know how many dids are in the system. + // Fortunately, we can guess an upper bound based on the number of + // mdirs in the mtree: + // + // mdirs + // dmask = 2 * ----- + // d + // + // Worst case (or best case?) each directory needs 1 name tag, 1 did + // tag, and 1 bookmark. With our current compaction strategy, each tag + // needs ~(5/2)t+2 bytes for tag+alts (see our attr_estimate). And, if + // we assume ~1/2 block utilization due to our mdir split threshold, we + // can multiply everything by 2: + // + // d = 3 * ((5/2)t+2) * 2 = 15t + 12 + // + // Assuming t=4 bytes, the minimum tag encoding: + // + // d = 15*4 + 12 = 72 bytes + // + // Rounding down to a power-of-two (again this is all arbitrary), gives + // us ~64 bytes per directory: + // + // mdirs mdirs + // dmask = 2 * ----- = ----- + // 64 32 // // This is a nice number because for common NOR flash geometry, // 4096/32 = 128, so a filesystem with a single mdir encodes dids in a