From 25c7831417e9896654a70a8f63e2c301727c919d Mon Sep 17 00:00:00 2001 From: Christopher Haster Date: Thu, 23 May 2024 17:48:37 -0500 Subject: [PATCH] Fixed clobbered shrubs after renaming over an mdir split Good news! test_wl_orphanzombie_fuzz found a rare and difficult to reach bug. Bad news, it found the bug only after changing littlefs's initial revision count, which is about as unrelated a change as you can possibly have... Oh well, at least now we can add specialized tests targeting this (and push them to hopefully cover anything similar): - test_files_mv_split - test_files_mv_split_backwards - test_forphans_rename_split - test_forphans_rename_split_backwards The bug occurs when a rename of a file to/from the same mdir triggers an mdir split, and you have that file opened, and the opened file handle tracks a bshrub or bsprout. Oh, and if that wasn't unlikely enough, this only breaks when the rename crosses from the new-right-sibling to the new-left-sibling (inverse order of mdir split compacts), left-to-right is fine. The problem is how we stage bshrubs/bsprouts. bshrubs/bsprouts are a bit tricky in that several unrelated operations can change their location, sometimes multiple times in the same lfsr_mdir_commit call: - mdir compaction - move bshrub/bsprout to new mdir - bshrub commit - append a new shrub trunk - rename commit - move bshrub/bsprout to a new mdir/mid To keep track of all of this, lfsr_file_t has a dedicated field, file.bshrub_, that holds the bshrub/bsprout's new location during lfsr_mdir_commit. This may be changed multiple times, but the last change wins. This works as long as changes occur in an expected order. Importantly, commits that change the bshrub, such as rename, need to play out after compactions. It turns out this is violated when splitting an mdir. Because we have single pcache, we need to write out the entire compact + commit of each mdir at a time. When we split, we arbitrarily do this left-to-right, which results in left commits being played out before right compactions. Here's how things play out when we rename right-to-left: 1. commit rename -> bshrub = src mid, orig mdir 2. commit fails because of ERANGE 3. compact left mdir -> bshrub = src mid, left mdir 4. commit left mdir -> bshrub = dst mid, left mdir 5. compact right mdir -> bshrub = src mid, right mdir 6. commit right mdir (skips rename) Oh no! Our staged bshrub ends up with the wrong location. --- This is quite tricky to solve. We can't just play out the rename again on the right mdir, because we've already lost the new bshrub trunk at this point. Other solutions involving the grm or extra "moved" flags get messy because, well, lfsr_mdir_commit's internals are quite messy. The solution here, which is a bit hacky, but also obnoxiously elegant in a way, is to reorder the split mdir compactions such that the new mdir containing the commit mid is always compacted last. The means any related attrs are played out after both compactions, allowing renames to resolve correctly: 1. commit rename -> bshrub = src mid, orig mdir 2. commit fails because of ERANGE 3. right mdir contains mid 4. compact right mdir -> bshrub = src mid, right mdir 5. commit right mdir (skips rename) 6. compact left mdir -> bshrub = src mid, left mdir 7. commit left mdir -> bshrub = dst mid, left mdir This only works as long as such commits only span a single mid, though we already rely on mdir commits being single-mid elsewhere, so maybe this won't be a problem? The only real remaining concern is how much complexity this adds to lfsr_mdir_commit. And while this feels logically messy, the resulting code cost is surprisingly little: code stack before: 33458 2640 after: 33482 (+0.1%) 2640 (+0.0%) Still, I'll have to scratch my head to see if there's a better way to solve this... --- lfs.c | 76 +++-- tests/test_files.toml | 154 +++++++++ tests/test_forphans.toml | 703 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 902 insertions(+), 31 deletions(-) diff --git a/lfs.c b/lfs.c index 9894c458..6dea1ae6 100644 --- a/lfs.c +++ b/lfs.c @@ -6697,42 +6697,54 @@ static int lfsr_mdir_commit(lfs_t *lfs, lfsr_mdir_t *mdir, } } - // compact into new mdir tags < split_rid - err = lfsr_mdir_alloc__(lfs, &mdir_, lfs_smax32(mdir->mid, 0)); - if (err) { - goto failed; - } + // order the split compacts so that that mdir containing our mid + // is committed last, this is a bit of a hack but necessary so + // shrubs are staged correctly + for (int i = 0; i < 2; i++) { + if ((i == 0) ^ (lfsr_mid_rid(lfs, mdir->mid) >= split_rid)) { + // compact into new mdir tags >= split_rid + err = lfsr_mdir_alloc__(lfs, &msibling_, + lfs_smax32(mdir->mid, 0)); + if (err) { + goto failed; + } - err = lfsr_mdir_compact__(lfs, &mdir_, mdir, 0, split_rid); - if (err) { - LFS_ASSERT(err != LFS_ERR_RANGE); - goto failed; - } + err = lfsr_mdir_compact__(lfs, &msibling_, + mdir, split_rid, -1); + if (err) { + LFS_ASSERT(err != LFS_ERR_RANGE); + goto failed; + } - err = lfsr_mdir_commit__(lfs, &mdir_, 0, split_rid, - mdir->mid, attrs, attr_count); - if (err && err != LFS_ERR_NOENT) { - LFS_ASSERT(err != LFS_ERR_RANGE); - goto failed; - } + err = lfsr_mdir_commit__(lfs, &msibling_, split_rid, -1, + mdir->mid, attrs, attr_count); + if (err && err != LFS_ERR_NOENT) { + LFS_ASSERT(err != LFS_ERR_RANGE); + goto failed; + } - // compact into new mdir tags >= split_rid - err = lfsr_mdir_alloc__(lfs, &msibling_, lfs_smax32(mdir->mid, 0)); - if (err) { - goto failed; - } + } else { + // compact into new mdir tags < split_rid + err = lfsr_mdir_alloc__(lfs, &mdir_, + lfs_smax32(mdir->mid, 0)); + if (err) { + goto failed; + } - err = lfsr_mdir_compact__(lfs, &msibling_, mdir, split_rid, -1); - if (err) { - LFS_ASSERT(err != LFS_ERR_RANGE); - goto failed; - } + err = lfsr_mdir_compact__(lfs, &mdir_, + mdir, 0, split_rid); + if (err) { + LFS_ASSERT(err != LFS_ERR_RANGE); + goto failed; + } - err = lfsr_mdir_commit__(lfs, &msibling_, split_rid, -1, - mdir->mid, attrs, attr_count); - if (err && err != LFS_ERR_NOENT) { - LFS_ASSERT(err != LFS_ERR_RANGE); - goto failed; + err = lfsr_mdir_commit__(lfs, &mdir_, 0, split_rid, + mdir->mid, attrs, attr_count); + if (err && err != LFS_ERR_NOENT) { + LFS_ASSERT(err != LFS_ERR_RANGE); + goto failed; + } + } } // adjust our sibling's mid after committing attrs @@ -10348,6 +10360,8 @@ static int lfsr_bshrub_commit(lfs_t *lfs, lfsr_file_t *file, if (err) { return err; } + LFS_ASSERT(file->bshrub.u.bshrub.blocks[0] + == file->o.mdir.rbyd.blocks[0]); // update _all_ shrubs with the new estimate for (lfsr_opened_t *o = lfs->opened; o; o = o->next) { diff --git a/tests/test_files.toml b/tests/test_files.toml index d98e6136..4efc3dc2 100644 --- a/tests/test_files.toml +++ b/tests/test_files.toml @@ -2043,6 +2043,159 @@ code = ''' lfsr_unmount(&lfs) => 0; ''' +# one particularly nasty case is renaming over an mdir split, since shrubs +# can be moved around quite a few times when that happens +# +# here we spam renames over an increasing number of files to hopefully hit +# that case +# +[cases.test_files_mv_split] +defines.N = [1, 2, 4, 8, 16, 32, 64] +defines.SIZE = [ + '0', + 'FBUFFER_SIZE/2', + '2*FBUFFER_SIZE', + 'BLOCK_SIZE/2', + 'BLOCK_SIZE', + '2*BLOCK_SIZE', + '4*BLOCK_SIZE', +] +defines.REMOUNT = [false, true] +if = '(SIZE*N)/BLOCK_SIZE <= 32' +code = ''' + lfs_t lfs; + lfsr_format(&lfs, CFG) => 0; + lfsr_mount(&lfs, CFG) => 0; + + // create this many files while renaming + uint32_t prng = 42; + for (lfs_size_t i = 0; i < N; i++) { + // always create as first file + lfsr_file_t file; + lfsr_file_open(&lfs, &file, "amethyst", + LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + lfsr_file_write(&lfs, &file, wbuf, SIZE) => SIZE; + lfsr_file_close(&lfs, &file) => 0; + + // rename! + char name[256]; + sprintf(name, "basalt%03x", i); + lfsr_rename(&lfs, "amethyst", name) => 0; + } + + if (REMOUNT) { + lfsr_unmount(&lfs) => 0; + lfsr_mount(&lfs, CFG) => 0; + } + + // check that renames worked + prng = 42; + struct lfs_info info; + lfsr_stat(&lfs, "amethyst", &info) => LFS_ERR_NOENT; + for (lfs_size_t i = 0; i < N; i++) { + // check with stat + char name[256]; + sprintf(name, "basalt%03x", i); + lfsr_stat(&lfs, name, &info) => 0; + assert(strcmp(info.name, name) == 0); + assert(info.type == LFS_TYPE_REG); + assert(info.size == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + + lfsr_file_t file; + uint8_t rbuf[SIZE]; + lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0; + lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + lfsr_file_close(&lfs, &file) => 0; + } + + lfsr_unmount(&lfs) => 0; +''' + +[cases.test_files_mv_split_backwards] +defines.N = [1, 2, 4, 8, 16, 32, 64] +defines.SIZE = [ + '0', + 'FBUFFER_SIZE/2', + '2*FBUFFER_SIZE', + 'BLOCK_SIZE/2', + 'BLOCK_SIZE', + '2*BLOCK_SIZE', + '4*BLOCK_SIZE', +] +defines.REMOUNT = [false, true] +if = '(SIZE*N)/BLOCK_SIZE <= 32' +code = ''' + lfs_t lfs; + lfsr_format(&lfs, CFG) => 0; + lfsr_mount(&lfs, CFG) => 0; + + // create this many files while renaming + uint32_t prng = 42; + for (lfs_size_t i = 0; i < N; i++) { + // always create as last file + lfsr_file_t file; + lfsr_file_open(&lfs, &file, "cobalt", + LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + lfsr_file_write(&lfs, &file, wbuf, SIZE) => SIZE; + lfsr_file_close(&lfs, &file) => 0; + + // rename! + char name[256]; + sprintf(name, "basalt%03x", (uint32_t)(N-1-i)); + lfsr_rename(&lfs, "cobalt", name) => 0; + } + + if (REMOUNT) { + lfsr_unmount(&lfs) => 0; + lfsr_mount(&lfs, CFG) => 0; + } + + // check that renames worked + prng = 42; + struct lfs_info info; + lfsr_stat(&lfs, "cobalt", &info) => LFS_ERR_NOENT; + for (lfs_size_t i = 0; i < N; i++) { + // check with stat + char name[256]; + sprintf(name, "basalt%03x", (uint32_t)(N-1-i)); + lfsr_stat(&lfs, name, &info) => 0; + assert(strcmp(info.name, name) == 0); + assert(info.type == LFS_TYPE_REG); + assert(info.size == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + + lfsr_file_t file; + uint8_t rbuf[SIZE]; + lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0; + lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + lfsr_file_close(&lfs, &file) => 0; + } + + lfsr_unmount(&lfs) => 0; +''' + + # fuzz test file creation and rename [cases.test_files_mv_fuzz] defines.N = [1, 2, 4, 8, 16, 32, 64] @@ -2237,6 +2390,7 @@ code = ''' lfsr_unmount(&lfs) => 0; ''' + # fuzz test file creation/deletion/rename [cases.test_files_mvrm_fuzz] defines.N = [1, 2, 4, 8, 16, 32, 64] diff --git a/tests/test_forphans.toml b/tests/test_forphans.toml index e1563c99..45d153c2 100644 --- a/tests/test_forphans.toml +++ b/tests/test_forphans.toml @@ -5944,6 +5944,709 @@ code = ''' ''' + +# one particularly nasty case is renaming over an mdir split, since shrubs +# can be moved around quite a few times when that happens +# +# here we spam renames over an increasing number of files to hopefully hit +# that case +# +[cases.test_forphans_rename_split] +defines.N = [1, 2, 4, 8, 16, 32, 64] +defines.SIZE = [ + '0', + 'FBUFFER_SIZE/2', + '2*FBUFFER_SIZE', + 'BLOCK_SIZE/2', + 'BLOCK_SIZE', + '2*BLOCK_SIZE', + '4*BLOCK_SIZE', +] +# keep unsynced data open? +defines.UNSYNC = [false, true] +# keep a desynced file open? +defines.DESYNC = [false, true] +# keep a zombie open? +defines.ZOMBIE = [false, true] +if = '(SIZE*N*(1+DESYNC+ZOMBIE))/BLOCK_SIZE <= 32' +code = ''' + lfs_t lfs; + lfsr_format(&lfs, CFG) => 0; + lfsr_mount(&lfs, CFG) => 0; + + // we need a file handle for each file + desync + zombie + lfsr_file_t files[N]; + lfsr_file_t desyncs[N]; + lfsr_file_t zombies[N]; + + // create this many files while renaming + uint32_t prng = 42; + uint32_t unsync_prng = 43; + uint32_t desync_prng = 44; + uint32_t zombie_prng = 45; + for (lfs_size_t i = 0; i < N; i++) { + // create a zombie? + if (ZOMBIE) { + lfsr_file_open(&lfs, &zombies[i], "batman!!!", + LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26); + } + lfsr_file_write(&lfs, &zombies[i], wbuf, SIZE) => SIZE; + lfsr_file_sync(&lfs, &zombies[i]) => 0; + lfsr_remove(&lfs, "batman!!!") => 0; + } + + // always create as first file + lfsr_file_open(&lfs, &files[i], "batman!!!", + LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE; + lfsr_file_sync(&lfs, &files[i]) => 0; + + // keep unsynced data? + if (UNSYNC) { + lfsr_file_rewind(&lfs, &files[i]) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&unsync_prng) % 26); + } + lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE; + } + + // keep a desynced file open? + if (DESYNC) { + lfsr_file_open(&lfs, &desyncs[i], "batman!!!", + LFS_O_RDWR | LFS_O_DESYNC | LFS_O_TRUNC) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26); + } + lfsr_file_write(&lfs, &desyncs[i], wbuf, SIZE) => SIZE; + } + + // rename! + char name[256]; + sprintf(name, "batman%03x", i); + lfsr_rename(&lfs, "batman!!!", name) => 0; + } + + // check that renames worked + prng = 42; + struct lfs_info info; + lfsr_stat(&lfs, "batman!!!", &info) => LFS_ERR_NOENT; + for (lfs_size_t i = 0; i < N; i++) { + // check with stat + char name[256]; + sprintf(name, "batman%03x", i); + lfsr_stat(&lfs, name, &info) => 0; + assert(strcmp(info.name, name) == 0); + assert(info.type == LFS_TYPE_REG); + assert(info.size == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + + lfsr_file_t file; + uint8_t rbuf[SIZE]; + lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0; + lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + lfsr_file_close(&lfs, &file) => 0; + } + + // check that file handles are as expected + prng = (!UNSYNC) ? 42 : 43; + desync_prng = 44; + zombie_prng = 45; + for (lfs_size_t i = 0; i < N; i++) { + // check size + assert(lfsr_file_size(&lfs, &files[i]) == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &files[i]) => 0; + lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + + // check any desynced files + if (DESYNC) { + assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &desyncs[i]) => 0; + lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + + // check any zombied files + if (ZOMBIE) { + assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &zombies[i]) => 0; + lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + } + + // try syncing any unsynced files + for (lfs_size_t i = 0; i < N; i++) { + lfsr_file_sync(&lfs, &files[i]) => 0; + } + + // check that sync worked + prng = (!UNSYNC) ? 42 : 43; + lfsr_stat(&lfs, "batman!!!", &info) => LFS_ERR_NOENT; + for (lfs_size_t i = 0; i < N; i++) { + // check with stat + char name[256]; + sprintf(name, "batman%03x", i); + lfsr_stat(&lfs, name, &info) => 0; + assert(strcmp(info.name, name) == 0); + assert(info.type == LFS_TYPE_REG); + assert(info.size == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + + lfsr_file_t file; + uint8_t rbuf[SIZE]; + lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0; + lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + lfsr_file_close(&lfs, &file) => 0; + } + + // check that file handles are as expected + prng = (!UNSYNC) ? 42 : 43; + desync_prng = 44; + zombie_prng = 45; + for (lfs_size_t i = 0; i < N; i++) { + // check size + assert(lfsr_file_size(&lfs, &files[i]) == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &files[i]) => 0; + lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + + // check any desynced files + if (DESYNC) { + assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &desyncs[i]) => 0; + lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + + // check any zombied files + if (ZOMBIE) { + assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &zombies[i]) => 0; + lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + } + + // try rewriting our open files + uint32_t rewrite_prng = 52; + for (lfs_size_t i = 0; i < N; i++) { + lfsr_file_rewind(&lfs, &files[i]) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26); + } + lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE; + lfsr_file_sync(&lfs, &files[i]) => 0; + } + + // check that rewrites worked + rewrite_prng = 52; + lfsr_stat(&lfs, "batman!!!", &info) => LFS_ERR_NOENT; + for (lfs_size_t i = 0; i < N; i++) { + // check with stat + char name[256]; + sprintf(name, "batman%03x", i); + lfsr_stat(&lfs, name, &info) => 0; + assert(strcmp(info.name, name) == 0); + assert(info.type == LFS_TYPE_REG); + assert(info.size == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26); + } + + lfsr_file_t file; + uint8_t rbuf[SIZE]; + lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0; + lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + lfsr_file_close(&lfs, &file) => 0; + } + + // check that file handles are as expected + rewrite_prng = 52; + desync_prng = 44; + zombie_prng = 45; + for (lfs_size_t i = 0; i < N; i++) { + // check size + assert(lfsr_file_size(&lfs, &files[i]) == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &files[i]) => 0; + lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + + // check any desynced files + if (DESYNC) { + assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &desyncs[i]) => 0; + lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + + // check any zombied files + if (ZOMBIE) { + assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &zombies[i]) => 0; + lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + } + + // cleanup files + for (lfs_size_t i = 0; i < N; i++) { + lfsr_file_close(&lfs, &files[i]) => 0; + } + if (DESYNC) { + for (lfs_size_t i = 0; i < N; i++) { + lfsr_file_close(&lfs, &desyncs[i]) => 0; + } + } + if (ZOMBIE) { + for (lfs_size_t i = 0; i < N; i++) { + lfsr_file_close(&lfs, &zombies[i]) => 0; + } + } + lfsr_unmount(&lfs) => 0; +''' + +[cases.test_forphans_rename_split_backwards] +defines.N = [1, 2, 4, 8, 16, 32, 64] +defines.SIZE = [ + '0', + 'FBUFFER_SIZE/2', + '2*FBUFFER_SIZE', + 'BLOCK_SIZE/2', + 'BLOCK_SIZE', + '2*BLOCK_SIZE', + '4*BLOCK_SIZE', +] +# keep unsynced data open? +defines.UNSYNC = [false, true] +# keep a desynced file open? +defines.DESYNC = [false, true] +# keep a zombie open? +defines.ZOMBIE = [false, true] +if = '(SIZE*N*(1+DESYNC+ZOMBIE))/BLOCK_SIZE <= 32' +code = ''' + lfs_t lfs; + lfsr_format(&lfs, CFG) => 0; + lfsr_mount(&lfs, CFG) => 0; + + // we need a file handle for each file + desync + zombie + lfsr_file_t files[N]; + lfsr_file_t desyncs[N]; + lfsr_file_t zombies[N]; + + // create this many files while renaming + uint32_t prng = 42; + uint32_t unsync_prng = 43; + uint32_t desync_prng = 44; + uint32_t zombie_prng = 45; + for (lfs_size_t i = 0; i < N; i++) { + // create a zombie? + if (ZOMBIE) { + lfsr_file_open(&lfs, &zombies[i], "batman???", + LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26); + } + lfsr_file_write(&lfs, &zombies[i], wbuf, SIZE) => SIZE; + lfsr_file_sync(&lfs, &zombies[i]) => 0; + lfsr_remove(&lfs, "batman???") => 0; + } + + // always create as last file + lfsr_file_open(&lfs, &files[i], "batman???", + LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE; + lfsr_file_sync(&lfs, &files[i]) => 0; + + // keep unsynced data? + if (UNSYNC) { + lfsr_file_rewind(&lfs, &files[i]) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&unsync_prng) % 26); + } + lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE; + } + + // keep a desynced file open? + if (DESYNC) { + lfsr_file_open(&lfs, &desyncs[i], "batman???", + LFS_O_RDWR | LFS_O_DESYNC | LFS_O_TRUNC) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26); + } + lfsr_file_write(&lfs, &desyncs[i], wbuf, SIZE) => SIZE; + } + + // rename! + char name[256]; + sprintf(name, "batman%03x", (uint32_t)(N-1-i)); + lfsr_rename(&lfs, "batman???", name) => 0; + } + + // check that renames worked + prng = 42; + struct lfs_info info; + lfsr_stat(&lfs, "batman???", &info) => LFS_ERR_NOENT; + for (lfs_size_t i = 0; i < N; i++) { + // check with stat + char name[256]; + sprintf(name, "batman%03x", (uint32_t)(N-1-i)); + lfsr_stat(&lfs, name, &info) => 0; + assert(strcmp(info.name, name) == 0); + assert(info.type == LFS_TYPE_REG); + assert(info.size == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + + lfsr_file_t file; + uint8_t rbuf[SIZE]; + lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0; + lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + lfsr_file_close(&lfs, &file) => 0; + } + + // check that file handles are as expected + prng = (!UNSYNC) ? 42 : 43; + desync_prng = 44; + zombie_prng = 45; + for (lfs_size_t i = 0; i < N; i++) { + // check size + assert(lfsr_file_size(&lfs, &files[i]) == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &files[i]) => 0; + lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + + // check any desynced files + if (DESYNC) { + assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &desyncs[i]) => 0; + lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + + // check any zombied files + if (ZOMBIE) { + assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &zombies[i]) => 0; + lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + } + + // try syncing any unsynced files + for (lfs_size_t i = 0; i < N; i++) { + lfsr_file_sync(&lfs, &files[i]) => 0; + } + + // check that sync worked + prng = (!UNSYNC) ? 42 : 43; + lfsr_stat(&lfs, "batman???", &info) => LFS_ERR_NOENT; + for (lfs_size_t i = 0; i < N; i++) { + // check with stat + char name[256]; + sprintf(name, "batman%03x", (uint32_t)(N-1-i)); + lfsr_stat(&lfs, name, &info) => 0; + assert(strcmp(info.name, name) == 0); + assert(info.type == LFS_TYPE_REG); + assert(info.size == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + + lfsr_file_t file; + uint8_t rbuf[SIZE]; + lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0; + lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + lfsr_file_close(&lfs, &file) => 0; + } + + // check that file handles are as expected + prng = (!UNSYNC) ? 42 : 43; + desync_prng = 44; + zombie_prng = 45; + for (lfs_size_t i = 0; i < N; i++) { + // check size + assert(lfsr_file_size(&lfs, &files[i]) == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &files[i]) => 0; + lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + + // check any desynced files + if (DESYNC) { + assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &desyncs[i]) => 0; + lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + + // check any zombied files + if (ZOMBIE) { + assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &zombies[i]) => 0; + lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + } + + // try rewriting our open files + uint32_t rewrite_prng = 52; + for (lfs_size_t i = 0; i < N; i++) { + lfsr_file_rewind(&lfs, &files[i]) => 0; + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26); + } + lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE; + lfsr_file_sync(&lfs, &files[i]) => 0; + } + + // check that rewrites worked + rewrite_prng = 52; + lfsr_stat(&lfs, "batman???", &info) => LFS_ERR_NOENT; + for (lfs_size_t i = 0; i < N; i++) { + // check with stat + char name[256]; + sprintf(name, "batman%03x", (uint32_t)(N-1-i)); + lfsr_stat(&lfs, name, &info) => 0; + assert(strcmp(info.name, name) == 0); + assert(info.type == LFS_TYPE_REG); + assert(info.size == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26); + } + + lfsr_file_t file; + uint8_t rbuf[SIZE]; + lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0; + lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + lfsr_file_close(&lfs, &file) => 0; + } + + // check that file handles are as expected + rewrite_prng = 52; + desync_prng = 44; + zombie_prng = 45; + for (lfs_size_t i = 0; i < N; i++) { + // check size + assert(lfsr_file_size(&lfs, &files[i]) == SIZE); + + // try reading the file + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &files[i]) => 0; + lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + + // check any desynced files + if (DESYNC) { + assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &desyncs[i]) => 0; + lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + + // check any zombied files + if (ZOMBIE) { + assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE); + + uint8_t wbuf[SIZE]; + for (lfs_size_t j = 0; j < SIZE; j++) { + wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26); + } + + uint8_t rbuf[SIZE]; + lfsr_file_rewind(&lfs, &zombies[i]) => 0; + lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE; + assert(memcmp(rbuf, wbuf, SIZE) == 0); + } + } + + // cleanup files + for (lfs_size_t i = 0; i < N; i++) { + lfsr_file_close(&lfs, &files[i]) => 0; + } + if (DESYNC) { + for (lfs_size_t i = 0; i < N; i++) { + lfsr_file_close(&lfs, &desyncs[i]) => 0; + } + } + if (ZOMBIE) { + for (lfs_size_t i = 0; i < N; i++) { + lfsr_file_close(&lfs, &zombies[i]) => 0; + } + } + lfsr_unmount(&lfs) => 0; +''' + + + # fuzz tests involving many orphans + zombies, this gets a bit crazy [cases.test_forphans_orphanzombie_fuzz] defines.N = [1, 2, 4, 8, 16, 32, 64]