Github/littlefs
1
Fork 0
Code Actions Activity

Fixed clobbered shrubs after renaming over an mdir split

Browse Source 
Good news! test_wl_orphanzombie_fuzz found a rare and difficult to reach
bug. Bad news, it found the bug only after changing littlefs's initial
revision count, which is about as unrelated a change as you can possibly
have...

Oh well, at least now we can add specialized tests targeting this (and
push them to hopefully cover anything similar):

- test_files_mv_split
- test_files_mv_split_backwards
- test_forphans_rename_split
- test_forphans_rename_split_backwards

The bug occurs when a rename of a file to/from the same mdir triggers an
mdir split, and you have that file opened, and the opened file handle
tracks a bshrub or bsprout. Oh, and if that wasn't unlikely enough, this
only breaks when the rename crosses from the new-right-sibling to the
new-left-sibling (inverse order of mdir split compacts), left-to-right
is fine.

The problem is how we stage bshrubs/bsprouts. bshrubs/bsprouts are a bit
tricky in that several unrelated operations can change their location,
sometimes multiple times in the same lfsr_mdir_commit call:

- mdir compaction - move bshrub/bsprout to new mdir
- bshrub commit - append a new shrub trunk
- rename commit - move bshrub/bsprout to a new mdir/mid

To keep track of all of this, lfsr_file_t has a dedicated field,
file.bshrub_, that holds the bshrub/bsprout's new location during
lfsr_mdir_commit. This may be changed multiple times, but the last
change wins.

This works as long as changes occur in an expected order. Importantly,
commits that change the bshrub, such as rename, need to play out after
compactions.

It turns out this is violated when splitting an mdir.

Because we have single pcache, we need to write out the entire compact +
commit of each mdir at a time. When we split, we arbitrarily do this
left-to-right, which results in left commits being played out before
right compactions.

Here's how things play out when we rename right-to-left:

1. commit rename                    -> bshrub = src mid, orig mdir
2. commit fails because of ERANGE
3. compact left mdir                -> bshrub = src mid, left mdir
4. commit left mdir                 -> bshrub = dst mid, left mdir
5. compact right mdir               -> bshrub = src mid, right mdir
6. commit right mdir (skips rename)

Oh no! Our staged bshrub ends up with the wrong location.

---

This is quite tricky to solve. We can't just play out the rename again
on the right mdir, because we've already lost the new bshrub trunk at
this point. Other solutions involving the grm or extra "moved" flags get
messy because, well, lfsr_mdir_commit's internals are quite messy.

The solution here, which is a bit hacky, but also obnoxiously elegant in
a way, is to reorder the split mdir compactions such that the new mdir
containing the commit mid is always compacted last. The means any
related attrs are played out after both compactions, allowing renames to
resolve correctly:

1. commit rename                    -> bshrub = src mid, orig mdir
2. commit fails because of ERANGE
3. right mdir contains mid
4. compact right mdir               -> bshrub = src mid, right mdir
5. commit right mdir (skips rename)
6. compact left mdir                -> bshrub = src mid, left mdir
7. commit left mdir                 -> bshrub = dst mid, left mdir

This only works as long as such commits only span a single mid, though
we already rely on mdir commits being single-mid elsewhere, so maybe
this won't be a problem?

The only real remaining concern is how much complexity this adds to
lfsr_mdir_commit. And while this feels logically messy, the resulting
code cost is surprisingly little:

           code          stack
  before: 33458           2640
  after:  33482 (+0.1%)   2640 (+0.0%)

Still, I'll have to scratch my head to see if there's a better way to
solve this...
This commit is contained in:
Christopher Haster
2024-05-23 17:48:37 -05:00
parent 0802115717
commit 25c7831417
3 changed files with 902 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lfs.c
+45 -31
View File
@@ -6697,42 +6697,54 @@ static int lfsr_mdir_commit(lfs_t *lfs, lfsr_mdir_t *mdir,
}
}
// compact into new mdir tags < split_rid
err = lfsr_mdir_alloc__(lfs, &mdir_, lfs_smax32(mdir->mid, 0));
if (err) {
goto failed;
}
// order the split compacts so that that mdir containing our mid
// is committed last, this is a bit of a hack but necessary so
// shrubs are staged correctly
for (int i = 0; i < 2; i++) {
if ((i == 0) ^ (lfsr_mid_rid(lfs, mdir->mid) >= split_rid)) {
// compact into new mdir tags >= split_rid
err = lfsr_mdir_alloc__(lfs, &msibling_,
lfs_smax32(mdir->mid, 0));
if (err) {
goto failed;
}
err = lfsr_mdir_compact__(lfs, &mdir_, mdir, 0, split_rid);
if (err) {
LFS_ASSERT(err != LFS_ERR_RANGE);
goto failed;
}
err = lfsr_mdir_compact__(lfs, &msibling_,
mdir, split_rid, -1);
if (err) {
LFS_ASSERT(err != LFS_ERR_RANGE);
goto failed;
}
err = lfsr_mdir_commit__(lfs, &mdir_, 0, split_rid,
mdir->mid, attrs, attr_count);
if (err && err != LFS_ERR_NOENT) {
LFS_ASSERT(err != LFS_ERR_RANGE);
goto failed;
}
err = lfsr_mdir_commit__(lfs, &msibling_, split_rid, -1,
mdir->mid, attrs, attr_count);
if (err && err != LFS_ERR_NOENT) {
LFS_ASSERT(err != LFS_ERR_RANGE);
goto failed;
}
// compact into new mdir tags >= split_rid
err = lfsr_mdir_alloc__(lfs, &msibling_, lfs_smax32(mdir->mid, 0));
if (err) {
goto failed;
}
} else {
// compact into new mdir tags < split_rid
err = lfsr_mdir_alloc__(lfs, &mdir_,
lfs_smax32(mdir->mid, 0));
if (err) {
goto failed;
}
err = lfsr_mdir_compact__(lfs, &msibling_, mdir, split_rid, -1);
if (err) {
LFS_ASSERT(err != LFS_ERR_RANGE);
goto failed;
}
err = lfsr_mdir_compact__(lfs, &mdir_,
mdir, 0, split_rid);
if (err) {
LFS_ASSERT(err != LFS_ERR_RANGE);
goto failed;
}
err = lfsr_mdir_commit__(lfs, &msibling_, split_rid, -1,
mdir->mid, attrs, attr_count);
if (err && err != LFS_ERR_NOENT) {
LFS_ASSERT(err != LFS_ERR_RANGE);
goto failed;
err = lfsr_mdir_commit__(lfs, &mdir_, 0, split_rid,
mdir->mid, attrs, attr_count);
if (err && err != LFS_ERR_NOENT) {
LFS_ASSERT(err != LFS_ERR_RANGE);
goto failed;
}
}
}
// adjust our sibling's mid after committing attrs
@@ -10348,6 +10360,8 @@ static int lfsr_bshrub_commit(lfs_t *lfs, lfsr_file_t *file,
if (err) {
return err;
}
LFS_ASSERT(file->bshrub.u.bshrub.blocks[0]
== file->o.mdir.rbyd.blocks[0]);
// update _all_ shrubs with the new estimate
for (lfsr_opened_t *o = lfs->opened; o; o = o->next) {
tests/test_files.toml
+154
View File
@@ -2043,6 +2043,159 @@ code = '''
lfsr_unmount(&lfs) => 0;
'''
# one particularly nasty case is renaming over an mdir split, since shrubs
# can be moved around quite a few times when that happens
#
# here we spam renames over an increasing number of files to hopefully hit
# that case
#
[cases.test_files_mv_split]
defines.N = [1, 2, 4, 8, 16, 32, 64]
defines.SIZE = [
'0',
'FBUFFER_SIZE/2',
'2*FBUFFER_SIZE',
'BLOCK_SIZE/2',
'BLOCK_SIZE',
'2*BLOCK_SIZE',
'4*BLOCK_SIZE',
]
defines.REMOUNT = [false, true]
if = '(SIZE*N)/BLOCK_SIZE <= 32'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
// create this many files while renaming
uint32_t prng = 42;
for (lfs_size_t i = 0; i < N; i++) {
// always create as first file
lfsr_file_t file;
lfsr_file_open(&lfs, &file, "amethyst",
LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
lfsr_file_write(&lfs, &file, wbuf, SIZE) => SIZE;
lfsr_file_close(&lfs, &file) => 0;
// rename!
char name[256];
sprintf(name, "basalt%03x", i);
lfsr_rename(&lfs, "amethyst", name) => 0;
}
if (REMOUNT) {
lfsr_unmount(&lfs) => 0;
lfsr_mount(&lfs, CFG) => 0;
}
// check that renames worked
prng = 42;
struct lfs_info info;
lfsr_stat(&lfs, "amethyst", &info) => LFS_ERR_NOENT;
for (lfs_size_t i = 0; i < N; i++) {
// check with stat
char name[256];
sprintf(name, "basalt%03x", i);
lfsr_stat(&lfs, name, &info) => 0;
assert(strcmp(info.name, name) == 0);
assert(info.type == LFS_TYPE_REG);
assert(info.size == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
lfsr_file_t file;
uint8_t rbuf[SIZE];
lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0;
lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
lfsr_file_close(&lfs, &file) => 0;
}
lfsr_unmount(&lfs) => 0;
'''
[cases.test_files_mv_split_backwards]
defines.N = [1, 2, 4, 8, 16, 32, 64]
defines.SIZE = [
'0',
'FBUFFER_SIZE/2',
'2*FBUFFER_SIZE',
'BLOCK_SIZE/2',
'BLOCK_SIZE',
'2*BLOCK_SIZE',
'4*BLOCK_SIZE',
]
defines.REMOUNT = [false, true]
if = '(SIZE*N)/BLOCK_SIZE <= 32'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
// create this many files while renaming
uint32_t prng = 42;
for (lfs_size_t i = 0; i < N; i++) {
// always create as last file
lfsr_file_t file;
lfsr_file_open(&lfs, &file, "cobalt",
LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
lfsr_file_write(&lfs, &file, wbuf, SIZE) => SIZE;
lfsr_file_close(&lfs, &file) => 0;
// rename!
char name[256];
sprintf(name, "basalt%03x", (uint32_t)(N-1-i));
lfsr_rename(&lfs, "cobalt", name) => 0;
}
if (REMOUNT) {
lfsr_unmount(&lfs) => 0;
lfsr_mount(&lfs, CFG) => 0;
}
// check that renames worked
prng = 42;
struct lfs_info info;
lfsr_stat(&lfs, "cobalt", &info) => LFS_ERR_NOENT;
for (lfs_size_t i = 0; i < N; i++) {
// check with stat
char name[256];
sprintf(name, "basalt%03x", (uint32_t)(N-1-i));
lfsr_stat(&lfs, name, &info) => 0;
assert(strcmp(info.name, name) == 0);
assert(info.type == LFS_TYPE_REG);
assert(info.size == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
lfsr_file_t file;
uint8_t rbuf[SIZE];
lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0;
lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
lfsr_file_close(&lfs, &file) => 0;
}
lfsr_unmount(&lfs) => 0;
'''
# fuzz test file creation and rename
[cases.test_files_mv_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64]
@@ -2237,6 +2390,7 @@ code = '''
lfsr_unmount(&lfs) => 0;
'''
# fuzz test file creation/deletion/rename
[cases.test_files_mvrm_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64]
tests/test_forphans.toml
+703
View File
@@ -5944,6 +5944,709 @@ code = '''
'''
# one particularly nasty case is renaming over an mdir split, since shrubs
# can be moved around quite a few times when that happens
#
# here we spam renames over an increasing number of files to hopefully hit
# that case
#
[cases.test_forphans_rename_split]
defines.N = [1, 2, 4, 8, 16, 32, 64]
defines.SIZE = [
'0',
'FBUFFER_SIZE/2',
'2*FBUFFER_SIZE',
'BLOCK_SIZE/2',
'BLOCK_SIZE',
'2*BLOCK_SIZE',
'4*BLOCK_SIZE',
]
# keep unsynced data open?
defines.UNSYNC = [false, true]
# keep a desynced file open?
defines.DESYNC = [false, true]
# keep a zombie open?
defines.ZOMBIE = [false, true]
if = '(SIZE*N*(1+DESYNC+ZOMBIE))/BLOCK_SIZE <= 32'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
// we need a file handle for each file + desync + zombie
lfsr_file_t files[N];
lfsr_file_t desyncs[N];
lfsr_file_t zombies[N];
// create this many files while renaming
uint32_t prng = 42;
uint32_t unsync_prng = 43;
uint32_t desync_prng = 44;
uint32_t zombie_prng = 45;
for (lfs_size_t i = 0; i < N; i++) {
// create a zombie?
if (ZOMBIE) {
lfsr_file_open(&lfs, &zombies[i], "batman!!!",
LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26);
}
lfsr_file_write(&lfs, &zombies[i], wbuf, SIZE) => SIZE;
lfsr_file_sync(&lfs, &zombies[i]) => 0;
lfsr_remove(&lfs, "batman!!!") => 0;
}
// always create as first file
lfsr_file_open(&lfs, &files[i], "batman!!!",
LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE;
lfsr_file_sync(&lfs, &files[i]) => 0;
// keep unsynced data?
if (UNSYNC) {
lfsr_file_rewind(&lfs, &files[i]) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&unsync_prng) % 26);
}
lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE;
}
// keep a desynced file open?
if (DESYNC) {
lfsr_file_open(&lfs, &desyncs[i], "batman!!!",
LFS_O_RDWR | LFS_O_DESYNC | LFS_O_TRUNC) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26);
}
lfsr_file_write(&lfs, &desyncs[i], wbuf, SIZE) => SIZE;
}
// rename!
char name[256];
sprintf(name, "batman%03x", i);
lfsr_rename(&lfs, "batman!!!", name) => 0;
}
// check that renames worked
prng = 42;
struct lfs_info info;
lfsr_stat(&lfs, "batman!!!", &info) => LFS_ERR_NOENT;
for (lfs_size_t i = 0; i < N; i++) {
// check with stat
char name[256];
sprintf(name, "batman%03x", i);
lfsr_stat(&lfs, name, &info) => 0;
assert(strcmp(info.name, name) == 0);
assert(info.type == LFS_TYPE_REG);
assert(info.size == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
lfsr_file_t file;
uint8_t rbuf[SIZE];
lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0;
lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
lfsr_file_close(&lfs, &file) => 0;
}
// check that file handles are as expected
prng = (!UNSYNC) ? 42 : 43;
desync_prng = 44;
zombie_prng = 45;
for (lfs_size_t i = 0; i < N; i++) {
// check size
assert(lfsr_file_size(&lfs, &files[i]) == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &files[i]) => 0;
lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
// check any desynced files
if (DESYNC) {
assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &desyncs[i]) => 0;
lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
// check any zombied files
if (ZOMBIE) {
assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &zombies[i]) => 0;
lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
}
// try syncing any unsynced files
for (lfs_size_t i = 0; i < N; i++) {
lfsr_file_sync(&lfs, &files[i]) => 0;
}
// check that sync worked
prng = (!UNSYNC) ? 42 : 43;
lfsr_stat(&lfs, "batman!!!", &info) => LFS_ERR_NOENT;
for (lfs_size_t i = 0; i < N; i++) {
// check with stat
char name[256];
sprintf(name, "batman%03x", i);
lfsr_stat(&lfs, name, &info) => 0;
assert(strcmp(info.name, name) == 0);
assert(info.type == LFS_TYPE_REG);
assert(info.size == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
lfsr_file_t file;
uint8_t rbuf[SIZE];
lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0;
lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
lfsr_file_close(&lfs, &file) => 0;
}
// check that file handles are as expected
prng = (!UNSYNC) ? 42 : 43;
desync_prng = 44;
zombie_prng = 45;
for (lfs_size_t i = 0; i < N; i++) {
// check size
assert(lfsr_file_size(&lfs, &files[i]) == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &files[i]) => 0;
lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
// check any desynced files
if (DESYNC) {
assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &desyncs[i]) => 0;
lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
// check any zombied files
if (ZOMBIE) {
assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &zombies[i]) => 0;
lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
}
// try rewriting our open files
uint32_t rewrite_prng = 52;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_file_rewind(&lfs, &files[i]) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26);
}
lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE;
lfsr_file_sync(&lfs, &files[i]) => 0;
}
// check that rewrites worked
rewrite_prng = 52;
lfsr_stat(&lfs, "batman!!!", &info) => LFS_ERR_NOENT;
for (lfs_size_t i = 0; i < N; i++) {
// check with stat
char name[256];
sprintf(name, "batman%03x", i);
lfsr_stat(&lfs, name, &info) => 0;
assert(strcmp(info.name, name) == 0);
assert(info.type == LFS_TYPE_REG);
assert(info.size == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26);
}
lfsr_file_t file;
uint8_t rbuf[SIZE];
lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0;
lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
lfsr_file_close(&lfs, &file) => 0;
}
// check that file handles are as expected
rewrite_prng = 52;
desync_prng = 44;
zombie_prng = 45;
for (lfs_size_t i = 0; i < N; i++) {
// check size
assert(lfsr_file_size(&lfs, &files[i]) == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &files[i]) => 0;
lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
// check any desynced files
if (DESYNC) {
assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &desyncs[i]) => 0;
lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
// check any zombied files
if (ZOMBIE) {
assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &zombies[i]) => 0;
lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
}
// cleanup files
for (lfs_size_t i = 0; i < N; i++) {
lfsr_file_close(&lfs, &files[i]) => 0;
}
if (DESYNC) {
for (lfs_size_t i = 0; i < N; i++) {
lfsr_file_close(&lfs, &desyncs[i]) => 0;
}
}
if (ZOMBIE) {
for (lfs_size_t i = 0; i < N; i++) {
lfsr_file_close(&lfs, &zombies[i]) => 0;
}
}
lfsr_unmount(&lfs) => 0;
'''
[cases.test_forphans_rename_split_backwards]
defines.N = [1, 2, 4, 8, 16, 32, 64]
defines.SIZE = [
'0',
'FBUFFER_SIZE/2',
'2*FBUFFER_SIZE',
'BLOCK_SIZE/2',
'BLOCK_SIZE',
'2*BLOCK_SIZE',
'4*BLOCK_SIZE',
]
# keep unsynced data open?
defines.UNSYNC = [false, true]
# keep a desynced file open?
defines.DESYNC = [false, true]
# keep a zombie open?
defines.ZOMBIE = [false, true]
if = '(SIZE*N*(1+DESYNC+ZOMBIE))/BLOCK_SIZE <= 32'
code = '''
lfs_t lfs;
lfsr_format(&lfs, CFG) => 0;
lfsr_mount(&lfs, CFG) => 0;
// we need a file handle for each file + desync + zombie
lfsr_file_t files[N];
lfsr_file_t desyncs[N];
lfsr_file_t zombies[N];
// create this many files while renaming
uint32_t prng = 42;
uint32_t unsync_prng = 43;
uint32_t desync_prng = 44;
uint32_t zombie_prng = 45;
for (lfs_size_t i = 0; i < N; i++) {
// create a zombie?
if (ZOMBIE) {
lfsr_file_open(&lfs, &zombies[i], "batman???",
LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26);
}
lfsr_file_write(&lfs, &zombies[i], wbuf, SIZE) => SIZE;
lfsr_file_sync(&lfs, &zombies[i]) => 0;
lfsr_remove(&lfs, "batman???") => 0;
}
// always create as last file
lfsr_file_open(&lfs, &files[i], "batman???",
LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE;
lfsr_file_sync(&lfs, &files[i]) => 0;
// keep unsynced data?
if (UNSYNC) {
lfsr_file_rewind(&lfs, &files[i]) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&unsync_prng) % 26);
}
lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE;
}
// keep a desynced file open?
if (DESYNC) {
lfsr_file_open(&lfs, &desyncs[i], "batman???",
LFS_O_RDWR | LFS_O_DESYNC | LFS_O_TRUNC) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26);
}
lfsr_file_write(&lfs, &desyncs[i], wbuf, SIZE) => SIZE;
}
// rename!
char name[256];
sprintf(name, "batman%03x", (uint32_t)(N-1-i));
lfsr_rename(&lfs, "batman???", name) => 0;
}
// check that renames worked
prng = 42;
struct lfs_info info;
lfsr_stat(&lfs, "batman???", &info) => LFS_ERR_NOENT;
for (lfs_size_t i = 0; i < N; i++) {
// check with stat
char name[256];
sprintf(name, "batman%03x", (uint32_t)(N-1-i));
lfsr_stat(&lfs, name, &info) => 0;
assert(strcmp(info.name, name) == 0);
assert(info.type == LFS_TYPE_REG);
assert(info.size == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
lfsr_file_t file;
uint8_t rbuf[SIZE];
lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0;
lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
lfsr_file_close(&lfs, &file) => 0;
}
// check that file handles are as expected
prng = (!UNSYNC) ? 42 : 43;
desync_prng = 44;
zombie_prng = 45;
for (lfs_size_t i = 0; i < N; i++) {
// check size
assert(lfsr_file_size(&lfs, &files[i]) == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &files[i]) => 0;
lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
// check any desynced files
if (DESYNC) {
assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &desyncs[i]) => 0;
lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
// check any zombied files
if (ZOMBIE) {
assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &zombies[i]) => 0;
lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
}
// try syncing any unsynced files
for (lfs_size_t i = 0; i < N; i++) {
lfsr_file_sync(&lfs, &files[i]) => 0;
}
// check that sync worked
prng = (!UNSYNC) ? 42 : 43;
lfsr_stat(&lfs, "batman???", &info) => LFS_ERR_NOENT;
for (lfs_size_t i = 0; i < N; i++) {
// check with stat
char name[256];
sprintf(name, "batman%03x", (uint32_t)(N-1-i));
lfsr_stat(&lfs, name, &info) => 0;
assert(strcmp(info.name, name) == 0);
assert(info.type == LFS_TYPE_REG);
assert(info.size == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
lfsr_file_t file;
uint8_t rbuf[SIZE];
lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0;
lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
lfsr_file_close(&lfs, &file) => 0;
}
// check that file handles are as expected
prng = (!UNSYNC) ? 42 : 43;
desync_prng = 44;
zombie_prng = 45;
for (lfs_size_t i = 0; i < N; i++) {
// check size
assert(lfsr_file_size(&lfs, &files[i]) == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &files[i]) => 0;
lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
// check any desynced files
if (DESYNC) {
assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &desyncs[i]) => 0;
lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
// check any zombied files
if (ZOMBIE) {
assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &zombies[i]) => 0;
lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
}
// try rewriting our open files
uint32_t rewrite_prng = 52;
for (lfs_size_t i = 0; i < N; i++) {
lfsr_file_rewind(&lfs, &files[i]) => 0;
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26);
}
lfsr_file_write(&lfs, &files[i], wbuf, SIZE) => SIZE;
lfsr_file_sync(&lfs, &files[i]) => 0;
}
// check that rewrites worked
rewrite_prng = 52;
lfsr_stat(&lfs, "batman???", &info) => LFS_ERR_NOENT;
for (lfs_size_t i = 0; i < N; i++) {
// check with stat
char name[256];
sprintf(name, "batman%03x", (uint32_t)(N-1-i));
lfsr_stat(&lfs, name, &info) => 0;
assert(strcmp(info.name, name) == 0);
assert(info.type == LFS_TYPE_REG);
assert(info.size == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26);
}
lfsr_file_t file;
uint8_t rbuf[SIZE];
lfsr_file_open(&lfs, &file, name, LFS_O_RDONLY) => 0;
lfsr_file_read(&lfs, &file, rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
lfsr_file_close(&lfs, &file) => 0;
}
// check that file handles are as expected
rewrite_prng = 52;
desync_prng = 44;
zombie_prng = 45;
for (lfs_size_t i = 0; i < N; i++) {
// check size
assert(lfsr_file_size(&lfs, &files[i]) == SIZE);
// try reading the file
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&rewrite_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &files[i]) => 0;
lfsr_file_read(&lfs, &files[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
// check any desynced files
if (DESYNC) {
assert(lfsr_file_size(&lfs, &desyncs[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&desync_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &desyncs[i]) => 0;
lfsr_file_read(&lfs, &desyncs[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
// check any zombied files
if (ZOMBIE) {
assert(lfsr_file_size(&lfs, &zombies[i]) == SIZE);
uint8_t wbuf[SIZE];
for (lfs_size_t j = 0; j < SIZE; j++) {
wbuf[j] = 'a' + (TEST_PRNG(&zombie_prng) % 26);
}
uint8_t rbuf[SIZE];
lfsr_file_rewind(&lfs, &zombies[i]) => 0;
lfsr_file_read(&lfs, &zombies[i], rbuf, SIZE) => SIZE;
assert(memcmp(rbuf, wbuf, SIZE) == 0);
}
}
// cleanup files
for (lfs_size_t i = 0; i < N; i++) {
lfsr_file_close(&lfs, &files[i]) => 0;
}
if (DESYNC) {
for (lfs_size_t i = 0; i < N; i++) {
lfsr_file_close(&lfs, &desyncs[i]) => 0;
}
}
if (ZOMBIE) {
for (lfs_size_t i = 0; i < N; i++) {
lfsr_file_close(&lfs, &zombies[i]) => 0;
}
}
lfsr_unmount(&lfs) => 0;
'''
# fuzz tests involving many orphans + zombies, this gets a bit crazy
[cases.test_forphans_orphanzombie_fuzz]
defines.N = [1, 2, 4, 8, 16, 32, 64]