This is an attempt to reduce the overhead of on-stack snapshots during
writes, by relaxing the gaurantees provided by lfsr_file_write during
errors.
Before, thanks to the on-stack snapshot, file writes could revert to the
previous state if an error occurred. Now, on-stack snapshots are limited
to lfsr_ftree_carve, so only the state change in lfsr_ftree_carve is
reverted.
This should behave relatively predictably, since lfsr_ftree_flush calls
lfsr_ftree_carve in a normal order. If an error occurs, some, none, or
all of the data is actually written. For truncate/fruncate, there is
only one call to carve, so these remain atomic.
It's tempting to want to push this lower. If you push the atomic
operations down to the btree/bshrub level, individual btree/bshrub commits
are already atomic, so tracking on-stack bshrubs could be dropped
completely. But failures at the sub-carve level get weird! Thanks to
block crystallization and the use of order-statistic operations, the
resulting file can be quite unpredictable. For example:
on-disk: abcdefghijklmnopqrstuvwxyz
write: JKLMN
error!
on-disk: abcdstuvwxyz
With atomic carves, worst case is something like this:
on-disk: abcdefghijklmnopqrstuvwxyz
write: JKLMN
error!
on-disk: abcdefghiJKlmnopqrstuvwxyz
Thanks to file-level snapshotting, the original file can still be
recovered (and is on-disk until sync is called), but the
unpredictability is concerning.
Alternatively, if we had btree range removals, lfsr_ftree_carve could be
entirely atomic (and more efficient).
Unfortunately, btree range removals still seem like they would be
difficult to implement, and will probably be out of scope for some time.
The added code cost will also likely outweigh any savings from dropping
on-stack bshrub tracking. Still, this is probably worth looking into in
the future.
Code changes:
code stack
before: 33356 3072
after: 33006 (-1.0%) 2992 (-2.6%)
The condition for checking if the left fragment could be coalesced was
wrong, preventing a common coalescing chance in linear rewrites, and
leaving a weird 1-flush-sized alignment issue in the fragments:
Was: lfsr_data_size(&bptr.data) < lfs->cfg->fragment_size
Should be: fragment_end - (bid-(weight-1)) <= lfs->cfg->fragment_size
This was correct for the right fragment coalescing, not sure how the
left ended up messed up.
(Actually I do know why, the math here is dense, complex, and subtle,
a nasty combo.)
Desynchronized files are a new concept intended to capture some useful
quirks of the previous multiple-open-file behavior.
This adds:
- LFS_O_DESYNC - Mark a file as desync during open
- lfsr_file_desync - Mark a file as desync whenever
- lfsr_file_sync - Mark a file as NOT desync, and sync the file
Desynced files:
1. Don't recieve updates from writes to other file handles. This makes
desynced files act as a sort of snapshot of the file at the time it
was marked desync.
2. Don't call lfsr_file_sync on close. Unless lfsr_file_sync is
explicitly called, changes to desynced files are not reflected on
disk and not broadcasted to other file handles.
A side-effect of 2., is that this gives you a quick way to abort a file
write. Marking a file as desync and then closing the file will never
error.
Additionally, if an error occurs during a write operation, the file is
implicitly marked as desync. This provides graceful write aborting in
unlikely error cases. This has actually always been a feature in
littlefs, it was just named differently and didn't have an optional
recovery mode.
Since littlefs actually has to do more work to keep files in sync, the
desync feature is quite cheap:
code stack
before: 33324 3072
after: 33360 (+0.1%) 3072 (+0.0%)
- Uninitialized warning in lfsr_mtree_namelookup because we adjust
mdir.mid unconditionally even if there is an error.
Putting the mid adjustment behind a check for ENOENT _should_ fix
this, but try as I might, GCC just can't figure out what's going on.
Adding a blanket rid_=0 in lfsr_mdir_namelookup is the best workaround
I can think of right now.
- Uninitialized warnings caused by not knowing if err is <0 in functions
where err gets merged with positive results.
I think this stems all the way from bd functions, since those poke
outside of littlefs. But it's a good idea to assert on this in
functions where err needs to be <0 anyways.
The __builtin_unreachable in lfs_util.h makes this double as a
compiler hint even in release mode, which is nice.
- GCC thinks rid is uninitialized in lfsr_btree_commit when checking
if left sibling can be merged.
As far as I can tell this actually can't happen because of the above
parent.trunk=0 check. It's not to hard to look at all code paths that
set parent.trunk=!0, they also set rid to some valid value.
I think the cause is GCC giving up because gotos are involved.
These uninitialized warnings are especially annoying since they are tied
to optimization passes. Which means they only show up when compiling -Os
and show up inconsistently (I don't know if there's more behind -O3 for
example).
This combined with frequent false positives, exacerbated by our heavy
use of out-pointers, makes this warning quite frustrating to deal with.
I would be tempted to turn this warning off if is wasn't so valuable.
Now setting estimate=-1 will force a recalculation on the next bshrub
commit. This centralizing the annoying bshrub estimate calculation, so
code that allocs/fetches bshrubs can be simplifies to just struct
assignment.
As a plus, you don't pay the estimate calculation cost when only reading
a file.
Also dropped lfsr_bshrub_alloc/fetch, these weren't really useful
functions and their naming could be misleading.
I wonder if bshrubs will ever stop feeling like a big hack.
Also tweaked bshrub estimates a bit so they only include
bsprouts/bshrubs. I think we can get away with this by including
max(bptr, trunk, btree) + shrub_size when we get around to calculating
file limits.
We are now including an extra tag when estimating bsprout size, but
that's not the end of the world. At least all calls to
lfsr_bsprout_estimate__ are consistent now.
These changes were more for code organization/runtime improvements, the
benefit to code cost is minimal:
code stack
before: 33364 3072
after: 33336 (-0.1%) 3072 (+0.0%)
We weren't updating other opened files with bshrub estimates, which
meant there was a risk bshrub estimates fall out of date, and mdir
commits lock up with ERANGE.
We need more testing over these estimate conditions. Unfortunately
they're a big headache in the new design...
The bshrub estimate now expects to include all synced struct contents,
so we need to set it to something greater than zero when converting from
a bsprout/bptr/btree.
But honestly, all of this bshrub estimate stuff needs work...
1. We used a temporary bshrub outside of the tracked ftree to commit
without clobbering the ftree. This breaks the invariant that all
bshrubs are tracked during mdir commit.
By luck, the bshrubs were still working. They normally don't get
staged properly without this invariant, but:
1. These are always new bshrubs, so no state for mdir compact to be
aware of.
2. These are always immediately SHRUBCOMMITed, so they happen to get
staged.
But we can avoid breaking this invariant by encoding the commit
first. We may clobber the ftree, but this is what our on-stack ftree
copies are for.
2. We weren't including becksums when converting bptrs to bshrubs.
Maybe we shouldn't care about this, since we don't create direct
bptrs in normal operation. But I guess we'll be compat friendly for
now...
Now, when files are synced, they broadcast their disk changes to any other
opened file handles. In effect, all open files match disk after a sync
call to any opened file handle pointing to that file.
This was a much requested feature, as the previous behavior (multiple
opened file handles maintain independent snapshots) is pretty different
from other filesystems. It's also quite difficult to implement outside
of the filesystem, since you need to track all opened files, requiring
either unbounded RAM or a known upper limit.
---
A bit unrelated, but this commit also changes bshrub estimate
calculation to include all opened file handles. This adds some annoying
complexity, but is necessary to prevent sporadic ERANGE errors when
the same file is opened multiple times.
The current implementation just refetches on-disk metadata. This adds
some maybe unnecessary metadata lookups, but simplifies things by
avoiding the tracking of on-disk sprout/shrub size, which risks falling
out of date. Keep in mind we only recalculate the estimate every
~inline_size/2 bytes written.
Just like lfsr_mdir_estimate, this scales O(n^2) with the number of
opened files (this are basically the same function... hmmm... can they
be deduplicated?). This is unlikely to be a problem for littlefs's use
case, but just something to be aware of.
Code changes:
code stack
before: 32920 3032
after: 33192 (+0.8%) 3048 (+0.5%)
- Reworked *_estimate functions to use swapping bounds/variables much
like lfsr_rbyd_appendattr.
- Merged *_estimate_ into *_estimate. Mainly for (perhaps misdirected)
code cleanliness reasons. The compiler is already going to be inlining
these single-calls since inlining is always worthwhile.
- Renamed internal mdir-commit-related functions to have the __ suffix
even if there's not a direct naming conflict.
This is a bit of a weird naming scheme, but it's useful to hinting
that all the *__ functions are at the same logic level. It also hints
that you probably shouldn't call these unless you're dealing with mdir
internals. Functions like lfsr_mdir_compact__ will probably just break
if called outside of lfsr_mdir_commit.
- Moved *_estimate functions to be closer to *_compact functions, since
these are closely related (*_estimate is basically a soft *_compact).
These changes saved a bit of code, but added a surprising amount stack
cost. I'm guessing this is related using swap functions. Maybe ptr
aliasingis causes problem, or swapping breaks compiler invariants about
variable locations:
code stack
before: 32948 2984
after: 32920 (-0.1%) 3032 (+1.6%)
Maybe we should consider an alternative impl for both the *_estimate
and *_appendattr variable swapping...
This adds more internal functions to deduplicate things:
- lfsr_bsprout_isbsprout
- lfsr_bsprout_isbleaf
- lfsr_bsprout_size
- lfsr_bsprout_cmp
- lfsr_bshrub_commit__
- lfsr_bsprout_estimate__
- lfsr_bshrub_estimate__
- lfsr_bsprout_compact__
- lfsr_bshrub_compact__
This doesn't actually save that much code, I guess our attr-list
iterations are quite cheap, but it does make mdir_commit__/compact__
a bit easier to read:
code stack
before: 32964 2984
after: 32948 (+0.0%) 2984 (+0.0%)
Since we don't need this for bshrub/bsprout negotiation, we might as
well drop it to keep concerns separate.
code stack
before: 32980 3008
after: 32964 (+0.0%) 2984 (-0.5%)
Now that bshrubs are limited to files, we might as well lean into it.
Instead of relying on extra traversals through our attr-list, we now
rely on bsprouts/bshrubs always existing in the opened mdir list. This
is a much more robust way to deduplicate bsprout/bshrub compaction
operations. The previous implementation already had a bug since
bsprout/bshrub calculation was forgotten from lfsr_mdir_estimate_.
This is a direct tradeoff of code and RAM. Though note the previous
impl was incomplete and would likely need a bit more code:
code stack
before: 33184 2944
after: 32980 (-0.6%) 3000 (+1.9%)
Now instead of relying on lfsr_f_isunsynced, which will probably have
issues with multiple opened files, we do O(n^2) scan sover the opened
file list to figure out what hasn't already been compacted.
What's interesting is the slight difference in lfsr_mdir_estimate_ and
lfsr_mdir_compact__. In lfsr_mdir_compact__, by updating all opened
bshrubs/bsprouts, we trivially prevent duplicate compactions. But for
lfsr_mdir_estimate_, we need to prevent duplicates without modification.
So instead, we only include the _last_ reference in the opened file
list. This avoids duplicates while also avoiding multiple mdir lookups.
The remaining puzzle is including in-flight in-attr bshrubs. Traversing
over two data-structures is already complicated/expensive. Traversing
over three might be a bit much...
Before, rdonly buffers were dropped, requiring a re-read for files open
RDWR. Now the buffer is updated with whatever data overlaps.
This doesn't add _that_ much code cost, and may be beneficial for
certain rd/wr patterns in different parts of a file (we don't drop
buffers at all in bypassing writes). Though it may be better to open two
separate file handles in this use case...
We will need this logic anyways for updating multiple in-sync opened
file handles. And maybe this logic can be deduplicated then.
code stack
before: 32848 2944
after 32908 (+0.2%) 2944 (+0.0%)
This can happend if we read some data into our buffer (or write+flush+seek
in some weird pattern) and then do a bypassing write that overlaps the
buffer.
The solution here is to make sure the buffer is cleared when doing
bypassing writes.
In theory, we could be a bit smarter by checking and only clearing when
overlap occurs, or even updating the buffer with the new contents like
we do in the bd cache layer, but the value would probably be minimal.
File writes are already expected to clobber buffered reads.
This is more a cludge than a real fix, we should probably be handled in
the bd-wrapper layer, but that's all a mess right now and slated for
future work.
The problem is that our "rewinding" of block-level progs for alignment
leaves our bd caches with stale data. This is normally fine, but then we
go read the stale data to calculate our becksum.
Dropping our caches before calculating the becksums fixes this, at least
temporarily.
This will always fail due to prog alignment, but we won't notice because
the align flag gets set in the flush loop. So it's not a _hard_ error,
but results in an unnecessary btree commit and a bunch of extra reading.
Some of these checks aren't necessary if crystal_thresh >= prog_size,
which should always be the case, but wanting crystal_size=0 to be a
shorthand for "no crystallization" makes it's possible. Maybe we should
set crystal_thresh = lfs_max32(crystal_thresh, prog_size) in the
future...
Unfortunately we don't really have a way to prove that block-level
erased-state checksums are working short of benchmarking, so sure
enough block-level erased-state checksums were not working.
Some of the issues were a bit silly:
- Forgot so zero-init the checksum.
- Typo meant we were calculating the becksum of the wrong block.
- Flushing when buffer is non-empty triggered redundant flushes if the
buffer was filled for reading.
The last one is a bit more fundamental.
Humorously, it wasn't caught earlier because the buffer is always
in-sync with disk. So redundant flushes aren't an _error_, but they sure
hurt performance.
The fix is a little bit tricky. We want to flush only when
LFS_F_UNFLUSHED is set, but this confuses lfsr_file_write into thinking
it can clobber small file buffers.
The solution here is to only flush when LFS_F_UNFLUSHED is set, always
set LFS_F_UNFLUSHED on small files, which makes a bit of sense if you
think about it.
This means LFS_F_UNFLUSHED can be set on read-only and in-sync files,
but that should hopefully not be an issue.
This is useful for debugging checksum mismatches on disk.
And since dbgblock.py has some relatively flexible options for slicing
the disk, this can be used to find the checksum of any on-disk data
pretty easily.
I think ctags's defaults may have changed recently or something, but
it's struggling to find function definitions when there's a
predeclaration in the same file.
Adding --fields=+n (line numbers) fixes this.
This lets us cache more of each btree entry during reads, since the
buffer is no longer filled with unlikely-to-be-relevant, earlier btree
entries.
Also added lfsr_ftree_readnext for reading one btree entry, which is a
bit more flexible than the hint mechanism.
Code changes:
code stack
before: 32776 2944
after: 32804 (+0.1%) 2944 (+0.0%)
Now all functions maintain the invariant that "small files" (inlinable,
bufferable, single-fragment files, these can be stored in a single data
attr) always reside entirely in the file's buffer.
This invariant wasn't strictly maintained before, unfortunately it's
tricky to maintain for truncate/fruncate while also providing error
recovery without unnecessary data flushes. But after dealing with a
number of problematic corner cases related to small files, I decided to
just do the unnecessary data flushes.
At the very least, by also checking for truncate/fruncate calls where
the data is already available, we can avoid pathological cases such as
small file logging. With this tweak, unnecessary data flushes should be
somewhat uncommon.
In addition to providing a strong invariant, handling small file caching
in truncate/fruncate is nice in that the complicated logic is entirely
contained in truncate/fruncate, meaning you don't pay the code cost if
you don't use these functions.
Code changes:
code stack
before: 32816 2936
after 32776 (-0.1%) 2944 (+0.3%)
This trades off fewer bus transactions when performing small linear
reads for the possibility of reading more data than is strictly
needed. It also introduces potential buffer thrashing when opening a
file RDWR, but this can be avoided by using two different file handles.
The current implementation only reads at most one fragment/block past
the requested read into the buffer. Reading any more doesn't really
save any reads, since we need to do an additional btree lookup for each
fragment/block.
This change was motivated by the observation that not using the buffer
in lfsr_file_read absolutely destroys performance when doing byte-level
linear reads.
In theory, buffering could be left up to the users, but buffering in the
filesystem allows us to be a bit smarter since we know the exact tree
layout.
---
This effectively reverts the file buffer changes to be much closer to
the previous version, though instead of flushing in lfsr_file_seek, we
flush in lfsr_file_read. This may be surprising to users, but we _must_
flush in one of these two functions. Flushing in lfsr_file_read moves
flushing to the last possible moment, increasing the chances we can
avoid flushing entirely.
This also makes lfsr_file_seek not touch disk at all, which is nice.
At the very least it may be useful to expose lfsr_file_flush to the
users so flushing in lfsr_file_read can be manually avoided...
This is intended to enable lfsr_file_read to use the buffer as well.
This adds LFS_F_UNFLUSHED and internal lfsr_file_flush to manage buffer
flushing. This also results in a nice reorganization of lfsr_file_sync.
Of course, small file caching proves to be a big pain again, with
several subtle corner cases in truncate/fruncate that need to set the
LFS_F_UNFLUSHED flag so we fix small files in lfsr_file_sync.
Note sure why we weren't hitting this earlier, but I've been hitting
this race condition a bunch recently and it's annoying.
Now every failed process kills the other test processes unconditionally.
It's not clear if this actually _fixes_ the race condition or just makes
it less likely, but it's good enough to keep the test script user
friendly.
This restores the previous handling of inlined files, which was a strict
requirement in earlier versions, now more of an optimization.
littlefs now tries to keep small inlinable files in RAM. This doesn't
always work because inlined files can be quite large, so littlefs only
tries if it knows the inlineable file fits in the file's buffer.
I had a bit of a hard time working truncate/fruncate into the scheme. It
seems simple on paper: if truncate/frunate make a file "small", move it
into our buffer. But if we already have data in our buffer, we can't make
the file small without potentially clobbering the buffer. This conflicts
with error recovery and leaves us in a bit of a bind.
Two options:
1. Flush the file on truncate/fruncate so we can use the buffer for
caching small files.
This risks unnecessary disk writes. Consider fruncating a small
log you know fits in the file buffer.
2. Allow small files to sometimes not be cached in the buffer. Fix this
state on sync.
This risks bugs caused by relying on the sometimes-incorrect small
file invariant.
This can still also cause unnecessary disk writes if you, say, write
to the file between truncate and sync.
This goes with the option 2. I think option 1 has more failure cases
that can be problematic and option 2 allows us to avoid disk writes in
more cases. But it isn't clear what the best option is.
While definitely winning cool points, uncrc32cs have a number of
problems:
1. uncrc32c is relatively unflexible, being limited to only CRC-related
checksums, and probably violating some properties of cryptographic
hashes if possible there.
2. Code savings are minimal, a reversed crc32c implementation is a only
a little less costly than the logic to save aligned CRCs, and since
it's not on the hot-path, the stack cost is ~zero.
3. uncrc32c may come with a high computation cost.
We aren't measuring this, but uncrc32c either operates at the
bit-level, or requires a second set of tables which is unreasonable
for littlefs's use case.
With uncrc32c you need to update the checksum based on every bit in
the extra unaligned data, up to prog_size. With flcksums it's just a
copy of a word, and prog_size has no impact.
So for now dropping uncrc32c, though this can always be reverted in the
future.
Checksumming unaligned data during block compaction is surprisingly
tricky. We don't know if our data will be aligned until after
a potentially unbounded number lookups, we need to write data into our
pcache as we go to avoid unnecessary lookups, but if we end up unaligned
we need to revert our checksum to the checksum of the aligned data.
The way I see it there are 4 options:
1. Calculate the checksum after writing data into the block.
This is the most expensive option, requiring a full second read of
the data to calculate the checksum. It is simple though.
2. Do a pass over the btree to figure out alignment before writing.
This at least only reads metadata twice, so is more efficient than
the 1st option.
3. Keep track of the aligned checksum on each flush, falling back to the
last flushed checksum if we need to correct alignment.
This solution is flexible though requires some extra state to track
multiple checksums.
4. Leverage the math behind CRCs to run the CRC backwards when we
truncate for alignment.
This works, though a bit inefficiently, but is strictly tied to
CRC-related checksums.
By inefficient I mean that we would likely be limited to a bit-level
"uncrc32c". It's possible to create nibble/byte tables for uncrc32c,
but this adds significant code cost for a relatively uncritical
function.
I was hopeful that we could leverage the existing tables in both
functions, but unfortunately it doesn't work out like that. You could
scan the crc32c table to find the constant to reverse, but this
requires ~16*2 or ~256 operations vs "naive" ~8 operations per byte.
This commit implements both 3 and 4, defaulting to 4 unless
LFS_NO_UNCRC32C is defined.
The current lfs_uncrc32c implementation is a simple bit-level
implementation, but does allow for crc32c truncation without any extra
state.
code stack
before: 32044 2880
uncrc32c: 32108 (+0.2%) 2880 (+0.0%)
flcksum: 32132 (+0.3%) 2880 (+0.0%)
Our crystallization threshold doesn't really describe the bounds of an
object, and I think it's a bit easier to think of it as a threshold for
block compaction.
Heck I've already been calling this the crystallization threshold all
over the code base.
An important change is this bumps the value by 1 bytes, so
crystal_thresh now describes the smallest size of a block our write
strategy will attempt to write.
Heuristically:
- data >= crystal_thresh => compacted into blocks
- data < crystal_thresh => stored as fragments
This trades a runtime check for a different function call. Enforcing
some minor semantics in the function's type/asserts.
This also makes it so there are no special tag bits used during rbyds
lookup, only rbyd commits.
In theory this saves a bit of code, we don't have a runtime check, but
in practice the extra function apparently outweighs the cost of the
runtime check:
code stack
before: 31956 2880
after: 32024 (+0.2%) 2880 (+0.0%)
We already get the leaf rbyd as a part of btree lookup, and since ids
can't be split across rbyd boundaries, we can be sure any bptr attrs
live in the same rbyd.
This can be extended to any future bptr attrs.
Aside from the small performance gain, this also means we can drop the
btree bid+tag lookups. All extra attr lookups to lookup the rbyd first.
This saves a bit of code but also avoids a set of issues with the btree
semantics where lookupnexting an extra attr can return ENOENT
prematurely when on an rbyd boundary.
As I'm typing this I realize this means we have no way to iterate over
all _tags_ in a btree, only over all _bids_. Fortunately I don't think
we will ever need the former.
code stack
before: 32136 2880
after: 31956 (-0.6%) 2880 (+0.0%)
This avoids a hacky shoehorning of our attr-list while also making
lfsr_btree_carve a nice reflection of lfsr_ftree_lookupnext.
I'm kind of surprised how little this impacts code size, maybe const
propagation already pushed the attr bounds through?
code stack
before: 32124 2888
after: 32128 (+0.0%) 2880 (-0.3%)
- Becksums in lfsr_ftree_carve can be preserved when slicing a right
sibling block.
The best way I can think to do this right now is to simply copy the
becksum tag as an extra attr. This unfortunately means
lfsr_ftree_carve doesn't exactly scale well, each new block-related
tag will increase the stack cost by at least one attr.
Fortunately we really should have that many block-related tags. The
only other planned one currently being content/parity ids.
- Fixed incorrect becksum check that meant we were only leveraging
becksums where bptr.off = 0.
Humorously, since lfsr_ftree_carve didn't support becksums, bptr.off
was always 0 when becksums were involved.
- Fixed issues where bptr.off != 0 in lfsr_ftree_flush (only caused by
becksums) was not accounted for, leading to progs beyond a block
during block compaction.
Code changes:
code stack
before: 32100 2832
after: 32124 (+0.1%) 2888 (+1.9%)
Block/fragment relevant variables are at least now localized to their
respective loops, though block writing still has some ugly gotos to
skip a few lookups when erased-state is found.
Though finding erased-state does sort of just break through the rest of
the block-writing heuristics, so maybe it's good that the code matches
the underlying logic...
Also made crystal_size lookups a bit more aggressive. The previous logic
assumed worst-case crystal size when near the beginning of a file, and
best best-case crystal size near the end. At the very least, this is
wildly inconsistent with crystals in the middle of sparse files.
Much like the erased-state checksums in our rbyds (ecksums), these
block-level erased-state checksums (becksums) allow us to detect failed
progs to erased parts of a block and are key to achieving efficient
incremental write performance with large blocks and frequent power
cycles/open-close cycles.
These are also key to achieving _reasonable_ write performance for
simple writes (linear, non-overwriting), since littlefs now relies
solely on becksums to efficiently append to blocks.
Though I suppose the previous block staging logic used with the CTZ
skip-list could be brought back to make becksums optional and avoid
btree lookups during simple writes (we do a _lot_ of btree
lookups)... I'll leave this open as a future optimization...
Unlike in-rbyd ecksums, becksums need to be stored out-of-band so our
data blocks only contain raw data. Since they are optional, an
additional tag in the file's btree makes sense.
Becksums are relatively simple, but they bring some challenges:
1. Adding becksums to file btrees is the first case we have for multiple
struct tags per btree id.
This isn't too complicated a problem, but requires some new internal
btree APIs.
Looking forward, which I probably shouldn't be doing this often,
multiple struct tags will also be useful for parity and content ids
as a part of data redundancy and data deduplication, though I think
it's uncontroversial to consider this both heavier-weight features...
2. Becksums only work if unfilled blocks are aligned to the prog_size.
This is the whole point of crystal_size -- to provide temporary
storage for unaligned writes -- but actually aligning the block
during writes turns out to be a bit tricky without a bunch of
unecesssary btree lookups (we already do too many btree lookups!).
The current implementation here discards the pcache to force
alignment, taking advantage of the requirement that
cache_size >= prog_size, but this is corrupting our block checksums.
Code cost:
code stack
before: 31248 2792
after: 32060 (+2.5%) 2864 (+2.5%)
Also lfsr_ftree_flush needs work. I'm usually open to gotos in C when
they improve internal logic, but even for me, the multiple goto jumps
from every left-neighbor lookup into the block writing loop is a bit
much...
Mainly by not looking up left neighbors after the first of many
block/fragment/crystal writes.
The right neighbors should already be avoiding redundant lookups since
redundant lookups imply a full fragment/block on the not-last write.
Additionally, once we fail our crystallization check, we can assume all
future fragment writes will fail, so we only need to do that lookup
once.
There's probably still more to optimize, but the way these heuristics
interact are tricky...
This combines the previous lfsr_data_truncate/lfsr_data_fruncate
behavior into a single flexible function, and makes truncate/fruncate
small aliases (drop in the future?).
The combined behavior lets us adopt lfsr_data_slice in more places.
Looking forward, bptr checksums provide an easy mechanism to validate
data residing in blocks. This extends the merkle-tree-like nature of the
filesystem all the way down to the data level, and is common in other
COW filesystems.
Two interesting things to note:
1. We don't actually check data-level checksums yet, but we do calculate
data-level checksums unconditionally.
Writing checksums is easy, but validating checksums is a bit more
tricky. This is made a bit harder for littlefs, since we can't hold
an entire block of data in RAM, so we have to choose between separate
bus transactions for checksum + data reads, or extremely expensive
overreads every read.
Note this already exists at the metadata-level, the separate bus
transactions for rbyd fetch + rbyd lookup means we _are_ susceptible
to a very small window where bit errors can get through.
But anyways, writing checksums is easy. And has basically no cost
since we are already processing the data for our write. So we might
as well write the data-level checksums at all times, even if we
aren't validating at the data-level.
2. To make bptr checksums work cheaply we need an additional cksize
field to indicate how much data is checksummed.
This field seems redundant when we already have the bptr's data size,
but if we didn't have this field, we would be forced to recalculate
the checksum every time a block is sliced. This would be
unreasonable.
The immutable cksize field does mean we may be checksumming more data
than we need to when validating, but we should be avoiding small
block slices anyways for storage cost reasons.
This does add some stack cost because our bptr struct is larger now:
code stack
before: 31200 2768
after: 31272 (+0.2%) 2800 (+1.1%)
This avoids redundant lookups when holes are involved.
And we don't really leverage data holes as an abstraction well. We use
data holes in two places, but they do two different things, so they may
as well be specialized operations.
code stack
before: 31092 2752
after: 31200 (+0.3%) 2768 (+0.6%)
Instead of writing every possible config that has the potential to be
useful in the future, stick to just writing the configs that we know are
useful, and error if we see any configs we don't understand.
This prevents unnecessary config bloat, while still allowing configs to
be introduced in a backwards compatible way in the future.
Currently unknown configs are treated as a mount error, but in theory
you could still try to read the filesystem, just with potentially
corrupted data. Maybe this could be behind some sort of "FORCE" mount
flag. littlefs must never write to the filesystem if it finds unknown
configs.
---
This also creates a curious case for the hole in our tag encoding
previously taken up by the OCOMPATFLAGS config. We can query for any
config > SIZELIMIT with lookupnext, but the OCOMPATFLAGS flag would need
an extra lookup which just isn't worth it.
Instead I'm just adding OCOMPATFLAGS back in. To support OCOMPATFLAGS
littlefs has to do literally nothing, so this is really more of a
documentation change. And who know, maybe OCOMPATFLAGS will have some
weird use case in the future...
Also:
- Renamed GSTATE -> GDELTA for gdelta tags. GSTATE tags added as
separate in-device flags. The GSTATE tags were already serving
this dual purpose.
- Renamed BSHRUB* -> SHRUB when the tag is not necessarily operating
on a file bshrub.
- Renamed TRUNK -> BSHRUB
The tag encoding space now has a couple funky holes:
- 0x0005 - Hole for aligning config tags.
I guess this could be used for OCOMPATFLAGS in the future?
- 0x0203 - Hole so that ORPHAN can be a 1-bit difference from REG. This
could be after BOOKMARK, but having a bit to differentiate littlefs
specific file types (BOOKMARK, ORPHAN) from normal file types (REG,
DIR) is nice.
I guess this could be used for SYMLINK if we ever want symlinks in the
future?
- 0x0314-0x0318 - Hole so that the mdir related tags (MROOT, MDIR,
MTREE) are nicely aligned.
This is probably a good place for file-related tags to go in the
future (BECKSUM, CID, COMPR), but we only have two slots, so will
probably run out pretty quickly.
- 0x3028 - Hole so that all btree related tags (BTREE, BRANCH, MTREE)
share a common lower bit-pattern.
I guess this could be used for MSHRUB if we ever want mshrubs in the
future?
I'm just not seeing a use case for optional compat flags (ocompat), so
dropping for now. It seems their *nix equivalent, feature_compat, is
used to inform fsck of things, but this doesn't really make since in
littlefs since there is no fsck. Or from a different perspective,
littlefs is always running fsck.
Ocompat flags can always be added later (since they do nothing).
Unfortunately this really ruins the alignment of the tag encoding. For
whatever reason config limits tend to come in pairs. For now the best
solution is just leave tag 0x0006 unused. I guess you can consider it
reserved for hypothetical ocompat flags in the future.
---
This adds an rcompat flag for the grm, since in theory a filesystem
doesn't need to support grms if it never renames files (or creates
directories?). But if a filesystem doesn't support grms and a grms gets
written into the filesystem, this can lead to corruption.
I think every piece of gstate will end up with its own compat flag for
this reason.
---
Also renamed r/w/oflags -> r/w/ocompatflags to make their purpose
clearer.
---
The code impact of adding the grm rcompat flag is minimal, and will
probably be less for additional rcompat flags:
code stack
before: 31528 2752
after: 31584 (+0.2%) 2752 (+0.0%)
Also renamed lfsr_rbyd_compact -> lfsr_rbyd_appendcompaction to make
room for a more high-level lfsr_rbyd_compact and emphasize that this is
an append operation.
lfsr_rbyd_appendshrub was a common pattern emerging in mdir commit
related functions for moving shrubs around. It's just a useful function
to have.
lfsr_rbyd_compact, on the other hand, is only useful for the evicting
bshrubs. Most other rbyd compaction code involves weird corner cases and
doesn't seem to generalize well (or at least I can't see it). But adding
lfsr_rbyd_compact is nice for consistency with the mdir commit/compact
functions. And it can be used by lfsr_bshrub_commit at least...
Code changes minimal:
before: 31596 2752
after: 31528 (+0.2%) 2752 (+0.0%)
Now that error recovery is well defined (at least in theory), and
high-level mdir/file functions create on-stack copies, the on-stack
copies for the low-level rbyd/btree/bshrub commit functions are
redundant and not useful.
Dropping the redundant on-stack copies in low-level functions saves a
bit for stack usage.
Additionally, we can adopt lfsr_rbyd_commit in more places where it was
avoided to avoid even more redundant on-stack copies.
code stack
before: 31676 2776
after: 31596 (-0.3%) 2752 (-0.9%)
Christopher Haster