Noticed a lot of duplicate conditions, so tried merging these two code
paths. This does risk a difficult to read/maintain function, since there
are some rather tricky subtleties with the sprout -> shrub transition.
On the other hand, the code reuse does mean less conditions to worry
about.
Merging these code paths also saves a bit of code:
code stack
before: 30960 2256
after: 30700 (-0.8%) 2256 (+0.0%)
Note we still end up with a shrub, even if the file could revert back to
a sprout. This is just a simplification for the inlined file logic. We
never implicitly revert to a sprout.
This mostly figures out how things might work with coalescing, without
fully implementing coalescing yet.
One thing noteworthy, previously when carving right data, we would
remove the right data and rewrite it. This was to accomidate implicit
splits:
buf: [bbbb]
shrub: [llrrrrrr]
1. rm [ll]
2. append [llrr]
3. append [llbbbbrr]
An implicit split being when the left sibling and right sibling are the
same data
buf: [bbbb]
shrub: [llllllll]
1. carve [ll]
2. append [llbbbb]
3. append [llbbbbll]
By separating out the split logic, this rm can be avoided:
buf: [bbbb]
shrub: [llrrrrrr]
1. carve [llrr]
2. append [llbbbbrr]
At the cost of making our implicit split have more steps (in code),
though, I believe it does have less subtle/more understandable behavior:
buf: [bbbb]
shrub: [llllllll]
1. carve [ll]
2. append [llll]
3. append [llbbbbll]
As a plus, we avoid looking up the same sibling twice when doing
implicit splits.
lfsr_file_carveinlined writes data into a shrub, while handling both the
carving logic of data we might be overlapping, and any hole logic we
need to fill out the tree.
This provides a nice, relatively simple but flexible, operation for all
shrub updates:
static int lfsr_file_carveinlined(lfs_t *lfs, lfsr_file_t *file,
lfs_off_t pos, lfs_off_t weight, lfs_soff_t delta,
lfsr_data_t data);
I'm quite happy with how these internal carveinlined/carvebtree
functions are coming together. It's nice to have all of that logic in
one place, even if it's a bit complex.
Ended up changing the name of lfsr_mtree_traversal_t -> lfsr_traversal_t,
since this behaves more like a filesytem-wide traversal than an mtree
traversal (it returns several typed objects, not mdirs like the other
mtree functions for one).
As a part of this changeset, lfsr_btraversal_t (was lfsr_btree_traversal_t)
and lfsr_traversal_t no longer return untyped lfsr_data_ts, but instead
return specialized lfsr_{b,t}info_t structs. We weren't even using
lfsr_data_t for its original purpose in lfsr_traversal_t.
Also changed lfsr_traversal_next -> lfsr_traversal_read, you may notice
at this point the changes are intended to make lfsr_traversal_t look
more like lfsr_dir_t for consistency.
---
Internally lfsr_traversal_t now uses a full state machine with its own
enum due to the complexity of traversing the filesystem incrementally.
Because creating diagrams is fun, here's the current full state machine,
though note it will need to be extended for any
parity-trees/free-trees/etc:
mrootanchor
|
v
mrootchain
.-' |
| v
| mtree ---> openedblock
'-. | ^ | ^
v v | v |
mdirblock openedbtree
| ^
v |
mdirbtree
I'm not sure I'm happy with the current implementation, and eventually
it will need to be able to handle in-place repairs to the blocks it
sees, so this whole thing may need a rewrite.
But in the meantime, this passes the new clobber tests in test_alloc, so
it should be enough to prove the file implementation works. (which is
definitely is not fully tested yet, and some bugs had to be fixed for
the new tests in test_alloc to pass).
---
Speaking of test_alloc.
The inherent cyclic dependency between files/dirs/alloc makes it a bit
hard to know what order to test these bits of functionality in.
Originally I was testing alloc first, because it seems you need to be
confident in your block allocator before you can start testing
higher-level data structures.
But I've gone ahead and reversed this order, testing alloc after
files/dirs. This is because of an interesting observation that if alloc
is broken, you can always increase the test device's size to some absurd
number (-DDISK_SIZE=16777216, for example) to kick the can down the
road.
Testing in this order allows alloc to use more high-level APIs and
focus on corner cases where the allocator's behavior requires subtlety
to be correct (e.g. ENOSPC).
This was a cludge due to needing lfs->mtree initialized to traverse the
mtree, the assumption being that future traversals should strictly
update the mtree/mroot to the existing state.
Moving code around (and adopting an actual state machine, which will be
needed for btree traversal) made this no longer necessary.
Now the mtree/mroot is only initialized in lfsr_mountinited, as it
should be.
Still needs testing, though the byte-level fuzz tests were already causing
blocks to crystallize. I noticed this because of test failures which are
fixed now.
Note the block allocator currently doesn't understand file btrees. To
get the current tests passing requires -DDISK_SIZE=16777216 or greater.
It's probably also worth noting there's a lot that's not implemented
yet! Data checksums and write validation for one. Also ecksums. And we
should probably have some sort of special handling for linear writes so
linear writes (the most common) don't end up with a bunch of extra
crystallizing writes.
Also the fact that btrees can become DAGs now is an oversight and a bit
concerning. Will that work with a closed allocator? Block parity?
First, realized the the LFS_UNREACHABLE logic was flipped after a
confusing test bug (damn double negatives). But also realized LFS_ASSERT
could be tweaked to "call" __builtin_unreachable() on assert failure to
act as a sort of compiler hint.
Turns out this hint saves a little bit of code, note both builds have
LFS_UNREACHABLE fixed:
code stack
without __builtin_unreachable: 28408 1928
with __builtin_unreachable: 28324 (-0.3%) 1920 (+0.0%)
Since __builtin_unreachable is a compiler extension, its usage respects
LFS_NO_INTRINSICS.
Added lfsr_bptr_t to represent block pointers (maybe we should rename
mblocks back to mptr), added fetching of btrees/bptrs in
lfsr_file_opencfg, added estimate tracking to our shrubs so we actually
know when to create a btree, and implemented most of the high-level
btree logic.
It's not working yet, but the biggest idea introduced here is how we
handle block alignment.
See, we really don't want awkward btree topologies to form where small
amounts of data get stuck between blocks:
.-----.--.-----.
| | | |
| | | |
'-----'--'-----'
This is wasteful, as the middle bit of data either gets represented as a
full block with its data partially covered, or as data inlined in the
btree, which comes with ~2x overhead.
The solution here is to scan for a block on either the left or right to
derive our block alignment from.
Unfortunately, since our sibling blocks could have been carved, this
requires scanning all the way from pos-2*B+1 to pos+2*B-1, a total of
4*B-2, to make sure we find a sibling if there is one.
worst case left worst case right
.-----.-----. .-----.-----.
| xxxx| | |p |xxxxx|
|xxxxx| p| | |xxxx |
'-----'-----' '-----'-----'
'----+----' '----+----'
pos-2*bs+1 pos+2*bs-1
Fortunately, at this stage, data should have had many chances to
coalesce, so hopefully the actual scan overhead should be much smaller
in practice.
Writing data to a file linearly, for example, only needs a single lookup
to find the previous block.
So now instead of needing:
./scripts/test.py ./runners/test_runner test_dtree
You can just do:
./scripts/test.py test_dtree
Or with an explicit path:
./scripts/test.py -R./runners/test_runner test_dtree
This makes it easier to run the script manually. And, while there may be
some hiccups with the implicit relative path, I think in general this will
make the test/bench scripts easier to use.
There was already an implicit runner path, though only if the test suite
was completely omitted. I'm not sure that would ever have actually
been useful...
---
Also increased the permutation field size in --list-*, since I noticed it
was overflowing.
Previously our lower/upper bounds were initialized to -1..weight. This
made a lot of the math unintuitive and confusing, and it's not really
necessary to support -1 rids (-1 rids arise naturally in order-statistic
trees the can have weight=0).
The tweak here is to use lower/upper bounds initialized to 0..weight,
which makes the math behave as expected. -1 rids naturally arise from
rid = upper-1.
This is an exciting new function, made possible by the order-statistic
nature of our rbyds and btrees.
lfsr_file_fruncation is like truncate, but from the front. It can trim
data off of the front of files, and grow files from the front,
effectively prefixing files with zeros cheaply.
This may have some niche use cases for prefixing files with headers, but
the real killer is making logging files trivial. Up until now logging
into a file has always resulted in awkward file-swapping code when a
file gets full. Now maintaining a log is just a single fruncate call.
---
Implementation wise, lfsr_file_fruncate is very similar to
lfsr_file_truncate, except we need to always inject holes into all file
trees to adjust file contents correctly.
What do you think a file's size becomes when you:
1. seek past the end of a file
2. call write with zero data!
POSIX/etc has this case explicitly mentioned, noting that zero-sized
writes should never update the file size.
This clashes with the assumption that file writes always update the file
position, but I suppose it makes a bit of practical sense if you want
zero-sized file writes to be idempotent.
This avoids the previous issues with block state for null inlined data,
and we're already testing the rid anyways for splits.
In theory we don't need the block for inlined data at all, but it is
convenient as it allows us to use the existing internal rbyd/data APIs
without needing to move data around. Though it may be worth looking into
alternative layouts at some point.
This deduplicates quite a bit of logic which is very satisfying.
It could be even better if the block field was located in the same place
for both sprouts and shrubs...
The logic behind relying on pre-commit inlined state to clear any failed
commits was sound, but built on the wrong assumption that file->inlined
would always contain the mdir's block. This was not true for
null-inlined, i.e. no inlined data, since this doesn't really live
anywhere.
Changed file's inlined state to track the mdir block, even when we have
no inlined data. A bit redundant, but a nice invariant to rely on in
lfsr_mdir_compact__.
This invariant also only affects lfsr_mdir_compact__, since this is the
only place inlined data can change blocks.
Once again another function we need to nearly-completely duplicate
thanks to the recursive nature of our shrubs.
I wasn't planning to test this at this stage, but it turns out
byte-level syncs quickly fill up mdirs, triggering early ERANGE asserts
unless we split.
A 32-byte, byte-level synced, shrub already takes up 928 bytes when
including tree overhead, 1856 bytes if you include the unsynced
copy, which is very close to the 2048 byte threshold for splitting
4KiB blocks.
We were not properly resetting the staged shrub in lfsr_mdir_commit__,
well, we were sometimes, but only when transitioning from a sprout to a
shrub.
Also tweaked the mdir commit logic to try to only use the staging
inlined state. This just simplifies how much state needs to be
considered when debugging and may result in less data fetches.
This turned out to have limited use for the tests themselves. I was
hoping to avoid the mount->format->mount fallback when powerloss
testing, but we still need it in case format was interrupted.
Still, TEST_PLS is very useful for debugging.
Previouly it was difficult to set a breakpoint at a specific location,
and after a specific powerloss event. Now all you need is this in gdb:
b <line> if test_pls == <pls>
Turns out it's hard to test file holes without seek.
It's interesting to note most of seek's buffer flush work actually
occurs lazily in lfsr_file_write, so lfsr_file_seek turns out to be a
relatively simple function.
This basically turns these functions into tiny bounded compilers, which
is interesting to think about. I wonder if this sort of evolution led to
how queries are compiled in modern databases.
This method of attr generation is both easier to use and more flexible.
It also saves some code, but note lfsr_file_flushbuffer underwent
significant tweaking leveraging this, so the actual code savings are a
bit muddy:
code stack
before: 25672 2024
after: 25452 (-0.9%) 1920 (-5.4%)
- coalesce_size - The amount of data allowed to coalesce into single
data entries.
- crystallize_size - How much data is allowed to be written to btree
inner nodes before needing to be compacted into a block.
Also deduplicated the test config is something I've been wanting to do
for a while. It doesn't make sense to need to modify several different
instantiations of lfs_config every time a config option is added or
removed...
This gets pretty ugly and mainly just involves a lot of subtle range
logic.
Our CAT data representation really shines here, but all of the scratch
datas do come with a code/ram cost:
code stack
before: 25448 1920
after: 25672 (+0.9%) 2024 (+5.1%)
This saves a bit of code:
code stack
before: 25552 1920
after: 25448 (-0.4%) 1920 (+0.0%)
But more importantly, this simplifies things and moves all of the
staging/updating logic into lfsr_mdir_commit, where most of the
subtle post-compaction interactions play out.
The main purpose of this change is to introduce LFSR_DATA_CAT, a
generalized way to concatenated various data references internally.
As a side-effect lfsr_data_t has been completely restructured. Now,
lfsr_data_t can be in one of 4 modes:
If the size field's sign bit=0, the lfsr_data_t points in-device. A new,
count field, determines the encoding:
sign(size)=0, count=0 => inlined:
.---+---+---+---.
| size |
|---+---+---+---|
|c=0| inlined d | note inlined data is just enough to hold
|---+ | one encoded leb128
| ata... |
'---------------'
sign(size)=1, count=1 => direct:
.---+---+---+---. .---+---+---+---.
| size | .>| data... |
|---+---+---+---| | | . |
|c=1| | | . . .
|---+---+---+---| | . . .
| direct ptr -----' . .
'---------------'
sign(size)=1, count>=2 => indirect:
.---+---+---+---. .---+---+---+---. .---+---+---+---.
| size | .>| size | .>| data... |
|---+---+---+---| | |---+---+---+---| | | . |
|c>1| | | |c=1| | | . . .
|---+---+---+---| | |---+---+---+---| | . . .
| indirect ptr ---' | direct ptr -----' . .
'---------------' '---------------' .---+---+---+---.
| size | .>| data... |
|---+---+---+---| | | . |
|c=1| | | . . .
|---+---+---+---| | . . .
| direct ptr -----' . .
'---+---+---+---'
| . |
| . |
. . .
. .
. .
note only one indirect layer is allowed due to no recursion
If the size field's sign bit=1, the lfsr_data_t points on-disk:
sign(size)=0 => on-disk:
.---+---+---+---. .....
| size | ..'' ''..
|---+---+---+---| : : :
| block ------+->| ..:|
|---+---+---+---| | |......( )::::::|
| off -------' |:::' : |
'---------------' :' : :
''.. :.''
'''''
My goal with this commit was to test the new implementation and see how
it would impact code/RAM size before adopting it in the actual file
handling code, and the results are... not great...
code stack
before: 24668 1840
after: 25552 (+3.5%) 1920 (+4.2%)
I think most of the new cost comes from the now correct handling of
read/cmp with concatentated datas, which previously would just assert.
This change gives us LFSR_DATA_CAT, so I will be working with it for
now, but this may be worth looking at again in the future. Maybe the
correct handling of read/cmp should just be reverted to an assert...
The *_is* functions for testing for bit flags have proven useful, maybe
mostly due to C's bad bitwise operator precedence, but the
*_set*/*_clear* functions just add extra code without much benefit.
Relying on explicit bitwise operations also lets us use the |=/&= shortcut
operators, which are nice.
This will stop being a problem when we actually have btrees, but for now
the fragmentation caused by byte-level syncs was easily enough to
overflow an mdir when cache size is big.
A smaller cache size is also nicer for debugging, since smaller cache
sizes results in data getting flushed to disk earlier, which is easier
to inspect than in-device buffers. And a 16-byte cache still provides
decent test coverage over cache interactions.
---
Also dropped inline_size to block_size/8. I realized while debugging
that opened shrubs take up additional space until we sync, so we need to
expect up to 2 temporary copies of shrubs when writing files.
- Added shrub tags to tagrepr
- Modified dbgrbyd.py to use last non-shrub trunk by default
- Tweaked dbgrbyd's log mode to find maximum seen weight for id padding
Get it? Because they're small trees!
Joking aside, having a new term for these helps structure and describe
the filesystem at a high-level without needing to say "inlined trees"
all the time.
Shrub trees are small rbyd trees inlined directly in a file's mdir.
The main issue was that we need to potentially overwrite our staging are
if we fail a compaction and need to split. This clobbers any inlined
state staged at higher-levels, such as creating new inlined trees.
The solution here is to just initialize new inlined trees in the low
level lfsr_mdir_commit__ commit. This is a bit of a hack, but makes
things work, which is always a plus.
The main improvement is moving the special inlined-file compaction logic
up into lfsr_mdir_compact__. We only need this logic for files stored in
mdirs, and thanks to its recursive nature, we weren't getting any
benefit from handling this at a lower level anyways.
This is a nice logical restructuring that probably saves a bit of code
cost in the end.
Another significant improvement is moving the staging copy of the
inlined tree's state up into the file struct itself. This solves the
problem of needed N copies of temporary inlined state when you have N
open files.
It also provides a central place to stage changes when compacting
inlined trees, which happens across several different places in the mdir
commit logic. Though some may see this as more a hack than a feature.
Also note-worthy, but minor: these changes required an additional
opened-mdir linked-list to know when the mdir is a file and may contain
an inlined tree.
Inlined files are unfortunately turning out to have more cost than
expected, mainly due to our strict no-recursion requirement.
It turns out recursively nesting (bounded) trees in a system without
recursion is a recipe for duplicating code. Though there may be other
ways to structure this.
One interesting hiccup during development is the need to have both NULL
tags and DEFERREDNULL tags in order to tell inlined trees apart from the
main tree during compaction.
And by working, I mean you can create inlined trees, just don't
compact/split/move/etc anything. But this does outline the path files
take when writing buffers into inlined trees.
"Inlined trees" in littlefs are entire small rbyd trees embedded as
secondary trees in an mdir's main rbyd tree. When fetching, we can
indicate if a given trunk belongs to the main tree or secondary tree by
setting one of the unused mode bits in the trunk's tag, now called the
"deferred" bit. This bit doesn't need to be included in the alt's "key"
field, so there's no issue with it conflicting with the alt's mode bits.
This requires a bit of tweaking lfsr_rbyd_fetch, since it needs to fall
back to the previous trunk if it discovers the most recent trunk belongs
to an inlined tree. But as a benefit we can leverage the full power of
rbyds in inlined files, including holes, partial updates, etc.
One downside is it looks like these inlined trees may involve more work
in maintining their state correctly, since they need to be sort of
"brought along" when mdirs are compacted, even if they don't actually
have a reference in the mdir yet. But the sheer amount of flexibility
this gives inlined files may make this overhead worth it.
Ran into an interesting macro-related bug. Turns out the way we are
doing implicit prefixing in TAG/ATTR macros sort of breaks how C macros
work a bit. The following does not compile:
lfsr_mdir_commit(lfs, &file->m.mdir, LFSR_ATTRS(
LFSR_ATTR(file->m.mdir.mid, DEFER, 0, DEFER(
(lfsr_rbyd_t*)&file->inlined,
LFSR_ATTR(file->buffer_pos,
DEFERRED(INLINED), +file->buffer_size, BUF(
file->buffer, file->buffer_size))))));
Or to distill it down, this does not compile:
#define LFSR_ATTR(_data) (LFSR_##_data)
#define LFSR_DEFER(_data) (LFSR_##_data)
#define LFSR_DATA(_data) (_data)
int a = LFSR_ATTR(DEFER(ATTR(DATA(1))));
But this does:
#define LFSR_ATTR(_data) (_data)
#define LFSR_DEFER(_data) (_data)
#define LFSR_DATA(_data) (_data)
int a = LFSR_ATTR(LFSR_DEFER(LFSR_ATTR(LFSR_DATA(1))));
Why? Well it turns out the whole way nested C macro's work is a big
hack.
A very reasonable design decision in C is to disallow recursive macro
expansions. Unlike C++, we don't want our preprocessor to suddenly stack
overflow. This rule is enforced by stopping macro expansion when a macro
contains itself. For example:
#define A() B()
#define B() A()
A()
Expands to:
A()
-> B()
-> A() (stops, probably erroring with 'A' undeclared)
But it _is_ common to want to recursively expand macro arguments. Macros
are a part of C's syntax after all, and users usually expect
expressions, such as arguments, to be context-free:
#define A(x) (x) + 1
A(A(A(A(A(0)))))
Naively this would expand to:
A(A(A(A(A(0)))))
-> (A(A(A(A(0))))) + 1 (stops)
The big hack that makes this work in C's preprocessor is the "Argument
prescan". Instead of expanding the "called" macro first, we expand any macro
inside our argument list, _then_ expand the "called" macro, and _then_
expand any new macros produced as a result of the expansion again just
for good measure.
So the above actually expands to:
A(A(A(A(A(0)))))
-> A(A(A(A((0) + 1))))
-> A(A(A(((0) + 1) + 1)))
-> A(A((((0) + 1) + 1) + 1))
-> A(((((0) + 1) + 1) + 1) + 1)
-> (((((0) + 1) + 1) + 1) + 1) + 1
This is still recursive actually! But the recursion is limited to the
actual length of the source code, so the developers likely thought this
was a reasonable tradeoff.
But what does this mean for our implicit prefixing?
#define P_A(x) P_##x
#define P_B(x) P_##x
#define P_C(x) (x)
P_A(B(A(C(0))))
None of A, B, C are in scope without prefixes, so they get expanded
after the "called" macro's expansion:
P_A(B(A(C)))
-> P_B(A(C(0)))
-> P_A(C(0)) (stops)
But this breaks when we hit the nested P_A macro.
---
For now I've gone with the temporary, and extra hacky, solution of
introducing a second LFSR_ATTR_ macro. This nesting of ATTR macros only
happens because of shrubs, and only ever goes 2 layers deep.
In the future maybe we should move away from implicit prefixing. They
have a few rough corners and may be a bit confusing for anyone new to
the code.
This is in order to support deferred-inlined files, which involves
intertwining secondary trees into an rbyd. In order to know which trunks
go to which trees, we need an additional bit to indicate if a tag is on
the primary tree or a secondary tree.
We were using pretty much all of our tag bits, but the rm and valid
bits can be combined. They more-or-less serve the same purpose.
New tag modes:
v000tttt 0ttttttt - normal tags
v001tttt 0ttttttt - deferred tags
v010tttt 0ttttttt - checksum tags
v1dckkkk 0kkkkkkk - alt tags
^'+''-----+-----'
'-|-------|- valid bit
'-------|- tag mode
'- tag type/key
Note that once we have a trunk, we don't need this deferred bit to
traverse the rbyd. This is why we can get away with using a mode bit
that would normally collide with the alt tag's encoding.
Also tweaked lfsr_rbyd_appendattr so GROW tags don't need to set the rm
bit anymore. This is just an internal usability thing.
Currently limited to inlined files and only simpler truncate-writes.
But still this lets us test file creation/deletion.
This is also enough logic to make it clear that, even though we have
some powerful high-level primitives, mapping file operations onto these
is still going to be non-trivial.
I had never noticed xxd has no header until comparing its output against
dbgblock.py. Turns out these headers aren't really all that useful, and
even sometimes wrong in dbglfs.py.
Now, instead of storing a single contiguous block of config data, config
is stored as tagged metadata like any other attribute.
This allows more flexibility towards adding/removing config in the
future, without cluttering up the config with deprecated entries (see
ATA's "IDENTIFY DEVICE" response).
Most of the config entries are single leb128 limits on various integer
types, with the exception of the magic string and version (major/minor
pair).
---
Note this also includes some semantic changes to the config:
- Limits are stored as size-1. This avoid issues with integer overflow
at extreme ranges.
This was also adopted for block size (block limit) and block count
(disk limit). This deviation between on-disk config and user-facing
config risks confusion, but allows the potential for the full 2^31 range
for these values.
- The default cksum type, crc32c, has been changed to 0.
Originally this was 2 to allow the type to map to the crc width for
crc8, crc16, crc32c, crc64, etc. But dropping this idea and numbering
checksums as they are implemented simplifies things.
May come back to this.
- Storing these configs as attributes opens up of the option of on-disk
defaults when configs are missing.
I'm being a bit conservative with this one, as it's not clear to me if
we should prefer default configs (less code/storage, risk of untested
config parsing) or prefer explicit on-disk configs.
Currently the following have defaults since they seem the most obvious
to me:
- cksum type => defaults to crc32c
- redund type => defaults to parity (TODO, should this default to
no redund?)
- utag_limit => defaults to 0x7f (no special tag decoding)
- uattr_limit => defaults to block_limit (implicit)
Christopher Haster