- BENCH_SIMTIME() => lfs3_kiwibd_simtime()
- BENCH_SIMRESET() => lfs3_kiwibd_simreset()
- BENCH_SIMPAUSE() => lfs3_kiwibd_simpause()
- BENCH_SIMRESUME() => lfs3_kiwibd_simresume()
- BENCH_RESET() => lfs3_kiwibd_simreset() + BENCH_STACK/HEAP_RESET()
- BENCH_PAUSE() => lfs3_kiwibd_simpause() + BENCH_STACK/HEAP_PAUSE()
- BENCH_RESUME() => lfs3_kiwibd_simresume() + BENCH_STACK/HEAP_RESUME()
This does two things:
1. Adds pause/resume counters to bd counters to make it easier to
exclude operations from the current bench (potentially useful for
seq+disk usage).
2. Exposes bd simtime operations as BENCH_* macros, to make it a bit
easier to interact with simtime without tying all the benches to
kiwibd. (Not that we'll ever probably not use kiwibd, but still).
Also adopted 32-bit counters for stack/heap pause state instead of a
32-bit stack. Not that either are at a risk of overflowing, but better
safe than sorry.
After letting it sit for a bit, the previous byte+op sim comes across as
overly clever in a way that is counter-productive. This is highlighted
by erase-timing scaling in a confusing way when per-op.
Fortunately, with a bit of tweaking, we can instead model the bd sim as
separate bus+buffer timings. This seems more intuitive and is closer to
how the actual hardware works.
---
In the bus+buffer model, bd operations are simulated using two sets of
timing estimates:
buffer timings (nor) bus timings (nor)
read_timing (0) readed_timing (40 ns/B)
prog_timing (1563 ns/B) progged_timing (19 ns/B)
erase_timing (10986 ns/B) erased_timing (0)
Bus timings are a simple multiplier of the bytes read/progged/erased,
while buffer timings are rounded up + aligned to the nearest "width":
bd geometry (nor) bd buffers (nor)
read_size (1 B) read_width (1 B)
prog_size (1 B) prog_width (256 B)
erase_size (4096 B) erase_width (4096 B)
For most purposes, the width should just be the device's read/prog/erase
buffer, but I went with the name width to try to keep it generic and
avoid confusion with "buffer" elsewhere in the codebase.
Some notes:
- Like the byte+op sim, the bus+buffer sim allows penalizing small
operations without artificially limiting what operations are possible.
- Because buffer timings depend on read/prog/erase alignment, there's no
simple equation from ops+bytes to bus+buffer. But as a tradeoff, this
new sim more accurately penalizes unaligned operations.
- All timings are still kept as per-byte instead of per-width. This has
proven to be more flexible when benchmarking, as you usually what
timings to scale with the relevant operation.
- Currently this implemented by changing reads/progs/erases to track the
number of "widths" read/progged/erased after alignment. Which makes
the simtime formula roughly:
simtime = reads*read_width*read_timing + readed*readed_timing
(per-butter) (per-bus)
I considered keeping separate counters for calls (read_calls/
prog_calls/erase_calls?), but not sure there's a good reason to. The
theory behind these widths is there no functional difference between
one big call vs multiple width sized calls, though maybe they would be
useful for debugging?
We can always add these later if they turn out to be useful.
- When widths are disable (0), reads/progs/erases reverts to the number
of read/prog/erase calls.
This is the behavior when BENCH_SIMPLE is defined at compile-time.
The main changes:
- Added TEST_DEFINES and BENCH_DEFINES to allow overriding the default
test/bench define header:
-DTEST_DEFINES=my_test_defines.h
Note these are VERY different from LFS_DEFINES upstream. They aren't a
typical header file, and are included multiple times with various
query macros.
It's hacky, but works surprisingly well.
Or maybe I'll just do anything to avoid having to write multiline
macros. Ugh, backslashes.
- Moved more logic into bench/test_defines.h, including everything
needed to integrate other filesystems out-of-tree.
This mostly meant moving all of the cfg initialization logic into its
own query macro (replacing the BENCH/TEST_CFG field macros).
But this also includes a bunch of small tweaks encountered while trying
to get external benchmarks running again.
The external benchmarks include several other filesystems (littlefs2,
SPIFFS, Yaffs2), and I'm hoping this injectable/queryable header thing
will do a good job at avoiding a maintenance headache. (At least a
better job than forking bench_runner.c, which was the previous
solution.)
This is based on some work in external benchmarks. What's worked well
there is emulating a global simtime based on per-byte estimates.
This moves the emulated simtime into emubd/kiwibd, and extends the idea
with both per-byte and per-op timing estimates for hopefully more
realistic results.
---
The problem is how NAND flash reads work.
Per-byte timing estimates are surprisingly accurate for NOR flash. There
is some overhead for sending the address, but it's mostly dominated by
bus cost (~20ns/B [1]).
NAND flash, on the otherhand, technically does support byte-level reads,
but first needs to read into 2KiB buffer. Surprisingly, these are pretty
close in cost (~19ns/B bus [2] vs ~12ns/B buffer [2]).
This close-ness makes modeling NAND flash difficult. If we set
read_size=1, we risk hiding the cost of small reads, which littlefs3 is
full of (rbyd lookups). If we set read_size=2048, we unfairly penalize
littlefs3 for the same reason.
---
The solution here is to expose both per-byte and per-op timing
estimates. This lets you model NAND reads using two data points:
^
| realtime --> ...............o
| : .....'''' :
| ...............:'''' ^ :
| :....''''' | :
| ..........:::::: simtime :
| .....:'''' :
|o....:::::.....: :
|: :
|: :
+:-----------------------------------------------------------:>
min read max read
Where:
bus_timing = 19ns
buffer_timing = 25us
buffer_size = 2KiB
erase_size = 128KiB
min_read = buffer_timing
max_read = (erase_size/buffer_size)*buffer_timing - buffer_timing
read_timing = min_read
readed_timing = ((max_read - min_read)/erase_size) + bus_timing
simtime = reads*read_timing + readed*readed_timing
(per-op) (per-byte)
This should correctly penalize small reads without complicating
emubd/kiwibd too much.
That's the idea anyways! It will take some use to understand if this is
a reasonable approach.
As a plus, this is a superset of the per-byte model, so both can be used
for realistic vs idealistic simulations (and to test the bus+buffer
model itself).
1: https://www.winbond.com/resource-files/W25Q256JV%20SPI%20RevQ%2002072025%20Plus.pdf
2: https://www.winbond.com/resource-files/W25N01GV%20Rev%20R%20070323.pdf
For some reason emubd had both a path argument to lfs3_emubd_create, and
a disk_path config option, with only the disk_path actually being used.
But the real curiosity is why did GCC only starting warning about it
when copied to kiwibd? path is clearly unused in lfs3_emubd_createcfg,
but no warning...
---
Anyways, not sure which one is a better API, but we definitely don't
need two APIs, so eeny meeny miny moe...
Went ahead and chose the lfs3_emubd_create path param for some
consistency with filebd.
Useful for emulating much larger disks in a file (or in RAM). kiwibd
doesn't have all the features of emubd, but this allows it to prioritize
disk size and speed for benchmarking.
kiwibd still keeps some features useful for benchmarking/emulation:
- Optional erase value emulation, including nor-masking
- Read/prog/erase trackers for measuring bd operations
- Read/prog/erase sleeps for slowing down the simulation to a human
viewable speed
Christopher Haster