Github/littlefs
1
Fork 0
Code Actions Activity

runners: bench: Added flags to control reading from bench probes

Browse Source 
- -S/--probe         - Specify a probe to sample.
- -x/--probe-step    - Sample probes every n steps.
- --probe-runfreq    - Sample probes at this frequency in hz.
- -X/--probe-simfreq - Sample probes at this frequency in simulated hz.

Also:

- --trace-simfreq    - Sample trace output at this frequency in
                       simulated hz.

These give finer grain control over which probes we measure during
benching, and how we measure them.

These also introduce several exciting bench features:

- -S/--probe provides the ability to easily filter which probes you're
  interested in at runtime.

  This should replace the growing use of MASK defines in the benches.

- -x/--probe-step makes it easy to relax sampling rate when the amount
  of data overwhelms later scripts.

  This should replace the growing use of STEP defines in the benches.

- The additional concept of simfreq, which allows perf-esque sampling in
  simtime. This provides another option for intuitively relaxing probe
  sampling rate without sacrificing reproducibility.

  (runfreq depends on wall time, so good bye reproducibility, though may
  still be useful in interactive contexts.)

Note -S/--probe and -x/--probe-step replace MASK/STEP defines, which
have already proved their usefulness, but required reimplementation in
every bench case. An obvious contender to move into the bench_runner!

---

Note note that -S/--probe also supports some simple sample expressions,
allowing flexible step/simfreq/runfreq at the per-probe level:

- -Swrite=100    - Sample probe "write" every 100 steps
- -Swrite=100rhz - Sample probe "write" 100 times a runtime second
- -Swrite=100shz - Sample probe "write" 100 times a simulated second

Though I wonder how long it will take before I forget this feature
exists.
This commit is contained in:
Christopher Haster
2026-02-09 13:43:55 -06:00
parent 81d681cab2
commit 4af4cf3212
8 changed files with 773 additions and 378 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benches/bench_rt.toml
+7 -7
View File
@@ -56,7 +56,7 @@ code = '''
lfs3_file_write(&lfs3, &file, wbuf, CHUNK) => CHUNK;
// taking too long?
if (SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME) {
if (SIM_TIME && BENCH_SIMTIME() >= SIM_TIME) {
return;
}
}
@@ -71,7 +71,7 @@ code = '''
lfs3_off_t size = 0;
uint64_t readed = 0;
while (!(SIM_SIZE && readed >= (uint64_t)SIM_SIZE)
&& !(SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME)) {
&& !(SIM_TIME && BENCH_SIMTIME() >= SIM_TIME)) {
// read from the file
uint8_t rbuf[CHUNK];
lfs3_file_read(&lfs3, &file, rbuf, CHUNK) => CHUNK;
@@ -123,7 +123,7 @@ code = '''
lfs3_file_write(&lfs3, &file, wbuf, CHUNK) => CHUNK;
// taking too long?
if (SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME) {
if (SIM_TIME && BENCH_SIMTIME() >= SIM_TIME) {
return;
}
}
@@ -137,7 +137,7 @@ code = '''
lfs3_file_open(&lfs3, &file, "bench_random", LFS3_O_RDONLY) => 0;
uint64_t readed = 0;
while (!(SIM_SIZE && readed >= (uint64_t)SIM_SIZE)
&& !(SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME)) {
&& !(SIM_TIME && BENCH_SIMTIME() >= SIM_TIME)) {
// seek to a random location
lfs3_off_t pos = BENCH_PRNG(&prng) % SIZE;
lfs3_file_seek(&lfs3, &file, pos, LFS3_SEEK_SET) => pos;
@@ -191,7 +191,7 @@ code = '''
lfs3_file_write(&lfs3, &file, wbuf, d) => d;
// taking too long?
if (SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME) {
if (SIM_TIME && BENCH_SIMTIME() >= SIM_TIME) {
return;
}
}
@@ -205,7 +205,7 @@ code = '''
BENCH_START("read");
uint64_t readed = 0;
while (!(SIM_SIZE && readed >= (uint64_t)SIM_SIZE)
&& !(SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME)) {
&& !(SIM_TIME && BENCH_SIMTIME() >= SIM_TIME)) {
// choose a random filename
lfs3_off_t pos = BENCH_PRNG(&prng) % ((SIZE+(CHUNK-1))/CHUNK);
char name[256];
@@ -224,7 +224,7 @@ code = '''
readed += d;
// taking too long?
if (SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME) {
if (SIM_TIME && BENCH_SIMTIME() >= SIM_TIME) {
break;
}
}
benches/bench_wt.toml
+5 -5
View File
@@ -54,7 +54,7 @@ code = '''
// ok, one of these needs to be non-zero
LFS3_ASSERT(SIM_TIME > 0 || SIM_SIZE > 0);
while (!(SIM_SIZE && written >= (uint64_t)SIM_SIZE)
&& !(SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME)) {
&& !(SIM_TIME && BENCH_SIMTIME() >= SIM_TIME)) {
// arguably we should just rewind and continue writing to the
// front of the file when we hit the end, but this overly
// penalizes littlefs2, so instead we truncate
@@ -122,7 +122,7 @@ code = '''
lfs3_off_t size = 0;
uint64_t written = 0;
while (!(SIM_SIZE && written >= (uint64_t)SIM_SIZE)
&& !(SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME)) {
&& !(SIM_TIME && BENCH_SIMTIME() >= SIM_TIME)) {
// seek to a random location
lfs3_off_t pos = BENCH_PRNG(&prng) % SIZE;
lfs3_file_seek(&lfs3, &file, pos, LFS3_SEEK_SET) => pos;
@@ -189,7 +189,7 @@ code = '''
// ok, one of these needs to be non-zero
LFS3_ASSERT(SIM_TIME > 0 || SIM_SIZE > 0);
while (!(SIM_SIZE && written >= (uint64_t)SIM_SIZE)
&& !(SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME)) {
&& !(SIM_TIME && BENCH_SIMTIME() >= SIM_TIME)) {
// append to log
uint8_t wbuf[CHUNK];
for (lfs3_size_t j = 0; j < CHUNK; j++) {
@@ -265,7 +265,7 @@ code = '''
// ok, one of these needs to be non-zero
LFS3_ASSERT(SIM_TIME > 0 || SIM_SIZE > 0);
while (!(SIM_SIZE && written >= (uint64_t)SIM_SIZE)
&& !(SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME)) {
&& !(SIM_TIME && BENCH_SIMTIME() >= SIM_TIME)) {
// choose a random filename
lfs3_off_t pos = BENCH_PRNG(&prng) % FILE_COUNT;
char name[256];
@@ -287,7 +287,7 @@ code = '''
written += d;
// taking too long?
if (SIM_TIME && BENCH_SIMTIME() >= (bench_ns_t)SIM_TIME) {
if (SIM_TIME && BENCH_SIMTIME() >= SIM_TIME) {
break;
}
}