make: Improved benchmarks/testmarks output

- Sort by -Si='min(enumerate())'

  This was an unexpectedly neat trick for ordering result based on input
  csv, without disabling folding like -i/-I.

  Note the min is needed because the field is summed before sorting, so
  an early case with many permutations can end up after later cases by
  default. Because -i/-I also disables folding, it doesn't have this
  problem.

- Adopted m -> probe rename.

- Adopted delta expr, so we can now show n correctly, without
  accidentally summing already cumulative results.

- Changed benchmark fields simtime+simthroughput -> n+t+throughput

- Adopted throughput=avg(throughput), so TOTAL is somewhat useful?
  Unsure if this is the correct way to merge throughput results.
This commit is contained in:
Christopher Haster
2026-02-02 13:39:50 -06:00
parent ff30368324
commit 026aee0139
+8 -4
View File
@@ -519,7 +519,7 @@ test-list list-tests: test-runner
## Summarize the test results ## Summarize the test results
.PHONY: testmarks .PHONY: testmarks
testmarks: SUMMARYFLAGS+=-spassed -Sruntime testmarks: SUMMARYFLAGS+=-Si='min(enumerate())'
testmarks: $(TEST_CSV) testmarks: $(TEST_CSV)
$(strip ./scripts/csv.py $^ \ $(strip ./scripts/csv.py $^ \
-bsuite \ -bsuite \
@@ -568,11 +568,15 @@ bench-list list-benches: bench-runner
## Summarize the bench results ## Summarize the bench results
.PHONY: benchmarks .PHONY: benchmarks
benchmarks: SUMMARYFLAGS+=-Si='min(enumerate())'
benchmarks: $(BENCH_CSV) benchmarks: $(BENCH_CSV)
$(strip ./scripts/csv.py $^ \ $(strip ./scripts/csv.py $^ \
-bcase='%(case)s+%(m)s' \ -bprobe='%(case)s+%(probe)s' \
-fsimtime='float(bench_simtime)/1.0e9' \ -fn='delta(n, case, probe)' \
-fsimthroughput='float(n)/max(float(bench_simtime)/1.0e9,1.0e-9)' \ -ft='float(bench_simtime)/1.0e9' \
-fthroughput='avg( \
float(delta(n, case, probe)) \
/ max(float(bench_simtime)/1.0e9, 1.0e-9))' \
$(SUMMARYFLAGS)) $(SUMMARYFLAGS))
## Save the bench results ## Save the bench results