Like codemapd3.py this include an interactive UI for viewing the
underlying filesystem graph, including:
- mode-tree - Shows all reachable blocks from a given block
- mode-branches - Shows immediate children of a given block
- mode-references - Shows parents of a given block
- mode-redund - Shows sibling blocks in redund groups (This is
currently just mdir pairs, but the plan is to add more)
This is _not_ a full filesystem explorer, so we don't embed all block
data/metadata in the svg. That's probably a project for another time.
However we do include interesting bits such as trunk addresses,
checksums, etc.
An example:
# create an filesystem image
$ make test-runner -j
$ ./scripts/test.py -B test_files_many -a -ddisk -O- \
-DBLOCK_SIZE=1024 \
-DCHUNK=10 \
-DSIZE=2050 \
-DN=128 \
-DBLOCK_RECYCLES=1
... snip ...
done: 2/2 passed, 0/2 failed, 164pls!, in 0.16s
# generate bmap svg
$ ./scripts/dbgbmapd3.py disk -b1024 -otest.svg \
-W1400 -H750 -Z --dark
updated test.svg, littlefs v0.0 1024x1024 0x{26e,26f}.d8 w64.128, cksu
m 41ea791e
And open test.svg in a browser of your choice.
Here's what the current colors mean:
- yellow => mdirs
- blue => btree nodes
- green => data blocks
- red => corrupt/conflict issue
- gray => unused blocks
But like codemapd3.py the output is decently customizable. See -h/--help
for more info.
And, just like codemapd3.py, this is based on ideas from d3 and
brendangregg's flamegraphs:
- d3 - https://d3js.org
- brendangregg's flamegraphs - https://github.com/brendangregg/FlameGraph
Note we don't actually use d3... the name might be a bit confusing...
---
One interesting change from the previous dbgbmap.py is the addition of
"corrupt" (bad checksum) and "conflict" (multiple parents) blocks, which
can help find bugs.
You may find the "conflict" block reporting a bit strange. Yes it's
useful for finding block allocation failures, but won't naturally formed
dags in file btrees also be reported as "conflicts"?
Yes, but the long-term plan is to move away from dags and make littlefs
a pure tree (for block allocator and error correction reasons). This
hasn't been implemented yet, so for now dags will result in false
positives.
---
Implementation wise, this script was pretty straightforward given prior
dbglfs.py and codemapd3.py work.
However there was an interesting case of https://xkcd.com/1425:
- Traverse the filesystem and build a graph - easy
- Tile a rectangle with n nice looking rectangles - uhhh
I toyed around with an analytical approach (something like block width =
sqrt(canvas_width*canvas_height/n) * block_aspect_ratio), but ended up
settling on an algorithm that divides the number of columns by 2 until
we hit our target aspect ratio.
This algorithm seems to work quite well, runs in only O(log n), and
perfectly tiles the grid for powers-of-two. Honestly the result is
better than I was expecting.
Might as well, since we already need to find this to calculate stack
info.
I've been considering adding -z/--depth to these scripts as well, but
that would require quite a bit more work. It's probably not worth the
added complexity/headache. Depth termination would need to happen on the
javascript side, and we'd still need cycle detection anyways.
But an error code is easy to add.
--no-header doesn't really deserve a shortform, and this risks conflicts
with -N/--notes in the future, not to mention any other number of flags
that can start with --no-*.
This should have no noticeable impact on plot.py, but shared classes
have proven helpful for maintaining these scripts.
Unfortunately, this did require some tweaking of the Canvas class to get
things working.
Now, instead of storing things in an internal high-resolution grid,
the Canvas class only keeps track of the most recent character, with
bitmasked ints storing sub-char info.
This makes it so sub-char draws overwrite full characters, which is
necessary for plot.py's axis/data overlap to work.
This only failed if "-" was used as an argument (for stdin/stdout), so
the issue was pretty hard to spot.
openio is a heavily copy-pasted function, so it makes sense to just add
the import os to openio directly. Otherwise this mistake will likely
happen again in the future.
- -*/--add-char/--chars -> -./--add-char/--chars
- -./--points -> -p/--points
- -!/--points-and-lines -> -P/--points-and-lines
Also fixed an issue in plot.py/Attr where non-list default were failing
to concatenate.
And added the optional --no-label to explicitly opt out.
This is a bit more consistent with treemapd3.py/codemapd3.py's handling
of labels, while still keeping the no-label default. It also makes it
easier to temporarily hide labels when editing commands.
So by default, instead of just using "." for tiles, we use interesting
parts of the tile's name:
- For treemap.py, we use the first character of the last by-field (so
"lfs.c,lfsr_file_write,1234" -> "1").
- For codemap.py, we use the first character of the non-subsystem part
of the function name (so "lfsr_file_write" -> "w").
This nice thing about this, is the resulting treemap is somewhat
understandable even without colors:
$ ./scripts/codemap.py lfs.o lfs_util.o lfs.ci lfs_util.ci -W60 -H8
code 35528 stack 2440 ctx 636
ffffffoooffaaaaaaaaaaaacccccccccttttccccrrrrpgffmmrraifmmcss
ffffffwwwttaaaaaaaaaaaacccccccccttttccccrprrpcscmmoommrrcepp
ffffffwwwttaaaaaaaaalllcccccccccttttccccrpppccscmmsrmmrrrrss
ccccssrrfclaaaaanneeasscccccccccgpppccccrpppsgsummstmmrrlfgf
ccccssrrfccaaaaanneeaaaccccccsaagpppcccccrrrfrrcccrrfiiilucs
ccccssrrtfcfffffaapplcccccccclssgnnllllcrrffrrrccccifssscmcm
ccccssrrtrdfffffaapppapcccfffllsgnnllllcrrrffrrcccorfsssicnu
Ok, so maybe the word "somewhat" is doing a lot of heavy lifting...
Like codemapd3.py, but with an ascii renderer.
This is basically just codemapd3.py and treemap.py smooshed together.
It's not the cleanest, but it gets the job done. codemap.py is not
the most critical of scripts.
Unfortunately callgraph and stack/ctx info are difficult (impossible?)
to render usefully in ascii, but we can at least do the script calling,
parsing, namespacing, etc, necessary to create the code cost tilemap.
Christopher Haster