One arrangement of a trace, instead of three

[tamnd]

Closes the kxray.models row of M1. Two things in here, and the second one is the reason for the first.

The split

kxray/models.py was 1424 lines holding six unrelated subjects, and it is now a package with one file each.

kxray/models/
├── __init__.py   re-exports all 50 names, so nothing outside had to change
├── lines.py      line accounting and the table printer, used by every parser in the project
├── tape.py       function_graph: frames, the events between them, and the whole tape
├── flat.py       the flat function tracer, and the flags column that gives the context
├── events.py     trace events, read through the format file the kernel publishes
├── proc.py       what comes out of /proc and /sys, each carrying its stability promise
└── types.py      what the kernel knows about its own types, out of BTF

Every one of the sixty odd call sites in the repository writes from kxray.models import ..., and a re-exporting __init__ keeps all of them working. The test suite passing unchanged is the proof, and that was checked before anything else was built on top of it.

The rule the split is protecting is what goes in the docstring: nothing in there imports a renderer. Models are the bottom of the stack, kxshapes works out geometry from them, and the two renderers draw what kxshapes returns. A model that knows how it is drawn is a model that two renderers will fight over.

The part that matters

The M1 exit criterion says all nine primitives have to be driven from the same objects. They were not. There was one parser and three renderers, which is the design, and three separate conversions from a trace to a picture, which is not:

• kxwidgets/tape.py walked the frames and built its own TraceCell for each one, and wrote its own hover text in a private _title method.
• tools/bpcgen.py printed tape.tree() as an indented block and drew no shapes at all, so a blueprint of a capture and a notebook of the same capture had nothing in common.
• A kxmanim storyboard listed the shapes it used in a hand written shows field with no trace behind it.

So a notebook, a blueprint and an animation of one recording were three separate readings of it that happened to agree, and nothing anywhere would have said so on the day they stopped.

kxshapes/scene.py is now the one reading. A Scene is lanes of Steps, each step holding one of the nine shapes and the sentence that goes with it, in call order. What each renderer does with that is the part allowed to differ: the widget draws every step at once, the blueprint prints them as a table, the animation reveals them along order.

The sentence matters as much as the numbers. Step.detail is the hover text on a box in a notebook, the note under a still and what a screen reader reads, and those three saying different things about the same call is exactly the failure this is trying to make impossible.

What a blueprint gets now

Section 5 of write-path.md used to print the indented tree and stop. It still prints it, because the tree is what a specification is about, and now it also prints the drawing as a table, generated from the same Scene the notebook draws.

3 calls from corpora/traces/tier0/write-1byte.txt: 27 calls in 3 call lanes, 5 levels deep at the
deepest, and some boxes were placed by call order because a duration is missing.
Primitives used: trace-cell.

#   lane                   depth  call                             took     width   note
--  ---------------------  -----  -------------------------------  -------  ------  --------------------
0   mutex_unlock on cpu 0  0      mutex_unlock                     22.625   100.0%
1   vfs_write on cpu 0     0      vfs_write                        516.167  100.0%
2   vfs_write on cpu 0     1      shmem_file_write_iter            503.875  97.6%
3   vfs_write on cpu 0     2      down_write                       4.250    0.8%
...
23  vfs_write on cpu 0     2      up_write                         3.500    0.7%
24  vfs_write on cpu 0     0      vfs_write                        unknown  100.0%
25  vfs_write on cpu 0     1      __copy_user_ll                   3.458    50.0%   placed by call order
26  vfs_write on cpu 0     1      mutex_lock                       3.542    50.0%   placed by call order

Those last two rows are the honest bit surviving the trip. The third call in that capture was cut off by the tracer window, so its children have no measured width and were placed by counting instead. The widget outlines them in red, the table says placed by call order, and both read it off the same flag.

The storyboard check has teeth now

shows was a list of words checked against the closed set of nine. A beat could say cpu-lane over a capture from a uniprocessor emulator and pass, because cpu-lane is one of the nine.

It is now also checked against a scene built from the file the storyboard actually names. cpu-interleave claims cpu-lane and points at corpora/traces/tier1/multi-cpu-write.txt, which really does have six of them. Pointing the same storyboard at a Tier 0 capture gives:

a beat shows cpu-lane, and corpora/traces/tier0/two-writes.txt does not have one in it:
4 calls from corpora/traces/tier0/two-writes.txt: 69 calls in 1 CPU lanes, 9 levels
deep at the deepest, and some boxes were placed by call order because a duration is missing.

Only the two shapes a trace can speak to are checked, and only when the storyboard names a capture. lock-cycle draws CPU lanes out of a lockdep report on a machine with one processor, and that mismatch is the subject of the beat rather than a mistake in it. A rule that went by the shape alone would fire on the one animation where the kernel drawing two CPUs that were not there is the whole point, and a rule that has to be argued with is a rule somebody switches off.

One invariant worth naming

_by_cpu walks the spans itself rather than calling kxshapes.lanes, because it needs the frame each cell came from to build the words and a CpuLane hands back cells with the frames already dropped. Matching the two lists up by position would be correct today and quietly wrong the first time either walk changed what it filters. So there is a test instead, test_a_cpu_scene_holds_the_same_cells_as_the_lane_function, asserting the two agree cell for cell across all 109 cells of the six CPU capture.

Checked

tests/test_scene.py is 21 tests. The ones carrying weight are the three that pin two things together: the widget draws the cells the scene produced, the CPU lanes are the cells lanes() builds one for one, and the hover text in the notebook is the string detail_of returns.

The whole gate:

ruff check, ruff format --check           clean, 175 files
pytest                                     all green
node --test tests/web/*.test.js            all green
baseline      38 artefact(s), 5224 line(s), 0 unparsed
claimledger   3 lesson(s) clean
coverage      30 subsystem(s), 23 path(s) cited
bpc           3 blueprint(s) clean
kconfig       7.2.2 pinned, 5 profile(s) clean
refcheck      paths clean, 73 citation(s)
lintnb        3 notebook(s) keep the contract
kxmanim       3 storyboard(s) clean
diagrams      8 source(s) up to date
nbbuild       C09, S05, Z02 up to date
sitebuild     3 lesson(s), 2 blueprint(s), clean
kxbox         3 recipe(s) clean
vendor        v86 0.5.455, 5 files match the pin
lintprose     36 file(s) clean

The blueprints were regenerated with --btf kxbox/kernel/build/A-full/vmlinux so sections 2 and 7 keep their real type information. Regenerating without it silently replaces both with a note saying there was no BTF, which is a thousand lines of measured layout turning into an apology, and it is worth writing down here because it is not obvious from the command.

What is still open

refcheck now has a directory where it had a file, which is why LAYOUT.md and kxray/source/__init__.py needed the three prose fixes in this diff. Scene covers the trace shapes. The other seven primitives are still built by their callers, and folding those in is the next row rather than this one.