Add the trace ring buffer blueprint, and do the arithmetic the stats file will not

[tamnd]

What this is

The fourth blueprint, trace-ring-buffer, and the one the other three quietly rest on. Every claim this project makes about what a trace contains is a claim about this file first and about the tracer second, so it is worth writing down what it actually promises.

It promises less than most people assume, and the whole blueprint is built around one fact.

A writer never fails, so a full buffer is a silent deletion

A writer into this buffer never blocks, never waits and never fails for lack of room. When the next page is the oldest page, it throws that whole page of events away, adds their count to a counter, moves the tail onto the space and carries on. The caller is handed a perfectly good pointer and is never told what it cost.

The capture in section 5 is that, on purpose. An eight kilobyte buffer, an unfiltered tracer, and about seven thousandths of a second of an idle machine running two small commands:

entries: 273
overrun: 44002
commit overrun: 0
dropped events: 0

44275 events were written. 273 are still there. 44002 are gone. That is 161 events discarded for every one kept, and 99.4% of everything the tracer recorded.

The part that is worth a blueprint

Why the trace file does not say so. The kernel does know the number, and it will tell you over one of the two ways of reading a trace and not the other.

trace_pipe does a consuming read, which computes the count as overrun minus last_overrun and carries it out to the caller, and it prints CPU:0 [LOST 44002 EVENTS].

The trace file uses an iterator, and an iterator asks a different question. It calls ring_buffer_iter_dropped, which reads a missed events flag, and that flag has exactly one source: a bit in a page's commit field. The kernel's own comment next to the one place that sets it says the main write buffer does not set these bits. So on an ordinary buffer that flag is always zero, the count is never consulted, and the file prints nothing about the 44002 events that are gone.

That is invariant 11 in section 4. Nothing in the kernel enforces it, nothing tests it, and it is not stated anywhere except in a comment about a different case. It is the difference between a trace that is quiet because nothing was lost and a trace that is quiet because nobody asked.

The generator does the arithmetic, because a reader who does it by hand does not

Section 5 comes from a new tracefs-stats reader in tools/bpcgen.py. The file it reads answers no question anybody actually has on its own. What was written is not in it. The share that was thrown away is not in it. Neither is how much of the machine's history the buffer was holding. All three are addition and division on numbers that are in the file, so the generator does them and prints them next to the table.

kxray/tracefs.py gains parse_timestamps and window for the two clock lines. account_stats is deliberately unchanged, so corpora/BASELINE.toml and the line accounting are untouched, and the generated text now says which two lines were skipped and why rather than leaving a bare number that looks like the parser gave up.

Two numbers that mislead, written down

The buffer was asked for as 8 kilobytes, reads back as 11, and cost 16. A request in bytes is divided by the usable size of a sub buffer, which is a page minus a twelve byte header, so 4084 rather than 4096. 8192 over 4084 rounds up to three. Three times 4084 is 12252, and the file prints kilobytes rounded down, so it says 11. The reader page is allocated on top, so four pages went. The kernel's own documentation says the number is a request rather than a setting.

And overrun and dropped events are not two views of the same thing. One counts events lost with overwrite on, the other counts events lost with overwrite off, exactly one of them can be non zero on a given buffer, and a reader who checks the zero one concludes that nothing was lost.

Seven names section 7 cannot find

Six are static functions the compiler inlined, which is the case the lock ordering blueprint already documents.

The seventh is ring_buffer_alloc, and it is a better story. It is not a function at all. It is a macro that declares a static struct lock_class_key and passes its address to the real function, so that every ring buffer in the kernel gets a lockdep class of its own. That is the lock ordering blueprint reaching into this one. Somebody who greps a build for that name and finds nothing draws exactly the wrong conclusion, so it gets a paragraph rather than a table row.

Evidence

75 citations in blueprints/trace-ring-buffer.refs.toml, all confirmed against the pinned 7.2.2 tree, every one anchored to text that matches exactly one line in the file it names. Three anchors were thrown out while writing them because they matched more than one place, which is the uniqueness check earning its keep. Several anchors are long comments on purpose: this is a lockless algorithm whose correctness argument is written in prose and nowhere else.

status is partial, and the reason is the usual one. The tooling has done everything it can and nobody has read it through yet.

One limit stated plainly in section 8 rather than glossed: the pinned machine has one processor, so the reader and writer races in section 3 were reasoned about from the source and not observed. The evidence here covers the counters and the loss behaviour.

Also in here

• The diagram, blueprints/assets/trace-ring-buffer-overrun.diagram.py, with alt text, an SVG and an excalidraw file.
• coverage.toml gains the blueprint and the two include/linux/ring_buffer*.h headers on the tracing entry.
• Tests for the new section 5 block and for the two clock readers.

Gates

Every one of them, locally: ruff check and format, pytest, the node tests, --check runs of diagrams, nbbuild, sitebuild, kxbox and vendor, and plain runs of baseline, claimledger, coverage, bpc, kconfig, refcheck, lintnb, kxmanim and lintprose.

bpc: 5 blueprint(s) clean, refcheck: paths clean, 207 citation(s), baseline: 41 artefact(s), 7968 line(s), 0 unparsed.