Make per-turn token usage observable, and say what it still cannot price

[MongLong0214]

bench/TOKEN-LEDGER.md §5 blocker B gave two reasons the drafting turn (W4)
is missing from the write side. One was that it needs a model call. The other
was sharper: the driver read a single session-total usage object, so the
harness could not have attributed an answer to the turn that produced it even
if a call were made. This closes the second reason. It is the instrument, not
the measurement — no drafting turn has been priced, and no figure in §9 moves.

What the CLI actually emits

Established by running it, not by designing against an expectation. Both raw
captures are committed under test/fixtures/claude-stream/ with local paths
redacted, so every claim below is checkable.

--output-format stream-json	accepted; requires --verbose under --print
assistant events	one per content block, all carrying the same message.usage and message.id — summing them double-counts every turn
input_tokens and both cache fields on those events	final; reconcile exactly with the session total
output_tokens on those events	the message_start snapshot
stream_event → message_delta	the turn's real usage, plus output_tokens_details.thinking_tokens — emitted only under --include-partial-messages

The gap is not small. On one probe the assistant events reported 4, 1 and 1
output tokens for three turns whose real outputs were 157, 193 and 36 — a
session total of 6 against 403. Nothing on the event marks that field
provisional while its neighbours are not.

So the one term the write side is missing is exactly the one term
stream-json alone reports wrongly, which is why the driver passes
--include-partial-messages and the parser keys off message_delta.

What is now measurable

--per-turn-usage writes a turn_usage object: one entry per assistant API
call, with its four usage fields, thinking tokens, stop reason, model, and the
kinds of its content blocks. Subagent turns are recorded with their
parent_tool_use_id rather than merged away.

Every ledger carries its own audit instead of asking to be trusted:
turn_total, session_total, and a reconciled boolean that is true only
when the turns sum to the total the CLI states for itself, field for field. A
parser that missed or duplicated a turn says so on the row.

How earlier runs are kept intact

• Opt-in. Off is the shape every row in bench/results/ was produced under,
  and no committed result file is touched.
• Both output formats end in the same result object. transcript, turns
  and tokens are still read from it, under the same field selection
  (cache_read_input_tokens still excluded). Checked on a live run: session
  total 26 + 649 + 752 = the 1,427 the row reported.
• The ledger is built from events the JSON format never printed. Nothing
  already measured is re-derived from them.

A pre-existing schema failure, fixed on the way past

reproposal_matches and the four rejected_path_* counts had reached the rows
without reaching result.schema.json, so bench/verify.mjs rejected all 80
rows of the most recent matrix on additionalProperties. Adding a sixth field
to that list without fixing it would have shipped a gate that gates nothing.
The gate now validates 252 rows across three current-shape result files.

What remains blocked

1. W4 is still unmeasured. No bench arm runs capture against a run's own
   transcript and diff. The instrument exists; the measurement has not been
   made.
2. Blocker A untouched. The session transcript is unrecoverable for this
   corpus — a usage ledger cannot recover a transcript nobody kept.
3. Blocker C untouched. Cache creation and cache reads are now separate
   counts, but a count is not a billing rate.
4. Two tokenizers. turn_usage carries the provider's; everything in
   TOKEN-LEDGER.md is in the product's own ceil(chars / 4) proxy. A future W4
   must be reported beside the floor, never added into it.
5. content_blocks says a turn called a tool, not which tool or on what.
   reproposed counts correct behaviour as failure — measure rejected work, not rejected mentions #141's rejected-path tool actions stay uncounted.

bench/TOKEN-LEDGER.md §5 was appended to, not edited — it is
pre-registration, and it is the reason §9's figures can be read as a floor. The
new §5.1 records what changed and what did not. The row constant in
bench/deterministic/ledger.ts was narrowed so future rows say the same thing;
the committed row keeps the text it was written with.

Verification

• npx vitest run — 12 new tests in test/bench-turn-usage.test.ts; full
  suite green
• npm run typecheck, npx tsc -p bench/tsconfig.json --noEmit,
  npm run build
• node scripts/check-readme-numbers.mjs — exit 0, BENCH block byte-identical
• node dist/commitlore.mjs validate -c HEAD — shape ok, references ok
• No benchmark matrix was run. Model calls were limited to the handful needed
  to inspect the stream format.

README.md and the translations are untouched.

[github-actions]

CommitLore — record lint

Trailers: clean — 1 commit in origin/dev..6d4e8288927a238f5732471cb25024ef29c615bf
Active constraints: 20 limits · 39 ruled-out · 15 warnings — from 24 records over 12 changed paths

Active constraints for the paths this PR touches

Limits (20)

• r-4e7c19 6d4e828 — turn_usage carries the provider's tokenizer; every figure in bench/TOKEN-LEDGER.md is in the product's own ceil(chars / 4) proxy. A future W4 must be reported beside the floor, never added into it, until both sides are on one tokenizer
• r-4e7c19 6d4e828 — content_blocks says a turn called a tool, not which tool or on what — the tool_use block's name and input are discarded, so reproposed counts correct behaviour as failure — measure rejected work, not rejected mentions #141's rejected-path tool actions stay uncounted
• r-tokenledger 298e47c — both sides are byte-derived token proxies under CHARS_PER_TOKEN=4, so the ratio cancels a uniform tokenizer error and does not cancel a differential one between prose and diffs
• r-ledgerpin b0da937 — the ancestry check proves this checkout continues the history the read side was measured on; it does not prove nothing between the two commits changed how a diff reconstructs
• r-ledgerresult bc31c90 — both sides are byte-derived proxies under CHARS_PER_TOKEN=4 rather than a provider tokenizer, so the ratio cancels a uniform error and not a differential one between diff text and prose
• r-ledgerresult bc31c90 — break-even in reads assumes reads land on the evaluation set the way the delivery run's per-path average describes, and real editing concentrates on a few files
• r-rejwork141 5f7af65 — the counts read the diff, so an agent that pursued a rejected approach entirely in conversation and committed nothing registers zero
• r-m4basis 5e2d2cb — the guard question stays unanswered until the exposure instrument is verified and M4 is rerun on it
• r-m4withdraw e5f9b73 — the guard question is now unanswered rather than answered null
• r-expomerge1 d6ad014 — M4's existing rows have no exposure field and must read as unknown, not as not-exposed — backfilling by inference would erase the finding
• r-exposure52 ba69411 — legacy JSONL artifacts predate model and guard-exposure fields | their absence remains unknown and is never inferred or backfilled
• r-4c73a2 1d82a5b — code inherits the limit of any lexical surface — an alternative implemented under another name is invisible, one named in a string literal is counted; both were true of artifacts
• r-3e8a41 2f0a8a0 — scoping is not implemented, so off-path records reach every arm
• r-2f8b90 a376808 — claude -p inherits the operator's MCP servers, settings, CLAUDE.md, skills and hooks unless every one is explicitly disabled
• r-9c2f74 d653153 — the ablation arms cannot discriminate on these fixtures -- no-grade and no-lifecycle are byte-identical to the treatment in 9 of 10 tasks, because the seeds carry one reconstructed record and one task with a lifecycle trailer between them
• r-9c2f74 d653153 — the harness assembles its own projection rather than calling the shipped injector, so what is measured is the harness's rendering of the records, not src/core/inject.ts (issue B-08 · Replace the benchmark harness injector with the actual src/core/inject.ts #36)
• r-3f7a29 49817dc — reconstruction reads text written before the protocol existed, so the evidence is thinner than a harvest and the discard rate is expected to be high
• r-5c8b31 60ddc39 — the agent CLI exposes no in-flight turn limit, so a per-task turn budget can only ever be observed with this driver
• r-0b7c44 d2b2ce3 — a command is only real once --help names it, because that is where users look before they read source
• r-6c2b95 5da793c — the CLI holds no API key, so any real driver runs through the user's own agent session and cannot be exercised in this environment

Ruled out (39)

• r-4e7c19 6d4e828 — turning per-turn capture on by default | it changes the CLI invocation, and a harness flag that quietly rewrote how a run is observed would make old and new rows two different things under one name
• r-4e7c19 6d4e828 — summing the usage on assistant events | they repeat once per content block and their output_tokens is a pre-generation snapshot; that reading is wrong by 397 of 403 on the one probe that checked it
• r-4e7c19 6d4e828 — writing turn_usage on every row with nulls when uninstrumented | an always-null field is worse than no field, and absent already reads as "not instrumented" everywhere else on this row
• r-4e7c19 6d4e828 — retaining the whole stream to parse at the end | partial messages emit an event per chunk, so a long run would hold tens of megabytes to produce a few hundred integers
• r-tokenledger 298e47c — Remeasuring decision_delivery inside this harness | it would price the read side on a corpus that already includes the commits adding this measurement, and docs/evidence.md would cite two corpora for one ratio
• r-tokenledger 298e47c — Reporting a single write-side accounting | issue The break-even figure does not say which token accounting it uses, and both accountings are defensible #138 established that a break-even which does not say what it counted is underspecified, so the diff term is reported both charged and free
• r-tokenledger 298e47c — Estimating the drafting turn from the 112 recorded runs | those rows carry a session total from --output-format json with no per-turn attribution, so any split across turns would be invented
• r-ledgerpin b0da937 — Falling back to HEAD when the recorded commit is unresolvable | it turns a loud provenance failure into a quiet corpus mismatch, which is the class of defect ADR-0018 exists to keep loud
• r-ledgerpin b0da937 — Keeping the HEAD run beside a pinned one as a sensitivity arm | it was never committed or quoted, so keeping it adds a second table a reader has to reconcile for a 3-record difference
• r-ledgerresult bc31c90 — Leading with the 99.4% reduction against the whole-repository dump | it is the largest number in the table and its denominator is a route no agent can run, which is the figure this market publishes
• r-ledgerresult bc31c90 — Omitting the code-only row because it has no percentage | a reduction table that drops the case where the product is the expensive option is the thing the method document was written to avoid
• r-ledgerresult bc31c90 — Publishing a single write-side accounting | The break-even figure does not say which token accounting it uses, and both accountings are defensible #138 established that a break-even which does not say what it counted is underspecified, so the diff term is reported both charged in full and charged at nothing
• r-rejwork141 5f7af65 — a single weighted rejected-work score | the weights would carry the argument, and a reader whose situation weights dependency additions above hunks cannot recover the components from a composite
• r-m4basis 5e2d2cb — keeping the non-delivery wording as a hedged claim | it asserts something I never established, and a verdict that overstates its own evidence is the defect this repository exists to catch
• r-m4withdraw e5f9b73 — retracting the dataset or calling M4 invalid | the data is valid and its provenance is clean; what it measured was not the treatment, and those are different words
• r-exposure52 ba69411 — compute a treatment effect with unknown guard exposure | an old row that never recorded whether the guard reached the run cannot distinguish no treatment from an ignored treatment
• r-m4docs1 32d5bf1 — keeping the withdrawal notice and only landing the verdict document | bench/report.ts already draws this line -- a provenanced dataset that still shows a withdrawal is a hard failure in check-readme-numbers.mjs (checked here), not a style choice left open
• r-2b58d4 4842356 — exempting datasets written before the fields existed | it is one line and it deletes the guarantee
• r-8e15c2 70f6bcf — hashing only the files the entry point transitively imports | it is the same judgement call that produced this bug, made with more machinery
• r-4a17d9 68d4c92 — guard versus the control as the primary test | it would confound the route with whether records exist at all, and the interesting question is which route works better once they do
• r-4a17d9 68d4c92 — a blocking guard arm | GUARD-CANNOT-BLOCK measured true and false positives in one score band; the only precision-safe threshold catches one in five
• r-3d61a8 b9f34b3 — adding --ablate-* flags to the shipped inject command | it would create exactly the path from user input to an ablation that the design forbids, and the one that matters injects a prompt-injection payload
• r-3d61a8 b9f34b3 — importing src/core/inject.ts from bench | src/ uses NodeNext .js specifiers, so --experimental-strip-types cannot load it. dist/ is committed (ADR-0011) and is always present
• r-3d61a8 b9f34b3 — letting the shim grow into a second injector | that is what bench/context.ts became, and then the benchmark measured the copy
• r-4c73a2 1d82a5b — re-scoring the recorded transcripts and reporting the result | the labels would be chosen with the outcome known, which §4 forbids; the recorded runs were used only to check the surface behaves as intended
• r-4c73a2 1d82a5b — moving violation_if to code as well | it is instrumented-only, and changing an unregistered metric alongside a registered one makes it unclear which change any difference belongs to
• r-3e8a41 2f0a8a0 — off-path records that advocate the ruled-out option | with no scoping they reach all three arms, raising re-proposal everywhere and compressing the grading and lifecycle contrasts the set is built to isolate
• r-2f8b90 a376808 — accept the inherited environment and document it | a memory server between runs is a confound, not a caveat
• r-2f8b90 a376808 — fail hard when the isolation flags are missing | an older CLI should still produce a run, as long as the log says it was not controlled
• r-9c2f74 d653153 — resume the pilot into the same file | a new process would load the edited code and create the mixing that had not happened
• r-9c2f74 d653153 — run the ablation arms as they stand | three nulls from comparing identical inputs read as "these guarantees do not matter"
• r-9c2f74 d653153 — keep the tasks that showed an effect and rewrite only the rest | the property is the criterion, not the direction of the result
• r-3f7a29 49817dc — repair a draft that fails verification | backfill's source material is weak enough that a repair loop would mostly be inventing
• r-3f7a29 49817dc — write reconstructed records into commit messages | history rewriting is irreversible and reaches every existing clone
• r-3f7a29 49817dc — post a fresh comment per push | it turns the signal into noise and the check gets muted
• r-5c8b31 60ddc39 — keep turns and explain it in prose | the JSONL outlives the prose, and whoever reads the rows later will not have it
• r-5c8b31 60ddc39 — drop the turn budget since it cannot be enforced | the overrun is still the signal that a run went off the rails
• r-6c2b95 5da793c — skip the dry-run driver | then nothing exercises the harness until a key exists, and the first real run debugs the harness instead of measuring anything
• r-6c2b95 5da793c — emit dry-run rows without a marker | indistinguishable from measurements the moment they leave the terminal

Warnings (15)

• r-4e7c19 6d4e828 (claim) — this closes the attribution half of blocker B only. W4 is still unmeasured, W3 is still unrecoverable for this corpus, and no figure in TOKEN-LEDGER.md section 9 changes
• r-4e7c19 6d4e828 (claim) — section 5 of TOKEN-LEDGER.md is pre-registration and was appended to, not edited — the floor has to stay visibly declared in advance
• r-exposure52 ba69411 (claim) — guard exposure is instrumented in the benchmark hook adapter, not by changing guard scoring | M4 rows remain unexposed and metrics refuses their effect estimate
• r-2b58d4 4842356 (claim) — this leaves the README with no measured numbers at all until M3-b runs. That is the honest state and it is also a worse first impression. The alternative was publishing numbers produced by a binary nobody recorded
• r-8e15c2 70f6bcf (claim) — this is the second time in two commits that a fix for the M3 failure did not actually cover the M3 failure. The first was caught by testing the fix against the incident rather than against its own tests
• r-4a17d9 68d4c92 (claim) — both hooked arms pay a subprocess per edit. M2 put all three of its timeouts in its hooked arm, and a timeout truncates a run, which flatters whichever arm suffers it — stated before the numbers exist
• r-3d61a8 b9f34b3 (claim) — this changes what the harness measures, so M1-b's numbers do not carry forward. A run under this delivery path is a new measurement and needs its own registration
• r-4c73a2 1d82a5b (claim) — SURFACES in task-loader.ts and MatchSurface in types.ts are two lists of the same thing — adding a surface means editing both
• r-8b41e6 0bbad66 (claim) — the payload share and the fixture share are not interchangeable — the payload adds framing and drops what grading and lifecycle withhold, so a reader who swaps one for the other will be wrong by a small, plausible margin
• r-3e8a41 2f0a8a0 (claim) — the ablation set is a different synthetic repository from bench/tasks, so its commitlore-on arm must never be pooled with the primary matrix's
• r-2f8b90 a376808 (claim) — if the runner log carries the "does not support" warning, the rows are not environment-controlled -- do not compare them to rows that are
• r-9c2f74 d653153 (claim) — after the measurement, check that git status is clean and the recorded sha is still HEAD -- an edit mid-run breaks reproducibility silently, and that check is the only thing that catches it
• r-3f7a29 49817dc (claim) — every backfilled record is Provenance: reconstructed, which the trust model always renders as a claim -- do not add a path that lets a draft override that field
• r-5c8b31 60ddc39 (claim) — a driver that gains a real turn limit should stop emitting over-turns and start emitting an enforced label -- do not reuse over-turns for something the harness actually stopped
• r-6c2b95 5da793c (claim) — never cite a row with simulated:true -- README numbers come from bench/results logs and those rows are not results

git log --follow accepts exactly one pathspec, so renames are not followed for 12 paths; query one path at a time to follow its rename chain

withheld the content of 1 record(s) graded blocked: a Ruled-out trailer matching an injection pattern is reported, never quoted (SPEC §7)

_{Trailer violations fail this check. Active constraints are informational — they are what the repository already decided, not a verdict on this PR.}