[docs] Record as ADR 0002: a guard must assert the SIZE of what it measured — the three that lied are exactly the three with no denominator assertion

[serge-ivo]

The three were not a coincidence, and the common cause is one line long

#426's overflow guard, runnerPanel.ts's tile invariant (#531) and jsx-tags.ts (#536) failed the same way — green while measuring a subset. I audited every scripts/check-*.mjs and every invariant test in the repo looking for a shared helper or a shared idiom, and the shared thing is not code. It is an absent assertion:

Every guard in this repo asserts its NUMERATOR — "the offender list is empty", "the count equals the pin". The ones that were caught lying are exactly the ones that never assert their DENOMINATOR — how many candidates they examined.

That is checkable, and I checked it both ways.

Guards that assert their denominator (none has been caught under-measuring):

guard	the assertion
scripts/check-design-tokens.mjs:99	declared.size < 10 → "the @theme block moved or changed shape, and this guard would pass by checking nothing"
scripts/check-test-isolation.mjs:131	isolatedCount === 0 → "green, and running none of the hard tests"
scripts/check-migrations.mjs:366	--require-history → "a history-dependent check that degrades to a silent no-op is worse than no check"
scripts/check-qa-config.mjs:47	ignore.length === 0 → "it is doing nothing"
workers/api/src/lib/security-invariants.test.ts:104	"finds the surfaces that verify inline at all" — plus five more toBeGreaterThan(0) pattern-still-matches checks (:251, :303, :370, :459, :500)
workers/api/src/lib/tool-reachability.test.ts:219	"reads the migrations it claims to read" — file count, parsed-name count, and one known declaration per parser shape
packages/sdk/src/safe-html-guard.test.ts:112	ALL.length > 100 && sinkHits.length > 0
workers/api/src/lib/coder2-parity.test.ts:685	checked >= 3 → "a cohesion rule that matched no agent would pass forever while meaning nothing"
scripts/lib/adr.mjs:45	throws rather than returning [] → "'no rules parsed' is indistinguishable from 'every rule covered' unless someone asserts otherwise"
store/console/src/pages/mute-touch-invariant.test.ts:191	discovers the surfaces and compares the set, rather than trusting a list

Guards that do not, and the two that were caught:

• store/console/src/lib/control-shapes.test.ts / control-labels.test.ts sweep three .tsx trees and assert emptiness and exact pins. Neither ever states how many tags, controls or files it examined — which is why 33 tags being invisible ([bug] jsx-tags.ts does not treat a backtick as a delimiter, so one apostrophe in a template literal drops a whole tag from two source guards #536, measured) changed nothing visible, and why the console's pin reads 42 when the tree holds 43.
• The [bug][mobile] Copy and Delete sit on top of the message timestamp — 42px of 110px covered in WebKit, and removing the year only recovers 30px #426 overflow spec measured an element with a zero rect. Nothing asserted the queried set was painted, or how large it was.

The rule generalises past lexers, which is why it is worth writing down rather than fixing three files: a directory walk blinded by an exclusion, a git log blinded by a shallow clone, and a regex blinded by a rename all present identically — a confident number over a set nobody sized.

The second half: a scanner that gives up must say so

The same idea one level down. store/console/src/lib/jsx-tags.ts:72:

if (j >= source.length) continue;

A tag whose end was never found is dropped in silence. That single continue is what turned a lexer bug into an under-count instead of a crash — had it counted the drops and had the test asserted zero, #536 would have failed on the day the first apostrophe landed rather than being noticed months later by a different lane. scripts/lib/bare-catch.mjs:118 (if (close === -1) continue;) has the same shape.

The good ones already do the opposite, and say why:

• workers/api/src/lib/tool-reachability.test.ts:158 — catch { // A blob this cannot parse is not silently skipped — the count assertion below fails. }
• workers/api/src/lib/coder2-parity.test.ts:56 — throw new Error(...) when no migration matches
• workers/api/src/lib/pipelines/seed-drift.test.ts:27 — expect(start).toBeGreaterThan(-1) before slicing

And a third finding, which is the reason to standardise rather than patch

Eight independent implementations of "strip JS source so a scanner can match it" ship in this repo, at four different fidelities:

#	implementation	handles
1	workers/api/src/lib/source-guard.ts:37	comments, quotes, templates with ${…} preserved as code, regex, escapes — the reference
2	workers/api/src/lib/prompt-claims.ts:319 promptTextOf	the deliberate inverse (keeps literal text, drops code); templates handled
3	scripts/lib/bare-catch.mjs:18	comments, quotes, backticks — but blanks the whole template span, so a catch {} inside ${…} is invisible
4	store/console/src/lib/jsx-tags.ts:42 + the quote tracker at :59	line-start comments only; no backtick, no escapes → #536
5	scripts/lib/design-tokens.mjs:122	comments only, // not preceded by :
6	workers/api/src/lib/usage-aggregates.ts:73	comments only
7	workers/api/src/lib/usage-claims.test.ts:65	comments only, incl. SQL -- — a second copy inside the same feature area as #6
8	store/console/src/pages/mute-touch-invariant.test.ts:97	comments + all three quote kinds, regex-based

Only #1 and #2 are separately unit-tested as lexers. Each of the others is one legal character away from measuring less than it prints, and nothing in the output distinguishes the fidelities: all eight produce a confident number.

Measured impact today: #4 loses 33 tags across 2 of 107 .tsx files and makes one pinned count wrong (#536, comment there has the numbers). #3's template gap costs nothing — I ran bareCatches over all eight pinned trees with #1's stripper substituted and both agree at zero everywhere, and the four unpinned trees are also at zero. #5–#8 are comment-only strippers whose subjects (class names, SQL comments, JSX guards) do not span template interpolations.

What to do

1. Write the ADR (full text below), so the next guard is written with a denominator from the start. This is the deliverable; the rest is follow-up.
2. Give scanTags a drop counter as part of [bug] jsx-tags.ts does not treat a backtick as a delimiter, so one apostrophe in a template literal drops a whole tag from two source guards #536, and have control-shapes.test.ts / control-labels.test.ts assert it is zero and that the tag/control totals are within a stated range. That converts the class of bug from "quietly smaller" to "loud".
3. Consolidate the lexers where the fidelity actually matters. [codex] Add PAGS browser runner runtime #1 is the reference and is already .ts in workers/api; the console cannot import across that boundary, so the honest options are (a) move a lexer to packages/sdk where both trees can consume it, or (b) leave the copies and require each to carry its own unit test naming what it does NOT handle. I would take (b) first — it is cheap, it is the discipline scripts/lib/bare-catch.mjs's header already claims ("a regex-shaped guard nobody tests is one edit away from silently passing"), and (a) is a package-boundary change that should not ride on a bug fix. Refactor: split console-instances.js (957 lines) into -core + -board #7 and Server-side deployment notifications (deploy succeeded/failed → push), not polling-only #6 should merge regardless; two comment strippers for one feature is not a boundary, it is a copy.
4. Print the denominator in every guard's success line. ✓ store/console: 43 hand-authored control shapes over 5473 tags in 69 files costs one interpolation and makes the next under-count visible in a passing build. check-file-size.mjs:1042 and check-test-isolation.mjs:140 already do this; most do not.

Alternatives considered and rejected

• Make it a CI lint (every guard file must contain a toBeGreaterThan). Rejected: it is satisfiable by writing expect(1).toBeGreaterThan(0), and a rule a machine grades on shape rather than meaning is how biome-ignore comments came to be suppressing nothing ([standards][tracking] Split worst large files into testable modules #305). This is a review constraint, which is what an ADR is for.
• A shared Guard harness that reports numerator and denominator. Rejected as premature: the guards legitimately measure different kinds of thing (files, tags, routes, migrations, phases), and a harness that fits all of them would be a framework nobody wanted. The ADR names the obligation; each guard meets it in its own vocabulary.
• Fix the eight lexers into one now. Rejected as the opening move — see 3(a) above; the package boundary makes it a design change, and [bug] jsx-tags.ts does not treat a backtick as a delimiter, so one apostrophe in a template literal drops a whole tag from two source guards #536 needs to land before it.

Acceptance criteria

1. platform/docs/adr/0002-a-guard-states-what-it-measured.md exists with the text below, and docs/adr/README.md's index gains its row.
2. Its rules parse with scripts/lib/adr.mjs — ids in the **G1 — … form the regex at adr.mjs:30 requires — so a later test can read them the way ADR 0001's two tests do. (No new test is required by this issue; the format is so that one is possible without editing the parser.)
3. The ADR is linked from scripts/lib/bare-catch.mjs and store/console/src/lib/jsx-tags.ts, one hop from where it would be broken — ADR 0001's third enforcement item, and the thing mute-touch-invariant.test.ts:299 asserts for its own.

Regression risk

None to running code. The risk is the ADR being read as "add a toBeGreaterThan to everything": G2 below is deliberately written as the guard states the size of the set it examined, not the guard contains an assertion of shape X, and the Consequences section says what it costs when the denominator is genuinely unstable.

---

Proposed text — platform/docs/adr/0002-a-guard-states-what-it-measured.md

# ADR 0002 — A guard states the size of what it measured

**Status:** Accepted (2026-08-13) · **Owner decision** · Supersedes nothing.

## Context

This repository defends itself with guards: eight `scripts/check-*.mjs` wired into `ci.yml`, and
around fifteen invariant tests that sweep source trees, migrations and documents. Almost every one
exists because someone was surprised once, and the good ones carry the incident in their header.

On 2026-08-12 three of them were found to be measuring less than they asserted, within a single day:

  * the **#426 mobile-overflow guard** — with both the phone and desktop layouts in the DOM, its
    aria-label query matched a `display:none` element with a zero rect and a computed opacity of 1.
    It passed while measuring nothing. Fixed under #514 by tagging every action control with
    `data-msg-action` and measuring what is painted.
  * **`store/console/src/lib/runnerPanel.ts`'s tile invariant** — its own header promised the tile
    was "derived from BOTH readings, so a disagreement is a test failure rather than a screenshot",
    but `machinesToShow` merged them only for the synthesised tile and the test iterated only the
    pinned node. A non-pinned node with a live socket was untested. Fixed under #531.
  * **`store/console/src/lib/jsx-tags.ts`** — the tag scanner does not treat a backtick as a string
    delimiter, so one apostrophe inside a template literal swallows the rest of a tag. Measured: 33
    tags invisible across two files, and a pinned ratchet reading 42 where the tree holds 43 (#536).

Each was fixed on its own terms. The class is what this record is for, because the three have one
property in common and it is not their subject matter: **each asserted only what it found, never
how much it looked at.** An empty offender list and an empty input set produce the same green tick,
and the guards where someone did assert the input set — `check-design-tokens.mjs`'s
`declared.size < 10`, `check-test-isolation.mjs`'s `isolatedCount === 0`,
`check-migrations.mjs --require-history`, `security-invariants.test.ts`'s "finds the surfaces that
verify inline at all", `tool-reachability.test.ts`'s "reads the migrations it claims to read",
`adr.mjs` throwing rather than returning `[]` — are, so far, none of the ones that failed.

A guard that under-reports is worse than no guard, because the number it prints is trusted. That is
the whole reason this is a constraint and not a style note: a subset measurement does not merely
fail to protect, it actively certifies ground it never walked.

## Decision

**A guard must make the size of the set it examined an assertion, not a by-product.**

Four rules. A guard that violates one is incomplete regardless of what it correctly detects.

**G1 — The input set is asserted, not assumed.** A guard that walks a directory, reads a config,
parses a document or matches a pattern must fail when that input is empty, implausibly small, or
structurally unrecognisable — with a message that says the guard has stopped measuring, not that
the code is clean. `parseThemeColorTokens` returning 3 tokens is not a clean tree, it is a moved
`@theme` block.

**G2 — The denominator is stated in the passing output.** A guard's success line names how many
files, tags, routes, migrations or phases it examined. A number that is only correct while nobody
reads it is how an under-count survives; printing it puts the evidence in every green build.

**G3 — A scanner that cannot parse something reports it.** Skipping an input the parser failed on
converts a bug in the parser into a silently smaller measurement. Count the failures and assert
they are zero, or throw. `if (j >= source.length) continue;` is the shape this forbids.

**G4 — A new guard is proven by watching it fail.** Before it lands, reintroduce the defect it was
written for — revert the fix, plant the fixture, delete the attribute — and record that the
assertion flipped. A guard whose only evidence is that it passes has evidence of nothing: every
guard passes on the day it is written, including the ones that measure the empty set.

## Consequences

Guards get slightly longer and slightly noisier. `✓ store/console: 43 hand-authored control shapes
over 5473 tags in 69 files` is a worse headline than `✓ clean` and a much better artefact, because
the next person to change the scanner can see the denominator move.

G1 has a real cost where the denominator is genuinely unstable: a threshold set too tight fails on
honest growth, and one set too loose asserts nothing. The answer is a bound with a reason beside it
("fewer than 10 colour tokens means the block moved"), never a bound chosen to make today's number
pass.

G4 costs one throwaway commit per guard, and it is the rule most likely to be skipped under time
pressure. It is also the only one of the four that would have caught all three of the 2026-08-12
failures before they shipped.

This record does NOT require a shared lexer, a guard framework, or a common harness. Guards measure
different kinds of thing and a framework that fitted all of them would fit none of them well. What
is required is that each states its own size in its own vocabulary.

## Enforcement

  * Review. The three failures above were all found by reading, and this document is what makes
    "where is the denominator?" a routine question at review rather than an insight.
  * The pattern is already established and can be copied rather than invented:
    `scripts/check-design-tokens.mjs` (G1), `scripts/check-file-size.mjs` and
    `scripts/check-test-isolation.mjs` (G2), `workers/api/src/lib/tool-reachability.test.ts` (G1
    and G3), `scripts/lib/adr.mjs` (G1 by throwing).
  * A pointer to this ADR belongs beside any hand-rolled source scanner, one hop from where it
    would be broken.
  * A mechanical check of G1 is deliberately NOT proposed: it would grade on the presence of an
    assertion rather than on what the assertion means, and this repository has already measured
    what that produces (#305's `biome-ignore` comments, which had drifted off the lines they
    covered and were suppressing nothing).

Verified vs inferred

Verified: every file:line in the tables above, read at cfe8ff7; the eight strippers and their
handling, read in full; the 33-tag / 42-vs-43 measurement (posted with its method on #536); the
bare-catch cross-check against source-guard.ts's stripper over all eight pinned trees plus four
unpinned ones, all zero.

Inferred: the causal claim — that the absence of a denominator assertion is why these three
failed rather than a correlate. It is supported by the split in the two tables (ten guards with the
assertion, none caught; the ones caught lacked it) and by the mechanism being sufficient in each
case, but three incidents is not a large sample and I have not audited a store outside PAGS for a
counter-example.

[serge-ivo]

Two corrections to line numbers in the tables above, and one addition that makes G4 easier to sell.

Corrections (the claims hold, the citations were off by a scroll):

• check-qa-config.mjs's empty-input assertion is at :28, not :47 — if (ignore.length === 0) problems.push(".vcqa.json has no \ignore` list — it is doing nothing.");`
• coder2-parity.test.ts's non-emptiness assertion is at :605, not :685 — expect(checked, "no agent declares github_create_issue — the walk or the parser is broken").toBeGreaterThanOrEqual(3);

G4 is already practised here, twice, and the ADR's Enforcement section should cite both — it is much easier to adopt a rule that has working examples in-tree than one that reads as new process:

• workers/api/src/lib/migration-hygiene.test.ts:1-12 drives the real check-migrations.mjs as a child process over throwaway git repositories and "asserts each arm goes RED for the mistake it is about" — with the reasoning spelled out: "an exception is exactly where a guard quietly stops firing", and "the exit code IS the contract (CI reads nothing else)". That is G4 in its strongest form: the guard's failure mode is executed, not argued.
• workers/api/src/lib/connection-outcome-guard.test.ts:24 runs the real statements against node:sqlite and states the inverse test in its own header: "Restore the unguarded WHERE id = ?1 (or the absolute attempts = ?) and every test here goes red."

Worth adding to the Enforcement bullet list as migration-hygiene.test.ts (G4) alongside the G1/G2/G3 exemplars already named.

Third data point for the same section, and the best example of G2 in the repo: workers/mcp/src/index.test.ts:496 — "blocks EVERY registered tool, not a hand-picked list" — runs the real registration path, asserts h.tools.size === MCP_TOOL_COUNT, then iterates every registered tool and collects the escapees. Denominator asserted against a constant that is itself asserted elsewhere, numerator compared to []. Nothing in this repo is harder to fool, and it is the shape the console's source sweeps (control-labels, control-shapes) could adopt directly.

[serge-ivo]

One more instance, and it is the cheapest possible demonstration of G1 — worth fixing in the same commit that lands the ADR, as its worked example.

scripts/check-agents-allowlist.mjs:12-25:

const dir = "agents";
let entries = [];
try {
	entries = readdirSync(dir).filter(…);
} catch {
	console.log(`✓ no ${dir}/ directory — nothing to check.`);
	process.exit(0);
}

A relative path plus a catch that prints a tick and exits 0. Run it from anywhere but the repo root and it certifies the allowlist while having read nothing. It cannot fire in CI today — ci.yml runs node scripts/check-agents-allowlist.mjs from the checkout root, and agents/ is tracked — so this is a latent G1 violation, not a live one, and I am recording it as such rather than dressing it up.

The fix is two lines: resolve the directory relative to import.meta.dirname like every other guard in scripts/ does (check-file-size.mjs, check-test-isolation.mjs, check-design-tokens.mjs all do), and make a missing agents/ a FAILURE — "the directory this guard exists to police is gone" is news, not a clean tree. The success line already prints its denominator (✓ agents/ contains only Tier-0 agents (coder, job-application-assistant, repo-chat)), so G2 is satisfied; only G1 is missing.

Verified: run at cfe8ff7 from the repo root it prints the tick with three entries; the catch arm is readdirSync's only failure path and the exit code is 0 either way.

[serge-ivo]

Landed as e021910 — docs/adr/0002-a-guard-states-what-it-measured.md, with the README index row.

Format decision (acceptance criterion 2). Prose, in the machine-readable shape. The four rules parse with scripts/lib/adr.mjs unchanged — readAdr(2).rules returns ["G1","G2","G3","G4"], verified — so a coverage test over them is possible later without touching the parser. No such test was added, and that is deliberate rather than deferred: ADR 0001's rules map onto specific code paths in the voice stack, so a test can ask "is M3 covered". 0002's rules are review constraints over every guard in the repo, and the Enforcement section rejects a mechanical check of G1 for the reason #305 already measured — a rule graded on the shape of an assertion is satisfied by expect(1).toBeGreaterThan(0).

Incorporated from the comments: check-qa-config.mjs:28 and coder2-parity.test.ts:605 (I re-read both — the corrected lines say what you quoted; the ADR body cites guards by name rather than line, so the record stays durable). Enforcement now names migration-hygiene.test.ts and connection-outcome-guard.test.ts as the G4 exemplars and workers/mcp/src/index.test.ts's "blocks EVERY registered tool" as the strongest G1+G2 one. All citations verified by reading them.

The worked example rode in the same commit, as you asked. scripts/check-agents-allowlist.mjs resolved agents/ against the CWD and answered an unreadable directory with a tick and exit 0. Now anchored to import.meta.url like every other guard under scripts/, and a missing agents/ exits 1. Proven both ways per G4 rather than asserted:

$ cd /tmp && node <pre-fix copy>/check-agents-allowlist.mjs
✓ no agents/ directory — nothing to check.        exit=0

$ cd /tmp && node <platform>/scripts/check-agents-allowlist.mjs
✓ agents/ contains only Tier-0 agents (coder, job-application-assistant, repo-chat).   exit=0

$ cd /tmp && node <copy in a root with no agents/>/check-agents-allowlist.mjs
✗ Could not read …/agents: ENOENT …                exit=1

Item 2 landed with #536 (1342c4f): scanTags throws instead of continue, scripts/lib/bare-catch.mjs:118 throws too, both sweeps assert and print their denominator, and the ADR is linked from both scanners' headers. Details on #536.

Item 3 (consolidate the lexers) is NOT done and is named in the ADR as the follow-on — it needs its own issue. The ADR takes your option (b) as the interim obligation: each hand-rolled scanner carries a unit test naming what it does not handle, until (a) happens. Two things I verified myself rather than repeating:

• jsx-tags.ts now has that test (it had none, which is how a lexer read by two ratchets stayed one character away from lying).
• The bare-catch stripper's ${…} gap: re-ran bareCatches over 628 source files across the eight pinned trees plus workers/host/src, e2e and scripts, with source-guard.ts's reference stripper substituted. Zero hits either way, zero disagreements. The gap is real and currently costs nothing.

Item 4 (print the denominator everywhere) is partially done — the two console sweeps and check-bare-catch.mjs now do; the rest of the guards do not, and that is left open as ordinary follow-on work, not claimed.

CI green on e021910.