docs-drift swept the file and missed a stale 124 — the detector matches phrasings, so its denominator counts files read, not claims checked

[serge-ivo]

A stale tool count survived a green gate in the file the gate swept — the detector matches phrasings, not claims

scripts/lib/mcp-split.mjs (#575) checks always-on/gated claims against MCP_TOOL_ALWAYS_ON / MCP_TOOL_GATED. Its success line reports 40 swept (3 required to state it).

workers/mcp/AGENTS.md is in that swept set. Line 15 says:

"18 of the 124 registrations are gated"

Actual: 19 of 136. The phrasing did not match the detector's pattern, so the file was swept, matched nothing, and passed. pnpm docs:drift reports all 13 checks green with that sentence present.

This is the file whose entire job is to be the calling agent's contract, and it has been telling every agent the wrong surface size.

This is the second instance of the same defect class today

#600 — state-vocabulary.ts's scanner returned [] for the two health descriptions that had actually drifted, because it reads (a/b/c) and backtick chains while those wrote `working` (gloss), `waiting` (gloss) — the shape a good description uses. The claim was never in the denominator, so no inventory entry could record it.

Same shape here: the denominator is "files swept", not "claims found", so a claim the detector cannot parse is invisible and indistinguishable from a file that makes no claim. Both guards report a healthy-looking number over a population that silently excludes the thing they exist to catch.

That is an ADR 0002 violation inside guards written to enforce ADR 0002: the success line states what was read, not what was checked.

Why "just widen the regex" is the wrong answer

#600's agent measured exactly that for the vocabulary scanner: 8 candidates over the surface, 6 false positives (result field names, write/destructive, a pair of tool names), and on the very description in question it recovered only two of four values. It chose generation over detection, which is that file's own stated doctrine.

The same is available here: MCP_TOOL_COUNT and MCP_TOOL_ALWAYS_ON are already constants. A doc could render them rather than restate them, or the check could assert that a file mentioning tool counts at all matches some known pattern — a shape requirement rather than a broader match.

Acceptance criteria

1. workers/mcp/AGENTS.md:15's claim is caught by a gate. Demonstrated red before the number is fixed (Agent-facing docs state four numbers the code disproves, and root AGENTS.md predates the output-schema change its first recipe depends on #602 fixes the number; this issue makes it catchable).
2. The success line states claims found, not files swept — or states both, distinguishing "no claim" from "claim not parsed". This is the load-bearing change.
3. A file in the swept set that mentions a guarded quantity in a shape the detector cannot parse fails rather than passing silently. If that produces false positives, report the count and rate rather than widening quietly.
4. Denominator per ADR 0002, and the same audit applied to mcp-split.mjs's sibling checks — the "N tools" check reported 5 claims in 39 files, which is the right shape; establish whether the others are.
5. Consider whether The vocabulary inventory's own citations are unchecked — three name files or symbols that do not exist #600's finding and this one warrant one fix. They are the same defect in two detectors, and two separate widenings is how a third appears.

Verified vs inferred

• Verified: AGENTS.md:15's text; tool-count.ts's constants; that docs:drift passes all 13 checks with the stale sentence present; mcp-split.mjs's 40 swept (3 required) output; The vocabulary inventory's own citations are unchecked — three name files or symbols that do not exist #600's measurement of the vocabulary scanner.
• Inferred: that the phrasing is why it was missed, rather than an exclusion elsewhere in the sweep. The file is in the swept set and the number is wrong and the gate is green — but the detector was not stepped through against this line. Confirm that before fixing; if the cause is an exclusion instead, the fix is different.

[serge-ivo]

Not implementing this (per instruction), but #602 settled the one item this issue lists as inferred, so recording it here rather than leaving it to be re-derived.

Confirmed: the cause is phrasing, not an exclusion

workers/mcp/AGENTS.md is in the swept set — docs-drift.mjs:119-120 adds README.md/AGENTS.md/CLAUDE.md for every subdirectory of workers, packages, agents, templates. Stepped the real parsers over the real file, both the stale and the corrected sentence:

BEFORE ("18 of the 124 registrations are gated")  | gated: []   | toolcount: []
AFTER  ("of the 136 tool registrations, 19 are gated") | gated: [19] | toolcount: [136]

GATED_CLAIM is /\b(\d+)\s+(?:tools?\s+)?are\s+(?:surface-)?gated\b/ — "of the 124 registrations" sits between the digit and "are gated", so no match. TOOL_COUNT_CLAIM is /\b(\d+)\s+tools?\b|\b(\d+)\s+tool registrations?\b/ — "124 registrations" lacks the word "tool", so no match either. Both detectors returned empty on a file that made two wrong claims. So AC1's premise holds exactly as written.

Fixing the phrasing moved the denominators — split 10 claims / 4 files → 11 / 5, count 5 → 6 — which is itself the evidence for AC2: those totals were the only signal that anything changed, and neither distinguishes "this file makes no claim" from "this file's claim was unparseable".

One observation on AC5, offered rather than acted on

The two detectors fail in opposite directions, which may matter for whether one fix covers both:

• The vocabulary inventory's own citations are unchecked — three name files or symbols that do not exist #600's scanner (state-vocabulary.ts) was too narrow against good prose — the drifted descriptions used the shape a well-written description naturally takes, and widening measured 6 false positives in 8 candidates. That is why it chose generation over detection.
• This one is too narrow against a specific word order, and the correct phrasings are few and already enumerated in doc-claims.mjs:74-76. The values are MCP_TOOL_COUNT / MCP_TOOL_ALWAYS_ON, already constants.

So the shared defect is the denominator (AC2/AC4) — "files swept" reported where "claims found" is meant — and that genuinely is one fix across both detectors. The detection halves may not be: generation suits #600, whereas here a shape requirement (a file mentioning a guarded quantity must match some known pattern or fail) looks sufficient and much cheaper. Splitting it that way would be one widening, not two, which is what AC5 is guarding against.

Untested claim, flagged as such: I did not measure the false-positive rate of a shape requirement over the 40 swept files. That measurement is what should decide it, the way #600's agent decided its own.

Note for whoever takes this

workers/mcp/AGENTS.md now states both quantities in parseable form and carries an inline comment saying why the shape matters. It is not in SPLIT_MUST_CLAIM, so it is checked-if-present but not required-to-state — a future rewrite could still silently retire it. Adding it to that list is a one-line change and is arguably part of AC3.

[serge-ivo]

Delivered in 50859d5, CI green, plus a detector fix in c74aaef (see the end).

The inferred item held — confirmed empirically, before building

workers/mcp/AGENTS.md is in the swept set (added by docFiles()'s per-worker AGENTS.md loop), and stepping both parsers through line 15 returns [] — for the line in isolation and for the whole file. TOOL_COUNT_CLAIM wants the word "tool" ("the 124 registrations" has none); GATED_CLAIM allows only "tools" between the number and "are gated", not "of the 124 registrations". Phrasing, not an exclusion. The assumed fix was the right one.

The design (AC5): one denominator convention, two detection strategies

Not one widening, and not two. The two detectors fail in opposite directions — #600's was too narrow against good prose, this one against a word order — so forcing their detection halves together would have made both worse. What is genuinely shared is the denominator:

no mention          -> silent, legitimately
mention -> parsed   -> checks 6 and 6b compare it to the constants (unchanged)
mention -> UNPARSED -> fails, naming file:line

findQuantityMentions answers "does this document raise the subject at all", which is a different question from "what does it claim". ADR 0002 explicitly does not ask for a shared harness, and these two live in different languages anyway.

Why not a wider regex (AC3): widening GATED_CLAIM leaves the class open — the next honest rewrite phrases past the wider regex too — and a widened extractor must also be right about what it pulls out. A shape requirement never extracts anything, so its remedy is to rephrase. #602 fixed the sentence by rephrasing, which is the prescribed remedy working.

AC2 — the success line

✓ MCP surface claim shapes: 15 mention(s) across 6 file(s) that quantify the surface
  == 15 parsed by a claim check + 0 unparsed, 61 swept (1 exempt, holding 23 unchecked mention(s))

Mentions = parsed + unparsed, and the exemption's size is printed too — "1 exempt file" hid 23 mentions, and an exclusion whose magnitude is invisible is the under-count ADR 0002 forbids.

AC3 — false-positive rate, measured not asserted

Over the real sweep: 0 false positives. Three unparsed at the time of writing — AGENTS.md:15 (the true positive) and two honest subset lines in workers/mcp/CLAUDE.md ("+ 3 loop tools", "86 instance tools"), which is the file already exempt from the count check for exactly that reason. The exemption is now shared from one place instead of restated.

A LINE-level rule was measured and rejected: 75 lines matched, 50 of them false — #84–#90 issue ranges, #22c55e colours, S256, CVE identifiers. The number must quantify the noun.

AC4 — the sibling checks audited

Check 6's "N tools" arm was already the right shape (5 claims in 39 files). 6b reported 40 swept with no claim count — now covered by the shared mention pass, which spans both arms, since AGENTS.md:15 was simultaneously a total claim and a split claim and neither saw it.

AC1 — red demonstrated

Watched fail against the pre-#602 text in a throwaway worktree; it names both halves of the sentence. Its own test also caught the detector missing "135 MCP tools" — the exact phrasing doc-claims.mjs's header names as the trap — because the filler required a lowercase initial.

Claims moved from unchecked to checked

2 → 0 unparsed on this file, and via #604 the same detector found two more real stale claims in docs/. The count check went 6 → 8 parsed claims once those were rephrased.

Honest limits, named in the test rather than left to be found

The detector does not read: a count that follows its subject (tools: 136, "a tool surface of 136"), a spelled-out numeral, or a number more than three words from its noun. Each is asserted as unread, per ADR 0002's obligation on a hand-rolled scanner.

Correction landed after the fact: the filler could not cross a backticked code span, which hid docs/mcp-instance-runtime.md:110 ("120 server.tool(...) registrations"). Fixed in c74aaef (#604) with a test that fails on the old regex.