Skip to content

A negative control written from the guard confirms the guard, not the claim - #822

Merged
MongLong0214 merged 3 commits into
mainfrom
cdeb-guard-mutation-registry
Aug 21, 2026
Merged

A negative control written from the guard confirms the guard, not the claim#822
MongLong0214 merged 3 commits into
mainfrom
cdeb-guard-mutation-registry

Conversation

@MongLong0214

Copy link
Copy Markdown
Owner

Five times in this session a guard was written, its negative control ran, the control failed as expected, and the guard still proved nothing.

argv assertion        mutating argv failed the test — while no probe ever ran in the container
refusal tests         passed while the attack could not run; "make it succeed" was impossible to construct
template validation   removing a field still failed — the control passed for the wrong property
bench-wide path grep  adding a path still failed — scope was wrong, so history got rewritten
exclusion index       record-id excluded correctly; the other 14 kinds were inert, with no control at all

Every control was derived from the guard's implementation, in the guard's vocabulary. That confirms the mechanism runs. It cannot confirm the claim, because the claim was never written anywhere a control could come from.

Claims, not functions

bench/cdeb/guards/registry.json indexes 21 claims — sentences about the study, not about the code. If a claim cannot be stated without naming a function, it is describing a mechanism and gets rewritten.

The runner reports four outcomes instead of pass/fail:

mutation applied, test failed    the guard binds this claim
mutation applied, test passed    the guard is inert for it
mutation cannot be applied       the claim has no expressible control
zero mutations                   nothing was ever attempted

The last two used to disappear. A property nobody could write a control for, and one nobody tried to, both looked identical to one that passed.

A ratchet, not a wall

The first honest run is red. Putting that in gate's needs makes the branch unmergeable, and a gate nobody can pass gets deleted — taking the instrument with it. (I made exactly this mistake earlier in this session with a CI-must-run assertion no runner could satisfy.)

So the job fails on regression against a baseline:

a bound property degrades                 FAIL
a new property arrives with no mutations  FAIL
a recorded gap stays a gap                reported, exit 0
a recorded gap becomes bound              FAIL — tighten the baseline

Without the fourth direction a ratchet rots: improvements accumulate while the record still claims the old gaps. Making repair fail until the baseline moves keeps measurement and record in step.

Current state, recorded per property with its reason

6 claims bound (15 mutations) · 1 inert · 1 unavailable · 13 uncovered

Each gap carries its reason in the claim's own words — "the census matches exclusion rows by value only, so this task identity is not represented in the candidate values the claim would need to make ineligible" — because a missing test and a missing feature need different work. Twelve of thirteen gaps are the second kind, which is the same conclusion an adversarial review reached independently by reading the code.

It caught its author on first use

While this was being built, a negative control removed the tampered-bundle mutation and did not restore it. A property the baseline recorded as bound measured uncovered, and the ratchet failed with exactly that sentence. Restored and re-verified.

Verification

Runner exits 0 on the current tree. Three negative controls observed and restored — a mutation removed from a bound property, a property added with an empty mutation list, and a baseline gap marked bound without repair — each failing with a distinct named reason. Both typechecks clean, ci-gate suite green.

Limits, in the commit

The baseline is a floor: a guard can bind against its one recorded mutation and still miss a different violation of the same claim. Reasons are written by the same author as the claims. Thirteen exclusion kinds remain uncovered and one scan inert — this records them and repairs neither.

… claim

Five times in this session a guard was written, its negative control was run, the control
failed as expected, and the guard still proved nothing.

    the isolation suite asserted the docker run argv; mutating the argv failed the test,
      while no probe had ever executed inside the container
    three refusal tests passed while the attack could not run at all, so the control
      "make the attack succeed" was impossible to construct and nobody wrote that down
    a schema was widened to accept placeholder strings so a template would validate;
      removing a required field still failed, so the control passed for the wrong property
    a path scan covered all of bench/, which forced four historical run logs to be
      rewritten; adding a path still failed the test
    the exclusion index matched only entry.value; kind was never read, so a record-id row
      excluded correctly and the other fourteen kinds were inert with no control at all

Each control was built from the guard's implementation, in the guard's vocabulary. That
confirms the mechanism runs. It cannot confirm the claim, because the claim was never
written down anywhere a control could be derived from it.

So the registry indexes claims, not functions, and the claim must be a sentence about the
study rather than about the code. If it cannot be stated without naming a function it is
describing a mechanism and gets rewritten. Each claim carries mutations that should break
it, and the runner reports four outcomes rather than pass and fail:

    mutation applied, test failed    the guard binds this claim
    mutation applied, test passed    the guard is inert for it
    mutation cannot be applied       the claim has no expressible control
    zero mutations                   nothing was ever attempted

The last two are the ones that used to disappear. A property nobody could write a control
for and a property nobody tried to write one for both looked exactly like a property that
passed.

The first run is red, and that is the runner working. Thirteen claims bind. One is inert:
the personal-path scan passes with its own check bypassed, because a clean tree satisfies
it either way and nothing constructs a violating file. One is unavailable. Thirteen
exclusion-index kinds have no expressible control, which is the same finding an adversarial
review reached from the other direction by reading the code -- two methods, one conclusion,
and only record-id actually excludes anything.

Nothing here fixes those. Surfacing them is the deliverable; a mutation runner whose first
run is green on a tree with a known inert guard is itself inert.

Record-Id: r-guardmutation
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: fixing the inert guards in the same change | a runner that has never reported a real failure is not known to report one, and the red run is the evidence that it can
Ruled-out: indexing guard functions instead of claims | that reproduces the exact failure this exists to stop, because the control comes back out of the implementation it is meant to test
Ruled-out: treating an unexpressible control as a skip | it is indistinguishable from a control nobody attempted, and both were silently green before
Ruled-out: folding this into the check job | it runs vitest once per mutation, so it belongs in its own job where its cost is visible
Limit: a mutation proves a guard reacts to one specific edit. A guard can bind its claim for that edit and miss a different violation of the same claim, so coverage here is a floor and not a proof
Limit: the claims were written by the same author as the guards, so a claim stated too narrowly produces a control that passes for a property nobody wanted
Limit: thirteen exclusion-index kinds remain inert; this change makes that visible and does not repair it
Verified: 15 mutations run against the real suites, producing 13 bound, 1 inert, 1 unavailable and 13 uncovered. Both typechecks clean and the CI gate suite passes at 7. The inert result was checked by hand: bypassing the personal-path scan leaves the suite green
CommitLore-Version: 2.0.0
…ument with it

The mutation runner's first honest run is red: thirteen exclusion-index kinds have no
expressible control, one path scan is inert, one control is unavailable. Wiring that into
the gate's `needs` made the branch unmergeable, and a gate nobody can pass is removed
rather than fixed. I made this mistake earlier in this session with a CI-must-run
assertion no runner could satisfy, so the same shape twice.

So the job fails on regression against a recorded baseline instead of on the backlog.
Four directions, and the fourth is the one that keeps the file honest:

    a bound property degrades                 FAIL
    a new property arrives with no mutations  FAIL
    a recorded gap stays a gap                reported, exit 0
    a recorded gap becomes bound              FAIL, tighten the baseline

Without the fourth a ratchet rots: improvements accumulate while the record still claims
the old gaps, and a reader cannot tell which gaps are real. Making repair fail until the
baseline moves keeps measurement and record in step.

Every gap carries its reason in the claim's own words -- "the census matches exclusion rows
by value only, so this task identity is not represented in the candidate values the claim
would need to make ineligible" -- because a missing test and a missing feature need
different work and the outcome alone cannot tell them apart. Twelve of the thirteen gaps
are the second kind.

The runner also caught a real regression while this was being built: the tampered-bundle
mutation had been deleted during a negative control and not restored, so a property the
baseline recorded as bound measured uncovered. The tool found its own author's mistake on
its first live use, which is the most useful thing it could have done.

Record-Id: r-guardratchet
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: leaving the job in the gate while it is red | branch protection would block every merge until someone deleted the job, and deleting it removes the only thing that can see these gaps
Ruled-out: allowing an improvement without updating the baseline | the record would drift below the measurement, and a baseline that overstates the gaps is as useless as one that understates them
Ruled-out: recording gaps as a count instead of per property with a reason | a count cannot distinguish a control nobody wrote from one that cannot exist, and those need different work
Limit: the baseline is a floor. A guard can bind its claim against the one mutation recorded for it and still miss a different violation of the same claim
Limit: the reasons are written by the same author as the claims, so a gap reasoned narrowly can look more settled than it is
Limit: thirteen exclusion kinds remain uncovered and one scan inert; this records them and repairs neither
Verified: the runner exits 0 against the current tree, prints all four outcome classes with per-property reasons, and the summary is 13 bound, 1 inert, 1 unavailable, 13 uncovered across 15 mutations. Three negative controls observed and restored -- a mutation removed from a bound property, a property added with an empty mutation list, and a baseline gap marked bound without repair -- each failing as a distinct named reason. Both typechecks clean, ci-gate suite green
CommitLore-Version: 2.0.0
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown

CommitLore — record lint

Trailers: clean — 3 commits in origin/main..17a580b5a2e9dadde8aa222a1c19799e98591352
Active constraints: 67 limits · 128 ruled-out · 15 warnings — from 95 records over 6 changed paths

Active constraints for the paths this PR touches

Limits (67)

  • r-reqchecksfixture 17a580b — the fixtures now agree with REQUIRED_CHECKS by construction, so a job that should be required but was never added to that list is still invisible here; the list itself remains the single named site
  • r-guardratchet 26b1989 — the baseline is a floor. A guard can bind its claim against the one mutation recorded for it and still miss a different violation of the same claim
  • r-guardratchet 26b1989 — the reasons are written by the same author as the claims, so a gap reasoned narrowly can look more settled than it is
  • r-guardratchet 26b1989 — thirteen exclusion kinds remain uncovered and one scan inert; this records them and repairs neither
  • r-guardmutation 551921d — a mutation proves a guard reacts to one specific edit. A guard can bind its claim for that edit and miss a different violation of the same claim, so coverage here is a floor and not a proof
  • r-guardmutation 551921d — the claims were written by the same author as the guards, so a claim stated too narrowly produces a control that passes for a property nobody wanted
  • r-guardmutation 551921d — thirteen exclusion-index kinds remain inert; this change makes that visible and does not repair it
  • r-ocimatrixci 6c738c8 — the matrix runs on the GitHub-hosted Linux runner and says nothing about Podman, rootless Docker, or any other runtime an operator might use. The refusals it observes are this daemon's, and the PRD's claim boundary should say so
  • r-lintcontextname 3e78376 — this pins where the name comes from, not that the check ever runs. A workflow-level failure before job dispatch produces no check run at all, and neither this nor the trigger assertion sees that
  • r-lintpullrequesttrigger 7617436 — this guards the trigger, not the job name. Renaming the lint job leaves the trigger intact and still breaks the required context, and nothing here catches that
  • r-cifaningateguards a22353e — the refined rule accepts exactly the string always(). A semantically equivalent expression -- ${{ always() }}, or always() && true -- is refused, which is stricter than the property being asserted and will read as arbitrary to whoever writes one
  • r-t1503revert 0c047a7 — merge-commit-only is still a sentence in a pull request body rather than a repository setting, so a squash remains one click away and would leave the source pull request open
  • r-t1503mode 34af906 — the mode is chosen from the pull request's changed paths, so a pull request that carries the artifact unchanged from base is checked in full and one that carries none is not checked against a committed bundle at all
  • r-macosevidence c2d0256 — the fixture and the assertions were exercised locally, so what is proven is that this mechanism fails under the restored defect on Darwin -- the job itself has still only ever run in this pull request
  • r-macosupgrade c9b0fca — the ubuntu twin is not added, and the review that suggested it was working from a claim in Committing dist is what makes the checkout the product — and what makes every second PR rebuild #719 that does not hold -- test/install-script.test.ts spawns the real installer, plants a current symlink and asserts readlinkSync plus the stray-link absence, so the GNU path is already pinned end to end
  • r-protoown cac5cde — the evidence block is the other four-way duplicate and is untouched here -- check-readme-numbers.mjs still owns it in all four READMEs, and moving it is its own change with its own negative control
  • r-plantmore722 f698236 — a shim that exits 0 for everything proves a branch was reached, not that the real host would agree -- these are detection assertions, not integration ones
  • r-plantmore722 f698236 — gemini-cli, windsurf and opencode still rest on the cursor fixture's shape rather than their own rows
  • r-plantedexit722 b6e28a9 — the second case's non-zero code is asserted, not its value -- a change from 1 to 2 passes here
  • r-plantedhost722 b6e28a9 — one host of seven, on one platform. gemini-cli, windsurf and opencode take the same JSON path and are not planted; codex, hermes and claude-code take different ones and are not covered at all
  • r-plantedhost722 b6e28a9 — this executes wiring, not a real agent -- a planted shim is a file the job wrote, and it says nothing about what Cursor itself does with the registration
  • r-rel110 d9a041f — this release does not make host wiring work on Windows -- detection still cannot see a .cmd and spawn still cannot run one (Windows: every detected host fails to wire — the temp filename carries the whole path, and hasCommand cannot see a .cmd #716)
  • r-rel110 d9a041f — 1.0.0 through 1.0.2 have no CHANGELOG entries; a pointer to the releases page stands in rather than reconstructing them
  • r-rellock110 d9a041f — nineteen version surfaces was already wrong before this -- the lockfile makes it twenty-one, and the count is only ever known after the gate says so
  • r-relmanifest110 d9a041f — this is the release commit's own repair, not a fix -- the next release will need the same regeneration for the same reason
  • r-childtreediag640 69c98a0 — the diagnosis is printed, not asserted — the next failure explains itself but the step still cannot say which outcome is acceptable
  • r-childtreepre640 69c98a0 — windows-latest is the only evidence for this path; nothing here was verified locally
  • r-coldstart640 69c98a0 — this measures the launcher's own child; it does not measure when cmd.exe started, so a stall before node is attributed to the same interval
  • r-launchernode640 69c98a0 — the earlier commit raising the probe budget to 15s stands on its own measurement — a healthy probe used 4478ms of 5000ms — but its message claimed that budget explained this flakiness, and it did not
  • r-launcherwhy640 69c98a0 — the deadline explanation is two independent diagnoses and a margin measurement, not a reproduction — the confirming evidence is the next run's own output
  • r-mcpidentity572 69c98a0 — Windows behaviour here rests on windows-latest runs and nothing else; a slower machine than three times the measured passing case will still be told it could not be verified, which is at least true
  • r-nodedrive640 69c98a0 — this reproduces the spawn, not the whole probe; a silent result narrows the cause without naming the fix
  • r-probefacts572 69c98a0 — the close-stdin outcome is measured, not guaranteed by the shell -- a platform whose shell defers the redirect past the parent's first write would produce the timeout code instead, and this case would then need its own seam rather than a looser assertion
  • r-proberace572 69c98a0 — this leaves one outcome unpinned, so a regression that swapped closed-input for timed-out on that fixture would not be caught here -- the four other cases still pin their codes exactly
  • r-reclaimnogate640 69c98a0 — windows-latest is the only evidence for this path, and one green run does not settle a race — this needs to hold across runs before The MCP probe's Windows behaviour is unverified: PATHEXT resolution and child-tree cleanup #640 can be called closed
  • r-sawinput640 69c98a0 — this records the arrival, not the response; if bytes arrive and nothing comes back, the next question is the probe's reader
  • r-slownotbroken640 69c98a0 — fifteen seconds is calibrated against one Windows runner's measurement; a machine slower than three times that will still be told it could not be verified, which is at least now true
  • r-canon605 f474cf4 — esbuild resolves a platform-specific binary
  • r-assertfile1 8a859c6 — this checks the report validate produced; it does not re-derive the range, so a range that silently covered nothing would pass here
  • r-pretag01 86e0153registers_commitlore reads the key, so a config that registers under a different key -- a host with its own naming -- still reads as unregistered and is wired again
  • r-dogfoodref1 f605dbb — this asserts the two checks validate performs; a third class added later is not required by name until somebody adds it here
  • r-mslquote1 5b23c44 — this is the second quoting layer in this step; a third -- a value with a double quote in it -- would need a different mechanism than more escaping
  • r-engfloor01 fe83524 — the parser covers the range shapes npm packages actually publish -- comparators like >=22 <23, and pre-release identifiers, are read by their first version and not by their bounds
  • r-readyhosts1 9db3c4d — the new jobs establish that an install runs and answers on those hosts, not that every command behaves identically there
  • r-wrongtag1 99667f3 — PowerShell cannot be run here, so this is reasoned from the shared design and proven against the shell twin; only the windows-latest job is evidence for install.ps1
  • r-vbind001 2c88d24 — this binds the requested tag to the runtime that answers, not the tag to its content -- a tag moved after publication installs whatever it now points at, which is a signing question rather than a version-binding one
  • r-insttxn1 afb7bfb — this establishes that the installed tree is complete and its commands run on this machine at this moment -- not that the machine will still have a working node tomorrow, and not that any agent host will load what was installed
  • r-pindigest c5a7cfa — pinning fixes the tree, not its behaviour -- a pinned action still runs with the job's token and can read this repository, and upgrading now requires deliberately resolving a new digest rather than inheriting one
  • r-pubprereq 8ffb31c — the gates qualify the commit and its CI, not the tag's authorship or timing -- anyone who can push a v* tag to a qualified commit on main can still publish, and expiring or re-pointing a tag after these jobs pass is outside what any of them observe
  • r-benchscope 67f4375 — nothing checks the shape of the eight metric-row files. This gate names them and steps over them, and bench/deterministic/types.ts is the only definition that family has -- there is no JSON schema for it, so drift on that side is still invisible
  • r-benchscope 67f4375 — the pre-provenance exemption reads started_at, which is data on the row rather than a fact about the file. A row that misreported it would be held to the shorter list of requirements; that is a deliberate falsification rather than the omission this fixes, and nothing here detects it
  • r-winsupported1124 6333251 — repositories that installed the hook before Windows: the commit-msg hook hangs instead of returning, and #71's containment can never match there #321 keep the old stub and must re-run commitlore hooks install; a corrected release does not reach them, and this row's claim is about a working install
  • r-winderive1124 282693b — a diagnostic that retypes the code it describes goes stale silently, because nothing fails when the two drift apart -- it has to be read out of the artifact that ships
  • r-winbound1124 156deed — an unbounded hang is not evidence; it is a job that dies at the runner timeout having printed nothing, so every hook-invoking commit here has to carry its own bound
  • r-winassert1124 616005d — an assertion whose only oracle is an absent side effect cannot distinguish "refused" from "the mechanism never worked", so each one needs a positive control that fires before the attack
  • r-winshell1124 aa68a9a — a GitHub Actions bash step is invoked as bash -eo pipefail, so set -uo pipefail inside the step does not clear -e and any bare command that fails ends the step at that line
  • r-ps1scope282 72f23df — the hook-runtime probe spawns /bin/sh, so it cannot report on a Windows hook at all until T-1124 changes it
  • r-ps1shell282 97735d6 — a step's shell key takes no expression, so a per-host matrix cannot select the interpreter
  • r-t1120nodeinst 14deeb4 — git and node are hard prerequisites now, so a host without them installs nothing and says which one is missing
  • r-instpath119 9e1fce7 — a user who ignores the printed line still gets "not found" on the next command
  • r-7a3e91 cf859e4 — better-sqlite3 stays external because it is native — the bundle degrades to --no-index without it, which only works because r-6f2a08 made that load lazy first
  • r-3b8f52 1f8b4be — the figure is one machine, one run — it establishes the order of magnitude, not a regression baseline anyone should tune against
  • r-0c5d38 aeb54a6 — the suite runs against the source tree, so no test in it can observe what packaging drops
  • r-6f2e58 ea9ae6d — a library test and a binary test cover different failures, and the packaging layer between them is exactly where a working module becomes a broken install
  • r-7e5f02 e5f5e00 — npm installs through an engine mismatch, so the ecosystem's own signal cannot be relied on to stop anything
  • r-3a9d68 6a3fc3b — a test runner reports what ran, and nothing in its summary distinguishes "did not run" from "does not exist"
  • r-8e2d51 ef93c0e — git is the parser, so a git version that folds or bounds trailers differently is a correctness problem for us, not a compatibility footnote

Ruled out (128)

  • r-reqchecksfixture 17a580b — adding guard-mutations to each fixture's copy of the list | it fixes today and leaves the next addition to break the same twenty tests
  • r-reqchecksfixture 17a580b — dropping guard-mutations from REQUIRED_CHECKS so the suites pass | the job exists to be required, and a required check the release gate does not know about is the gap this list closes
  • r-guardratchet 26b1989 — leaving the job in the gate while it is red | branch protection would block every merge until someone deleted the job, and deleting it removes the only thing that can see these gaps
  • r-guardratchet 26b1989 — allowing an improvement without updating the baseline | the record would drift below the measurement, and a baseline that overstates the gaps is as useless as one that understates them
  • r-guardratchet 26b1989 — recording gaps as a count instead of per property with a reason | a count cannot distinguish a control nobody wrote from one that cannot exist, and those need different work
  • r-guardmutation 551921d — fixing the inert guards in the same change | a runner that has never reported a real failure is not known to report one, and the red run is the evidence that it can
  • r-guardmutation 551921d — indexing guard functions instead of claims | that reproduces the exact failure this exists to stop, because the control comes back out of the implementation it is meant to test
  • r-guardmutation 551921d — treating an unexpressible control as a skip | it is indistinguishable from a control nobody attempted, and both were silently green before
  • r-guardmutation 551921d — folding this into the check job | it runs vitest once per mutation, so it belongs in its own job where its cost is visible
  • r-ocimatrixci 6c738c8 — asserting the image-digest refusal anyway | the runner does not implement it, so the assertion would have described an intention; naming it unverified keeps the gap visible until the runner changes
  • r-ocimatrixci 6c738c8 — gating the matrix on the daemon alone | it turned "cannot run here" into a red suite on any machine with Docker and an interactive sudo, because beforeAll runs under a skipped describe
  • r-ocimatrixci 6c738c8 — letting the job skip on CI when a daemon is missing | a required check that executed nothing is worse than an absent one, because it reports success
  • r-lintcontextname 3e78376 — asserting the name equals "lint" | the job has no name: today and should not gain one, so the property is its absence; asserting equality would invite someone to add name: lint, which is redundant and one edit from wrong
  • r-lintcontextname 3e78376 — leaving it because renaming the id is already caught | that catches one of the two ways the context can stop reporting, and the uncaught one leaves a file that still reads correctly
  • r-lintpullrequesttrigger 7617436 — asserting the trigger inside the release gate instead | that gate deliberately does not know about lint, and teaching it would blur the reason it requires ten
  • r-lintpullrequesttrigger 7617436 — making lint report on pushes to main so it is visible there | it would then run on every squash for no purpose, and the release gate would have to start expecting a context it was written to do without
  • r-cifaningateguards a22353e — exempting gate from the unconditional rule by name | the next job with a condition would be exempted by the next person for the same reason, and the rule would become a list of names rather than a property
  • r-cifaningateguards a22353e — dropping if: always() so the existing rule passes unchanged | that is the defect the gate exists to prevent, adopted to keep a test green
  • r-cifaningateguards a22353e — replacing the ten entries in REQUIRED_CHECKS with gate alone | the release gate would then accept a run where the ten never reported, since it verifies the names it is given rather than what gate transitively required
  • r-plantmore722 f698236 — plant hermes too | its wiring never runs a hermes binary, so the fixture would exercise nothing that distinguishes it
  • r-plantedhost722 b6e28a9 — assert on the existing "hosts":[] output | it is the same on a working installer and a deleted one, which is the defect this replaces
  • r-plantedhost722 b6e28a9 — a stub that always registers cleanly | it proves the installer reports what the stub told it, and cannot fail, so it restores the vacancy under a new name
  • r-rel110 d9a041f — fold the second Windows cause into this release | it arrived as Fix Windows host resolution and batch spawning (#716) #720 with real Windows evidence and needs its own judgement, and holding this back would make the note about what is still broken false in both directions
  • r-rellock110 d9a041f — replace the version string throughout the lockfile | it matches four dependencies that are really at 1.0.2, and nothing in the suite would have caught it
  • r-bundleprobe636 69c98a0 — sidecar MCP helper | it is outside the declared single-file runtime
  • r-childtreediag640 69c98a0 — guess between the two suspects and change the fixture again | the previous change was a reasonable guess that did not hold, and a second guess costs another Windows round trip with no more information than the first
  • r-childtreepre640 69c98a0 — weaken or delete the failing step | it is the only coverage of Windows child-tree reclamation in the repository, and no unit test touches taskkill
  • r-coldstart640 69c98a0 — read the node-spawn driver's silence as proof the spawn loses bytes | it shares the same five-second window, so slowness would make it silent too and I would have read a timing result as a delivery result
  • r-initiator636 69c98a0 — asset readiness preflight | F-002 and prepare_capture fails with ENOENT on a dev-hash asset root that is never installed #633/commitlore_prepare_capture fails with ENOENT on a 0.6.0 spec path; the Claude plugin cache only has 0.8.0 and the codex cache has 0.8.2 #635 own that separate runtime check
  • r-launchernode640 69c98a0 — keep the hard-coded path and widen the timeout | no budget reaches an interpreter that is not on disk, and three attempts had already shown that
  • r-launchernode640 69c98a0 — treat this as the product defect | a registration naming an interpreter that does not exist should be reported unhealthy, and the probe running the registered command verbatim is the behaviour to keep
  • r-launcherwhy640 69c98a0 — revert to the hard-coded path | it works on this image and testing against the Node the job installed is still the more honest fixture; the correction here is to the reason, not to the code
  • r-mcpid572 69c98a0 — a separate MCP protocol probe | reuse the Both installers still skip a host whose registration cannot run, and exit 0 #595 implementation to prevent drift
  • r-mcpidentity572 69c98a0 — widen the budget and keep the wording | the message would still send an operator to repair something that works, just less often
  • r-mcpidentity572 69c98a0 — retry the probe | a retry pays another full budget to answer a question the first attempt already answered, and doubles doctor's worst case
  • r-mcpidentity572 69c98a0 — gate the child-tree assertion on the server being healthy first | reclamation runs on every probe exit including the timeout, so requiring health made the precondition depend on the behaviour under test and could not tell a broken product from a broken fixture
  • r-nodedrive640 69c98a0 — instrument mcp-probe.ts itself | product code should not carry diagnostics for one platform's fixture, and the mechanism is reproducible without touching it
  • r-probefacts572 69c98a0 — accepting either code in one case | it makes the regression pass whether or not the probe still separates the two facts, which is the property under test
  • r-proberace572 69c98a0 — accepting either code in both fixtures | that is what makes the pair meaningless, since neither would then hold its own fact
  • r-proberace572 69c98a0 — forcing determinism by having the fixture signal readiness before closing stdin | the probe writes immediately on spawn, so there is no point at which a fixture could report ready first without changing the protocol
  • r-probesync572 69c98a0 — treating stdout as completion | spawnSync waits for helper exit after stdout
  • r-reclaimnogate640 69c98a0 — keep gating on probe status and retry the step | the gate races on cmd.exe stdin ownership, so retries would convert a real signal into a slower coin flip
  • r-reclaimnogate640 69c98a0 — mark the path unsupported | reclamation is implemented and unconditional in the source; what was missing was a test that could reach it
  • r-sawinput640 69c98a0 — drive the wrapper through a pipe instead of a file as the discriminator | PowerShell closes the pipe after writing, so it would test EOF rather than the held-open pipe the probe uses
  • r-slownotbroken640 69c98a0 — widen the window and keep the wording | the message would still tell an operator to repair something that works, just less often
  • r-slownotbroken640 69c98a0 — retry the probe | a retry pays another full budget to answer a question the first attempt already answered — the server was not reached in time — and doubles doctor's worst case
  • r-slownotbroken640 69c98a0 — keep the sidecar's outer bound as a constant | it was 7000ms against a 5000ms budget, so raising one without the other would kill the helper before it could answer and report that death as the server's fault
  • r-stub572 69c98a0 — retaining the fixed one-line stub | it is not an MCP server and the new probe correctly rejects it
  • r-trimdiag640 69c98a0 — keep the drivers for the next investigation | they are cheap to write again from the pull request comment, and a permanent cost for a question already answered is the wrong trade
  • r-winprobe640 69c98a0 — process manager framework | taskkill is a bounded Windows helper
  • r-canon605 f474cf4 — remove the legacy dist sidecars now | that changes test harnesses and needs a separate reviewed reduction
  • r-verdict607 5fcae0c — check-run names and app slugs | they do not bind a result to a workflow run or attempt
  • r-engfloor01 fe83524 — adding semver as a dependency to parse this | one regex over a handful of published shapes does not justify a runtime dependency in a check that runs before install
  • r-vbind001 2c88d24 — deleting a checkout that fails the version check | a directory the installer cannot identify may not be its own, and refusing costs an operator one command while destroying it may cost them something unrecoverable
  • r-insttxn1 afb7bfb — treating "verification could not run" as fatal | it fails a good install over a missing optional tool, which is the failure the non-fatal policy was introduced to stop
  • r-pindigest c5a7cfa — upgrading to the latest major while pinning | two changes in one, and a version bump that arrives inside a security fix gets reviewed as a security fix
  • r-pindigest c5a7cfa — a dependency bot to keep digests current | the pin is what removes silent movement, and a bot that updates it automatically restores the property being removed
  • r-pubprereq 8ffb31c — expressing either check as an if: on publish | a condition in YAML has no test, and a release gate whose logic cannot fail in a suite is a claim rather than a control
  • r-pubprereq 8ffb31c — treating a missing required check as nothing to report | that is the empty-set inversion above, and it is exactly how a release with no CI at all would have published
  • r-benchscope 67f4375 — A declared list of result files, in the gate or in a manifest | opt-in under another name, and a file left off it is silently ungated -- which is precisely the failure bench/verify.mjs is not run by anything, so the result schema drifted unnoticed #392 reports
  • r-benchscope 67f4375 — Validating only the three files that pass today | a description of the present rather than a rule, and it goes stale the first time a matrix is added or the schema moves
  • r-benchscope 67f4375 — A naming convention on results filenames | the runner's default output is an ISO stamp plus hex and the metric writers' is a family name plus an ISO stamp, but every run-record file in the tree was named by hand through --out (t702-m2, m5-off-design-20-tasks), so the convention is enforced nowhere and cannot carry a gate
  • r-benchscope 67f4375 — A marker file dropped beside each gated result | still a step somebody has to remember, and a forgotten step is the defect being fixed
  • r-benchscope 67f4375 — Widening result.schema.json to describe both row families | they share no required field, so the union would accept a run record with no task and a metric row with no metric, which is a schema that rejects nothing
  • r-benchscope 67f4375 — Making harness_commit and dist_digest optional in the schema | weakens the gate for every future row in order to admit six historical files, and Make per-turn token usage observable, and say what it still cannot price #390's drift was an additionalProperties failure that a laxer schema catches later rather than sooner
  • r-benchscope 67f4375 — Skipping the six pre-provenance files whole | cheaper to write, but it stops checking types, patterns and the closed property set on 380 rows in order to excuse two absent fields
  • r-winsupported1124 6333251 — Flipping the cell on the throwaway verification head | the ticket requires the attacks to pass in a required job in this ticket's own pull request, and a head that exists to be discarded is not that
  • r-winsupported1124 6333251 — Leaving the assertion pinned and editing only the document | the two contradict each other, so the suite would fail; a guard that blocks the change it was guarding for is a defect in the guard
  • r-winsupported1124 6333251 — Widening this ticket to the hook fix so it could be self-contained | the stop condition sent that to T-1127 for good reason, and this ticket's job was to measure and then say the word once measurement allowed it
  • r-winderive1124 282693b — Updating the two hardcoded lines to match the fixed stub | it would be correct today and wrong at the next change, for the same reason it is wrong now
  • r-winderive1124 282693b — Dropping the two diagnostics and relying on the commit alone | the commit says whether the hook worked, not which branch decided it, and that is the reading the next Windows question starts from
  • r-winbound1124 156deed — Fixing the walk in src/hooks/commit-msg.ts as part of this change | the stub is not this ticket's to own, and the ticket says to record a Windows failure as its own issue rather than absorb it
  • r-winbound1124 156deed — Instrumenting the stub to report its internal state | same ownership objection, and the chained-hook slot already reads the same environment without touching it
  • r-winbound1124 156deed — Raising the job timeout and letting the hang run to completion | it never completes, and the assertions behind it would never execute
  • r-winassert1124 616005d — Keeping the extensionless payload for attack 2 | the stub execs a recorded value only inside the .mjs|.js arm, so an extensionless target is refused by a branch that was never going to run it, and the step asserted its own control flow back to itself
  • r-winassert1124 616005d — Calling attack 2 evidence of containment once the payload is a .cmd | it guards a loosening, it does not exercise the install-root check; conflating the two is what would let a green sticker land on an untested branch
  • r-winassert1124 616005d — Keeping the chained-hook slot as the probe | it was never invoked, and a scratch repo with its own hook measures the platform without depending on the stub's argv0 handling
  • r-winassert1124 616005d — A trailing rm for the planted file | it is skipped by every early exit, and the install root is shared by every later step on the runner
  • r-winshell1124 aa68a9a — set +e at the top of each step | it also disables the abort for genuinely unexpected failures, where stopping is the right answer; the || form is scoped to the one command whose failure is a measurement
  • r-winshell1124 aa68a9a — Reading the outcome from the step's own exit code instead | 124 tells you a bound fired somewhere, not which commit, and it discards the probe output that explains why
  • r-ps1scope282 72f23df — Making the hook probe pass by installing a shim named commitlore with no extension | Windows will not execute it, and inventing a second artifact to satisfy a probe is the wrong direction; the probe is what needs to learn about Windows
  • r-ps1scope282 72f23df — Committing through the hook in this job to prove the hook works | the hook path on Windows is exactly what is unestablished, so the assertion would be testing T-1124's subject in T-1121's job
  • r-ps1scope282 72f23df — Dropping the doctor call so the log stays clean | its output is the evidence T-1124 starts from, and a clean log that hides it would cost the next ticket the finding
  • r-ps1shell282 97735d6 — Duplicating the job once per host | eighty lines twice, kept in step by hand, to assert the same things about two interpreters
  • r-ps1shell282 97735d6 — Running everything through cmd with the host as a variable | every assertion would become a quoted one-liner passed to -Command, which is the least readable form of the most important job here
  • r-ps1shell282 97735d6 — Testing only PowerShell 7 | 5.1 is what a Windows machine has before anyone installs anything, and a 7-only construct is a parse error for the whole file
  • r-instps1121 7bf6ced — Writing the user-scope PATH with SetEnvironmentVariable | two active records on install.sh reject an installer that changes the user's environment behind their back, and the Windows registry spelling of that act is not a different decision
  • r-instps1121 7bf6ced — An extensionless shim matching the shell wrapper's name | Windows will not execute it, and the git hook resolves through recorded config rather than PATH, so the name would buy nothing
  • r-instps1121 7bf6ced — Testing only under PowerShell 7 | 5.1 is the version a Windows machine already has, and a 7-only construct is a parse error for the whole file rather than a failure in one branch
  • r-instps1121 7bf6ced — Requiring a helper for JSON merges the way install.sh needs jq | PowerShell parses and writes JSON natively, so the cannot-merge branch has no counterpart and inventing one would be a copied limitation
  • r-instps1121 7bf6ced — Passing the version through $args so irm | iex could take one | iex gives a piped script no arguments, so the pinned form is documented as a scriptblock invocation and COMMITLORE_VERSION is accepted as well
  • r-nobin284 ab2f08f — Keeping the binary classification arm for the installer's wrapper | the wrapper is a shell script that execs node, so trusting it by name would re-admit any extensionless executable called commitlore while the bundle it runs is the thing with an interpreter to check
  • r-nobin284 ab2f08f — Relaxing the dogfood Evidence check to allow a missing path | it exists to catch a citation nobody can follow, and dropping the check to accommodate a deletion would retire the guarantee instead of correcting the question it asked
  • r-nobin284 ab2f08f — Leaving src/core/paths.ts alone because ADR-0026's inventory did not name it | it was the only remaining code that read files out of a compiled artifact, so leaving it would have left the invariant false while the test asserting it passed
  • r-nobin284 ab2f08f — Keeping /dist/commitlore in gitignore in case a binary returns | an ignore rule for an artifact nothing produces is a claim that something might, and the ADR says otherwise
  • r-t1120fatal 0412668 — Reading the remaining failure by inference from the generic advice line | two cycles already went that way, and the informative line was present in the log the whole time
  • r-t1120cijob 1871106 — Keeping the separate staged clone and adding more mount configuration | each addition would be configuration protecting a copy that exists for no reason once the checkout can be cloned directly
  • r-t1120cijob 1871106 — Dropping the container steps and relying on the vitest suite | the suite runs on a machine that has node and git; the point of a bare image is that the prerequisite is genuinely absent, which is the assertion the active record on this file asks for
  • r-t1120clonemsg 55ffb0a — Keeping the guessed message and fixing only the CI mount | the guess would still be printed for every other clone failure, and this one was found only because a container happened to hit it
  • r-t1120clonemsg 55ffb0a — Letting the clone's stderr through to the terminal unfiltered | it interleaves with the installer's own log lines and a piped installer should speak in one voice; the last lines quoted into one named error keep both
  • r-t1120nodeinst 14deeb4 — Hardcoding the release version in install.sh as the default | it would add a bump obligation to every release and a stale default between a tag and the bump; git ls-remote resolves the newest tag with no API token and no rate limit
  • r-t1120nodeinst 14deeb4 — Defaulting to a branch when no tag is given | installing a moving target is what pinning exists to prevent, and the release gate already forbids a dev-referencing one-liner
  • r-t1120nodeinst 14deeb4 — Writing the wrapper with cp over the destination | an in-place overwrite of a file that may be executing is the defect that forced a same-day patch release; the wrapper is written beside the target and renamed
  • r-t1120nodeinst 14deeb4 — Letting post-install verification decide the exit code | that was the other half of the same defect, where a killed --version became the installer's exit status; verification now retries once and reports, and an install that succeeded exits 0
  • r-t1120nodeinst 14deeb4 — Editing the shell profile when the wrapper directory is not on PATH | an active record on this file rejects it, and printing the line is what this script does about PATH
  • r-t1120nodeinst 14deeb4 — A test-only environment hook to force a verification failure | a source repository whose bundle exits non-zero is a real broken release and needs no seam in shipped code
  • r-t1120nodeinst 14deeb4 — Checking for git with command -v alone | a git that cannot execute is as useless here as a missing one, so the check runs git --version and catches both
  • r-instpath119 9e1fce7 — editing the user's shell profile from the installer | printing the line is honest, rewriting .bashrc silently is what makes people distrust curl-to-shell installers, and this project sells verifiability
  • r-instci99a 318b9a2 — pre-installing curl/jq/tar/git in the containers before running install.sh | would hide exactly the gap this work exists to find — Debian ships neither curl nor wget by default, and asserting that failure cleanly is the point of the bare debian:stable-slim step, not an obstacle to route around
  • r-instci99a 318b9a2 — publishing a -musl release target so Alpine could install successfully | a release.yml/build-matrix change, not an install.sh or CI-verification fix; out of scope here (DO NOT: no Docker in the release build matrix) and orthogonal to making the existing failure clear instead of a raw crash
  • r-instci99a 318b9a2 — heuristically detecting musl before download (checking for /lib/ld-musl-*.so.1) | executing the freshly extracted, already checksum-verified binary directly is a stronger, more general check — it catches any reason the binary cannot run on this machine, not only the one musl signature this repository happens to know the path for
  • r-instci99a 318b9a2 — a matrix job splitting debian/alpine | the two images need different assertions (debian: bare failure, then success once curl is added; alpine: one clean failure, nothing more to add) and share one staged release + HTTP server; one job with named, ordered steps keeps that shared setup and reads linearly in the log
  • r-5e91c7 f991389 — leaving pull_request as the only check for develop | a direct push to an integration branch is exactly what nobody reviews
  • r-7a3e91 cf859e4 — inlining spec/SPEC.md and the schema into the bundle | SPEC.md would need a codegen step that itself needs a drift guard, and the package-root walk removes the reason to want it
  • r-7a3e91 cf859e4 — replacing the tsc output with the bundle | test/cli.test.ts, test/hooks.test.ts and test/mcp.test.ts import dist internals by path
  • r-3b8f52 1f8b4be — enabling the large fixture for the whole default suite | generating 100k commits costs ~22s and the other tests in that file gain nothing from it, so it runs as its own step against its own file
  • r-7f31c9 750ab17 — reporting both datasets from one source list | readSources groups by condition and cannot separate repositories, so any second dataset with a commitlore-on arm silently corrupts the headline test
  • r-0c5d38 aeb54a6 — smoke with --version only | it passes with the schema missing, which is the failure worth catching
  • r-0c5d38 aeb54a6 — assert a file list in package.json | it goes stale silently, and installing is the actual question
  • r-6f2e58 ea9ae6d — pin the adoption sha in the workflow | it is the same stale cutoff the dogfooding suite was written to avoid
  • r-6f2e58 ea9ae6d — spawn git per commit to read trailers | one pass with %(trailers:key=…) uses git's parser and does not scale with history
  • r-7e5f02 e5f5e00 — hold Node 20 and downgrade both dependencies | paying maintenance cost to keep supporting an unpatched runtime
  • r-7e5f02 e5f5e00 — floor at 24 | excludes Node 22 users today for no benefit, and 22 outlives v0.1.0
  • r-7e5f02 e5f5e00 — drop engines entirely | the failure moves from install time to runtime, where it is someone else's confusing bug
  • r-3a9d68 6a3fc3b — pin an expected test count | it goes stale on the next file added, and a stale count reads as passing
  • r-3a9d68 6a3fc3b — keep threads and skip the index tests in CI | that is the same hole with the sign flipped, made deliberate
  • r-8e2d51 ef93c0e — pin one git version in CI | it would hide exactly the divergence the matrix exists to surface
  • r-8e2d51 ef93c0e — allow shallow checkout and skip dogfooding | a skipped gate reports the same green as a passing one

Warnings (15)

  • r-pindigest c5a7cfa (claim) — no comment can keep a digest honest, so the # v4 beside each is a label rather than a check; the version it names is verified once, here
  • r-pubprereq 8ffb31c (claim) — the required-check list is a literal, so a job renamed in the CI workflow becomes a check this gate waits for and never finds -- it fails closed, which is right, and the failure will read as a CI problem rather than a rename
  • r-instps1121 7bf6ced (claim) — nothing on this machine can execute this file -- there is no PowerShell here, so every claim about its behaviour comes from the Windows runner in this pull request rather than from a local run
  • r-nobin284 ab2f08f (claim) — an extensionless COMMITLORE_BIN is now ignored rather than honoured -- the hook falls through to the recorded install and to PATH, so a caller who pointed it at the wrapper still gets a working hook, by a different route
  • r-t1120nodeinst 14deeb4 (claim) — keep install.sh ASCII only -- a non-ASCII character in a string silently terminated /bin/sh here while sh -n accepted the file
  • r-instci99a 318b9a2 (claim) — the new binary-execution guard's die() message names musl/glibc specifically because that is the only way this repository's own release actually fails to execute today (glibc-only "-gnu" targets, verified against Alpine). If a -musl target is ever published, or the binary can fail to execute for an unrelated reason, this message will misdiagnose it — revisit the wording if that guard ever needs to distinguish causes.
  • r-7f24a9 96d960a (claim) — this file is still NOT validated as a GitHub workflow. actionlint is unavailable and the sandbox cannot reach the API. Conflating a YAML parse with workflow validation broke gitseed's CI earlier today; the first run is the proof
  • r-7f24a9 96d960a (claim) — the same task was told to update the factory skill's Gitflow section and its phase-gate.py, and could not — it was launched rooted at this repository and had no write access to ~/.claude/skills. The gate still checks for a branch named develop and fails both repositories for having dev. My spec error, not the delegate's; it is the next task
  • r-5e91c7 f991389 (claim) — these files are NOT validated as GitHub workflows. actionlint is unavailable and the sandbox cannot reach the API, so only the YAML parse and structural assertions ran. Conflating those two broke gitseed's CI four commits ago; the first GitHub run is the proof
  • r-7a3e91 cf859e4 (claim) — hardcoding ../ counts back to the package root is what broke this — new code reads assets through installedPath(), never through import.meta.url
  • r-7f31c9 750ab17 (claim) — adding a file to README_SOURCES pools it into every aggregate in the block, including the significance test — check the arm names first
  • r-0c5d38 aeb54a6 (claim) — spec/ ships on purpose -- alternative implementations are told the fixtures are the contract, and this step is what keeps that true
  • r-7e5f02 e5f5e00 (claim) — raising the floor again needs an ADR superseding ADR-0010 -- a package.json-only change will pass tests and lie to users
  • r-3a9d68 6a3fc3b (claim) — any native addon added later needs the fork pool -- moving back to threads will silently drop whatever loads it
  • r-8e2d51 ef93c0e (claim) — keep fetch-depth 0 on any job that runs npm test -- the dogfooding suite fails deliberately on a shallow clone, and "fixing" that by relaxing the assertion removes the gate

git log --follow accepts exactly one pathspec, so renames are not followed for 6 paths; query one path at a time to follow its rename chain

withheld the content of 4 record(s) graded blocked: a Limit, Ruled-out, Verified trailers matching an injection pattern is reported, never quoted (SPEC §7)

Trailer violations fail this check. Active constraints are informational — they are what the repository already decided, not a verdict on this PR.

…list of eleven

`REQUIRED_CHECKS` names the jobs a release must see. The release-gate suites built their
job fixtures from a hand-written copy of that list, so adding `guard-mutations` made the
two disagree: the verifier reported the missing job first, and every test asserting a
specific downstream message never reached it.

    × refuses a required job that failed
      → expected 'ERROR: exact-head CI did not pass for…' to contain 'conclusion "failure"'

A list written beside the thing it checks agrees with itself by construction. That is the
shape already recorded in this repository from `check (22.23.2)` and from `lint`'s
pull_request trigger, and it is why a one-line addition took twenty tests down.

The fixtures derive their job lists from `REQUIRED_CHECKS` now, mutating only the single
job each case is about. A thirteenth required job will not touch them.

Record-Id: r-reqchecksfixture
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: adding guard-mutations to each fixture's copy of the list | it fixes today and leaves the next addition to break the same twenty tests
Ruled-out: dropping guard-mutations from REQUIRED_CHECKS so the suites pass | the job exists to be required, and a required check the release gate does not know about is the gap this list closes
Limit: the fixtures now agree with REQUIRED_CHECKS by construction, so a job that should be required but was never added to that list is still invisible here; the list itself remains the single named site
Verified: 36 cases across the ci-gate and release-prerequisite suites, both typechecks clean, and the mutation ratchet still exits 0 at 13 bound, 1 inert, 1 unavailable, 13 uncovered. Negative control observed and restored -- an unknown name added to REQUIRED_CHECKS leaves 19 exact-head cases and the tag-binding case passing, which is the property being bought
CommitLore-Version: 2.0.0
@MongLong0214
MongLong0214 merged commit 4a0c215 into main Aug 21, 2026
15 checks passed
@MongLong0214
MongLong0214 deleted the cdeb-guard-mutation-registry branch August 29, 2026 01:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant