fix(gate-61): an empty scope has two causes and only one of them is a diff (.github#347) - #371
Merged
Merged
Conversation
… diff (.github#347) `bin/hydra-gates` forwards `--base` only on a diff-scoped run, so on `--full` the runner keeps its own `origin/development` default and hands it over anyway. On `development` itself that diffs the branch against itself, the checker returns 3, and gate-61 printed NOT APPLICABLE — the diff against 'origin/development' put every post-event registration out of scope two lines after the run's own preamble said `Base ref: n/a — --full requested`. THERE WAS NO DIFF. Sibling of #361: check_listener_placement.py:555 carries the same non-empty argparse default. THE VERDICT DOES NOT CHANGE, AND THAT IS DELIBERATE. Sweeping the tree on an unscoped run was tried here before and reverted -- the BUILDER runs unscoped, so `--all` surfaces the fleet's whole registration backlog as blocking findings on every build. `na` with an HONEST reason is the third answer: it cannot go false-RED, and it stops the run claiming an exclusion nothing performed. BLAST RADIUS: ZERO. No verdict changes in any repo, at any scope. What changes is the sentence, and the fact that the size of what went unread is now stated. WHAT WAS ACTUALLY MISSING was that number. `0 of 1 inspected` and `0 of 45 inspected` printed identically. An advisory whole-tree sweep now supplies it -- informational, exit status discarded, and quoted only when the helper printed its own terminal summary, so a sweep that finds inherited debt cannot become this run's verdict and a sweep that CRASHES cannot either. Measured on openregister @ development: `--all` reaches 45 post-event registrations carrying 3 findings. That is exactly the number SHARED-LESSONS records as hidden by this defect, and it is now on the verdict line.⚠️ THE ADVISORY SUMMARY IS RECOMPOSED, NOT QUOTED. The helper's own line ends "…, 0 out of scope: 3 failure(s)", and pasting it in put the phrase `out of scope` straight back into a reason on a run that computed no scope -- the exact sentence this fix removes, smuggled in as a quotation. The suite's gate-agnostic property (a NOT APPLICABLE may not blame a diff on a --full run) caught it, listing gate-61 among the blamers again after the "fix". FIXTURE, observed red then green, same tree, only the scope flag changed: before 3 assertions RED -- "#347 is LIVE: gate-61 still claims 'the diff … put every post-event registration out of scope'", reason does not state the run computed no diff, backlog size not stated after 13 passed / 0 failed, and gate-61 drops off the "gates blaming a diff on a --full run" list, which now reads `6 7 8 9` The known-defect entry for #347 in test_gate_scope_matrix.sh is replaced by the positive assertions, as its own text demanded ("flip this assertion"). The gates 6/7/8/9 wording defect is a separate, still-unfiled entry and is untouched.
This was referenced Aug 11, 2026
rubenvdlinde
added a commit
that referenced
this pull request
Aug 12, 2026
* feat(gates)!: scan the whole tree by default — ADR-020 superseded The gates now scan the ENTIRE tracked codebase. Diff scoping becomes opt-in via --scope-to-diff / --diff / HYDRA_GATE_SCOPE=diff. Ruben's rationale: when a gate changes or is added, the next push to beta should fail unless the old code is brought up to the new standard, so gate updates travel with releases instead of applying only to code written after them. Recorded in hydra-gates/ADR-020-SUPERSEDED.md, including what it costs: the first development-to-beta run after this lands surfaces the fleet backlog at once, and the last wide-scope measurement was roughly 3,900 findings. The scope is now two independent, named inputs rather than one. File scope controls which files the state gates open and defaults to full; the delta base is resolved separately and is what the five delta gates compare against. Gates 16, 29, 47, 48 and 61 are keyed on whether a base was resolved, not on the file scope, so a pull request keeps every delta gate and gains whole-tree state coverage. Without a base they report NOT APPLICABLE by name, never PASS. An unresolvable base stays fatal on a diff-scoped run and is non-fatal at full scope, where it costs five gates rather than all sixty-four. BASE_REF no longer carries a hardcoded origin/development default, which was the shared root cause of the gate-16 and gate-61 empty-scope defects. Eighteen gates could report PASS over a scope they never opened. Full scope removes the empty set in the common case but leaves the fall-through latent, so it is fixed at the fall-through through one shared helper: gates 14, 17, 18, 20, 21, 22, 34 through 44 and 52. Gates 14 and 20 were found by sweeping the table and are not in the issue's list of sixteen. Gate 40's inline note arguing against this was false on both of its claims and is deleted with the fix; gate 52's uncomputed ratchet half and gate 53's uncomputed orphan-promotion half now say so on stdout instead of hiding behind a PASS. APP_DIR is absolutised before the cd and the resolved absolute path is printed. Gate 17 was the only gate handing APP_DIR to its checker afterwards, so a relative app-dir made the scan root resolve a second time against the app dir itself: same tree, absolute path FAIL 1, relative path PASS. CI was never exposed because bin/hydra-gates absolutises; the documented human invocation was. The acceptance suite now enforces the empty-scope property across the package instead of for seven named gates, and carries a relative-path arm the standing driver is structurally unable to express because it always builds absolute paths. Both new arms were proven able to fail by deliberate breakage. Helper suites 74 passed / 2 quarantined / 1 failed before and after, the failure being the pre-existing unresolvable-ajv needs-external skip that fails closed locally and is green in CI. Entry-point tests 61 passed / 0 failed before, 69 passed / 0 failed after. Refs: .github#374, #361, #347, #364, #371 * docs(gates): the runner header described the old default The Options block still said --scope-to-diff was the mode and named origin/development as the base default. Both are gone: full scope is the default, and BASE_REF starts empty because that hardcoded default is the shared root cause of .github#347 and #361. * fix(gates): a crashed gate-61 read as an empty scope, and gate-24 invented a count Two repairs, both from the same rule: a crashed checker must never read as an empty scope, and an empty scope must never read as a pass. gate-61 — the first draft of the delta-base change skipped the checker invocation entirely when no base was resolved and short-circuited to the empty-scope status. Nothing ran, so nothing could crash, and test_gate_crashed_checker_is_not_a_finding.sh caught it immediately: with a python3 that cannot run, gate-61 reported NOT APPLICABLE where it must report SKIPPED (wiring). That is the same regression #364's first draft made in gate-16, caught by the same suite, and the repair is the same one: always run the checker, evaluate wiring first and scope second. With no base the helper is now invoked in its whole-tree mode purely to learn whether it can run, and its findings are discarded — sweeping on a baseless run was tried before and reverted because the builder runs that way. gate-24 — the parity wrapper is app-owned and declares no terminal marker, so a wrapper that exited non-zero having written nothing parseable was reported as one parity violation: a fabricated number with a plausible message and nothing behind it, the same shape as gates 22, 25 and 28. The verdict deliberately stays FAIL, because nothing here can tell a crash from a differently-phrased finding and guessing wiring would turn a real parity failure into a green hole. It simply stops claiming a count nobody measured. gate-61 carries the same clamp idiom and it is left alone with the reason written down: its wiring branch sits immediately above, so a crash cannot reach it, and zero counted lines with a non-zero exit is format drift rather than a dead checker. Two suites updated, neither weakened. The crashed-checker suite now accepts gate-17's NOT APPLICABLE on its E2BIG arm only when the terminal count marker proves the interpreter ran over the oversized scope list, which is a stronger assertion than the two it sits beside; its verdict regex also gained the multi-word form that made NOT APPLICABLE parse as NOT. The route-auth suite's gate-14 expectation moves from PASS to NOT APPLICABLE for the same reason its gate-5 neighbour moved in August: a pass here is scoping, not absence, and that reclassification was only half done. Refs: .github#374, #330, #364, #379 * fix(gate-61): the scope-matrix invariant read the FILE scope, not whether a diff existed The gate-agnostic property — a NOT APPLICABLE may not blame a diff on a run that computed none — is exactly right and stays. Its TRIGGER was wrong: it keyed on --full, when the property is about whether a diff was COMPUTED. Those are two independent inputs now, so the suite was reading the wrong one. That is the same conflation .github#347 was, one layer up, inside the test written to catch it. It fired on this branch and it fired correctly. Arm 3 now removes the BASE rather than the file scope, so every assertion it already carried holds verbatim. Arm 3b covers what used to be unreachable and is now what every PR in the fleet runs: full file scope with a real base. There, naming the diff is a true statement, and what has to hold instead is the falsifiability condition whose absence made #347 unreadable for weeks — a reason may name a diff only on a run that PRINTED the base it diffed against, so a reader can check the claim rather than trust it. Asserted from both sides. The advisory whole-tree sweep now runs in BOTH empty-scope branches. It ran only on the no-base path, because that used to be the only way to reach a state where the size of the unread backlog mattered. Full scope is the default now, so "a base resolved and excluded every registration" is the common case, and the ambiguity the sweep was added to remove had simply moved to the path everyone runs: 0 of 1 and 0 of 45 still printed identically there. The whole point of the reversal is that inherited debt stops being invisible, and leaving this gate's backlog unstated on the common path would have exempted it from exactly that. Refs: .github#374, #347 * ci(gates): the unresolvable-base check asserted the contract this PR supersedes The required "Unresolvable base ref must exit 99 with no green" step encoded ADR-020's rule as the ONLY acceptable behaviour, so this PR could not merge: under the new default an unresolvable base no longer aborts, it falls back to full scope. The old assertion is kept BYTE-FOR-BYTE and simply run in the mode it was always about (--scope-to-diff): with an explicit diff request and no usable base, the runner must still exit 99, print NOTHING WAS CHECKED, and emit no gate lines. Every clause of it still holds. A second step covers the new default: the run must say the base was unresolvable, must still emit gate lines, and none of gates 16/29/47/48/61 may print PASS over a scope that was never read. Positive-controlled against this runner's real captured output four ways — injecting "[gate-16] ... PASS" fails it, deleting a gate line fails it, and the diff-mode arm fails if the 99/NOTHING WAS CHECKED behaviour regresses.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #347.
bin/hydra-gatesforwards--baseonly on a diff-scoped run, so on--fulltherunner keeps its own
origin/developmentdefault and hands it over anyway. Ondevelopmentitself that diffs the branch against itself, the checker returns 3,and gate-61 printed
two lines after the run's own preamble said
Base ref: n/a — --full requested.There was no diff. Sibling of
#361:check_listener_placement.py:555carriesthe same non-empty argparse default.
The verdict does not change, and that is deliberate
Sweeping the tree on an unscoped run was tried here before and reverted — the
runner's own invocation comment records it. The builder runs unscoped, so
--allsurfaces the fleet's whole registration backlog as blocking findings onevery build.
nawith an honest reason is the third answer: it cannot gofalse-RED, and it stops the run claiming an exclusion nothing performed.
Blast radius: zero. No verdict changes in any repo, at any scope.
What was actually missing was the number
0 of 1 inspectedand0 of 45 inspectedprinted identically. An advisorywhole-tree sweep now supplies it — informational, exit status discarded, and quoted
only when the helper printed its own terminal summary, so a sweep that finds
inherited debt cannot become this run's verdict and a sweep that crashes cannot
either.
Measured on openregister @ development:
--allreaches 45 post-eventregistrations carrying 3 findings — exactly the number SHARED-LESSONS records as
hidden by this defect. It is now on the verdict line.
The helper's own line ends
…, 0 out of scope: 3 failure(s). Pasting it in put thephrase
out of scopestraight back into a reason on a run that computed noscope — the exact sentence this fix removes, smuggled in as a quotation. The suite's
gate-agnostic property (a
NOT APPLICABLEmay not blame a diff on a--fullrun) caught it, listing gate-61 among the blamers again after the "fix". Worth
recording: the generic invariant caught a regression the gate-specific assertions
would have passed.
Fixture — red before, green after
Same tree, only the scope flag changed:
--fullrun" list, which now reads6 7 8 9The
_known_defectentry for#347is replaced by positive assertions, as its owntext demanded ("flip this assertion and delete the entry"). The gates 6/7/8/9
wording defect is a separate, still-unfiled entry and is untouched.
ShellCheck clean.