test(gate-16): the fixture #364 shipped without - #366
Merged
Conversation
#364 changed a verdict with no fixture behind it. This adds one, driven through the real bin/hydra-gates over a purpose-built two-commit repository, and observed RED before the change and GREEN after: before (81c8c97) 3 failed / 9 passed -- arm 3: "#361 is LIVE: gate-16 printed PASS on a --full run that computed no diff, over a tree whose two untagged methods the positive control just named" after (274b926) 0 failed / 12 passed Both of #364's load-bearing claims were re-derived here rather than taken on trust, by patching the runner and re-running this same fixture: * category `scope` instead of `na` -> [gate-16] FAIL - internal error: _skip called with reason category 'scope'. A fleet-wide false RED, exactly as the PR body says. The suite now asserts against it by name. * a full sweep on --full -> [gate-16] FAIL - 2 changed method(s) missing @SPEC, naming LegacyDebtController, which no author touched. ADR-020 violated. The suite also asserts the legacy surface never reaches the log, so a sweep cannot creep back in wearing a NOT APPLICABLE. REFINEMENT to the second claim, for the record: a LITERAL copy of gate-19's else-branch (dropping HYDRA_GATE_BASE_REF=) would not have swept at all -- check_spec_coverage.py:508 defaults the env lookup to 'origin/development', so it returns `# count=0` and changes nothing. The false-RED danger is real but arrives one line later, with the check_visual_coverage.py `or None` precedent. The arms, all through bin/hydra-gates on one tree: positive control --mode report names LegacyDebtController, 2 uncovered arm 1 diff, docs only PASS (ADR-020: inherited debt is not mine) arm 2 diff ADDS untagged work FAIL, and NAMES NewWorkController.php arm 3 --full, arm 1's tree NOT APPLICABLE, reason names the ABSENCE of a diff, no legacy finding in the log, gate-16 listed in the coverage summary's NA block Coverage ratchet: gate-16 moves from UNCOVERED.md (`needs-diff`) to COVERED-ELSEWHERE.md. 12 of 64 fixtured -> 13 of 64.
This was referenced Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#364(merged as 765d081) changed a gate verdict fleet-wide and shipped with nofixture behind it. This is the fixture, driven through the real
bin/hydra-gatesover a purpose-built two-commit repository, and observed RED before the change
and GREEN after:
81c8c97(pre-#364)765d0817(post-#364)Both of #364's load-bearing claims were re-derived, not taken on trust
Each by patching the runner and re-running this same fixture:
_skip 16 … scopeinstead ofna[gate-16] spec-coverage: FAIL — internal error: _skip called with reason category 'scope'— a fleet-wide false RED, exactly as #364 states. The suite now asserts against it by name.--full(the gate-19 else-branch shape)[gate-16] FAIL — 2 changed method(s) missing @spec, namingLegacyDebtController, which no author touched. ADR-020 violated.gate-19's else-branch — merely dropping
HYDRA_GATE_BASE_REF=— would not haveswept at all:
check_spec_coverage.py:508defaults the env lookup toorigin/development, so it returns# count=0and changes nothing. Thefalse-RED danger is real, but it arrives one line later, with the
check_visual_coverage.pyor Noneprecedent. Nothing in #364 is wrong; themechanism is one step removed from where the PR body places it.
The arms
All through
bin/hydra-gates, one tree, only the scope input changed:Arms 1 and 2 are what make arm 3 readable: without them a
NOT APPLICABLEisambiguous between "gate-16 is honest about an empty scope" and "this fixture
contains nothing to find".
Arm 3 asserts the absence of
LegacyDebtControllerfrom the gate's log, notjust the absence of a FAIL verdict — so a full sweep cannot creep back in wearing
a NOT APPLICABLE.
Coverage ratchet
gate-16 moves from
UNCOVERED.md(needs-diff, whose stated reason — "it needs apurpose-built two-commit history, not a fixture tree" — was accurate and is now
satisfied) to
COVERED-ELSEWHERE.md. 12 of 64 fixtured → 13 of 64.Verified green on
origin/mainat 765d081:test_gate16_spec_coverage_scope.sh12/12,
test_gate_acceptance_matrix.sh4/4,test_gate_crashed_checker_is_not_a_finding.shALL PASS, ShellCheck clean (koalaman/shellcheck:stable against the repo's own
.shellcheckrc).