fix(gates 35,40,42,44): four a11y gates excused themselves from a repo full of markup, and three reported PASS over a crashed checker - #272
Merged
rubenvdlinde merged 2 commits intoAug 8, 2026
Conversation
…o full of markup, and three reported PASS over a crashed checker Measured at package sha cdfbd7a against opencatalogi (93 .vue) and nldesign (zero .vue, one PHP template), one textbook true positive planted per gate in BOTH — the asymmetry that made #225/#261 possible. All 11 gates in the 34-44 band fired and named the plant in both arms, and all returned to their exact prior verdict on removal. Two defects survive that. 1. FOUR GATES GO `na` ON A TEMPLATES-ONLY REPO Gates 35, 40, 42 and 44 still guarded on `[ -d src ]` while 34/36/37/39/43 had moved to `_a11y_has_markup_dir`, and the central applicability table listed the whole family under `[ -d src ]`. On a repo with a `templates/` full of markup and no `src/`, same run, same files: gate-34/36/37/38/39/41/43 ran; four of them FAILED on the plants gate-35/40/42/44 NOT APPLICABLE — "this repo ships no frontend, so there is no .vue/.js/.ts source for this gate to inspect" `na` is the one verdict that removes a gate from coverage accounting, and the reason was contradicted by the same run's own output three lines above it. No fleet app is templates-only today; nldesign is one `rm` away, since its `src/` holds a single `manifest.json` — the exact shape that made twelve gates pass over nothing in #225. The guards now call `_a11y_has_markup_dir`, and the applicability declaration calls THE SAME FUNCTION rather than restating it, so the two cannot drift again. No third scope definition was added. 2. A CRASHED CHECKER REPORTED PASS (#147 / #249) — gates 40, 42, 44 With a `python3` on PATH that exits 1 on every call, run against opencatalogi: gate-40 PASS gate-42 PASS gate-44 PASS <- the three inline ones gate-34/37/38/39/41/43 SKIPPED (wiring) <- the six behind a helper gate-40 printed PASS over the 13 real findings it had reported one run earlier. gate-40 discarded its status with `2>/dev/null || true`; 42 and 44 ran per-file inline heredocs and never had one. 42 and 44 move to scripts/lib/check_link_text.py and scripts/lib/check_autocomplete.py — one interpreter for the whole file set, findings on stdout, exit code as a status — and 40 gains the same return-code guard. FOUND WHILE WRITING THE TESTS * gate-44 judged an input on the FIRST of name/id/v-model and stopped, so `<input id="e" type="text" name="email">` — the plainest textbook case this gate has — passed. Fleet effect, measured across 15 repos: openregister 0 -> 1 (an OpenAI Organization ID field), pipelinq 4 -> 5 (a "Colleague email" field). Both genuine, nothing lost. * gates 35, 36 and 44 read attribute values out of DOUBLE QUOTES ONLY. `tabindex='5'`, `alt=''` and `name='telephone'` render identically and reported PASS in both arms. Zero occurrences in the fleet today, which is why they could sit there indefinitely. * `[^>]*` in gates 42 and 44: a `>` inside an attribute value is not the end of a tag — the parse that hid 19 buttons from gate-39 (#259, #198, #236). * gates 42 and 44 scanned RAW text, so a commented-out `<a>click here</a>` or `<input name="email">` counted. That is gate-64's defect (#184), the one gate-38 (#247) and gate-41 (#266) each shipped a fix for. MEASURED AFTER, NOT ONLY BEFORE * 15 repos, gates 34-44, before vs after: every verdict and every finding count identical except the two new gate-44 true positives above. The rewrites of 42 and 44 removed nothing. * opencatalogi and nldesign return to their exact pre-plant baselines. * ARM 4 of test_gate_a11y_markup_scope.sh was mutation-checked: reverting gate-42's guard to `[ -d src ]` turns it red with the finding it was written for. TESTS * scripts/lib/test_check_link_text.py, test_check_autocomplete.py — 32 assertions; every relaxation ships with the true positive it must not swallow, comment/script exclusions ship with their positive control, and each ends with the whole PRE-FIX checker replayed as the mutant, asserting it answers DIFFERENTLY on every fixture. * test_gate_a11y_helper_wiring.sh gains gates 39, 40, 42, 44 (39 was wired correctly but never listed, so nothing held it to that) — 70 assertions. * test_gate_a11y_markup_scope.sh gains ARM 4, the templates-only repo. * Full discovered suite: 49 passed, 0 failed, 2 pre-existing quarantines. tests/test-hydra-gates-bin.sh: 59 passed, 0 failed.
`case " 38 45 " in *" ${_g} "*)` matches a constant against a pattern
built from the variable, which is the comparison written backwards. It
happened to work, and ShellCheck is right that it reads as a mistake.
Verified with shellcheck 0.10.0 at full severity: clean.
This was referenced Aug 8, 2026
rubenvdlinde
added a commit
that referenced
this pull request
Aug 9, 2026
…or, and gate-20 had never fired at all Every gate in the 12-22 band was given one textbook true positive in a real fleet repo and asked to catch it. Measured against package sha cdfbd7a, and re-measured on top of fef032b (#272), across openregister (207 .vue, 794 spec scenarios), pipelinq (202 .vue, type:"dashboard" pages) and shillinq (an ADR-040 AppHost adopter that keeps its own SettingsController). Seven fired, named the plant, and returned to their exact prior verdict when it was removed: 12, 13, 14 (both invariants), 15, 16, 18, 19, 21, 22. Four did not. 1. GATE-20 HAD NEVER FIRED. NOT RARELY - NEVER. Its search was `grep -nE "->${_pat//(/\\(}" "${_file}"`, and the expanded pattern `->findObjects\(` begins with `-`, so grep parses it as OPTIONS: `grep: invalid option -- '>'`, exit 2, no output. `2>/dev/null || true` discarded the message and the status, every file came back with zero hits, and the gate printed PASS. Planting `$this->objectService->findObjects(...)` in openregister's ActionsController: PASS. Repairing the grep ALONE is not the fix. The receiver test was "the FILE mentions ObjectService somewhere", and with the grep working that yields 14 findings on openregister and 5 on shillinq, ALL FALSE - `createFromArray()` is a real method on OpenRegister's *Mappers*. The receiver is now part of the pattern. Fleet measurement across the three repos afterwards: ONE finding, and it is real - shillinq's BookingNotificationController resolves OCA\OpenRegister\Service\ObjectService from the container inside its non-admin authorisation guard and calls findObject(), which does not exist on it (real surface: find/findAll/saveObject/createObject/updateObject/ deleteObject). That is a BadMethodCallException in an auth guard, shipped. 2. GATE-17 WAS WRONG ABOUT THE SAME API, IN THE OTHER DIRECTION. Four of the six names in OBJECT_SERVICE_CRUD - findObjects, createFromArray, updateFromArray, deleteFromId - do not exist on ObjectService; they are precisely what gate-20 flags as fabricated. findAll/createObject/ updateObject/deleteObject were absent. And `^\s*return\s+new\s+JSONResponse` sits in WRAPPER_NOISE_PATTERNS, which was tested BEFORE the ObjectService check - so the commonest pass-through spelling of all, `return new JSONResponse($this->objectService->findAll([]));`, was discarded as "response wrapping" with the call still inside it. It did not merely miss the shape: RESCUE_PATTERNS' `\$this->\w+Service->\w+\(` then matched it and returned False. The gate rescued its own subject. Blast radius of both changes across openregister/pipelinq/shillinq: 0 new findings. The CRUD-name filter and the `@spec exclude` escape hatch are untouched. 3. GATE-14 NEVER JUDGED THE TEN ROUTES AppHost SUPPLIES (#265, closed). #223 taught invariant 1 about `Routes::standard()`. Invariant 2 asks the opposite question and still read route names as literals out of the leaf's own appinfo/routes.php, where those ten names never appear. Deleting `SettingsController::update()` from shillinq - which keeps its own controller, as aliasControllerUnlessLeafDefinesIt explicitly allows - leaves PUT /api/settings resolving to nothing. That is not a 404: the router matches, ControllerMethodReflector reflects, the request dies 500. shillinq's own docblock on update() spells the hazard out. Gate-14's findings log came back EMPTY and the gate said PASS. 4. THREE GATES REPORTED PASS OVER A CRASHED CHECKER. With python3 replaced by a stub that always exits 1 - so not one file was inspected by any python-backed gate: gate-12 SKIPPED (wiring) gate-15 PASS gate-17 SKIPPED (wiring) gate-16 PASS gate-18 PASS gate-19 FAIL - "an unreported number of scenario(s)" 15/16/18 wrote `2>/dev/null || true` and counted lines in an empty log. 19 read exit 1 as EXIT_FAIL from a helper that never printed its own summary, producing a blocking verdict with a count nobody measured. The three helpers now print a terminal `# count=` marker and the runner requires it; gate-18's helper contract is "always exit 0", so any non-zero exit is a wiring skip; gate-19 treats "EXIT_FAIL with no FAIL - N summary line" as wiring. check_spec_coverage.py and check_dashboard_antipattern.py also stop returning their finding COUNT as an exit status (#209). 5. GATES 12 AND 13 PASSED OVER A src/ WITH ZERO .vue (#274). `[ -d src ]` is not `src/ contains a Vue component`. nldesign's src/ holds one manifest.json, and at fef032b both gates print PASS there - the shape that let twelve gates certify nldesign in #225, still current, not one `rm` away. Now `na`, with the judgement #274 asked for stated in the reason: NcSelect / NcModal / NcDialog are Vue SFC components, a PHP template cannot instantiate one, so these two stay .vue-only and gate-40 keeps the language-agnostic input-label rule for templates/. 6. GATE-22's VERDICT DEPENDED ON WHERE THE GATES WERE CHECKED OUT. `require('ajv/dist/2020')` resolves relative to check_manifest.js, never to the repo under test. Run from openregister's own root with node_modules/ajv PRESENT in that root, the validator printed "Ajv is not resolvable from this process (no node_modules, no NODE_PATH)" - false - and gate-22 went FAIL; exporting NODE_PATH to that same directory flipped it to PASS. Same tree, same package, two verdicts. Resolution is now anchored on the manifest's own repo root, then cwd, then this package, and the degradation names every directory it searched. 7. GATE-15's SLOT MATCHER HAD AN ATTRIBUTE-ORDER BLIND SPOT. `<template #widget-x class="wide">` and `<template #widget-x='{i}'>` did not match - the pattern assumed the slot binding was the last thing before `>`. Same class as gate-44's first-attribute stop in #272. TESTS test_gate_or_objectservice_surface.sh NEW - gate-20 and gate-17, each with the silent sibling that must stay silent (a mapper's real createFromArray; a domain-named method with an identical body), plus a direct assertion that a grep pattern starting with `-` is parsed as options test_gate_crashed_checker_is_not_a_finding.sh all six python-backed gates under a broken interpreter, generically, so a future gate inherits the assertion test_gate_empty_scope_never_passes.sh gates 12/13 on an nldesign-shaped repo, with the one-.vue control test_gate_route_registration.sh new fixture routes-standard-missing- update/, differing from its sibling by exactly the deleted method test_check_manifest.sh ajv resolved from the subject's node_modules with cwd and NODE_PATH pointing elsewhere test_check_dashboard_antipattern.py four slot-tag spellings that were silent misses Every one mutation-checked against the pre-fix tree: the crashed-checker arm is 7 failures red at cdfbd7a, the 12/13 arm 3 failures red at fef032b, the slot-tag arm 4 subtests red, and the ajv anchor red - all green after. 50 helper suites pass, 59 bin-level assertions pass.
rubenvdlinde
added a commit
that referenced
this pull request
Aug 9, 2026
…ver a dead interpreter, and three could not see the defect they exist for (#280) * fix(gates 45-55): eleven gates passed over an unopened scope, eight over a dead interpreter, and three could not see the defect they exist for Every gate in this band was given ONE textbook true positive of exactly what it exists to catch, planted in a real fleet repo, then removed again. Where a gate could not fail, it was repaired; where it could, the plant is now a regression test. Measured at package sha 34370f6. ## 1. All eleven reported PASS over a scope they never opened (#242/#240/#258/#268) On a README-only diff against larpingapp, gates 45-55 printed eleven PASS lines and the summary read "53 of 53 applicable gates ran". Not one of them had opened a file. Gates 4/6/7/19/25/28/62/63 have answered the identical situation with NOT APPLICABLE since #268; this band never adopted it. Gates 47 and 48 are the sharper case: they can only answer a question about a CHANGE SET, so on every builder full-repo run in the fleet — no base ref at all — they printed a co-change verdict they had not formed. ## 2. Eight reported PASS over a crashed interpreter (#147/#249/#262) A planted defect only fires when the gate runs, so no plant can see this. With a `python3` on PATH that exits 1 on every call, on a tree carrying real findings: gate-46 PASS — over the 277 unresolved @SPEC findings, across 104 distinct targets, it had reported one run earlier on the same files gate-47 PASS — on the same diff where it had just reported FAIL gate-45/49/50 PASS (`2>/dev/null` discarded status and traceback) gate-51/54/55 PASS (`|| true` discarded the status) gate-52 FAIL — "1 custom-widget finding(s)", a fabricated finding: the helper returned its COUNT as its exit status, the same channel Python uses for a traceback (#209). The count was also clamped to 99 to fit in a byte. It now prints `findings=N` on stdout and exits boolean; no `findings=` line means the helper died. gate-54 was the quietest: its advisory WARN half reads the same log, so a dead helper silenced both halves at once. ## 3. gate-45 was the residue of #272's fix (.github#274) #272 migrated gates 35/40/42/44 off `[ -d src ]` onto `_a11y_has_markup_dir` and left the twelfth member of the family behind. On a templates-only app gate-45 reported NOT APPLICABLE — "this repo ships no frontend" — over a `<style>` block with `transition:` and no reduced-motion fallback, in the same file gate-43 FAILED on in the same run. `na` is the one verdict that removes a gate from coverage accounting. The regression test was already written and gate-45 was excluded from it by name, with a comment explaining why. Removing the name from ARM 4's skip list in test_gate_a11y_markup_scope.sh IS the test; it fails against 34370f6. ## 4. gate-47: prose satisfied it, and a qualified attribute did not `_ANNOTATION_RE` was an unanchored alternation of string literals, and it was wrong in both directions from that one regex — the pairing #269 found in gate-48 and never carried to its sibling. FALSE POSITIVE rewording ONE docblock sentence that merely NAMES the annotation ("becomes `@NoAdminRequired` again, paired with a real ownership check") made the gate demand a test co-change. A gate satisfiable by prose manufactures the appearance of a security review (#191). FALSE NEGATIVE `#[\OCP\AppFramework\Http\Attribute\NoAdminRequired]` was invisible. A commit adding exactly that to a controller — opening an admin-only endpoint to every authenticated user — with no test in the diff reported PASS. Now position-anchored, by the same rule check_csrf_removal.py already used. ## 5. gate-50: a false positive and a false negative in the same regex FALSE NEGATIVE the app-id argument had to be a QUOTED STRING, so every read written the fleet-standard way — `getValueString( Application::APP_ID, 'listing_register', '')` — was invisible. Identical code with `'larpingapp'` FAILED. Same family as #184. 7 security-relevant reads across 5 repos sit behind a constant today. FALSE POSITIVE the empty-compare guard required a closing paren immediately after the empty string, so the correct compound guard `if ($reg === '' || $sch === '')` was reported as unguarded — twice, on code the gate was asking for. A guard that is a boolean `return` rather than an `if` was rejected too. Both directions are now asserted, including the opencatalogi#86 shape that mixes them: one read guarded, the next unguarded two lines later. ## 6. gate-53 did not block the PR that creates larpingapp#286 Reintroducing #286 exactly — the check-in tab deleted from src/manifest.json, `EventRoster` left registered in src/registry.js — reported PASS. Direction 1 of the registry cross-reference stays advisory for LEGACY orphans, correctly: the gate cannot tell "wire it" from "delete it". But when the DIFF ITSELF removed the last reference it can, and that finding now blocks. Pre-existing orphans are untouched (larpingapp carries one today), so this is prevention, not a burn-down list nobody can close. ## Verified working, repaired nothing gate-46 (dangling file, dangling fragment, valid anchor), gate-48 (short and fully-qualified attribute removal; a comment reword correctly stays green), gate-49, gate-51 (title, description and nested items.properties independently), gate-52's ratchet (growth fails, shrink passes), gate-54 (flat, nested and $ref-carrying), gate-55. ## Deliberately NOT enforced `title == key` on a schema property is a real gate-51 defect — the renderer uses `prop.title || key`, so the user sees the raw technical key. Measured across 10 repos: 148 occurrences, ALL of them in softwarecatalog, where they are VNG-standardised element names (`identifier`, `type`, `name`) that must not be renamed. Enforcing it would produce 148 findings with no legitimate end state in the one repo that has them. Reported rather than gated (#252). ## Divergence to reconcile gate-45 now answers an empty in-scope set with `na`; gate-40 answers it with PASS, by a deliberate choice in #272 that cited the invariant test this PR reworks. The invariant now discriminates on the REASON — the applicability table's own phrasing must not appear once its prerequisite holds — so both behaviours are expressible. The family should pick one. ## Testing New: hydra-gates/scripts/lib/test_gate_45_to_55_acceptance.sh — 31 arms across six families, discovered by run-helper-suites.sh. Against the package as merged on main it fails 20 of 31; the 11 that pass are exactly the anti-widening and no-regression controls. Every mutation asserts its anchor is present before it plants. Repos used, chosen for different shapes: larpingapp (register-owning, manifest-driven, ships registry.js), nldesign (PHP templates, no .vue, no register), doriath (ships no phpcs SpecTagSniff — the #246 control, held at 81 findings across 46 targets before and after the plant), openconnector (41 register files). Full package suite: 52 discovered suites pass, 2 quarantined as documented; 60/60 entry-point invariants. * fix(gate-50): the fail-mode window started where the call began, not where it ended The constant-app-id fix in the parent commit made procest's config reads visible for the first time and immediately produced 3 findings on lib/Service/AiService.php — all three false positives, and both causes are ordinary code the window could never have seen: multi-line call PHPCS formats each read across five lines. Two of them plus a blank line put the guard on the ELEVENTH line, one outside a window counted from the line the match BEGAN on. The guard being missed is a textbook `if (empty($registerId) === true || empty($schemaId) === true) { $this->logger->warning(...); return; }` (AiService.php:580, :967). same-line guard `'ai_api_key_set' => ...getValueString(APP_ID, 'ai_api_key', '') !== ''` handles the empty default ON the match line, and the window started after it (AiService.php:710). The window now anchors to the END of the call expression — parentheses balanced forward from the `(` — and includes the remainder of that line. A single-line read keeps exactly the ten lines it always had. Caught by a before/after sweep of 12 fleet repos: 26 of 121 verdicts changed, 25 of them PASS -> NOT APPLICABLE (the truthfulness correction), and this was the only one that changed to FAIL. procest is PASS again, correctly. Three arms added: the multi-line shape, the same-line shape, and the reverse control — the same multi-line shape with the guard DELETED must still FAIL, so the window cannot have been widened until the gate finds nothing. Also: shellcheck SC2181 in gate-45's new status check, and a file-scoped SC2016 suppression for the acceptance suite, whose PHP fixtures are single-quoted on purpose.
rubenvdlinde
added a commit
that referenced
this pull request
Aug 9, 2026
…or, and gate-20 had never fired at all (#277) * fix(gates 12-22): four gates could not detect the defect they exist for, and gate-20 had never fired at all Every gate in the 12-22 band was given one textbook true positive in a real fleet repo and asked to catch it. Measured against package sha cdfbd7a, and re-measured on top of fef032b (#272), across openregister (207 .vue, 794 spec scenarios), pipelinq (202 .vue, type:"dashboard" pages) and shillinq (an ADR-040 AppHost adopter that keeps its own SettingsController). Seven fired, named the plant, and returned to their exact prior verdict when it was removed: 12, 13, 14 (both invariants), 15, 16, 18, 19, 21, 22. Four did not. 1. GATE-20 HAD NEVER FIRED. NOT RARELY - NEVER. Its search was `grep -nE "->${_pat//(/\\(}" "${_file}"`, and the expanded pattern `->findObjects\(` begins with `-`, so grep parses it as OPTIONS: `grep: invalid option -- '>'`, exit 2, no output. `2>/dev/null || true` discarded the message and the status, every file came back with zero hits, and the gate printed PASS. Planting `$this->objectService->findObjects(...)` in openregister's ActionsController: PASS. Repairing the grep ALONE is not the fix. The receiver test was "the FILE mentions ObjectService somewhere", and with the grep working that yields 14 findings on openregister and 5 on shillinq, ALL FALSE - `createFromArray()` is a real method on OpenRegister's *Mappers*. The receiver is now part of the pattern. Fleet measurement across the three repos afterwards: ONE finding, and it is real - shillinq's BookingNotificationController resolves OCA\OpenRegister\Service\ObjectService from the container inside its non-admin authorisation guard and calls findObject(), which does not exist on it (real surface: find/findAll/saveObject/createObject/updateObject/ deleteObject). That is a BadMethodCallException in an auth guard, shipped. 2. GATE-17 WAS WRONG ABOUT THE SAME API, IN THE OTHER DIRECTION. Four of the six names in OBJECT_SERVICE_CRUD - findObjects, createFromArray, updateFromArray, deleteFromId - do not exist on ObjectService; they are precisely what gate-20 flags as fabricated. findAll/createObject/ updateObject/deleteObject were absent. And `^\s*return\s+new\s+JSONResponse` sits in WRAPPER_NOISE_PATTERNS, which was tested BEFORE the ObjectService check - so the commonest pass-through spelling of all, `return new JSONResponse($this->objectService->findAll([]));`, was discarded as "response wrapping" with the call still inside it. It did not merely miss the shape: RESCUE_PATTERNS' `\$this->\w+Service->\w+\(` then matched it and returned False. The gate rescued its own subject. Blast radius of both changes across openregister/pipelinq/shillinq: 0 new findings. The CRUD-name filter and the `@spec exclude` escape hatch are untouched. 3. GATE-14 NEVER JUDGED THE TEN ROUTES AppHost SUPPLIES (#265, closed). #223 taught invariant 1 about `Routes::standard()`. Invariant 2 asks the opposite question and still read route names as literals out of the leaf's own appinfo/routes.php, where those ten names never appear. Deleting `SettingsController::update()` from shillinq - which keeps its own controller, as aliasControllerUnlessLeafDefinesIt explicitly allows - leaves PUT /api/settings resolving to nothing. That is not a 404: the router matches, ControllerMethodReflector reflects, the request dies 500. shillinq's own docblock on update() spells the hazard out. Gate-14's findings log came back EMPTY and the gate said PASS. 4. THREE GATES REPORTED PASS OVER A CRASHED CHECKER. With python3 replaced by a stub that always exits 1 - so not one file was inspected by any python-backed gate: gate-12 SKIPPED (wiring) gate-15 PASS gate-17 SKIPPED (wiring) gate-16 PASS gate-18 PASS gate-19 FAIL - "an unreported number of scenario(s)" 15/16/18 wrote `2>/dev/null || true` and counted lines in an empty log. 19 read exit 1 as EXIT_FAIL from a helper that never printed its own summary, producing a blocking verdict with a count nobody measured. The three helpers now print a terminal `# count=` marker and the runner requires it; gate-18's helper contract is "always exit 0", so any non-zero exit is a wiring skip; gate-19 treats "EXIT_FAIL with no FAIL - N summary line" as wiring. check_spec_coverage.py and check_dashboard_antipattern.py also stop returning their finding COUNT as an exit status (#209). 5. GATES 12 AND 13 PASSED OVER A src/ WITH ZERO .vue (#274). `[ -d src ]` is not `src/ contains a Vue component`. nldesign's src/ holds one manifest.json, and at fef032b both gates print PASS there - the shape that let twelve gates certify nldesign in #225, still current, not one `rm` away. Now `na`, with the judgement #274 asked for stated in the reason: NcSelect / NcModal / NcDialog are Vue SFC components, a PHP template cannot instantiate one, so these two stay .vue-only and gate-40 keeps the language-agnostic input-label rule for templates/. 6. GATE-22's VERDICT DEPENDED ON WHERE THE GATES WERE CHECKED OUT. `require('ajv/dist/2020')` resolves relative to check_manifest.js, never to the repo under test. Run from openregister's own root with node_modules/ajv PRESENT in that root, the validator printed "Ajv is not resolvable from this process (no node_modules, no NODE_PATH)" - false - and gate-22 went FAIL; exporting NODE_PATH to that same directory flipped it to PASS. Same tree, same package, two verdicts. Resolution is now anchored on the manifest's own repo root, then cwd, then this package, and the degradation names every directory it searched. 7. GATE-15's SLOT MATCHER HAD AN ATTRIBUTE-ORDER BLIND SPOT. `<template #widget-x class="wide">` and `<template #widget-x='{i}'>` did not match - the pattern assumed the slot binding was the last thing before `>`. Same class as gate-44's first-attribute stop in #272. TESTS test_gate_or_objectservice_surface.sh NEW - gate-20 and gate-17, each with the silent sibling that must stay silent (a mapper's real createFromArray; a domain-named method with an identical body), plus a direct assertion that a grep pattern starting with `-` is parsed as options test_gate_crashed_checker_is_not_a_finding.sh all six python-backed gates under a broken interpreter, generically, so a future gate inherits the assertion test_gate_empty_scope_never_passes.sh gates 12/13 on an nldesign-shaped repo, with the one-.vue control test_gate_route_registration.sh new fixture routes-standard-missing- update/, differing from its sibling by exactly the deleted method test_check_manifest.sh ajv resolved from the subject's node_modules with cwd and NODE_PATH pointing elsewhere test_check_dashboard_antipattern.py four slot-tag spellings that were silent misses Every one mutation-checked against the pre-fix tree: the crashed-checker arm is 7 failures red at cdfbd7a, the 12/13 arm 3 failures red at fef032b, the slot-tag arm 4 subtests red, and the ajv anchor red - all green after. 50 helper suites pass, 59 bin-level assertions pass. * fix(tests): the bin suite asserted that src/ existing means gates 12/13 ran test-hydra-gates-bin.sh looped 12 and 13 into 'with src/ present, every src-guarded gate really runs'. That fixture's src/ holds one .js file and no .vue, so the assertion encoded the belief this PR removes: that `[ -d src ]` is the same as 'the gate has a subject'. Both gates printed PASS there — an empty glob under an existing src/, nldesign's exact shape. They are now asserted separately and more strictly: not 'did not say na', but 'said na and NAMED the empty .vue glob'. PASS is called out by name as the regression it would be. 61 bin assertions pass, 50 helper suites pass. * fix(gates 12-22): four gates could not detect the defect they exist for, and gate-20 had never fired at all Every gate in the 12-22 band was given one textbook true positive in a real fleet repo and asked to catch it. Measured against package sha cdfbd7a, and re-measured on top of fef032b (#272), across openregister (207 .vue, 794 spec scenarios), pipelinq (202 .vue, type:"dashboard" pages) and shillinq (an ADR-040 AppHost adopter that keeps its own SettingsController). Seven fired, named the plant, and returned to their exact prior verdict when it was removed: 12, 13, 14 (both invariants), 15, 16, 18, 19, 21, 22. Four did not. 1. GATE-20 HAD NEVER FIRED. NOT RARELY - NEVER. Its search was `grep -nE "->${_pat//(/\\(}" "${_file}"`, and the expanded pattern `->findObjects\(` begins with `-`, so grep parses it as OPTIONS: `grep: invalid option -- '>'`, exit 2, no output. `2>/dev/null || true` discarded the message and the status, every file came back with zero hits, and the gate printed PASS. Planting `$this->objectService->findObjects(...)` in openregister's ActionsController: PASS. Repairing the grep ALONE is not the fix. The receiver test was "the FILE mentions ObjectService somewhere", and with the grep working that yields 14 findings on openregister and 5 on shillinq, ALL FALSE - `createFromArray()` is a real method on OpenRegister's *Mappers*. The receiver is now part of the pattern. Fleet measurement across the three repos afterwards: ONE finding, and it is real - shillinq's BookingNotificationController resolves OCA\OpenRegister\Service\ObjectService from the container inside its non-admin authorisation guard and calls findObject(), which does not exist on it (real surface: find/findAll/saveObject/createObject/updateObject/ deleteObject). That is a BadMethodCallException in an auth guard, shipped. 2. GATE-17 WAS WRONG ABOUT THE SAME API, IN THE OTHER DIRECTION. Four of the six names in OBJECT_SERVICE_CRUD - findObjects, createFromArray, updateFromArray, deleteFromId - do not exist on ObjectService; they are precisely what gate-20 flags as fabricated. findAll/createObject/ updateObject/deleteObject were absent. And `^\s*return\s+new\s+JSONResponse` sits in WRAPPER_NOISE_PATTERNS, which was tested BEFORE the ObjectService check - so the commonest pass-through spelling of all, `return new JSONResponse($this->objectService->findAll([]));`, was discarded as "response wrapping" with the call still inside it. It did not merely miss the shape: RESCUE_PATTERNS' `\$this->\w+Service->\w+\(` then matched it and returned False. The gate rescued its own subject. Blast radius of both changes across openregister/pipelinq/shillinq: 0 new findings. The CRUD-name filter and the `@spec exclude` escape hatch are untouched. 3. GATE-14 NEVER JUDGED THE TEN ROUTES AppHost SUPPLIES (#265, closed). #223 taught invariant 1 about `Routes::standard()`. Invariant 2 asks the opposite question and still read route names as literals out of the leaf's own appinfo/routes.php, where those ten names never appear. Deleting `SettingsController::update()` from shillinq - which keeps its own controller, as aliasControllerUnlessLeafDefinesIt explicitly allows - leaves PUT /api/settings resolving to nothing. That is not a 404: the router matches, ControllerMethodReflector reflects, the request dies 500. shillinq's own docblock on update() spells the hazard out. Gate-14's findings log came back EMPTY and the gate said PASS. 4. THREE GATES REPORTED PASS OVER A CRASHED CHECKER. With python3 replaced by a stub that always exits 1 - so not one file was inspected by any python-backed gate: gate-12 SKIPPED (wiring) gate-15 PASS gate-17 SKIPPED (wiring) gate-16 PASS gate-18 PASS gate-19 FAIL - "an unreported number of scenario(s)" 15/16/18 wrote `2>/dev/null || true` and counted lines in an empty log. 19 read exit 1 as EXIT_FAIL from a helper that never printed its own summary, producing a blocking verdict with a count nobody measured. The three helpers now print a terminal `# count=` marker and the runner requires it; gate-18's helper contract is "always exit 0", so any non-zero exit is a wiring skip; gate-19 treats "EXIT_FAIL with no FAIL - N summary line" as wiring. check_spec_coverage.py and check_dashboard_antipattern.py also stop returning their finding COUNT as an exit status (#209). 5. GATES 12 AND 13 PASSED OVER A src/ WITH ZERO .vue (#274). `[ -d src ]` is not `src/ contains a Vue component`. nldesign's src/ holds one manifest.json, and at fef032b both gates print PASS there - the shape that let twelve gates certify nldesign in #225, still current, not one `rm` away. Now `na`, with the judgement #274 asked for stated in the reason: NcSelect / NcModal / NcDialog are Vue SFC components, a PHP template cannot instantiate one, so these two stay .vue-only and gate-40 keeps the language-agnostic input-label rule for templates/. 6. GATE-22's VERDICT DEPENDED ON WHERE THE GATES WERE CHECKED OUT. `require('ajv/dist/2020')` resolves relative to check_manifest.js, never to the repo under test. Run from openregister's own root with node_modules/ajv PRESENT in that root, the validator printed "Ajv is not resolvable from this process (no node_modules, no NODE_PATH)" - false - and gate-22 went FAIL; exporting NODE_PATH to that same directory flipped it to PASS. Same tree, same package, two verdicts. Resolution is now anchored on the manifest's own repo root, then cwd, then this package, and the degradation names every directory it searched. 7. GATE-15's SLOT MATCHER HAD AN ATTRIBUTE-ORDER BLIND SPOT. `<template #widget-x class="wide">` and `<template #widget-x='{i}'>` did not match - the pattern assumed the slot binding was the last thing before `>`. Same class as gate-44's first-attribute stop in #272. TESTS test_gate_or_objectservice_surface.sh NEW - gate-20 and gate-17, each with the silent sibling that must stay silent (a mapper's real createFromArray; a domain-named method with an identical body), plus a direct assertion that a grep pattern starting with `-` is parsed as options test_gate_crashed_checker_is_not_a_finding.sh all six python-backed gates under a broken interpreter, generically, so a future gate inherits the assertion test_gate_empty_scope_never_passes.sh gates 12/13 on an nldesign-shaped repo, with the one-.vue control test_gate_route_registration.sh new fixture routes-standard-missing- update/, differing from its sibling by exactly the deleted method test_check_manifest.sh ajv resolved from the subject's node_modules with cwd and NODE_PATH pointing elsewhere test_check_dashboard_antipattern.py four slot-tag spellings that were silent misses Every one mutation-checked against the pre-fix tree: the crashed-checker arm is 7 failures red at cdfbd7a, the 12/13 arm 3 failures red at fef032b, the slot-tag arm 4 subtests red, and the ajv anchor red - all green after. 50 helper suites pass, 59 bin-level assertions pass. * fix(tests): the bin suite asserted that src/ existing means gates 12/13 ran test-hydra-gates-bin.sh looped 12 and 13 into 'with src/ present, every src-guarded gate really runs'. That fixture's src/ holds one .js file and no .vue, so the assertion encoded the belief this PR removes: that `[ -d src ]` is the same as 'the gate has a subject'. Both gates printed PASS there — an empty glob under an existing src/, nldesign's exact shape. They are now asserted separately and more strictly: not 'did not say na', but 'said na and NAMED the empty .vue glob'. PASS is called out by name as the regression it would be. 61 bin assertions pass, 50 helper suites pass. * fix(gate-14): a namespaced route name resolves to the app root, not to lib/Controller/ NC's RouteParser::buildControllerName() does not prefix the app namespace when the route name already contains a backslash, so `AppHost\Controller\GenericHealth#index` is looked up as the bare class `AppHost\Controller\GenericHealthController`. PSR-4 maps `OCA\<App>\` onto `lib/`, so that class lives at lib/AppHost/Controller/ — not under lib/Controller/, which is the only place the resolver looked. Reproduced against this package's own gates-23-33/planted fixture: lib/Controller/AppHost/Controller/GenericHealthController.php route='AppHost\Controller\GenericHealth#index' rule=controller-class-not-found a path that cannot exist, reported as a missing class, INSIDE the repository that ships the file. Same shape as the gate-30 finding in #276: the path the gate derives is not the path the app uses. The false FAIL was only half of it. Where a DI binding rescued the absence the loop `continue`d, so the method-existence check never ran — measured on two new fixtures differing by exactly one renamed method: at origin/main BOTH report PASS, including the one whose route resolves to nothing. That is #265's defect at a different address. Both candidate paths are probed and the one that EXISTS wins; when neither does the lib/Controller/ spelling is reported exactly as before, so a genuinely missing controller reads the same as it always has. Also: test_gate_route_registration.sh hardcoded its runner, so pointing HYDRA_GATES_RUNNER_UNDER_TEST at a pre-fix tree silently kept running the FIXED one and reported all-green. A mutation check that cannot fail is the same defect as a gate that cannot fail. With the override honoured, the suite is 5 failures red against origin/main and green after. And test-hydra-gates-bin.sh's 'with src/ present, every src-guarded gate really runs' loop had a stale premise: #276 gave that fixture two real .vue files (because gates 26/31/32 now distinguish 'src/ exists' from 'src/ contains what I read'), so gates 12 and 13 correctly run and PASS there. They go back in the loop, and the loop now asserts its own premise first. The zero-.vue case is asserted where its premise actually holds — the nldesign-shaped fixture in test_gate_empty_scope_never_passes.sh. 52 helper suites pass, 60 bin assertions pass. * fix(gate-16): deleting an @SPEC tag was the one edit the gate could not see `_overlaps` walks FORWARD from the declaration line through the method body. The docblock sits ABOVE the declaration, and the docblock is the only place `@spec` can live — so the scope window never contained the tag. Measured on a two-method fixture: delete the `@spec openspec/...` line, leave the body byte-identical, and the helper prints `# count=0` and exits 0. Every @SPEC tag in a repository can be stripped and gate-16 stays green. `run_gate` also skipped any file whose `added` set was empty, and a pure deletion produces exactly that, so the file was never even opened. Same family as filter_preexisting_methods.py filing an auth-attribute removal as pre-existing: a body-shaped scope cannot see a change that is not in the body. Gate-16 does NOT route through that helper — its four call sites are gates 6, 7, 8 and 30, and check_spec_coverage.py references it zero times — it arrives at the same blind spot by its own path. The fix keys on 'a tag was TAKEN AWAY', not on 'a docblock was touched'. A file that lost a tag is evaluated twice — as it is, and as it was at the base, with the same walkers and therefore the same exemptions — and only the DIFFERENCE is reported. So: delete a tag -> the method that lost it, and only it typo-fix a legacy untagged docblock -> clean (no inherited debt surfaced) add a new untagged method -> still caught, unchanged All three asserted, the second being the anti-widening control. --------- Co-authored-by: Conduction Release Bot <release-bot@conduction.nl>
rubenvdlinde
added a commit
that referenced
this pull request
Aug 9, 2026
The branch was rebased from cdfbd7a onto 48c88ba after four gate-package releases landed mid-session (#272, #275, #276, #280/#282). The rebase gave the same content a new history, which is a force-push, and force-push is blocked on shared branches for good reason. The tree here is IDENTICAL to the rebased HEAD — this commit only re-attaches the old tip as a second parent so the push is a fast-forward.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
The acceptance test for gates 34–44 (the accessibility band), run at package sha
cdfbd7a: plant one textbook true positive per gate in both a.vueapp (opencatalogi, 93.vue) and a PHP-template app (nldesign, zero.vue, one template) — the asymmetry that made #225/#261 possible — require the gate to FAIL and name the plant in both, then remove it and require the prior verdict back.All 11 gates fired and named the plant in all 22 arms, and all 22 returned to baseline exactly. The
_a11y_markup_filesfix holds. Two defects survive that test, and neither is visible from it.1. Four gates declare themselves NOT APPLICABLE over a repo full of markup
Gates 35, 40, 42, 44 still guarded on
[ -d src ]while 34/36/37/39/43 had moved to_a11y_has_markup_dir, and the central applicability table listed the whole family under[ -d src ]. On a repo with atemplates/full of markup and nosrc/— same run, same files:nais the one verdict that removes a gate from coverage accounting, and the reason is contradicted by the same run's own output three lines above it. The a11y family has read.phpand.htmlsince #261; the declaration never caught up.No fleet app is templates-only today. nldesign is one
rmaway — itssrc/holds a singlemanifest.json, the exact shape that made twelve gates pass over nothing in #225.The four guards now call
_a11y_has_markup_dir, and the applicability declaration calls the same function instead of restating it, so the two cannot drift again. No third scope definition was added (#247/#261).2. A crashed checker reported PASS — gates 40, 42, 44 (#147 / #249)
With a
python3on PATH that exits 1 on every invocation, against opencatalogi:gate-40 discarded its status with
2>/dev/null || true; 42 and 44 ran per-file inline heredocs and never had one. 42 and 44 move intocheck_link_text.py/check_autocomplete.py— one interpreter for the whole file set, findings on stdout, exit code as a status — and 40 gains the same guard.Found while writing the tests
name/id/v-modeland stopped, so<input id="e" type="text" name="email">— the plainest textbook case this gate has — passed. Fleet effect, measured over 15 repos: openregister 0 → 1 (an OpenAI Organization ID field), pipelinq 4 → 5 (a "Colleague email" field). Both genuine.tabindex='5',alt='',name='telephone'render identically and reported PASS in both arms. Zero fleet occurrences today — which is why they could sit there indefinitely.[^>]*in gates 42 and 44 — a>inside an attribute value is not the end of a tag; the parse that hid 19 buttons from gate-39 (fix(gate-39): a BOUND name is still a name, and[^>]*could not see 19 buttons #259, fix(gate-9): the admin rule matched no real isAdmin() call, and prose bought the self-auth exemption #198, gates 12/32/38: two text-extraction false positives and one that scores a comment #236).<a>click here</a>counted. gate-64's defect (fix(hydra-gates): gate-64 read spelling, not code — 2 false REDs and 1 false GREEN #184), the one fix(gate-38): a mount point is not a page root, and the remedy it demanded regressed a11y (#214, #216, #227) #247 and gate-41 (html-lang) has gate-38's defect verbatim: a PHP COMMENT mentioning <html> makes a mount point a page root #266 each shipped a fix for.Measured after, not only before
Tightening a gate creates its own false positives, so the after-measure is the load-bearing one:
[ -d src ]turns it red with the finding it was written for. A check that cannot fail is not a check.What the controls confirmed still holds
Verified independently, not assumed from the PR that claimed them: gate-34 both arms (#269 — the comment does not fire,
window['confirm']()does) · gate-37tabindex="-1"is not focusable (#251) · gate-38/41 a PHP comment mentioning<html>does not make a mount point a page root, and a commented-out<html lang>does not satisfy a real unlangged one (#247/#266) · gate-39 a bound:title/:aria-labelledbyis a name, and a>inside an attribute does not hide the button (#259) · gate-40 a slottedNcCheckboxRadioSwitchis not flagged, a self-closing one is (#251) · gate-43 anaria-hiddenspacer<th>is not flagged (the 8 false openconnector findings).One finding NOT fixed here, reported instead
gate-40 flags
<input type="file" aria-hidden="true" tabindex="-1">— the exact element gate-37's #222 fix declared canonical and correct. Two gates in the same band contradict each other, andaria-labelon anaria-hiddenelement is inert, so the finding cannot be closed honestly. Zero fleet occurrences today (nldesign's three aredisplay:none, notaria-hidden), and excludingdisplay:nonewould be blinding — a style can be toggled by JS. Filing rather than guessing.Tests
test_check_link_text.py,test_check_autocomplete.py— 32 assertions. Every relaxation ships with the true positive it must not swallow; each comment/script exclusion ships with its positive control; each file ends with the whole pre-fix checker replayed as the mutant, asserting it answers differently on every fixture.test_gate_a11y_helper_wiring.shgains gates 39, 40, 42, 44 — 39 was wired correctly but never listed, so nothing held it to that. 70 assertions.test_gate_a11y_markup_scope.shgains ARM 4, the templates-only repo.test-hydra-gates-bin.sh59 passed / 0 failed.🤖 Generated with Claude Code