Skip to content

test(gates): refuse to run the 45–55 suite when ajv is unresolvable (it reported 4 gate-53 defects that do not exist) - #292

Merged
rubenvdlinde merged 1 commit into
mainfrom
fix/g53-suite-ajv-preflight
Aug 9, 2026
Merged

test(gates): refuse to run the 45–55 suite when ajv is unresolvable (it reported 4 gate-53 defects that do not exist)#292
rubenvdlinde merged 1 commit into
mainfrom
fix/g53-suite-ajv-preflight

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

What this fixes

test_gate_45_to_55_acceptance.sh reports four gate-53 defects that do not exist when ajv is not resolvable — and one of its arms passes for the wrong reason while doing so.

Evidence

Fresh clone of main (c51a225, git rev-list --left-right --count origin/main...HEAD = 0 0), 0-byte stderr on both runs.

Without NODE_PATH — 4 failures:

FAIL — gate-53 failed without naming EventRoster
FAIL — gate-53 accepts removing the component and its registry entry together → expected 'PASS', got 'FAIL'
FAIL — gate-53 leaves a PRE-EXISTING orphan advisory (WARN), not blocking → expected 'PASS', got 'FAIL'
FAIL — gate-53 swallowed the pre-existing orphan entirely

With NODE_PATH setALL GREEN, exit 0.

So gate-53 is correct and the suite is not stale. Both helpers behave properly in isolation: check_manifest_crossref.js on the D3 fixture exits 0 and emits the expected WARN for the pre-existing orphan. The gate fails closed on [gate-53] ... ajv not resolvable from <lib> — refusing to run fail-open, which is the right behaviour — but it makes every downstream assertion describe a gate that never ran.

This was previously reported upstream as "the suite expects blocking behaviour for pre-existing orphans that #250/#260/#280 deliberately made advisory". That reading is wrong in both halves: the suite expects advisory, and the shipped gate delivers advisory.

The part worth keeping

🔑 Arm D1 still printed ok. It expects FAIL and got FAIL — because gate-53 failed closed, not because it detected the orphan. A false pass inside the suite built to catch false passes.

That is why this aborts (exit 2, zero verdict lines) rather than warning and continuing. A wiring fault must not be able to produce a verdict at all.

The check itself

⚠️ node -e "require('ajv')" is not a valid check on its own — Node resolves upward from the cwd, so it can succeed against a node_modules belonging to an ancestor directory rather than to the gates package. (Measured the same day: an agent's ajv check passed for exactly this reason.)

So ajv is resolved from the helpers' own directory — the same place the runner resolves it from — and the absolute path is printed as the evidence:

preflight — ajv resolves to /…/node_modules/ajv/dist/ajv.js

The path is the evidence; the exit code cannot tell you which ajv answered.

Proven both directions

arm result
ajv absent exit 2, 0 verdict lines emitted, wiring message on stderr naming the fix
ajv present exit 0, ALL GREEN, empty stderr

No gate logic changed. No assertion relaxed. One file touched.

With ajv unresolvable, gate-53 fails closed ("ajv not resolvable ...
refusing to run fail-open") and four FAMILY D arms then read as gate
defects: D2 and D3 flip PASS->FAIL, the EventRoster name never appears,
and the pre-existing WARN never surfaces. That was reported upstream as
"the suite expects blocking where the gate was deliberately made
advisory". It does not, and the gate is correct -- with NODE_PATH set the
suite is ALL GREEN on main (c51a225).

Worse, arm D1 still printed 'ok'. It expects FAIL and got FAIL for a
completely unrelated reason: a false pass inside the suite built to catch
false passes. So this aborts (exit 2, zero verdicts) rather than warning.

ajv is resolved from the helpers' own directory -- the same place the
runner resolves it -- and the absolute path is printed as evidence,
because 'node -e "require(...)"' can succeed against an ANCESTOR
directory's node_modules and the exit code cannot tell you which.

Proven both directions: ajv absent -> exit 2, 0 verdict lines; ajv
present -> exit 0, ALL GREEN, empty stderr.
@rubenvdlinde
rubenvdlinde merged commit 651e5c5 into main Aug 9, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant