test(gates): refuse to run the 45–55 suite when ajv is unresolvable (it reported 4 gate-53 defects that do not exist) - #292
Merged
Conversation
With ajv unresolvable, gate-53 fails closed ("ajv not resolvable ...
refusing to run fail-open") and four FAMILY D arms then read as gate
defects: D2 and D3 flip PASS->FAIL, the EventRoster name never appears,
and the pre-existing WARN never surfaces. That was reported upstream as
"the suite expects blocking where the gate was deliberately made
advisory". It does not, and the gate is correct -- with NODE_PATH set the
suite is ALL GREEN on main (c51a225).
Worse, arm D1 still printed 'ok'. It expects FAIL and got FAIL for a
completely unrelated reason: a false pass inside the suite built to catch
false passes. So this aborts (exit 2, zero verdicts) rather than warning.
ajv is resolved from the helpers' own directory -- the same place the
runner resolves it -- and the absolute path is printed as evidence,
because 'node -e "require(...)"' can succeed against an ANCESTOR
directory's node_modules and the exit code cannot tell you which.
Proven both directions: ajv absent -> exit 2, 0 verdict lines; ajv
present -> exit 0, ALL GREEN, empty stderr.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
test_gate_45_to_55_acceptance.shreports four gate-53 defects that do not exist whenajvis not resolvable — and one of its arms passes for the wrong reason while doing so.Evidence
Fresh clone of
main(c51a225,git rev-list --left-right --count origin/main...HEAD=0 0), 0-byte stderr on both runs.Without
NODE_PATH— 4 failures:With
NODE_PATHset —ALL GREEN, exit 0.So gate-53 is correct and the suite is not stale. Both helpers behave properly in isolation:
check_manifest_crossref.json the D3 fixture exits 0 and emits the expectedWARNfor the pre-existing orphan. The gate fails closed on[gate-53] ... ajv not resolvable from <lib> — refusing to run fail-open, which is the right behaviour — but it makes every downstream assertion describe a gate that never ran.This was previously reported upstream as "the suite expects blocking behaviour for pre-existing orphans that #250/#260/#280 deliberately made advisory". That reading is wrong in both halves: the suite expects advisory, and the shipped gate delivers advisory.
The part worth keeping
🔑 Arm D1 still printed
ok. It expectsFAILand gotFAIL— because gate-53 failed closed, not because it detected the orphan. A false pass inside the suite built to catch false passes.That is why this aborts (exit 2, zero verdict lines) rather than warning and continuing. A wiring fault must not be able to produce a verdict at all.
The check itself
node -e "require('ajv')"is not a valid check on its own — Node resolves upward from the cwd, so it can succeed against anode_modulesbelonging to an ancestor directory rather than to the gates package. (Measured the same day: an agent's ajv check passed for exactly this reason.)So ajv is resolved from the helpers' own directory — the same place the runner resolves it from — and the absolute path is printed as the evidence:
The path is the evidence; the exit code cannot tell you which ajv answered.
Proven both directions
ALL GREEN, empty stderrNo gate logic changed. No assertion relaxed. One file touched.