Skip to content

fix(quality): an empty frontend-checks list DELETED the job from the run; frontend-tests ignored frontend-path - #150

Merged
rubenvdlinde merged 6 commits into
mainfrom
fix/frontend-checks-matrix-guard
Aug 4, 2026
Merged

fix(quality): an empty frontend-checks list DELETED the job from the run; frontend-tests ignored frontend-path#150
rubenvdlinde merged 6 commits into
mainfrom
fix/frontend-checks-matrix-guard

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

What was actually wrong

The reported symptom was quality / Frontend Check (${{ matrix.script }}) skipping on procest
(run 30894060736), read as "the matrix did not expand".

That diagnosis was wrong, and the diagnostic rule behind it is wrong. Measured:

  • frontend-checks expands and runs correctly wherever a caller supplies the input.
    petstore run 30897009428: Frontend Check (test:unit) and Frontend Check (check:specs), both green.
  • procest simply never sets frontend-checks, so it defaults to "[]" and the job's if: is false.
    That is a correct opt-out.
  • A literal ${{ matrix.x }} in a check list is the signature of a false if:, not of an empty
    matrix.
    In self-test run 30898711087, PHP Quality (${{ matrix.tool }}),
    License (${{ matrix.ecosystem }}) and PHPUnit (PHP ${{ matrix.php-version }}, ...) rendered
    unexpanded in exactly the same way — and all three of those matrices are hard-coded literal lists
    that cannot be empty. They were gated off by their if:, nothing more.

So I did not patch the reported symptom. Chasing it did surface two real defects.

Fix 1 — frontend-checks: the guard disagreed with the matrix

if: inputs.frontend-checks != '[]' was a two-character string comparison against a value the
matrix consumes through fromJSON. Every other spelling of empty — '[ ]', '[\n]', '[""]'
passes that test and then resolves to an empty matrix vector.

An empty matrix vector does not skip. Measured, run 30898927145 with frontend-checks: '[ ]':

jobs in run Frontend Check rows
before ('[ ]') 19 0 — job deleted outright
after ('[ ]') 20 1, skipped

No "skipping" row, no failure, no annotation. report has if: always(), so the run still goes
green. The gate disappears without a trace — strictly worse than the visible skip a false if:
produces. Routing both sides through fromJSON makes that state unreachable.

Fix 2 — frontend-tests ignored frontend-path

Found by running the self-test with frontend-path: ci-selftest. This was the only frontend job
with neither working-directory nor cache-dependency-path, so it probed the repo root:

  • measured outcome: setup-node hard-failed, "Dependencies lock file is not found in
    /home/runner/work/.github/.github"
    — a falsely RED gate blaming the app for the workflow's own
    mislocation;
  • quieter outcome where a root package.json exists: no test script found there, records
    "skipped", goes GREEN while the frontend's real suite never runs — reopening, for subdirectory
    repos, exactly the hole this job was added to close.

Now mirrors frontend-build: detect first (including the lockfile), then setup-node gated on the
result with cache-dependency-path.

Latent, not live — org-wide code search on 2026-08-04 found no caller setting a non-default
frontend-path; the only hit is this file. Fixed so that adopting frontend-path cannot silently
switch the unit gate off.

Evidence

.github has no caller of quality.yml, so I added a temporary self-test caller
(uses: ./.github/workflows/quality.yml) plus a fixture with a deliberately passing and a
deliberately failing npm script, ran it, then removed it — this PR touches only quality.yml.

run config result
30898711087 ["selftest:pass","selftest:fail"] (selftest:pass) ✅ · (selftest:fail) ❌ · Quality Report ❌
30898927145 '[ ]', pre-fix job vanished (19 jobs, 0 rows)
30899179109 post-fix, both scripts legs still red/green; Frontend Tests (unit) ❌→✅
30899314680 '[ ]', post-fix visible skipped row (20 jobs)
30899630764 failure removed all green, Quality Report ✅

Red and green in the same run, with a same-run control, and the failure propagated to
Quality Report — so the legs genuinely execute their script and genuinely block.

actionlint clean. Positive-controlled first: renaming the matrix key to scriptTYPO made it emit
5 errors, so the clean result is meaningful (this also rules out a key-name mismatch and wrong
nesting as causes).

Left alone deliberately

  • Features Extract skipping is legitimate. if: ... && github.event_name != 'pull_request',
    and the run was a PR. Its PR-side twin Features Check ran and passed. Not the same defect class;
    no change made.
  • No repo's enable-hydra-gates / enable-axe touched. No suppression baseline added or modified.
  • #148 (enable-axe/axe-routes/axe-version, needs: [playwright] on hydra-gates) verified
    intact; branched from main at 0d3e589.

@rubenvdlinde
rubenvdlinde merged commit dd6953f into main Aug 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant