ci(quality): add a standalone frontend build gate and hand its output to the Playwright job - #133
Conversation
… to Playwright
`npm run build` lived in exactly two places in this workflow — inside the
Playwright job and inside the journeydoc capture job — and both are opt-in
and default OFF. Of the 21 repos that call quality.yml, 2 enable Playwright
(doriath, pipelinq) and 0 enable journeydoc capture. So for 19 repos the
frontend has never been compiled in CI at all: a repo could ship a bundle
that does not build and every gate stayed green, because nothing in the
pipeline ever ran webpack.
hermiq is the proof. `node-polyfill-webpack-plugin: ^4.0.0` floated to
4.1.0 and webpack emitted 25 errors; CI had nothing to say. Nine repos
still carry an unpinned caret on that same plugin, so any lockfile
regeneration can reproduce it silently.
New `frontend-build` job, modelled on the existing auto-detected
`frontend-tests`: no opt-in input, runs `npm ci && npm run build` when the
repo has a package.json with a `build` script, blocks on failure, and is
wired into the `Quality Report` gate. Repos without a package.json, without
a lockfile, or without a `build` script skip cleanly — and unlike
frontend-tests they skip *before* setup-node, so a repo without a lockfile
cannot be newly broken by this job.
timeout-minutes: 30. The step has never run standalone, so this is sized
from its embedded proxies: `npm run build` measured at 211 s under heavy
load, and the frontend-checks legs that wrap it peak at 2.1 min end to end.
30 min is ~8x the worst observed build. Deliberately generous — a bound
that fires under normal contention yields a CANCELLED job, which returns no
verdict at all, which is strictly worse than no bound.
The build output is uploaded as `frontend-build-output` and the Playwright
job now restores it instead of running its own identical build. The
hand-off is a fast path with a full fallback, never a replacement:
- the download is `continue-on-error`, keyed off a job output that is only
'true' when the build ran AND produced files;
- the restore asserts the tarball is non-empty and actually contains
JavaScript before extracting;
- anything else — no artifact, empty artifact, no .js, or a repo with a
non-default `frontend-path` — falls through to the pre-existing build
step, byte for byte.
That asymmetry is deliberate. A missing or wrong bundle does not 404: the
front controller answers 200 text/html, so a status-code probe reads it as
success and the only symptom is a selector timeout naming an element. The
repos whose ci-seed.sh asserts the response is really JavaScript keep that
guard meaningful, because the reuse path proves the same thing before it is
taken. Cost of the fallback: ~3 minutes. Cost of a bad hand-off: a false
verdict on every UI spec.
What this cannot break, explicitly:
- It adds no URL and no path assumption. Nothing here references
/apps/<app> or /custom_apps/<app>; the tarball is extracted into the app
checkout at the exact relative paths the build wrote, discovered with a
timestamp marker rather than a hardcoded js/ or dist/.
- `frontend-build` is in playwright's `needs` only so the artifact is
downloadable. The job's `if:` contains `!cancelled()`, which suppresses
the implicit success() on needs, so a failed or skipped frontend-build
still lets Playwright run and reach its own verdict.
- No existing job's steps, order or conditions change apart from the added
`if:` on Playwright's build step, whose false branch is the current
behaviour.
Verified: actionlint clean (and positive-controlled — an injected bad
`needs` is reported, so the file is genuinely being checked); shellcheck
findings unchanged at 14 before and after; YAML re-parsed, job count 17 -> 18
and report.needs 14 -> 15, exactly +1 each; the package/restore tar
round-trip exercised locally including the negative control that a tarball
with no .js sets restored=false and falls back to building.
Live verification on the branch — both controls firedTwo throwaway PRs pinned Happy path + artifact hand-off — ConductionNL/keepiq#141 (closed unmerged)Run 30831935876. doriath is the only repo in the fleet whose Playwright job is currently green, so it is the one target where a regression would be visible. The Playwright job's step outcomes are the actual evidence — a green E2E alone would not prove the reuse happened: Controlled against the identical head SHA. Baseline run 30822156638 ran on
Zero new failures. The job-count delta is fully accounted for: Negative control — ConductionNL/petstore#15 (closed unmerged)Run 30832016276. petstore Injected
Both test branches have been deleted. |
Post-merge verification — 19 fresh
|
| repo | Frontend Build | Download | Restore | Build app frontend | Playwright |
|---|---|---|---|---|---|
| doriath 30833303244 | ✅ | ✅ | ✅ | skipped | ✅ 56 tests, 53 passed (4.8m) |
| pipelinq 30833307592 | ✅ | ✅ | ✅ | skipped | pre-existing red, see below |
doriath's own history is the control: 56 tests / 51 passed / 2 flaky when it built in-job, 56 / 53 reusing the artifact.
pipelinq's E2E was already red before this change — run 30830273021 at 16:02, 38 minutes pre-merge: 211 tests, 61 failed, on ObjectService::find(): Argument #2 ($_extend) must be of type ?array, string given at Customer360Controller.php:142. An app bug, not a bundle problem.
Fleet coverage
| outcome | n | repos |
|---|---|---|
Frontend Build ✅ |
13 | doriath, pipelinq, openregister, opencatalogi*, docudesk, planix, decidesk, launchpad, openbuild, petstore, portaliq, scholiq, shillinq, zaakafhandelapp |
Frontend Build ❌ |
3 | deskdesk, app-versions, nextcloud-app-template |
| skipped / not observed | 4 | hydra, mydash, larpingapp, opencatalogi@main |
All three failures are npm ci, not npm run build, and none changes a repo's verdict — every one of those runs was already red from Vue Quality, Security (npm) and License (npm), which run the same npm ci:
- deskdesk —
ERESOLVE,@nextcloud/eslint-config@8.4.2vs@typescript-eslint/utils@7.18.0 - app-versions — lockfile out of sync,
Missing: picomatch@4.0.5 from lock file - nextcloud-app-template — lockfile out of sync,
Missing: text-table@0.2.0 from lock file
petstore went fully green end to end with the new job in place.
Not caused by this change (verified, not assumed)
- opencatalogi@main / hydra@main →
startup_failure. Their callers declare apermissions:ceiling narrower than the shared workflow'scontents: write/actions: writejobs, which GitHub validates statically even for disabled jobs. Those declarations are in00c11a1, before this PR, and this commit adds zeropermissionslines. Proof rather than assertion: dispatching opencatalogi ondevelopment— whose caller has the wide ceiling — against the very same merged@maingives a fully successful 27-job run (30833733008) withFrontend Build✅. hydra has beenstartup_failuresince 08:33, 8 hours before the merge. - mydash — run queued 30+ minutes and never picked up a runner; last successful run was in May. Unobserved, unrelated.
- larpingapp — its caller has no
workflow_dispatchtrigger, so it could not be dispatched. The gate will run on its next push or PR.
The build row renders in the Quality Report and result-frontend-build is downloaded by it.
Correction / addendum — pipelinq's Playwright job was cancelled at the 45-minute capRun 30833307592: Not caused by this change, and the evidence is in the step timings. The whole setup phase — checkout, Nextcloud install, npm ci, Playwright install, artifact download, restore — took 2m10s, of which this change accounts for 3 seconds ( pipelinq
The 16:02 baseline I quoted earlier ran So doriath is the one repo where the artifact hand-off can be shown end-to-end against a green Playwright suite, and it is green. On pipelinq the hand-off demonstrably works (download ✅, restore ✅, build skipped) but the suite itself cannot reach a verdict inside its cap — a pre-existing pipelinq problem, worth its own issue: a 1-worker 232-test suite against a 45-minute bound. |
Why
npm run buildlived in exactly two places inquality.yml— inside the Playwright job and inside the journeydoc capture job — and both are opt-in and default OFF.Surveying all 21 repos that call this workflow: 2 enable Playwright (doriath, pipelinq) and 0 enable journeydoc capture. So for 19 repos the frontend has never been compiled in CI at all. A repo could ship a bundle that does not build and every gate stayed green, because nothing in the pipeline ever ran webpack.
That is not theoretical. hermiq's build broke when the floating
node-polyfill-webpack-plugin: ^4.0.0resolved to 4.1.0 and webpack emitted 25 errors — with CI green throughout. Nine repos still carry an unpinned caret on that same plugin, so any lockfile regeneration reproduces it silently:^4.0.0^4.1.0^3.0.0What changes
1. New
frontend-buildjobModelled on the existing auto-detected
frontend-tests— no opt-in input. It runsnpm ci && npm run buildwhenenable-frontendis true and the repo has apackage.jsonwith abuildscript plus apackage-lock.json, and it blocks on failure through theQuality Reportgate.Repos missing any of those skip cleanly — and unlike
frontend-teststhey skip beforesetup-node, so a repo without a lockfile cannot be newly broken by this job.timeout-minutes: 30. The job has never existed, so this is sized from its embedded proxies:npm run buildmeasured at 211 s under heavy load, and thefrontend-checkslegs that wrap it peak at 2.1 min end to end. 30 min is ~8x the worst observed build. Deliberately generous — a bound that fires under normal contention produces a cancelled job, and a cancelled job returns no verdict at all, which is strictly worse than no bound.2. Playwright consumes the artifact instead of rebuilding
The output is uploaded as
frontend-build-output(tarred, 1-day retention) and the Playwright job restores it in place of its own identicalnpm run build.The hand-off is a fast path with a full fallback, never a replacement:
continue-on-error, keyed off a job output that is'true'only when the build ran and produced files;.js, or a repo with a non-defaultfrontend-path— falls through to the pre-existing build step, byte for byte.That asymmetry is deliberate. A missing or wrong bundle does not 404 — the front controller answers
200 text/html, so a status-code probe reads it as success and the only symptom is a selector timeout naming an element. The repos whoseci-seed.shasserts the response is really JavaScript keep that guard meaningful, because the reuse path proves the same thing before it is taken. Cost of the fallback: ~3 minutes. Cost of a bad hand-off: a false verdict on every UI spec in the fleet.The output location is discovered, not assumed: a timestamp marker before the build, then everything the build touched outside
node_modules/.git. Nextcloud apps emit tojs/, vite apps todist/, portaliq writes two bundles from a composite script — hardcoding any of those is exactly how this would ship an empty bundle.What this change cannot break
/apps/<app>or/custom_apps/<app>. The tarball extracts into the app checkout at the exact relative paths the build wrote.frontend-buildis inplaywright.needsonly so the artifact is downloadable — it is not a gate. The Playwright job'sif:contains!cancelled(), which suppresses the implicitsuccess()onneeds, so a failed or skippedfrontend-buildstill lets Playwright run and reach its own verdict.if:on Playwright's build step — whose false branch is today's behaviour.code-quality.yml) are untouched.Verification
needsis reported at the new job, proving the file is genuinely being checked rather than silently skipped.actionlint -shellcheck).report.needs14 → 15 — exactly +1 each, nothing else moved..jssetsrestored=falseand falls back to building.Both are throwaway PRs on new branches — genuinely new runs, not re-runs, since a re-run replays the original workflow resolution and would not exercise this change at all.
Blast radius
All 21 callers pin
quality.yml@main, so this takes effect everywhere on merge. Expected: the gate runs on 20 repos and skips 1 (hydra —enable-frontend: false, nopackage.json). Any repo whose frontend does not currently compile will newly go red. That is the point of the change, and it is a true finding rather than a regression, but it is a real change in fleet verdicts.