fix(axe): the axe runner could never resolve @playwright/test - #338
Merged
Conversation
The runner is executed by absolute path from `${GITHUB_WORKSPACE}/gates/`
while the packages it needs are installed into
`server/apps/<app>/node_modules`. CommonJS resolves a module's own
`require` calls by walking up from the directory of the ENTRY FILE, not
from the process cwd, and those two paths are siblings — so the walk
never reaches the app's node_modules and `require('@playwright/test')`
throws MODULE_NOT_FOUND on the first repo that ever opted in.
That is docudesk#423: `enable-axe: true`, Playwright green, 82 e2e tests
passed, and then the axe step died on `Cannot find module
'@playwright/test'` — a failure with nothing to do with accessibility.
The step above it printed `@axe-core/playwright resolved.` on the same
run. It used `node -e`, which resolves from the cwd the step had just
`cd`-ed into, so it answered a question nobody asked: it measured
whether the package was on disk, never whether the runner could reach
it. It would have printed the same line on every future break of this
kind.
Two changes, one root cause:
* `NODE_PATH="${PWD}/node_modules"` on the runner invocation. NODE_PATH
is a global fallback consulted regardless of where the entry file
sits, which is exactly the property the cross-tree layout needs.
* the control now resolves through `createRequire` anchored ON THE
RUNNER, and checks both `@axe-core/playwright` and `@playwright/test`.
It therefore fails precisely when the runner would.
Measured on a reconstruction of the runner's real layout, with the
actual `axe-run.cjs`:
runner, no NODE_PATH -> Error: Cannot find module '@playwright/test'
runner, NODE_PATH set -> ::error::axe-runner: AXE_BASE_URL is empty.
old control -> PASS in both arms (this is the defect)
new probe, no NODE_PATH-> exit 1, MODULE_NOT_FOUND
new probe, NODE_PATH -> PASS
The second line is the proof: the runner gets past resolution and
reaches its own env validation. The third is why this went unseen.
Both `run:` blocks stay far under the ~20 KB per-step limit that made
this workflow unresolvable fleet-wide (2.1 KB and 2.6 KB).
This was referenced Aug 11, 2026
rubenvdlinde
added a commit
to ConductionNL/zaakafhandelapp
that referenced
this pull request
Aug 11, 2026
gate-33 (axe-core) has reported NOT APPLICABLE on every run this repo has ever made, because `enable-axe` was never set. That is not a pass: runtime accessibility — contrast, landmarks, ARIA validity, live regions — has never been measured here at all, by a choice recorded in this file. The recorded reason was "a vanilla Nextcloud 34 already carries serious/critical violations from core's own UI". An exemption's reason is a testable claim, and this one names a STATE OF THE WORLD rather than a test artifact — so it rots. It has: 1. ConductionNL/.github#161 (re-landed as #168) gave the axe runner an AxeBuilder.include SCOPE, so core's chrome is no longer in the measurement. It is a scope, not a mute, and the runner proves that on every run — it injects a deliberate violation INSIDE the container and asserts axe reports it, then injects the same violation OUTSIDE and asserts axe does not. A selector matching nothing is a hard failure, not an empty pass. 2. ConductionNL/.github#338 fixed an axe runner that had NEVER resolved @playwright/test in any repo — so nobody could have enabled this before tonight even if they had wanted to. Measured rather than inherited: docudesk runs with `enable-axe: true` today and its gate run reports `[gate-33] axe-core: PASS` (run 31461514843). The blocker is gone, so the excuse does not hold here either. `axe-include-selector: '#zaakafhandelapp-app'` — NOT the `#content-vue, #content` default. templates/index.php:34 renders exactly `<div id="zaakafhandelapp-app">` and src/main.js:151 mounts there, because Vue 3's mount() renders INSIDE the match and mounting on `#content` would nest the app inside Nextcloud's own wrapper. Core's layout wraps our div in `#content`, so the DEFAULT selector would match and drag core's navigation, header and app menu back into scope — re-attributing core's violations to this app, which is exactly what the old opt-out was worried about. This selector is this app's own rendered DOM and nothing else. No baseline is taken with this. DOM scoping excludes TERRITORY; a baseline suppresses FINDINGS, including future ones on our own markup. If this run reports violations in our own DOM they are ours, and the fix is the markup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
gate-33 has never been able to run in any caller repo
enable-axe: trueexecutes the runner by absolute path:The packages it requires are installed by the step above into
server/apps/<app>/node_modules. CommonJS resolves a module's ownrequirecalls by walking up from the directory of the entry file — never from the process cwd — andgates/is a sibling ofserver/. The walk goesgates/hydra-gates/scripts/node_modules,gates/hydra-gates/node_modules,gates/node_modules,<workspace>/node_modules, and stops. It never entersserver/.So
require('@playwright/test')throwsMODULE_NOT_FOUNDon the first repo that ever opted in — ConductionNL/docudesk#423, where Playwright itself was green and all 82 e2e tests passed before the axe step died on a message that names neither accessibility nor the layout that caused it.The control above it passed for the wrong reason
node -eresolves from the cwd the step had justcd-ed into. It measured whether the package was on disk, not whether the runner could reach it — and printed@axe-core/playwright resolved.on the very run whose next step could resolve nothing. It would print the same line on every future break of this kind.The fix
NODE_PATH="${PWD}/node_modules"on the runner invocation — NODE_PATH is a global fallback consulted regardless of where the entry file sits, which is the property this cross-tree layout needs.createRequireanchored on the runner, and checks@playwright/testtoo. It fails exactly when the runner would.Both directions, on the real
axe-run.cjsReconstructed the runner's real directory layout (
gates/sibling toserver/apps/dd/node_modules) and ran the actual file:NODE_PATHError: Cannot find module '@playwright/test'NODE_PATHset::error::axe-runner: AXE_BASE_URL is empty.PASS— this is the defectNODE_PATHMODULE_NOT_FOUNDNODE_PATHsetPASSRow 2 is the proof the fix works: the runner gets past resolution and reaches its own env validation. Row 3 is why this was invisible.
Step size
The ~20 KB per-step limit that once made this workflow resolve to zero jobs is respected: the two touched
run:blocks are 2.1 KB and 2.6 KB.