Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -129,16 +129,18 @@ jobs:

browser:
if: github.event_name != 'workflow_dispatch'
name: Browser journeys (${{ matrix.engine }}, ${{ matrix.shard }}/2)
name: Browser journeys (${{ matrix.engine }}, ${{ matrix.shard }}/3)
# Measurements stay isolated on their own runner, not on our critical path.
# CI required still requires their success independently of these jobs.
runs-on: ubuntu-24.04
# Keep enough shards that slow runner setup cannot push a passing shard
# past this limit; two shards ran 11-14 minutes and were often cancelled.
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
engine: [chromium, webkit]
shard: [1, 2]
shard: [1, 2, 3]
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
Expand All @@ -161,7 +163,7 @@ jobs:
- name: Functional journeys
env:
PLAYWRIGHT_JSON_OUTPUT_FILE: test-results/browser/ci-report.json
run: node scripts/ci-test-report.mjs kind=playwright report=test-results/browser/ci-report.json evidence=test-results/browser/ci-timing.json title="Browser journeys (${{ matrix.engine }}, ${{ matrix.shard }}/2)" -- pnpm test:browser:ci --project ${{ matrix.engine }} --no-deps --shard=${{ matrix.shard }}/2 --reporter=list,json
run: node scripts/ci-test-report.mjs kind=playwright report=test-results/browser/ci-report.json evidence=test-results/browser/ci-timing.json title="Browser journeys (${{ matrix.engine }}, ${{ matrix.shard }}/3)" -- pnpm test:browser:ci --project ${{ matrix.engine }} --no-deps --shard=${{ matrix.shard }}/3 --reporter=list,json
- name: Browser evidence
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
Expand Down
8 changes: 4 additions & 4 deletions docs/browser-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
`pnpm test` and therefore `just scan` now run the checked-in Chromium and WebKit
journeys in `tests/browser/`. `just iterate` remains the fast, browser-free loop.
[GitHub Actions](../.github/workflows/ci.yml) runs these gates on PRs and main
pushes: an isolated serial measurement job runs alongside four functional jobs
(two file-level shards per engine, two workers per runner). Both engines remain
pushes: an isolated serial measurement job runs alongside six functional jobs
(three file-level shards per engine, two workers per runner). Both engines remain
mandatory for the [CI-selected coverage](#ci-coverage-and-local-only-webkit-checks);
`CI required` fails if any lane or shard fails, is cancelled, or is skipped.
Owner-specific tests are colocated in `src/` and `dev/`; separate manual diagnostic
Expand Down Expand Up @@ -106,9 +106,9 @@ instances and every untagged WebKit case remain required. Both engines, serial
measurement order, zero retries, the documented functional assertions and
measurement ceilings, and the strict `CI required` aggregate remain in place.

CI shards each functional engine across two runners, without waiting for the
CI shards each functional engine across three runners, without waiting for the
separate measurement runner. Each job selects its engine with `--no-deps` and
`--shard=N/2`; measurement success is enforced by `CI required`, not job ordering.
`--shard=N/3`; measurement success is enforced by `CI required`, not job ordering.
This preserves measurement isolation while spending more setup/runner minutes,
including when measurements fail. Local same-runner dependencies remain unchanged.
Artifacts include engine and shard so parallel jobs never overwrite one another.
Expand Down
2 changes: 1 addition & 1 deletion docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,7 @@ the complete suite still runs with `pnpm test` / `just scan`:
tests build Rust and install scaffold dependencies; they are intentionally CI-only
rather than part of pre-push.
- **Browser measurements:** Chromium then WebKit, serially on an isolated runner.
- **Browser journeys:** four runners (Chromium and WebKit, two file-level shards
- **Browser journeys:** six runners (Chromium and WebKit, three file-level shards
per engine), each with two workers. They start alongside measurements on separate
runners; `CI required` still requires both lanes. Each runner builds the native
plugin-manager fixture in a separately logged setup step before starting
Expand Down
4 changes: 2 additions & 2 deletions tests/integration/browser-ci.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ const matrixValues = (key) => {
return values.split(",").map((value) => value.trim());
};

test("four independent browser jobs retain isolated measurements and native setup", () => {
test("six independent browser jobs retain isolated measurements and native setup", () => {
assert.deepEqual(matrixValues("engine"), ["chromium", "webkit"]);
assert.deepEqual(matrixValues("shard"), ["1", "2"]);
assert.deepEqual(matrixValues("shard"), ["1", "2", "3"]);
assert.doesNotMatch(browser, /^ {4}(needs|continue-on-error):/m);
assert.doesNotMatch(browser, /^ {8}(include|exclude):/m);
assert.match(browser, /^ {6}fail-fast: false$/m);
Expand Down
Loading