ci: shard Vitest and run browser jobs in the pinned Playwright image - #466
Closed
kalvinnchau wants to merge 5 commits into
Closed
kalvinnchau wants to merge 5 commits into
kalvinnchau wants to merge 5 commits into
Conversation
Signed-off-by: tulsi <tulsi@block.xyz> (cherry picked from commit 3314dfb) Signed-off-by: peon <9ac6794b000690b7e814eb1805ad32405d0bec7d52838de3a86cf967565dacc0@buzz.block.builderlab.xyz>
Signed-off-by: tulsi <tulsi@block.xyz> (cherry picked from commit cf4a66b) Signed-off-by: peon <9ac6794b000690b7e814eb1805ad32405d0bec7d52838de3a86cf967565dacc0@buzz.block.builderlab.xyz>
Signed-off-by: tulsi <tulsi@block.xyz> (cherry picked from commit 8f4346a) Signed-off-by: peon <9ac6794b000690b7e814eb1805ad32405d0bec7d52838de3a86cf967565dacc0@buzz.block.builderlab.xyz>
Signed-off-by: tulsi <tulsi@block.xyz> (cherry picked from commit b07233d) Signed-off-by: peon <9ac6794b000690b7e814eb1805ad32405d0bec7d52838de3a86cf967565dacc0@buzz.block.builderlab.xyz>
The JavaScript job ran all Vitest files on one runner with two workers and set the floor for CI wall time together with the WebKit shards. Move Vitest into its own three-way matrix job so lint, types and the frontend build still run once. Each shard keeps the two-worker contention limit. CI required now needs the vitest job, and a guard pins the shard denominator, the worker limit and the single sharded invocation. Co-authored-by: peon <9ac6794b000690b7e814eb1805ad32405d0bec7d52838de3a86cf967565dacc0@buzz.block.builderlab.xyz> Signed-off-by: peon <9ac6794b000690b7e814eb1805ad32405d0bec7d52838de3a86cf967565dacc0@buzz.block.builderlab.xyz>
kalvinnchau
marked this pull request as ready for review
September 30, 2026 18:33
kalvinnchau
requested review from
a team,
comp615 and
wesbillman
as code owners
September 30, 2026 18:33
kalvinnchau
marked this pull request as draft
September 30, 2026 18:33
kalvinnchau
marked this pull request as ready for review
September 30, 2026 18:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two CI changes, in one PR so their effect can be watched together.
vitestmatrix job runsvitest run --shard=N/3on three runners.JavaScriptkeeps Biome, the TypeScript check and the frontend build, so those still run once. Each shard keepsBUZZ_TEST_WORKERS=2.CI requirednow needsvitest.3314dfbc,cf4a66be,8f4346a0,b07233d9; import-block conflicts with Show agent activity in navigation #423 inbrowser-ci.test.mjsresolved) so they can land separately from the inbox feature.measurementsandbrowserrun inmcr.microsoft.com/playwright:v1.63.0-noblepinned by digest, with noapt-getorplaywright install. A newbrowser_fixturejob buildsfixture-bridgeonce; the twelve shards download and verify it instead of each installing Rust and building it.No test, assertion, engine or shard count is removed. Playwright workers and the 2×6 browser matrix are unchanged.
Why
Hosted CI on
main, 77 successful first-attempt runs between 2026-09-29T21:29Z and 2026-09-30T17:42Z:JavaScriptjobJavaScriptjob, on one runner with two workers (run 36671118189,5d2b08e2: 475 s).apt-getagainstazure.archive.ubuntu.com. Onmainevery browser job runs apt; after this change onlyRust and tool integrationdoes.Timings
Vitest, local only (Apple M3 Max, Node 24.18.0, at
1c75f6c6),BUZZ_TEST_WORKERS=2 bin/pnpm exec vitest run --shard=N/3 --reporter=json:The three JSON reports together contain all 463 files from
vitest list --filesOnlyexactly once (no duplicates, none missing), 5,698 tests, all passing. Hosted estimate, not yet measured: about 3–3.5 min per shard job against 8.3 min today.Browser jobs, hosted CI. Time from run start to the start of the
Functional journeysstep, across the twelve shards:main5d2b08e2, healthy mirror)mainbfe4c7f1, slow mirror)982ce489)In the #422 run the fixture job took 78 s and each shard then spent 47–71 s before its journeys started. With a healthy mirror, journeys therefore start about 50 s later than on
maintoday, because every shard waits forbrowser_fixture. With a slow mirror they start several minutes earlier. #422 carries different browser tests, so its journey durations are not comparable tomain; the first run on this PR is the like-for-like measurement.Known risks
browser_fixtureis on the critical path for all twelve shards. It took 288 s in run 36751775314 (233 s insideSwatinem/rust-cachebefore it printed its configuration, then a cache miss) and 78 s in run 36755437303. The Rust cache is saved only on pushes tomain, so runs on this PR will be cold.message-actions.spec.mjs(Chromium 2/6) andsidenav-polish.spec.mjs(WebKit 3/6) failed once. feat: add inbox conversations and draft previews #422 later added982ce489, which waits for popup and wheel completion in those two specs. It is not included here. It applies cleanly to this branch if the failures reproduce.vitestjob gets its own Hermit cache key, so its first run provisions tools cold.Validation
bin/node --test tests/integration/browser-ci.test.mjs: 11 pass, including the new guard on the shard denominator, worker limit and single sharded invocation.bin/node --test tests/integration/ci-test-report.test.mjs: 5 pass.bin/pnpm exec biome check --error-on-warningson the changed JS files: clean..githooks/pre-pushrun by hand (the worktree hook installer refuses because a globalcore.hooksPathis set): no JS, design-system or Rust inputs selected.Deferred checks
vitestshards and for the browser lane onmain's test set: this PR's first CI run.browser_fixtureonmainafter merge.JavaScriptcheck by name for Vitest coverage;CI requiredcovers the new job.