ci(efficiency): reduce redundant cost and latency without weakening required quality/security gates

[qnbs]

Purpose

Create one bounded S12 owner for CI efficiency, latency and duplication. The repository already has strong granular quality/governance trackers; what is missing is an integration-level optimization contract that decides where each expensive check belongs, how often it needs to run, and how to reduce repeated work without weakening evidence.

This issue is explicitly about execution efficiency. It must not become a pretext to lower thresholds, remove coverage/security checks, or bypass branch protection.

Existing specialist owners to reconcile, not duplicate

• test(ci): expand coverage scope to executable api, workers, and contexts #446 — executable coverage scope and ratchet graduation;
• ci(performance): make bundle budgets evidence-based regression budgets #450 — evidence-based bundle budgets;
• workflow-policy-check: compound if: expressions + cross-workflow gate coverage #506 — workflow-policy parser/cross-workflow coverage gaps;
• ci: write-scoped governance jobs (pr-size, workflow-policy) enforce from PR-controlled code, not a trusted ref #510 — trusted execution boundary for write-scoped governance jobs;
• tooling: define hermetic exact-tree validation / compiler sandbox boundary #504 — hermetic exact-tree validation architecture question;
• native: promote macos-15-intel qualification lane into the release matrix #507 — Intel macOS qualification/release-cost decision;
• supply-chain: document trust-policy gaps across Docker/GHCR, Tauri release artifacts, and pnpm dependency provenance #529 — supply-chain/provenance checks where CI is part of enforcement.

This issue owns the whole-pipeline cost/latency model and de-duplication plan, not those specialist semantics.

Inventory first

For every workflow/job on main, pull requests, tags, schedules and manual dispatch, record:

• trigger;
• required vs advisory status;
• runner/OS/runtime matrix;
• median/p95 queue time and execution time where observable;
• duplicated setup/build/test work;
• cache inputs/outputs and hit assumptions;
• artifact production/consumption;
• whether result is needed for every PR, only affected paths, main, release, schedule, or evidence cadence;
• security/trust reason a job cannot safely reuse PR-controlled artifacts;
• failure signal uniqueness (what defect class only this job catches).

Optimization principles

1. Preserve evidence before reducing frequency. A slow check that uniquely catches a real class of failures is not redundant merely because another green job looks similar.
2. Share deterministic artifacts only across compatible trust boundaries. Do not let untrusted PR artifacts become authority for write-scoped/release/security jobs.
3. Prefer one build, many consumers when platform/runtime semantics genuinely match.
4. Use path/change awareness only when sound. Shared configuration, generated files, workspace packages and transitive dependencies must invalidate affected gates correctly.
5. Separate fast merge feedback from deep evidence cadence. Expensive mutation/soak/nightly/browser/platform work can be scheduled if merge-safety coverage remains adequate.
6. Do not optimize by weakening thresholds, skipping required matrices, or hiding flaky failures.

Specific audit targets

• duplicate dependency installation/setup across jobs;
• repeated production builds that can safely consume one immutable artifact;
• repeated i18n/docs/parity/governance scans with identical inputs;
• Node 22/24 matrix value versus duplicated work, preserving compatibility evidence;
• Storybook/VRT/Lighthouse execution topology;
• Playwright vs Deep E2E overlap and unique scenarios;
• CodeQL/Semgrep/OSV/Socket/GitGuardian boundaries and trigger cadence;
• Tauri platform builds versus qualification/release-only lanes;
• cache-key breadth and stale-cache correctness;
• concurrency cancellation of superseded PR runs;
• generated artifact reuse and retention;
• merge-queue/main/tag duplicate cycles;
• reviewer/provider integrations that trigger repeatedly on correction commits but add little new signal.

Required measurements

Before and after each optimization, record at minimum:

PR feedback median/p95
full required-check wall clock
runner minutes by OS/runtime
cache hit/miss where measurable
number of duplicated build/install operations
failure-detection coverage retained

Cost optimization should be expressed as evidence, not intuition.

Safety gates

Any proposed optimization must answer:

• Can a changed file influence a skipped job indirectly?
• Does artifact reuse cross a trust boundary?
• Could cancellation leave an old/stale result satisfying a required context?
• Does a matrix reduction remove a supported runtime/platform oracle?
• Does cache reuse hide dependency/config drift?
• Does the change weaken test(ci): expand coverage scope to executable api, workers, and contexts #446/ci(performance): make bundle budgets evidence-based regression budgets #450/workflow-policy-check: compound if: expressions + cross-workflow gate coverage #506/ci: write-scoped governance jobs (pr-size, workflow-policy) enforce from PR-controlled code, not a trusted ref #510 or supply-chain guarantees?

If uncertain, preserve the stronger current behavior until deterministic proof exists.

Acceptance criteria

• A current workflow/job cost and latency inventory exists.
• Every required check has a documented unique purpose or is consolidated safely.
• Duplicate setup/build/test work is reduced where trust/input semantics allow it.
• Superseded PR runs cancel safely where appropriate.
• Cache/artifact reuse has deterministic invalidation and trust-boundary rules.
• Expensive advisory/deep evidence runs have an explicit cadence rather than accidental duplication.
• Node/platform/browser matrices retain evidence-backed support coverage.
• No quality, Codecov, security, signatures, governance or branch-protection requirement is weakened merely for speed/cost.
• Before/after wall-clock and runner-minute evidence is recorded.
• Specialist follow-ups (test(ci): expand coverage scope to executable api, workers, and contexts #446/ci(performance): make bundle budgets evidence-based regression budgets #450/workflow-policy-check: compound if: expressions + cross-workflow gate coverage #506/ci: write-scoped governance jobs (pr-size, workflow-policy) enforce from PR-controlled code, not a trusted ref #510/tooling: define hermetic exact-tree validation / compiler sandbox boundary #504/native: promote macos-15-intel qualification lane into the release matrix #507/supply-chain: document trust-policy gaps across Docker/GHCR, Tauri release artifacts, and pnpm dependency provenance #529) remain individually owned and reconciled.

Non-goals

• making CI green by deleting tests;
• lowering coverage/bundle/security thresholds;
• bypassing protected-branch/ruleset requirements;
• trusting PR-controlled code/artifacts in privileged workflows;
• eliminating deep scheduled evidence merely because it is slow;
• optimizing reviewer/model quotas by suppressing genuinely useful findings.

[qnbs]

Progress reconciliation — CI-performance Waves 1–2 landed

This umbrella remains open, but two concrete optimization slices are already complete and should not be rediscovered as future work:

Wave 1 — pnpm bootstrap variance: DONE via #604

pnpm/action-setup's npm-based self-install was measured as the dominant setup-variance source (roughly 20s–6m in sampled jobs while warm-store dependency install itself stayed much smaller). #604 replaced both relevant bootstrap call sites with pnpm/setup's standalone binary path while preserving the existing dependency-store cache and trust-boundary semantics.

Wave 2 — Browser Quality duplicate setup: DONE via #608

The former separate VRT and Lighthouse jobs were consolidated into one browser-quality job, sharing setup/build-artifact acquisition while preserving independent diagnostic execution and required/advisory failure semantics. The change reduced duplicated runner work without removing VRT or Lighthouse evidence.

Remaining #550 scope

Keep this issue open for the still-unfulfilled whole-pipeline acceptance criteria: current cost/latency inventory, any further safe artifact/setup reuse, cancellation/cache analysis, matrix evidence decisions, and before/after runner-minute/critical-path measurements.

Do not repeat #604/#608, and do not optimize by weakening #446/#506/#510, browser/platform coverage, security gates, or required ✅ CI Success semantics.

[qnbs]

Playwright scale-out addendum — measure first, shard before raising workers or changing frameworks

#736 now owns the required Playwright gate architecture. #550 remains the CI cost/latency/duplication authority and should supply the evidence that determines when/how that gate scales.

Preserve current runner-resource discipline

Current playwright.config.ts deliberately uses:

workers: process.env['CI'] ? 1 : 4

Do not treat workers: 1 as an accidental inefficiency. On GitHub-hosted runners it is currently a resource-safety choice that limits concurrent browser RAM/CPU pressure and makes startup/persistence races easier to diagnose.

Any proposal to raise CI workers must first measure:

runner memory pressure / OOM incidence
CPU saturation
first-attempt flake rate
startup/persistence race incidence
wall-clock change
runner-minute change

Preferred first scale-out mechanism

If the required ordinary E2E suite exceeds its acceptable feedback budget, prefer Playwright-native sharding before a framework migration or high in-process worker count.

Initial candidate topology:

matrix shard: [1/2, 2/2]
command: playwright test --shard=<matrix value>
workers: 1 inside each process

Do not implement sharding merely because #736 exists. First establish the trigger from measured current behavior.

Required measurements for the sharding decision

Record at minimum over a representative recent window:

required E2E p50 wall clock
required E2E p95 wall clock
GitHub queue time
runner minutes per successful run
Desktop Chromium runtime
Mobile Chrome runtime
slowest specs / long-tail distribution
retry / flaky incidence
browser cache hit/miss behavior
artifact upload size/time
cancelled superseded PR-run frequency

Set the scale-out trigger from evidence, not a fixed test-count threshold.

Examples of possible evidence-based triggers to evaluate, not pre-authorize:

• required E2E p95 materially dominates merge feedback;
• job approaches the configured timeout with insufficient safety margin;
• new test(ai-e2e): qualify Copilot, Character Interviews, ProForge and mutation-capable AI user flows #710/test(opt-in): qualify Settings admission, persistence and network/runtime safety for default-off features #711 required scenarios create a persistent long tail;
• two shards reduce wall clock materially without unacceptable runner-minute/cost increase or new flakes.

Sharding safety invariants

Any sharded implementation must prove:

• every required spec executes exactly once across the matrix;
• every shard is part of the blocking aggregate / ci-success contract;
• a cancelled or failed shard cannot be hidden by another shard succeeding;
• no test relies on execution order or shared mutable state across shards;
• JUnit/Codecov Test Analytics can identify shard + Playwright project;
• traces/screenshots/HTML reports remain discoverable per shard or merge deterministically;
• artifact names cannot collide;
• browser cache keys remain coupled to toolchain(e2e): reconcile Playwright test/core/browser versions and define deterministic browser-update policy #571's Playwright/browser revision authority;
• shard count can be adjusted without changing test semantics;
• PR concurrency cancellation remains safe.

Prefer Playwright-supported report/artifact merge facilities if report aggregation is needed rather than custom brittle concatenation.

Required gate stays lean even after sharding

Sharding is a latency optimization, not permission to move all deep tests into required CI.

#736 keeps the required gate focused on smoke + critical journeys. #623 continues to govern advisory promotion. #709 decides coverage disposition. #710/#711 add functional scenarios under those authorities.

Web/native cost boundary

Do not duplicate the browser suite in Tauri/Qt merely to increase coverage. Shared React behavior should remain browser-qualified once; native jobs should target only genuinely distinct filesystem/updater/native-dialog/window/IPC/packaging semantics.

Native matrix/runtime cost should be measured separately from web Playwright cost.

Additional acceptance implications for #550

• required Playwright p50/p95 and runner-minute baseline is recorded before topology changes;
• current workers: 1 policy is retained or changed only with evidence;
• initial scale-out evaluates 2-way Playwright sharding before framework replacement;
• all-shards-required aggregation and artifact naming are validated;
• sharding does not weaken first-attempt failure evidence or E2E WelcomePortal entry remains nondeterministic across startup states #532 diagnostics;
• cost optimization does not broaden the required test set by itself;
• browser and native qualification costs remain separate evidence planes.

[qnbs]

Current-main addendum — main-push concurrency still loses per-SHA diagnostic evidence — 2026-09-16

Revalidated against main 4137286887275ca03aa5b75906e6b529c68c786c.

Current workflow policy is still:

group: ci-${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.head_ref || github.ref_name }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

docs/CI.md describes this as “main branch pushes are never cancelled — each push builds its own run”. That wording is too strong for GitHub concurrency semantics: cancel-in-progress: false prevents cancellation of the currently running job, but a concurrency group permits only one pending run; a newer queued main push can replace/cancel an older pending main run before it starts.

The 2026-09-16 audit observed this during the recent Dependabot burst: intermediate main SHAs such as the jsdom-30 merge and several adjacent dependency merges had their queued resulting-main runs superseded, while the final cumulative main state was tested green.

Why this matters

Final cumulative-green evidence protects current main, but it weakens per-merge causality:

merge A
merge B quickly after
A's queued main run is replaced
B/final combined state passes
→ current main is green
→ exact regression attribution for A is lost

That is especially undesirable for major/toolchain/security-sensitive dependency updates.

#550 decision to make

Measure and compare a small set of policies rather than changing concurrency blindly:

1. Per-SHA main concurrency group for changes where every resulting-main SHA must receive a run;
2. preserve branch-group concurrency but enforce sequential merge/resulting-main verification operationally;
3. hybrid policy — ordinary low-risk bursts may use cumulative evidence, while major/security/runtime-sensitive merges require terminal resulting-main before the next merge;
4. merge queue if/when repository governance adopts it.

Account for runner-minute cost and deploy duplication. Do not simply disable all concurrency optimization without measurements.

Required truth/documentation correction

Until a stronger policy lands, current docs should not claim every main push necessarily builds its own run. State the actual GitHub pending-run behavior and distinguish:

• running main run is not cancelled by cancel-in-progress;
• an older pending run in the same group can be superseded by a newer one;
• the final cumulative head may still be fully qualified even when intermediate SHAs are not.

Add this to #550's cost/latency/evidence inventory and preserve the standing requirement that high-risk merges retain causal resulting-main evidence where practical.

[qnbs]

Current-main CI-evidence addendum — burst merges can cancel queued per-SHA main runs — 2026-09-16

A fresh audit of the 2026-09-15 Dependabot merge burst found a distinct evidence/cost tradeoff that belongs in #550's cancellation/topology analysis.

The current workflow concurrency behavior can replace/cancel queued main push runs when several merges land faster than the full pipeline drains, even with the intended cancel-in-progress: false behavior for a run already executing. During the dependency burst, intermediate resulting-main runs for merges including #752/#753/#754/#756/#758 did not all reach terminal full-pipeline evidence before a newer main SHA became authoritative; the final cumulative main state was subsequently green.

This is not evidence that those merged states were broken, and a later cumulative green main proves the final combined tree. It does mean the repository cannot always claim independent full-CI qualification for every intermediate merge SHA, which reduces bisect/attribution precision when a burst contains a breaking dependency change.

Add this to the #550 topology decision

Measure and explicitly choose the desired invariant for main:

A. EVERY_MERGED_MAIN_SHA_TERMINAL
   each merge SHA gets an independent blocking CI/CodeQL result

or

B. LATEST_MAIN_TERMINAL_WITH_BOUNDED_SUPERSESSION
   queued intermediate SHAs may be superseded, but the final cumulative tree is mandatory and the loss of per-SHA evidence is explicit

Do not change concurrency blindly. First quantify:

• frequency of burst merges / superseded queued main runs;
• runner-minute and queue-time cost of forcing every SHA to drain;
• how much per-SHA evidence is already supplied by exact-head PR CI before merge;
• whether dependency/security/major migrations need stricter resulting-main serialization than ordinary docs/low-risk merges;
• effect on incident bisection and release evidence.

A useful bounded design may be risk-routed rather than globally expensive, but it must not allow a final release gate to proceed without terminal CI + CodeQL on the actual final main SHA.

This supplements #550's existing PR-run cancellation/sharding work; it is specifically about post-merge main evidence retention, not permission to weaken exact-head PR convergence.