Skip to content

ci: bump the coverage-run matrix from 4 to 6 shards - #4944

Merged
JSONbored merged 1 commit into
mainfrom
perf/shard-coverage-6-way
Jul 11, 2026
Merged

ci: bump the coverage-run matrix from 4 to 6 shards#4944
JSONbored merged 1 commit into
mainfrom
perf/shard-coverage-6-way

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Why

Measured from the 4-way rollout: ~2,026s of aggregate test+import work spread across 4 shards, landing at ~150s wall time each (~85% parallel efficiency), with ~35s of fixed per-shard overhead (checkout/cache/build-engine + upload steps). At 6 shards, that projects to ~135s (~2.2min) wall time per shard, cutting the overall CI critical path from ~3.5-4min down to roughly ~2.7-2.9min.

Deliberately not going straight to 8: checked this repo's actual concurrent-run history and found genuine bursts of 15-20+ simultaneous ci.yml runs (confirmed only 2 of 17 concurrent runs in one sampled burst were from my own session's retriggers -- the rest were other contributors'/automation's real traffic, mostly automatic push-to-main runs). Since each PR's shard phase multiplies the job-slots that run demands from the shared concurrent-runner pool, doubling shard count (4→8) doubles that per-run demand during exactly the bursts this repo already produces regularly -- risking queuing that could offset or reverse the wall-clock win. 6 shards is a smaller (1.5x, not 2x) step to validate before considering 8.

Test plan

  • npx tsc --noEmit clean
  • npm run actionlint clean
  • YAML parses
  • codecov-policy.test.ts / workflow-runner-labels.test.ts pass (no shard-count-specific assertions broke, only comment text updated)
  • Watch real CI timing after merge, and watch for any shard jobs sitting queued rather than in_progress during a busy window -- that would be the signal to back off rather than push to 8

Real per-shard timing from the 4-way rollout: ~2026s of aggregate
test+import work across 4 shards, ~150s wall time each, ~35s of that
fixed setup/teardown overhead per shard job. At 6 shards, projected
wall time per shard drops to ~135s (~2.2min), for a total CI critical
path around 2.7-2.9min vs the current ~3.5-4min -- while keeping the
per-PR job-slot multiplier increase to 1.5x rather than 2x.

Deliberately not going straight to 8: this repo sees genuine bursts of
15-20+ concurrent ci.yml runs (measured directly from run history, not
an artifact of this session's own churn -- only 2 of 17 concurrent runs
in one sampled burst were mine). Doubling per-run job-slot demand during
those bursts risks queuing that could offset or reverse the wall-clock
win; 6 shards is a smaller, safer step to validate before considering 8.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.24%. Comparing base (65422c8) to head (9f0be78).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4944   +/-   ##
=======================================
  Coverage   94.24%   94.24%           
=======================================
  Files         444      444           
  Lines       38854    38854           
  Branches    14157    14157           
=======================================
  Hits        36617    36617           
  Misses       1577     1577           
  Partials      660      660           
Flag Coverage Δ
shard-1 46.72% <ø> (-4.75%) ⬇️
shard-2 33.31% <ø> (-12.12%) ⬇️
shard-3 31.39% <ø> (-10.50%) ⬇️
shard-4 33.01% <ø> (-18.90%) ⬇️
shard-5 33.48% <ø> (?)
shard-6 45.20% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 11, 2026
@JSONbored JSONbored self-assigned this Jul 11, 2026
@loopover-orb

loopover-orb Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-11 02:02:06 UTC

2 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI green · unstable

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
This PR mechanically bumps the validate-tests shard matrix from [1,2,3,4] to [1,2,3,4,5,6], updating every shard-count reference in lockstep: the --shard=N/6 CLI arg, step display names, failure-guidance echo text, and prose comments describing the merge job's behavior. It's a pure CI/infra change confined to .github/workflows/ci.yml and a comment-only edit in test/unit/workflow-runner-labels.test.ts, with no src/** impact. The rationale is data-backed (real per-shard timing from the prior 4-shard rollout) and deliberately conservative (6 rather than 8, reasoned against this repo's observed concurrent-runner-pool contention), and CI already shows all six new shard jobs plus validate-tests-merge passing on this exact commit, confirming the merge/coverage-threshold step correctly consumes 6 shards' artifacts rather than silently assuming 4.

Nits — 3 non-blocking
  • The description cites prior PR ci: shard the full-suite coverage run across a matrix #4815 as precedent rather than linking an eligible open issue — worth a one-line confirmation from the author per repo convention, though this is narrow operator-facing CI tuning rather than a feature addition.
  • The test plan leaves 'watch real CI timing after merge / watch for queued shard jobs' unchecked as a stated follow-up — not blocking, but worth tracking so the post-merge validation isn't lost.
  • Once merged, capture the observed 6-shard wall-clock number in a follow-up comment or issue so the eventual 6→8 bump has the same empirical grounding this PR used for 4→6.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 48 registered-repo PR(s), 40 merged, 312 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 48 PR(s), 312 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 48 PR(s), 312 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 11, 2026
@JSONbored
JSONbored merged commit b761241 into main Jul 11, 2026
18 checks passed
@JSONbored
JSONbored deleted the perf/shard-coverage-6-way branch July 11, 2026 02:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant