Skip to content

fix(ci): skip the heavy job fan-out for draft pull requests - #6448

Merged
JSONbored merged 1 commit into
mainfrom
fix/ci-skip-draft-prs
Jul 16, 2026
Merged

fix(ci): skip the heavy job fan-out for draft pull requests#6448
JSONbored merged 1 commit into
mainfrom
fix/ci-skip-draft-prs

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • validate-code, the 6-way validate-tests shard matrix, validate-tests-merge, and security were running their full 11-job fan-out on every draft PR open and push, identical to a ready PR.
  • This directly worsens queue contention on the account's shared GitHub-hosted runner pool (already flagged in this file's own comments as capacity-constrained, not billing-constrained) — confirmed via live workflow-run data showing multi-minute gaps between shard start times under current load (11 workflow runs queued/in-progress at time of investigation, most stuck in queued).
  • Each affected job's if: now requires the PR not be a draft; push runs are unaffected (github.event.pull_request is unset there).
  • The pull_request trigger now explicitly lists ready_for_review alongside the prior default types — the default list omits it, so without this change, marking a draft ready would silently wait for the next push before CI ran at all.
  • This is one half of the draft-PR fix; the other half (stopping the bot's AI review/labels/slop-gate from running on drafts at the app level, and a new opt-in draft-close policy) is tracked separately.

Test plan

  • New test/unit/ci-skip-draft-prs.test.ts (6 tests) asserting the trigger types and each job's draft-guard condition
  • All existing ci.yml-parsing tests still pass unmodified (21 tests across 6 files)
  • Full local gate (npm run test:ci) green

validate-code, the 6-way validate-tests shard matrix, validate-tests-merge,
and security were running their full 11-job fan-out on every draft PR open
and push, identical to a ready PR. Combined with the account's shared
GitHub-hosted runner pool (already flagged in this file's own comments as
capacity-constrained, not billing-constrained), this meaningfully worsens
queue contention for every other PR in flight whenever a contributor opens
or pushes to a draft -- confirmed via live workflow-run data showing
multi-minute gaps between shard start times under load.

Each affected job's if-condition now requires the PR not be a draft; push
runs are unaffected since github.event.pull_request is unset there. The
pull_request trigger now explicitly lists ready_for_review alongside the
prior default types, since the default list omits it -- without this,
marking a draft ready would silently wait for the next push before CI ran
at all.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.61%. Comparing base (8b85ce8) to head (189db3c).
⚠️ Report is 17 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6448   +/-   ##
=======================================
  Coverage   95.61%   95.61%           
=======================================
  Files         598      598           
  Lines       47308    47310    +2     
  Branches    15053    15056    +3     
=======================================
+ Hits        45233    45235    +2     
  Misses       1291     1291           
  Partials      784      784           
Flag Coverage Δ
shard-1 44.10% <ø> (+0.17%) ⬆️
shard-2 36.49% <ø> (-0.41%) ⬇️
shard-3 32.27% <ø> (+0.04%) ⬆️
shard-4 35.20% <ø> (+0.87%) ⬆️
shard-5 30.78% <ø> (-0.67%) ⬇️
shard-6 45.27% <ø> (-0.15%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 16, 2026
@loopover-orb

loopover-orb Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Warning

⏸️ LoopOver review result - manual review recommended

Review updated: 2026-07-16 08:19:59 UTC

2 files · 1 AI reviewer · 2 blockers · CI green · clean

⏸️ Suggested Action - Manual Review

Review summary
This PR gates the four heavy CI jobs (validate-code, validate-tests, validate-tests-merge, security) with `github.event.pull_request.draft != true` so draft PRs no longer trigger the full 11-job fan-out, and adds `ready_for_review` to the pull_request trigger types so marking a PR ready fires CI immediately instead of waiting for the next push. The null-coalescing logic for push events is correctly reasoned in the comment (`github.event.pull_request` is unset on push, so the property access is null, and `null != true` is true) and is verified by the new test suite parsing the actual YAML. The change is narrowly scoped, well-tested against the real workflow file rather than a fabricated fixture, and the `validate` aggregation job's `needs`/`if: always()` is unchanged so it still correctly reflects skipped-as-success semantics.

Nits — 4 non-blocking
  • The test file's regression-guard comment (test/unit/ci-skip-draft-prs.test.ts:15-20) claims a security motivation ('farm bot labels/AI review/screenshots for free') not mentioned in the PR description, which focuses solely on queue contention — worth aligning the comment with the actual stated rationale.
  • The `validate-tests` and `validate-tests-merge` if-conditions are identical in shape (`.github/workflows/ci.yml:455,636`) and duplicated the draft-guard logic three times across jobs; consider a reusable `needs.changes.outputs.*` style flag or a job-level default if the workflow gains more heavy jobs later.
  • Consider confirming whether `security`'s job-level if (`.github/workflows/ci.yml:687`) should also account for `github.event_name == 'push'` for symmetry with the others, or whether that's intentionally push-exempt as before.
  • The new test's `it.each` loop (test/unit/ci-skip-draft-prs.test.ts:39) is a nice way to avoid triplicated near-identical test bodies — good pattern to keep for future job additions.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
📋 Copy for AI agents — paste into your coding agent
Fix the following blocker(s) from this PR review:

1. No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.

2. Maintainer requires a linked issue — Link the relevant issue (for example `Closes #123`) before opening the PR.

Decision drivers

  • ❌ Code review — 2 blockers (1 reviewer)
  • ❌ Gate result — Blocking (Repo-configured hard blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 39 registered-repo PR(s), 32 merged, 370 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 39 PR(s), 370 issue(s).
Improvement ℹ️ Insufficient signal risk: clean · value: insufficient-signal
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 39 PR(s), 370 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Then work through the remaining 2 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 16, 2026
@JSONbored
JSONbored merged commit 7ebeb78 into main Jul 16, 2026
17 checks passed
@JSONbored
JSONbored deleted the fix/ci-skip-draft-prs branch July 16, 2026 08:20
nghetienhiep pushed a commit to nghetienhiep/gittensory that referenced this pull request Jul 16, 2026
…farming

Contributors were opening PRs directly as draft (or converting to draft
immediately after opening) to get labels, AI review, and slop-gate feedback
from the bot without ever being subject to a real one-shot disposition --
distinct from the existing reviewEvasionProtection family, which only
enforces after a review has already run against the PR's current head, or
on the 2nd+ ready<->draft conversion.

Adds draftPrClosePolicy ("off" | "close", off by default) as a new
RepositorySettings field (migration + schema + resolver + openapi, mirroring
reviewEvasionProtection's exact wiring) and a new maybeCloseDraftPr guard in
src/queue/review-evasion.ts, closely modeled on the existing draft-conversion
review-evasion guard but firing on ANY draft (opened directly as one, or
converted to one) rather than requiring a prior review pass. Deliberately
does not record a moderation strike -- this is a blanket repo policy against
ordinary GitHub draft usage, not a detected abuse pattern. Enabled for this
repo's own dogfood config (.loopover.yml + the bundled fallback manifest).

Companion to the CI-side fix (JSONbored#6448, merged) that stops the heavy job
fan-out from running on draft PRs in the first place.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant