Skip to content

orb(ci): check-run pagination cap exhaustion is silent — a failing check beyond 1000 runs reads as CI passed (wrong-merge path) #9051

Description

@JSONbored

Every failed page read is correctly fail-closed (sets checkRunsIncompleteciState = "pending", src/github/backfill.ts ~3052). But exhausting the page cap is not detected at all:

for (let page = 1; page <= PR_DETAIL_MAX_PAGES; page += 1) {
  const result = await githubJsonWithHeaders(...).catch(() => undefined);
  if (!result) { checkRunsIncomplete = true; break; }   // failure → detected
  checkRuns.push(...(result.data.check_runs ?? []));
  if (!hasNextPage(result.link)) break;
}                                                        // cap hit → NOT detected

(backfill.ts ~3097-3112 for check-runs, ~3116-3130 for statuses.) At 10 pages x 100 = 1000 runs with rel="next" still present, the loop exits normally, checkRunsIncomplete stays false, and reduceLiveCiAggregate treats a partial set as complete. hasMissingRequiredContext is likewise computed as if the read were whole (~3066).

Result: a red check on page 11+ is invisible → ciState = "passed" → planner reviewGoodmerge. The executor's act-boundary recheck calls the same function, so it reproduces the same wrong verdict rather than catching it. The false passed is then persisted into the durable cross-job CI cache.

Every other capped loop in this file warns at the cap (~1848, ~1924, ~4450). This one is the outlier.

Related, same class: the check-suites backstop (backfill.ts ~3140-3149, /commits/{sha}/check-suites?per_page=100) reads page 1 only with no Link follow — and it is the last gate before a commit is certified settled. A first-party github-actions suite still running on page 2 is invisible → anyPending never set → stays passed. The GraphQL twin has the same hole (checkSuites(first: 100) at ~3180/~3205 has no hasNextPage guard, unlike contexts which correctly bails at ~3227).

Reachability note: on repos with no branch protection and no expectedCiContexts, enforceRequiredOnly is false and mergeable_state === "clean" is satisfied without CI, so ORB's own aggregate is the sole CI gate — which is what makes this reachable rather than theoretical.

Fix

  1. GET /commits/{sha}/check-runs returns total_count and we never read it (grep confirms zero uses). Compare it to checkRuns.length, or set checkRunsIncomplete = true when page === PR_DETAIL_MAX_PAGES && hasNextPage(result.link).
  2. Paginate the check-suites backstop, or bail to null (the reducer already fails closed on null) when total_count > 100.
  3. Add the same hasNextPage guard to the GraphQL checkSuites selection.

Acceptance

  • A commit with >1000 check-runs where a failure sits past the cap resolves to pending, never passed.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions