Skip to content

fix(github): paginate the GraphQL PR-detail fallback's files/reviews past 100 nodes #7453

Description

@JSONbored

Context

src/github/backfill.ts's fetchPullRequestDetailsFromGraphQl (~line 4125) is the GraphQL fallback
used by fetchPullRequestFiles (~line 2391) and fetchPullRequestReviews (~line 2455) whenever the
REST-side githubPaginatedList (~line 2363) fails to fetch page 1 of /pulls/{n}/files or
/pulls/{n}/reviews.

The REST path already handles this correctly: githubPaginatedList walks up to
PR_DETAIL_MAX_PAGES = 10 pages of per_page=100 (up to 1,000 items), following Link: rel="next",
specifically because (per that function's own comment) "GitHub caps list endpoints at 100
items/page, so a single per_page=100 fetch silently truncates a large PR's files/reviews/checks."
pr-actions.ts's dismissLatestBotApproval makes the identical point for reviews specifically,
walking up to REVIEW_PAGE_LIMIT = 10 pages because "a single per_page:100 fetch would only see the
bot's earliest reviews on a PR with a long review history."

The GraphQL fallback does not follow this same convention. Its query:

files(first: 100) { nodes { path additions deletions changeType } }
reviews(first: 100) { nodes { databaseId author { login } state authorAssociation submittedAt } }

requests no pageInfo at all, so there is no way for the caller to even detect truncation, let alone
follow it. GitHubPullRequestDetailsResponse (~line 260) has no pageInfo field in its type either.

Requirements

  • Add pageInfo { hasNextPage endCursor } to both the files and reviews connections in
    fetchPullRequestDetailsFromGraphQl's query.
  • Loop with GraphQL cursor pagination (after: $cursor) for each connection independently, mirroring
    the REST-side githubPaginatedList bound: cap at the same effective ceiling already used elsewhere
    in this file for PR detail data (10 pages of 100 = 1,000 items per connection) so a pathological PR
    can never turn this into an unbounded fetch loop — introduce a shared or connection-scoped page-cap
    constant next to the function, following the existing PR_DETAIL_MAX_PAGES naming convention.
  • A later-page failure must keep the items already fetched (same "don't drop a successful partial
    result" semantics githubPaginatedList already documents), not discard everything.
  • Do not change the function's existing fail-safe contract: any total failure still surfaces the same
    way to callers (fetchPullRequestFiles/fetchPullRequestReviews already .catch(() => undefined)
    this call and fall through to a warning + empty result).

Deliverables

  • fetchPullRequestDetailsFromGraphQl paginates files and reviews past 100 nodes each, bounded
    by an explicit page cap.
  • GitHubPullRequestDetailsResponse's type updated to carry pageInfo for both connections.
  • A regression test proving a >100-node files or reviews GraphQL response (two-page fixture,
    second page returned via a mocked after cursor) is fully collected, not truncated at 100.
  • A regression test proving the existing single-page (hasNextPage: false) case is unaffected.

Test Coverage Requirements

This repo's Codecov patch gate is 99%+ branch-counted on every changed line/branch in src/**. The
new pagination loop, its page-cap bound, and the "keep partial results on a later-page failure" branch
all need explicit test coverage — mirror the existing test shapes in test/unit/backfill.test.ts /
test/unit/backfill-2.test.ts for fetchPullRequestFiles/fetchPullRequestReviews's GraphQL-fallback
paths.

Expected Outcome

A PR whose REST files/reviews fetch fails on page 1 (rate limit, transient network error) and falls
back to GraphQL no longer silently loses files/reviews beyond the 100th when the PR has more than 100
changed files or more than 100 reviews — the downstream churn/size scoring and review-decision dedup
that consume this data see the complete list, matching what the REST path already guarantees.

Links & Resources

  • src/github/backfill.ts: githubPaginatedList (~2363), PR_DETAIL_MAX_PAGES (~2361),
    fetchPullRequestFiles (~2381), fetchPullRequestReviews (~2447),
    fetchPullRequestDetailsFromGraphQl (~4125), GitHubPullRequestDetailsResponse (~260).
  • src/github/pr-actions.ts: dismissLatestBotApproval's REVIEW_PAGE_LIMIT pagination (~8-12,
    ~144-151) — the precedent this issue asks the GraphQL path to match.

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions