Skip to content

feat(selfhost): cache stable GitHub GET reads #1842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JSONbored merged 1 commit into from
Jun 30, 2026
Merged

feat(selfhost): cache stable GitHub GET reads #1842

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,7 @@ REDIS_URL=redis://redis:6379 # REQUIRED for the self-host review
# GITTENSORY_IMAGE=ghcr.io/jsonbored/gittensory-selfhost:latest # image used by scripts/deploy-selfhost-image.sh;
# # pin production rollouts to a release tag such as :orb-v0.1.0
# # or to an immutable @sha256 digest.
# GITHUB_CACHE_TTL_SECONDS=20 # Short default Redis TTL for safe GitHub GET response caching. Set 0 to disable.
# QDRANT_URL= # set to http://qdrant:6333 to use Qdrant as the RAG vector store
# # (--profile qdrant). Overrides the built-in sqlite-vec / pgvector.
# DISCORD_WEBHOOK_URL= # one Discord channel for per-action notifications (merged/closed/
Expand Down
17 changes: 17 additions & 0 deletions apps/gittensory-ui/src/routes/docs.self-hosting-configuration.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,23 @@ INTERNAL_JOB_TOKEN=<random-32-byte-token>`}
<code>FOO</code> wins over the file variant.
</p>

<h2>GitHub API cache</h2>
<p>
Redis backs shared caching for stable GitHub GET responses, including repeated installation,
repo/user metadata, and branch-protection required-status reads. Keys include the caller
identity and response-shaping headers, and cold misses are single-flighted so concurrent
jobs do not stampede GitHub.
</p>
<CodeBlock filename=".env" code={`GITHUB_CACHE_TTL_SECONDS=20`} />
<Callout variant="note">
<code>GITHUB_CACHE_TTL_SECONDS</code> is the short default for repeated safe GitHub GETs.
Stable repo/user metadata and branch-protection required-status reads use longer internal
TTLs. Live CI status, check-run, check-suite, pull/issue subresources, pull mergeability,
token minting, rate-limit, and collaborator-permission endpoints are never served from this
cache. Prometheus exports <code>gittensory_github_response_cache_total</code>, and the
bundled self-host Grafana dashboard includes the hit/miss/coalesced/error breakdown.
</Callout>

<h2>Per-PR feature flags</h2>
<p>
Most review capabilities need both their own flag and the repo in{" "}
Expand Down
1 change: 1 addition & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ services:
# DATABASE_URL: postgres://gittensory:${POSTGRES_PASSWORD:-CHANGEME}@pgbouncer:5432/gittensory
# Required shared transient state for review correctness, webhook dedup, and rate limiting.
REDIS_URL: "${REDIS_URL:-redis://redis:6379}"
GITHUB_CACHE_TTL_SECONDS: "${GITHUB_CACHE_TTL_SECONDS:-20}"
# Uncomment for Qdrant RAG vector store (--profile qdrant):
# QDRANT_URL: http://qdrant:6333
# Uncomment for Ollama AI (--profile ollama):
Expand Down
118 changes: 118 additions & 0 deletions grafana/dashboards/gittensory.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1376,6 +1376,124 @@
"unit": "short"
}
}
},
{
"collapsed": false,
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 90
},
"id": 113,
"title": "GitHub API Cache",
"type": "row"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"lineWidth": 2,
"fillOpacity": 10,
"stacking": {
"mode": "normal",
"group": "A"
}
},
"unit": "reqps"
}
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 91
},
"id": 114,
"options": {
"legend": {
"calcs": [
"lastNotNull"
],
"displayMode": "table",
"placement": "bottom"
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"expr": "sum by (result) (rate(gittensory_github_response_cache_total[5m]))",
"legendFormat": "{{result}}",
"refId": "A"
}
],
"title": "GitHub Cache Rate",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"lineWidth": 2,
"fillOpacity": 10
},
"unit": "short"
}
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 91
},
"id": 115,
"options": {
"legend": {
"calcs": [
"lastNotNull"
],
"displayMode": "table",
"placement": "bottom"
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"expr": "sum by (class, result) (gittensory_github_response_cache_total)",
"legendFormat": "{{class}} {{result}}",
"refId": "A"
}
],
"title": "GitHub Cache Totals",
"type": "timeseries"
}
],
"refresh": "30s",
Expand Down
139 changes: 13 additions & 126 deletions src/github/app.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@ import {
fetchBrokeredInstallationToken,
isOrbBrokerMode,
} from "../orb/broker-client";
import { makeInstallationOctokit } from "./client";
import {
clearGitHubResponseCacheForTest,
makeInstallationOctokit,
timeoutFetch,
} from "./client";
import { maintainerControlPanelUrl } from "./footer";
import type { AgentActionMode } from "../settings/agent-execution";
import { signRs256Jwt } from "../utils/crypto";
Expand All @@ -29,6 +33,13 @@ export {
GITTENSORY_GATE_CHECK_NAME,
GITTENSORY_LEGACY_GATE_CHECK_NAME,
} from "../review/check-names";
export type { CachedGitHubResponse, GitHubResponseCache } from "./client";
export {
isCacheableGithubUrl,
isRateLimitedResponse,
rateLimitRetryMs,
setGitHubResponseCache,
} from "./client";

type CheckRunResponse = {
id: number;
Expand All @@ -55,130 +66,6 @@ type GitHubCheckConclusion =
| "skipped";
type GitHubCheckStatus = "queued" | "in_progress" | "completed";

/** Hard cap on a single GitHub API request. Without it a slow/half-open GitHub connection can hang the
* Worker — e.g. the Gate's own completing PATCH stalling after the pending check was posted, which leaves
* the check in_progress forever. A bounded timeout turns a hang into a catchable error the caller can
* finalize. Applied to every raw fetch here and to the Octokit instances (via a timeout-injecting fetch). */
const GITHUB_FETCH_TIMEOUT_MS = 12_000;

/** A short-TTL cache for safe GitHub GET responses (e.g. Redis on the self-host). Stores only status/body/
* content-type — never rate-limit or encoding headers. Set on the self-host; the Worker leaves it null. */
export interface CachedGitHubResponse {
status: number;
body: string;
contentType: string;
}
export interface GitHubResponseCache {
get(url: string): Promise<CachedGitHubResponse | null>;
set(url: string, value: CachedGitHubResponse): Promise<void>;
}
let responseCache: GitHubResponseCache | null = null;
export function setGitHubResponseCache(
cache: GitHubResponseCache | null,
): void {
responseCache = cache;
}

/** Only cache safe GETs to the GitHub REST API. Never cache token-minting, rate-limit, or
* authorization/permission endpoints whose response must reflect the live caller context. Exported for tests. */
export function isCacheableGithubUrl(url: string): boolean {
if (!url.startsWith("https://github.com/ghapi/")) return false;
if (url.includes("/access_tokens") || url.includes("/rate_limit"))
return false;
return !/\/repos\/[^/]+\/[^/]+\/collaborators\/[^/]+\/permission(?:$|[?#])/.test(
url,
);
}

// Transient GitHub rate-limit handling (#ratelimit-resilience). A primary (x-ratelimit-remaining:0) or secondary
// (Retry-After / "secondary rate limit" body) limit returns 403/429. Instead of surfacing it as a failure — or
// MISCLASSIFYING a 403 as a permission gap — back off a few times and retry. A sustained limit exhausts the
// retries and the response is returned so the caller (and the queue) handles it. Bounded so a review never stalls.
const GITHUB_RATE_LIMIT_MAX_RETRIES = 3;
const GITHUB_RATE_LIMIT_MAX_DELAY_MS = 8_000;

const sleep = (ms: number): Promise<void> =>
new Promise((resolve) => setTimeout(resolve, ms));

/** Does this GitHub response signal a rate limit (primary or secondary)? 403/429 with a Retry-After header, an
* exhausted x-ratelimit-remaining, or a secondary-limit/abuse body. A 403 with NONE of these is a real
* permission/other error and must surface — not retry, not be mistaken for a rate limit. Exported for tests. */
export async function isRateLimitedResponse(
response: Response,
): Promise<boolean> {
if (response.status !== 403 && response.status !== 429) return false;
if (response.headers.get("retry-after") != null) return true;
if (response.headers.get("x-ratelimit-remaining") === "0") return true;
try {
return /secondary rate limit|\babuse\b|api rate limit exceeded/i.test(
await response.clone().text(),
);
/* v8 ignore next 3 -- defensive: a cloned Response body that fails to read isn't reachable in practice */
} catch {
return false;
}
}

/** How long to wait before the next rate-limit retry: honor a valid Retry-After (seconds), else exponential
* backoff — each capped so a review can never stall on one call. A sustained PRIMARY limit (reset up to an hour
* out) simply exhausts the few inline retries and the queue retries the job later. Exported for tests. */
export function rateLimitRetryMs(response: Response, attempt: number): number {
const retryAfterHeader = response.headers.get("retry-after");
if (retryAfterHeader != null) {
const retryAfter = Number(retryAfterHeader);
if (Number.isFinite(retryAfter) && retryAfter >= 0)
return Math.min(retryAfter * 1000, GITHUB_RATE_LIMIT_MAX_DELAY_MS);
}
return Math.min(500 * 2 ** attempt, GITHUB_RATE_LIMIT_MAX_DELAY_MS);
}

async function timeoutFetch(
input: RequestInfo | URL,
init?: RequestInit,
): Promise<Response> {
const method = (init?.method ?? "GET").toUpperCase();
const url = String(input); // timeoutFetch is only ever called with string URLs (app template strings + octokit)
const useCache =
responseCache !== null && method === "GET" && isCacheableGithubUrl(url);
if (useCache) {
const hit = await responseCache!.get(url).catch(() => null); // a cache read must never break the fetch
if (hit)
return new Response(hit.body, {
status: hit.status,
headers: { "content-type": hit.contentType },
});
}
let response: Response;
for (let attempt = 0; ; attempt += 1) {
response = init?.signal
? await fetch(input, init)
: await fetch(input, {
...(init ?? {}),
signal: AbortSignal.timeout(GITHUB_FETCH_TIMEOUT_MS),
});
// Retry a transient rate-limit (with backoff) instead of surfacing it; stop once exhausted or it's not a limit.
if (
attempt >= GITHUB_RATE_LIMIT_MAX_RETRIES ||
!(await isRateLimitedResponse(response))
)
break;
await sleep(rateLimitRetryMs(response, attempt));
}
if (useCache && response.status === 200) {
try {
const body = await response.clone().text(); // clone leaves the returned response readable
await responseCache!.set(url, {
status: 200,
body,
contentType: response.headers.get("content-type") ?? "application/json",
});
} catch {
/* caching is best-effort */
}
}
return response;
}

// In-isolate installation-token cache. GitHub installation tokens are valid ~1h; minting a fresh one on EVERY
// call (the previous behavior) multiplied GitHub API usage enormously — each review path mints several tokens,
// and across the sweep + re-reviews that exhausted the hourly rate limit (observed min_remaining=0 → reviews
Expand Down Expand Up @@ -402,7 +289,7 @@ export function isForeignAppInstallation(
export function clearInstallationTokenCacheForTest(): void {
installationTokenCache.clear();
externalTokenStore = null;
responseCache = null;
clearGitHubResponseCacheForTest();
}

export async function getAppInstallation(
Expand Down
9 changes: 6 additions & 3 deletions src/github/backfill.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ import {
} from "../review/check-names";
import { buildReviewThreadBlocker, type ReviewThreadBlocker } from "../review/review-thread-findings";
import { delayUntil, shouldWaitForGitHubRateLimit } from "./rate-limit";
import { isGitHubResponseCacheReplay, timeoutFetch } from "./client";

type GitHubLabelPayload = {
name: string;
Expand Down Expand Up @@ -2687,10 +2688,12 @@ async function githubJsonWithHeaders<T>(
): Promise<{ data: T; link: string | null; etag: string | null; lastModified: string | null }> {
const { owner, name } = repoParts(repoFullName);
const url = `https://github.com/ghapi/repos/${encodeURIComponent(owner)}/${encodeURIComponent(name)}${path}`;
let response = await fetch(url, { headers: githubRestHeaders(token) });
await recordGitHubResponse(env, repoFullName, path, response, "rest");
let response = await timeoutFetch(url, { headers: githubRestHeaders(token) });
if (!isGitHubResponseCacheReplay(response)) {
await recordGitHubResponse(env, repoFullName, path, response, "rest");
}
if (response.status === 404 && token && token === env.GITHUB_PUBLIC_TOKEN) {
response = await fetch(url, { headers: githubRestHeaders() });
response = await timeoutFetch(url, { headers: githubRestHeaders() });
// Do not persist unauthenticated fallback rate-limit headers into the shared REST backoff state.
// GitHub's unauthenticated REST bucket is capped below LOW_REST_RATE_LIMIT_REMAINING, so recording
// successful fallback responses can incorrectly stall later token-backed segment jobs.
Expand Down
Loading
Loading