Skip to content

feat(task-board): push PR checks and preview url from a GitHub webhook - #7039

Closed
pedrofrxncx wants to merge 6 commits into
mainfrom
fix/pr-card-webhook-push
Closed

pedrofrxncx wants to merge 6 commits into
mainfrom
fix/pr-card-webhook-push

Conversation

@pedrofrxncx

@pedrofrxncx pedrofrxncx commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

The problem

Checks go green on GitHub and the deploy bot posts a preview url, but the card
shows neither for 1–3 minutes. Nothing is stuck — every layer is one window
behind, and the windows stack:

layer window
card cache (DECOCMS_PR_CARDS) 30s
raw reads (get_status, get_check_runs, get_comments) 55s
dialog poll 60s

#7033 zeroed the first two while CI is pending, which is the wrong half of the
timeline: the preview comment lands after the checks turn green, and at that
instant every layer reverts to its full window.

Turning the polling up further is not the fix — at a 10s poll one open dialog on
a mid-flight PR costs ~30 GitHub calls/min, and an App installation's floor is
5,000/hr (~83/min). Three open dialogs would exceed it, against the same limit
review-sweeper.ts says already took the App out once.

The fix: use the events GitHub already sends

Most of this was already built and dormant:

  • The receiver exists/api/_github/webhook has been HMAC-verifying push
    deliveries for tenant warm pools. It had no consumer for PR-card events.
  • The App already emits them — deco-cms (owner deco-cx) subscribes to
    issue_comment and push; check_suite was added for this (it covers
    Cloudflare Pages / Workers Builds, which Actions-only workflow_run does not).
  • The SSE lane exists/api/:org/watch, same hub the board already uses
    for task-board.item.updated.

What this PR adds:

  1. check_suite / issue_comment deliveries resolve the PR → its cards via a
    new findPrLinks, plus an index for that lookup (migration 204 — the
    table's only index was (organization_id, task_board_item_id) and its PK
    (task_board_item_id, url), so a repo/number lookup was a full scan on a
    path that fires for every repo the App is installed on).
  2. The refresh re-reads GitHub bypassing the read cache — the event is the
    invalidation, and a 55s-old read is exactly what it supersedes — writes the
    result to the card cache, and emits task-board.item.prs.updated.
  3. The open dialog writes those cards straight into its query cache. Checks and
    the Preview button land in ~1s.
  4. Polling stays as the fallback for repos the App isn't installed on.

Also: a preview url that only the deploy check knows

Reported alongside: "sometimes the bot doesn't comment the preview url, but the
info is available on the deploy check."
True — extractPreviewUrlFromCheckRuns
only understood Workers Builds: <name> + Version ID:. Now any deploy check
that prints its url in output.summary / output.text / output.title /
details_url counts, successful runs first, still gated through
isTrustedPreviewHost. The exact Workers Builds path still wins when present.

Bounded / reversible

  • No secret → the route still answers 503 and everything falls back to
    today's polling. Nothing here is load-bearing until GITHUB_WEBHOOK_SECRET is
    deployed.
  • An event for a repo no card links costs one indexed lookup — the common
    case in prod.
  • The poll ladder is unchanged as the fallback, and its preview chase is bounded
    to 10 min past the PR's updated_at (a repo that never publishes a preview
    stops costing reads).

Deploy steps (not code)

  1. GITHUB_WEBHOOK_SECRET into AWS SM prod/studio/application (us-west-2) —
    ESO syncs it into deco-studio-secrets within 1h. Confirmed missing today:
    POST https://studio.decocms.com/api/_github/webhook503 {"error":"github webhook not configured"}.
  2. The same string in the App's Webhook secret field, URL pointing at
    /api/_github/webhook.

Side effect worth knowing: this switches on the warm-pool push acceleration
that has been dormant since it shipped, for the same reason (no secret).

Testing

Unit (bun test, 91 pass in the touched files): prRefsFromGithubEvent
payload shapes (check_suite fan-out + dedupe, PR comment vs plain-issue comment,
wrong shape); the generalized check-run preview scan (output, details_url,
success-preferred, untrusted host rejected, Workers Builds still wins);
isCardNotReady including its bound.

E2Epackages/e2e/tests/task-board-pr-webhook.spec.ts walks the whole
chain black-box, with GitHub as a local MCP server (the card path reads GitHub
through an mcp-github connection, not the REST stub) whose answers flip
mid-test from "CI running, no preview" to "CI passed, url in the deploy check's
output" — and never a bot comment:

HMAC-signed delivery → reverse lookup → fresh read → card cache write → SSE push
→ the already-open dialog showing Checks passing and a preview control.

Plus: bad signature → 400, unhandled event → 200 ignored, unlinked PR →
refreshed: 0, issue_comment on a plain issue → not treated as a PR.

Migration 204 verified against real Postgres (task_board_item_prs_repo_idx
present after bun run migrate).

bun run fmt, bun run lint (0 errors) and apps/api typecheck clean. The 10
failing task-board unit tests are the Postgres-backed ones and fail identically
on main in this shell; the 2 remaining apps/web type errors are the
pre-existing duplicate-prosemirror ones.

One open decision

ciMaxStaleMs (from the commit before this one) gives a pending read a zero
stale ceiling
, so a rate-limited get_status returns null
checksStatus: nullprReadyForReview passes → the reviewer gets handed a PR
whose CI is still running. With this webhook path in place that blocking read
buys much less; I'd give it a short ceiling instead of 0, or gate the reconciles
on a positively-live checks answer. Left as-is here — say the word and it's a
two-line follow-up.


Summary by cubic

Task-board PR cards were poll-only, so CI results and preview URLs could appear 1–3 minutes late. This adds GitHub webhook refreshes plus SSE updates so open dialogs receive changes in about a second, while polling remains the fallback.

New Features

  • Handles signed check_suite and issue_comment events and refreshes linked cards with uncached GitHub reads; failures are logged and the delivery returns 200, so GitHub doesn't retry transient errors, and one failing card doesn't block the others on the same PR.
  • Adds an indexed PR reverse lookup and emits task-board.item.prs.updated to update the open dialog immediately.
  • Extracts trusted preview URLs from deploy check output and details links, even when no bot comment exists.
  • Bounds preview polling to 10 minutes after the PR's last update and never parks pending CI or unenriched card reads.

Migration

  • Run migration 204 to add task_board_item_prs_repo_idx.
  • Set GITHUB_WEBHOOK_SECRET in AWS Secrets Manager and use the same value for the GitHub App webhook at /api/_github/webhook.
  • Without the secret, the endpoint returns 503 and existing polling continues; enabling it also activates warm-pool push refreshes.

Written for commit aa0a028. Summary will update on new commits.

Review in cubic

Pedro França added 6 commits September 8, 2026 12:52
…serving

#7033 gave the pending reads a zero HIT window, which only starts a background
refresh — the poll still served the previous read, so a card assembled from it
needed a second poll to show green, and never moved at all if that one
background write failed. Give the same reads a zero STALE CEILING so a pending
read is a miss and asks GitHub on the spot. Dialog polls a pending card at 10s.
A PR card's CI state and preview url were poll-only, so they landed 1-3
minutes after GitHub had them: the card cache, the raw reads and the dialog
poll each held a window, and the windows stacked.

GitHub already emits what we need and the receiver already exists — the
`deco-cms` App subscribes to `issue_comment`/`push` (and now `check_suite`),
and `/api/_github/webhook` has been HMAC-verifying push deliveries for warm
pools. It just had no consumer for the PR-card events, and prod has no
`GITHUB_WEBHOOK_SECRET`, so the route answers 503.

- `check_suite` / `issue_comment` deliveries resolve the PR to its cards
  (new `findPrLinks` + an index for it — the table had none for a repo/number
  lookup), re-read GitHub bypassing the read cache (the event IS the
  invalidation), write the card cache, and emit
  `task-board.item.prs.updated` on the org's `/watch` stream.
- The open dialog writes those cards straight into its query cache, so checks
  and the Preview button land in about a second instead of on a poll.
- Preview urls are now also read off a deploy CHECK, not only a bot comment:
  the check-run scan was limited to Workers Builds' version id, and any deploy
  check that prints its url in `output` or `details_url` now counts. Several
  providers never comment one.
- Polling stays as the fallback for repos the App isn't installed on.

Bounded: an event for a repo no card links costs one indexed lookup. The
webhook is optional — without the secret the route still 503s and everything
falls back to the polling behavior it has today.
# Conflicts:
#	apps/api/src/tools/task-board/prs-get.ts
#	apps/web/src/hooks/use-task-board-item-prs.ts
pedrofrxncx added a commit that referenced this pull request Sep 9, 2026
…#7094)

Reverts the polling escalation from #7033 and #7037. Together they turned a
pending-CI card into a live GitHub read on EVERY poll, at a 10s poll:

- #7037 gave the CI reads a zero STALE CEILING, so a pending read was a MISS
  and blocked on the provider instead of being served from KV.
- #7033 gave the same reads a zero HIT window (a detached live read per poll),
  widened the card cache's zero window from "pending AND no preview" to any
  pending card, and cut the dialog poll from 60s to 15s; #7037 cut it to 10s.

Six polls a minute x a live detail read each, per open dialog, against an App
installation floor of 5,000/hr - which is the rate limit we are now hitting in
production. Back to the pre-#7033 windows: 55s reads / 30s cards / 60s poll.

The freshness this bought is coming back off webhooks in #7039, which costs no
polling at all.

Co-authored-by: Pedro França <pedrofrxncx@deco.cx>
@pedrofrxncx

Copy link
Copy Markdown
Collaborator Author

Superseded by #7100.

This branch is stacked on #7037, which #7094 reverts — we were rate-limited against GitHub in production, and the 10s poll here plus #7037's zero stale ceiling was the cause. It also no longer applies: #7022 replaced the get_status / get_check_runs MCP read path this builds on with the git-providers abstraction, so prs-get.ts conflicted across 8 hunks.

#7100 is the same feature ported onto current main, on top of the revert, with two changes:

  • the dialog poll stays at 60s (the webhook is the freshness path now, so the 10s tier isn't needed and is what caused the incident)
  • check_suite is narrowed to action: "completed" — a suite fires 3× per commit per app and every refresh is an uncached provider read

It also settles the open question in this PR's description: ciMaxStaleMs is gone with the revert, so a rate-limited pending read can no longer report checksStatus: null and let prReadyForReview hand a reviewer a PR whose CI is still running.

One thing that did not survive the port: the full delivery→SSE→dialog e2e walk. It stubbed GitHub as an mcp-github connection, which isn't how the card path reads any more, and there's no fixture yet for the GraphQL detail read that replaced it. #7100 keeps the route-contract half and flags the gap.

pedrofrxncx pushed a commit that referenced this pull request Sep 9, 2026
Ports #7039 onto post-#7022 main, on top of the #7094 revert.

Checks go green and the deploy bot posts a preview url, but the card shows
neither for 1-3 minutes. Polling harder is not the fix - it is the incident
#7094 just reverted. Use the events GitHub already sends instead:

- `/api/_github/webhook` gains `check_suite` / `issue_comment` consumers; they
  reverse-look-up the PR's cards (new `findPrLinks`, indexed by migration 207),
  re-read the provider bypassing the read cache, write the card cache and emit
  `task-board.item.prs.updated` on the org's SSE stream.
- The open dialog writes those cards straight into its query cache.
- A deploy check that prints its url in `summary` counts as a preview source,
  so a repo whose bot never comments still gets a Preview button.

Changes from #7039, both about not re-creating the rate limit:

- The dialog poll stays at 60s. #7039 kept a 10s tier for in-flight cards,
  which is what #7094 reverted, and the webhook is the freshness path now. A
  repo the App is not installed on falls back to the minute.
- `check_suite` is narrowed to `action: "completed"`. A suite fires three times
  per commit per app and every refresh is an UNCACHED provider read, so the
  other two tripled this path's cost to learn "pending" - which the card
  already shows.

Rebased onto the git-providers abstraction: `updatedAt` is now on the neutral
`ChangeRequest` (both adapters), the `fresh` bypass is a flag on the read cache
rather than a zero-stale-ceiling predicate, and the obsolete `get_status` /
`get_check_runs` window overrides are gone with the MCP read path.

Dormant until deployed: no `GITHUB_WEBHOOK_SECRET` means 503 and today's
polling, unchanged.
pedrofrxncx pushed a commit that referenced this pull request Sep 9, 2026
Ports #7039 onto post-#7022 main, on top of the #7094 revert.

Checks go green and the deploy bot posts a preview url, but the card shows
neither for 1-3 minutes. Polling harder is not the fix - it is the incident
#7094 just reverted. Use the events GitHub already sends instead:

- `/api/_github/webhook` gains `check_suite` / `issue_comment` consumers; they
  reverse-look-up the PR's cards (new `findPrLinks`, indexed by migration 207),
  re-read the provider bypassing the read cache, write the card cache and emit
  `task-board.item.prs.updated` on the org's SSE stream.
- The open dialog writes those cards straight into its query cache.
- A deploy check that prints its url in `summary` counts as a preview source,
  so a repo whose bot never comments still gets a Preview button.

Changes from #7039, both about not re-creating the rate limit:

- The dialog poll stays at 60s. #7039 kept a 10s tier for in-flight cards,
  which is what #7094 reverted, and the webhook is the freshness path now. A
  repo the App is not installed on falls back to the minute.
- `check_suite` is narrowed to `action: "completed"`. A suite fires three times
  per commit per app and every refresh is an UNCACHED provider read, so the
  other two tripled this path's cost to learn "pending" - which the card
  already shows.

Rebased onto the git-providers abstraction: `updatedAt` is now on the neutral
`ChangeRequest` (both adapters), the `fresh` bypass is a flag on the read cache
rather than a zero-stale-ceiling predicate, and the obsolete `get_status` /
`get_check_runs` window overrides are gone with the MCP read path.

Dormant until deployed: no `GITHUB_WEBHOOK_SECRET` means 503 and today's
polling, unchanged.
pedrofrxncx added a commit that referenced this pull request Sep 9, 2026
#7100)

Ports #7039 onto post-#7022 main, on top of the #7094 revert.

Checks go green and the deploy bot posts a preview url, but the card shows
neither for 1-3 minutes. Polling harder is not the fix - it is the incident
#7094 just reverted. Use the events GitHub already sends instead:

- `/api/_github/webhook` gains `check_suite` / `issue_comment` consumers; they
  reverse-look-up the PR's cards (new `findPrLinks`, indexed by migration 207),
  re-read the provider bypassing the read cache, write the card cache and emit
  `task-board.item.prs.updated` on the org's SSE stream.
- The open dialog writes those cards straight into its query cache.
- A deploy check that prints its url in `summary` counts as a preview source,
  so a repo whose bot never comments still gets a Preview button.

Changes from #7039, both about not re-creating the rate limit:

- The dialog poll stays at 60s. #7039 kept a 10s tier for in-flight cards,
  which is what #7094 reverted, and the webhook is the freshness path now. A
  repo the App is not installed on falls back to the minute.
- `check_suite` is narrowed to `action: "completed"`. A suite fires three times
  per commit per app and every refresh is an UNCACHED provider read, so the
  other two tripled this path's cost to learn "pending" - which the card
  already shows.

Rebased onto the git-providers abstraction: `updatedAt` is now on the neutral
`ChangeRequest` (both adapters), the `fresh` bypass is a flag on the read cache
rather than a zero-stale-ceiling predicate, and the obsolete `get_status` /
`get_check_runs` window overrides are gone with the MCP read path.

Dormant until deployed: no `GITHUB_WEBHOOK_SECRET` means 503 and today's
polling, unchanged.

Co-authored-by: Pedro França <pedrofrxncx@deco.cx>
@pedrofrxncx

Copy link
Copy Markdown
Collaborator Author

Closing as stale: this PR sat past the bot's 48h merge window, main has moved on, and its CI results no longer reflect the current base. This is a housekeeping close, not a rejection of the change — if the underlying problem still exists, the bot will find it again and open a fresh, rebased PR.

[studio-bot:stale-close]

@pedrofrxncx
pedrofrxncx deleted the fix/pr-card-webhook-push branch September 14, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant