reviews: scope Codex connector review to high-blast-radius PRs — its advisory threads block the merge gate on every PR

[kyle-sexton]

Control-tower bottleneck finding (2026-07-20, operator-directed latency work). Measured today: review-thread clearance, not CI runtime, is the dominant PR queue time. Every PR currently gets BOTH the claude-review action AND the chatgpt-codex-connector review; Codex threads are advisory in intent but count as unresolved threads, which the babysit merge gate (and #534's future gate script) treats as blockers — so routine/docs PRs pay a full adversarial-review thread-clearing cycle twice.

Proposal (research-first): determine where the Codex connector's trigger is configured (GitHub App config vs workflow), then scope it to high-blast-radius PRs only (the rule-10 class: skill frontmatter/trigger keywords, cross-plugin contracts, hooks, merge-gate/classifier/security surfaces, area: security label) while keeping claude-review universal. Alternative if scoping is impossible at the app layer: an explicit policy that clean Codex COMMENTED reviews with zero inline threads never gate, codified in #534's classifier.

Cross-model review diversity for high-blast-radius PRs is deliberately KEPT (#536's premise) — this narrows where it applies, it does not remove it.

Filed by the control-tower session.

[kyle-sexton]

Research findings — trigger mechanism, scoping surface, and fallback

Investigated empirically (this repo + Codex GitHub integration docs). Summary up front: the Codex trigger is app/cloud-level, not repo-file configurable — there is no supported per-path/label/branch filter. And the merge blocker is a GitHub org ruleset, not the babysit classifier. That relocates both the "scope it" action and the fallback.

1. Where the Codex review comes from (verified)

• Reviews are posted by the chatgpt-codex-connector[bot] GitHub App — confirmed on recent PRs (feat(autonomy): standing-routine catalog, v1 leaves, setup routine slice (WP6) #600, feat(source-control): surface babysit bot-to-personal identity fallback as an attribution-drift finding (#450) #606, feat(ci): add portability-lint gate for ecosystem/forge-agnostic skills (#531) #609, feat(repo-hygiene): tree-batch multi-repo clean/tree mode with skip-list + dirty guard #614, …), all state: COMMENTED.
• No repo-side trigger exists: no Codex workflow in .github/workflows/, no repo webhook (gh api …/hooks → []), no .github/codex.* config. AGENTS.md is present but the docs are explicit that it only controls review guidelines (what Codex flags — P0/P1), not which PRs get reviewed.
• The connector has two trigger modes: Automatic reviews (a per-repo toggle in Codex cloud settings — chatgpt.com/codex/settings/code-review) and the manual @codex review PR comment. The current volume (a Codex review on essentially every PR, with no @codex mention) is consistent with Automatic reviews = ON for this repo.

Sources (official OpenAI/ChatGPT docs): Codex code review in GitHub · Code review · Review GitHub pull requests. All three confirm automatic review is all-or-nothing per repository — no documented path/label/branch/draft filter; AGENTS.md tunes focus, not scope.

2. Scoping is NOT possible via a repo config file — operator 2-minute action

Because the trigger is the cloud-settings toggle, scoping Codex to high-blast-radius PRs cannot be done by committing a file here. The only repo-side way to approximate it:

1. Operator turns OFF Automatic reviews in chatgpt.com/codex/settings/code-review (requires whoever administers the melodic-software Codex connector — the org owner), and
2. something posts @codex review only on the rule-10 class. The babysit skill already has the dormant generic review-trigger module (request_review.py, four unset config slots) that could post that phrase — but it has no blast-radius predicate today, and posting from github-actions[bot] may not trigger the connector (unverified — the docs don't say bots can invoke it).

→ Operator 2-minute test that unblocks the whole workflow path: on a throwaway PR, post @codex review from a GitHub Action (github-actions[bot]) and watch for the 👀 reaction + a review. If it responds, workflow-driven scoping becomes viable; if not, scoping is only achievable by the cloud toggle + manual human mentions. Not safely in reach tonight either way (needs the toggle + a new predicate in request_review.py + this verification), so no PR filed — flagging as operator-owned.

3. The actual merge blocker (this reframes the fallback)

The unresolved-thread gate is not the babysit classifier and not babysit-readiness-gate.sh (that's a finding-decomposition predicate). It is the org-level branch ruleset:

• Ruleset base (source_type: Organization, melodic-software), pull_request rule → required_review_thread_resolution: true.

So GitHub itself blocks the squash-merge until every Codex advisory inline thread is resolved — independent of what the babysit classifier decides. Two consequences:

• The babysit classifier already treats a clean COMMENTED (non-CHANGES_REQUESTED, no P0/P1/required-fix) Codex review as Nonblocking But Material (feedback.md), and the worker tier already auto-resolves bot threads it has addressed (babysit_resolve_thread.py, review-discipline.md D7.5). But each advisory thread still costs a full per-finding address cycle before it can be resolved — and that happens on every PR because Codex auto-reviews everything. That per-thread cycle is the latency this issue measured.
• Changing the ruleset lives in github-iac / Pulumi (org-owned), not here — and turning required_review_thread_resolution off would remove thread-resolution enforcement for all reviewers (claude + humans too), which is too blunt. Not recommended.

4. Fallback assessment (re: #534)

The fallback as worded — "clean Codex COMMENTED reviews with zero inline threads never gate" — is the trivial case: zero threads means nothing to resolve, so it already doesn't gate. It does not touch the measured cost, which comes from advisory threads that do carry content.

The correctly-located, non-trivial version — and it is worth doing, in #534's shared classifier:

When the shared classifier tags a Codex review thread as advisory (opening comment is the chatgpt-codex-connector bot, review is COMMENTED, no P0/P1/required-fix/regression severity), the worker tier fast-path resolves it — bot-authored resolution is already permitted by D7.5 — without running the full per-finding D1–D7 verification cycle. That satisfies the org ruleset's required_review_thread_resolution autonomously and removes the human/worker clearing latency, while leaving the ruleset (and thread-resolution enforcement for genuine findings) untouched.

This keeps cross-model diversity on every PR (over-delivering vs #536, which only asked for it on high-blast-radius) and kills the latency without the fragile cloud-toggle + bot-trigger path. Recommended primary fix; the §2 scoping is the alternative if the goal shifts to reducing Codex volume/noise rather than thread-clearing latency.

Net: app-level trigger (no repo-file scoping) + org-ruleset gate (not the classifier) ⇒ no PR filed tonight. The durable, in-repo win is the §4 advisory fast-path in #534; the §2 scoping is operator-owned and needs the bot-trigger test first.

Filed by the research worker for #618.

[chatgpt-codex-connector]

To use Codex here, create an environment for this repo.

[kyle-sexton]

Control-tower routing (2026-07-20). Research verdict accepted: repo-file scoping impossible (app-level all-or-nothing toggle); the durable fix is the ADVISORY-THREAD FAST-PATH, now specced on #534 (comment posted there) where the shared classifier owns it. This issue stays open as the app-level ALTERNATIVE, deferred-with-trigger: revisit the chatgpt.com/codex settings toggle only if the goal becomes cutting Codex review volume/noise rather than thread-clearing latency. Operator's optional 2-minute action (cloud toggle ownership check + bot-trigger test) stands as documented above but is NOT required for the primary fix.

[kyle-sexton]

This was generated by AI during triage.

T2 triage planning comment. Confidence: LOW → needs-human (both LOW conditions hold: the Codex trigger is not repo-controllable, and the lever that actually saves queue time narrows review coverage — the exact tradeoff #536 guards, needing operator sign-off).

What I found

1. The Codex connector has no trigger in this repo. Path (a) is not implementable from repo files.

• chatgpt-codex-connector[bot] is a GitHub App. A whole-repo grep for codex/chatgpt returns only unrelated research docs; .github/workflows/ contains exactly one review trigger — claude-review.yml — and nothing invokes Codex. Confirmed on recent PRs: Codex posts COMMENTED reviews as chatgpt-codex-connector[bot].
• Its auto-review trigger lives in the Codex App's own installation/environment settings (github.com / ChatGPT-Codex console), not in any file here. Whether that console even offers path- or label-scoped auto-review is an external capability question for the operator — I cannot verify or change it from repo-visible files, and won't guess at a console setting I can't see. So "scope Codex to the rule-10 class" cannot be done as an in-repo change; it is operator/App-console territory.

2. GitHub itself enforces thread resolution — this reframes path (b).
gh api repos/melodic-software/claude-code-plugins/rules/branches/main shows the pull_request rule has required_review_thread_resolution: true (and required_approving_review_count: 0, squash-only). So every unresolved review thread — including advisory Codex inline threads — blocks merge at the GitHub layer, independent of any babysit logic. A policy codified only in our classifier cannot unblock a merge GitHub is holding.

3. Path (b) as literally worded is near-redundant; the real cost is elsewhere.
"A clean Codex COMMENTED review with zero inline threads never gates" is already effectively true today:

• babysit_merge.py::evaluate() only adds a thread blocker via if threads: over unresolved_threads() — zero inline threads ⇒ no thread blocker; a COMMENTED state sets no reviewDecision block and required_approving_review_count is 0.
• babysit_feedback.py::collect_feedback() already routes a clean COMMENTED bot body (no CRITICAL/IMPORTANT, passes approval_downgrade()) to ignored.

The queue-time cost #618 actually names comes from Codex's advisory inline threads (P1/P2/P3 nits). Two things treat each as blocking regardless of author-class or severity:

• GitHub: required_review_thread_resolution: true (above).
• babysit merge gate: babysit_merge.py::evaluate() if threads: counts every thread from unresolved_threads() as a blocker; the readiness gate (babysit-readiness-gate.sh) additionally counts each Codex /badge/P[0-3]- marker as a finding requiring a classification row.

So the lever that saves real queue time is downgrading advisory Codex inline threads (auto-resolve, or exempt by author-class/severity) — and that is precisely the review-coverage-narrowing call that needs operator sign-off.

Realistic path & concrete next steps

Path (a) is out as an in-repo change. The work splits into one operator decision and one implementable-once-decided change, both riding #534's shared-classifier umbrella (do not spin up a parallel classifier).

Operator decision (blocks the rest — needs-human): choose how to relieve advisory-Codex thread cost, accepting the coverage-vs-security tradeoff #536 flags:

• (a-console) In the Codex App/console, scope Codex auto-review to high-blast-radius PRs (the rule-10 class: skill frontmatter/trigger-keyword changes, cross-plugin contracts, hooks, merge-gate/classifier/security surfaces, or area: security). Requires first confirming the console supports path/label scoping — an autonomous research step. If supported, this is the cleanest fix and needs no repo change.
• (worker-resolve) Keep Codex universal; let the babysit worker tier auto-resolve outdated/advisory Codex bot threads so both the GitHub gate and our gate clear. This narrows nothing about where Codex runs but stops advisory threads from blocking.
• (ruleset) Relax required_review_thread_resolution — an operator ruleset change in the relevant github-iac repo, not here. Highest blast radius; least recommended.

Implementable change once the decision lands (rides #534):

• Codify the merge-ready boolean and thread classification in the single shared classifier source-control:babysit-prs: one shared authorship/finding/approval classifier + golden fixtures; gate script owns the merge-ready boolean (6-issue class) #534 extracts. Add the agreed policy as a golden fixture case, not a new code path. Current call sites to unify: babysit_merge.py::unresolved_threads() + the if threads: blocker in evaluate() (merge-ready boolean), and babysit_feedback.py::collect_feedback() (snapshot bucketing).
• The narrow "clean COMMENTED + zero inline threads never gates" line is safe to encode verbatim as a regression fixture (documents intent, near-noop today).
• The broader "advisory Codex inline thread does not gate" rule is only sound if the operator picks (a-console) or (worker-resolve); encode it in the classifier only under that decision, since it changes what merges without a resolved Codex thread — the review/work-items: cross-vendor (codex) verification for high-blast-radius PRs; round-robin judge diversity for panels (self-preference bias is model-level) #536 coverage tradeoff.

Cross-model review diversity for high-blast-radius PRs stays (per #536): every option above keeps Codex reviewing the rule-10 class; they only change whether its advisory threads on routine PRs block the gate.

Suggested labels

priority: medium, type Task, needs-human, wayfind: interview (the binding open item is the operator's coverage-vs-security decision).