fix(source-control): drop Refs #N from pull-request create opt-out gate (#630) by kyle-sexton · Pull Request #651 · melodic-software/claude-code-plugins

kyle-sexton · 2026-07-20T08:44:49Z

Summary

The pull-request create flow's §2.4.2 pre-create gate treated a bare Refs #N line as a valid closing-keyword opt-out, but the real pr-issue-linkage reusable CI workflow does not accept Refs #N. A PR body that satisfied the skill's own local gate could therefore still fail the CI gate on push. This is the narrow, same-repo fix (issue #630, option 1): align the skill's local opt-out marker set to the validator by dropping Refs #N.

Fix

In plugins/source-control/skills/pull-request/reference/create.md:

§2.4.2 OPTOUT_REGEX drops Refs #N: ^(Refs #[0-9]+|No related issue:) → ^No related issue:. A rationale comment explains why Refs #N is excluded. The two abort-message lines no longer list Refs #N as an opt-out.
§2.4.0 orphan-PR prompt reduced from three options to two (Closes #N / No related issue:); a Refs #N link-without-close reference is now explicitly routed to the ## Related section instead of the closing-keyword line.
§2.4.1 asymmetry note rewritten: a Refs #N line satisfies the closing-keyword half of neither the §2.4.2 gate nor the real validator.
The §2.4.2 "gate passes silently" prose no longer names Refs #N as a silent-pass path.

Refs #N remains valid where it always belonged — as a link-without-close reference inside the ## Related section (§2.4.1) and in the multi-issue prompt where a primary Closes #N already carries the gate. Extending the upstream validator to accept Refs #N (option 2) requires a change in melodic-software/ci-workflows and is out of scope for this lane.

source-control bumped 0.13.1 → 0.13.2 (patch — contract-alignment bug fix) with a matching CHANGELOG entry at the top.

Verification

The real validator's accepted set — read live from the pinned reusable source (melodic-software/ci-workflows/.github/workflows/pr-issue-linkage.yml @ 90f1c54935203fa31b5b3d1f41531228be2c2b7f, the SHA this repo's .github/workflows/pr-issue-linkage.yml pins). Its closing-keyword half accepts exactly two forms:

const CLOSING_KEYWORD =
  /\b(close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s*:?\s*(?:[\w.-]+\/[\w.-]+)?#\d+\b/i;
const NO_ISSUE_MARKER = /\bno (?:linked|related) issue\b/i;

Refs #N matches neither.

The skill's local gate after this change (§2.4.2):

KEYWORD_REGEX='^(close|closes|closed|fix|fixes|fixed|resolve|resolves|resolved):? #[0-9]+'
# Only `No related issue:` — a bare `Refs #N` links without closing and does NOT
# satisfy the real pr-issue-linkage validator's closing-keyword half, so accepting
# it here would clear a body the CI gate then rejects.
OPTOUT_REGEX='^No related issue:'

Refs #N no longer matches OPTOUT_REGEX — the specific false-accept this issue reports is eliminated.

Marker-set relationship. The two sets are not literally identical, and this PR does not claim they are: the validator additionally honors No linked issue, owner/repo-qualified keywords, and looser-whitespace keyword forms that the line-anchored local gate does not. What now holds is the safety property that was broken: every body the local gate passes, the validator also passes (the local gate is a strict safe subset). The one marker the local gate accepted that the validator rejects — Refs #N — has been removed, so a "skill-gate-satisfied" body can no longer fail the real CI gate on this axis. The skill only ever emits No related issue: (never the sibling No linked issue phrasing), so the local gate is deliberately mirrored to what the skill produces, not widened to the validator's full superset.

No stale references remain. Grep of the whole plugin for Refs #:

create.md:144  multi-issue prompt — `Refs #Y` links additional issues (primary `Closes #N` carries the gate)
create.md:153  new prose — routes link-without-close refs to `## Related`
create.md:200  `## Related` section usage (correct)
create.md:202  rewritten asymmetry note — `Refs #N` satisfies neither gate
create.md:214  new OPTOUT_REGEX rationale comment

Every surviving Refs # is either legitimate ## Related / multi-issue linking or prose that now states Refs #N does not satisfy the gate. No occurrence presents Refs #N as a closing-keyword opt-out.

Closes #630

source-control:pull-request: bare 'Refs #N' opt-out clears the skill's local gate but not the real pr-issue-linkage validator #630 — the issue this PR closes (option 1, the narrow same-repo fix).
source-control:pull-request: emit ## Related + closing-keyword skeleton in create flow #487 / PR fix(source-control): scaffold ## Related section in pull-request create flow (#487) #629 — the sibling work this builds on: fix(source-control): scaffold ## Related section in pull-request create flow (#487) #629 added the ## Related scaffold to the create flow and documented this Refs #N opt-out asymmetry in §2.4.1 as a known, deliberately-out-of-scope gap; this PR is the follow-up that closes it.

🤖 Generated with a Claude Code implementation subagent (issue #630)

…te (#630) The §2.4.2 pre-create gate's OPTOUT_REGEX accepted a bare `Refs #N` as a closing-keyword opt-out, but the real pr-issue-linkage reusable CI workflow accepts only a native closing keyword or a `No linked issue` / `No related issue:` phrase — `Refs #N` is not in that set. A `Refs #N`-only body cleared the skill's own gate yet failed CI on push. Drop `Refs #N` from OPTOUT_REGEX (`^No related issue:` only) so any body the local gate passes the validator also passes (strict safe subset). Reconcile the §2.4.0 orphan-PR prompt, §2.4.1 asymmetry note, and §2.4.2 gate messages; `Refs #N` remains valid as a link-without-close reference in `## Related`. Bump source-control 0.13.1 -> 0.13.2 with a CHANGELOG entry. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

claude · 2026-07-20T08:45:04Z

Claude finished @kyle-sexton's task in 1m 53s —— View job

PR Review

Read changed files and PR diff
Analyze correctness of the opt-out gate alignment
Check version bump and CHANGELOG
Post review findings

Summary

This is a clean, narrow bug fix that restores the safety property of the local pre-create gate: every body the skill's gate passes, the real pr-issue-linkage CI validator also passes. The root cause was simple — Refs #N was in OPTOUT_REGEX but is not accepted by the real validator. The fix is correct and the change surface is well-scoped to exactly the files that needed updating.

Correctness

OPTOUT_REGEX change — correct.

The old ^(Refs #[0-9]+|No related issue:) allowed a Refs #N-only body to pass the local gate, but the upstream CI validator's NO_ISSUE_MARKER = /\bno (?:linked|related) issue\b/i doesn't match Refs #. The new ^No related issue: eliminates the false-accept exactly.

The safety-subset invariant is now satisfied in both directions:

Closing keyword path: ^(close|closes|...|resolved):? #[0-9]+ with -iE is line-anchored (more restrictive than the validator's \b-anchored form), so anything passing the local gate passes the real validator.
Opt-out path: ^No related issue: is a strict-cased match for the one phrase the skill itself emits, and the real validator's \bno related issue\b/i is a superset of that — any body passing the local opt-out check passes the CI opt-out check.

Case-sensitivity asymmetry — intentional and correct.

KEYWORD_REGEX uses grep -iE (case-insensitive); OPTOUT_REGEX uses grep -E (case-sensitive). This is a deliberate narrowing: the skill only ever emits the exact-case No related issue:, so the local gate mirrors what the skill produces rather than widening to the full case-insensitive superset the validator accepts. This is documented clearly in §2.4.2. permalink

Prose Consistency

All five remaining Refs # occurrences in create.md were verified:

Line	Role
144	Multi-issue prompt — `Refs #Y` to link-without-close (correct)
153	Orphan-PR prose — explicitly routes `Refs #N` to `## Related`, states it satisfies neither gate (correct)
200	`## Related` scaffold — `Refs #N` as a related-but-not-closed reference (correct)
202	§2.4.1 asymmetry note — `Refs #N` satisfies neither gate (correct)
214	§2.4.2 comment — rationale for exclusion from `OPTOUT_REGEX` (correct)

Zero occurrences present Refs #N as a closing-keyword opt-out. The §2.4.0 orphan-PR prompt correctly shows two options (reduced from three), with the Refs #N case now routed to prose guidance instead of the opt-out list.

CHANGELOG and Version Bump

Version 0.13.1 → 0.13.2 is appropriate: this is a contract-alignment bug fix with no API or behavior changes beyond removing the false-accept. permalink
The CHANGELOG entry is accurate, cites the upstream validator regex forms, names both options considered, and correctly scopes the fix. permalink

No Findings

No bugs, security concerns, stale references, or prose inconsistencies found. The PR does exactly what it says, no more.

Approved.

claude · 2026-07-20T08:48:28Z

Claude finished @kyle-sexton's task in 1m 29s —— View job

Read changed files and PR diff
Analyze correctness of the opt-out gate alignment
Check for stale references in prose and inline comments
Check version bump and CHANGELOG
Post review findings

Summary

This is a correct, well-scoped fix that restores the safety-subset invariant: every body the skill's §2.4.2 gate passes, the real pr-issue-linkage CI validator also passes. The root cause (Refs #N in OPTOUT_REGEX) is cleanly removed; the surrounding prose in §2.4.0, §2.4.1, and §2.4.2 has been reconciled to match.

Gate Logic — Correct

OPTOUT_REGEX after change: ^No related issue:

The old ^(Refs #[0-9]+|No related issue:) would pass a Refs #N-only body through the local gate, but the real validator's NO_ISSUE_MARKER = /\bno (?:linked|related) issue\b/i does not match Refs #. The false-accept path is eliminated.

Case-sensitivity split is intentional and correct. KEYWORD_REGEX uses grep -iE (case-insensitive across all 9 closing-keyword forms); OPTOUT_REGEX uses grep -E (case-sensitive). The skill only ever emits the exact phrase No related issue:, so mirroring that exact case in the gate is a deliberate narrowing — it cannot produce a false-reject for a skill-generated body. This is clearly documented in the §2.4.2 rationale comment. permalink

One Stale Inline Comment

Line 134 still reads:

ISSUE_NUM=""   # fall through to orphan-PR 3-option prompt below

The orphan-PR prompt was reduced from three options to two in this PR (lines 148–152 now list only Closes #N and No related issue:). The 3-option count is stale and should read 2-option. Minor, but since the PR explicitly calls out removing the third option, this comment is now misleading.

Fix this →

Prose Consistency — Clean

All five remaining Refs # occurrences in create.md verified:

Line	Role	Status
144	Multi-issue prompt — `Refs #Y` link-without-close (primary `Closes #N` carries the gate)	✓ Correct
153	Orphan-PR prose — routes `Refs #N` to `## Related`, states it satisfies neither gate	✓ Correct
200	`## Related` scaffold — `Refs #N` as related-but-not-closed reference	✓ Correct
202	§2.4.1 asymmetry note — `Refs #N` satisfies neither gate	✓ Correct
214	§2.4.2 rationale comment — explains exclusion from `OPTOUT_REGEX`	✓ Correct

Zero occurrences present Refs #N as a closing-keyword opt-out.

CHANGELOG and Version Bump

0.13.1 → 0.13.2 is the correct semver level for a contract-alignment bug fix with no API/behavior changes beyond removing the false-accept. The CHANGELOG entry is accurate and well-detailed, citing the upstream validator regex forms and naming both options considered. permalink

Verdict

The core fix is correct. One stale 3-option inline comment at line 134 should be updated to 2-option — everything else is clean. Happy to apply that one-liner if you'd like.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a030d47da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…an-PR eval (#630) Resolve two internal contradictions the §2.4.2 opt-out fix left behind: - §2.4.0 multi-issue prompt appended accepted `Refs #Y` lines to ${CLOSES_LINE} (rendered on the closing-keyword line), contradicting the new "Refs never on the closing-keyword line" invariant. Refs lines now route into `## Related`; `Closes #X` still carries the closing-keyword line. - evals.json eval 8 (closed-branch-issue-does-not-autoclose) still described a three-option orphan prompt (Closes, Refs, no-related-issue); aligned to the two-option prompt (Closes or No related issue:). - Stale inline comment at the branch-issue fallback said "3-option prompt"; corrected to "2-option". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

claude · 2026-07-20T09:13:35Z

Claude finished @kyle-sexton's task in 1m 17s —— View job

Read changed files and PR diff
Evaluate Codex findings (evals.json and multi-issue Refs routing)
Check the stale 3-option comment at line 134
Post review findings

Summary

The second commit (bf00083) correctly resolves all three open items from earlier review passes — including both Codex P2 findings and the stale 3-option inline comment flagged in the prior Claude review. The PR is complete.

Codex Finding 1 — evals.json eval 8: Resolved

evals.json L97: expected_output now reads "explicit Closes or no-related-issue choice", dropping Refs from the enumeration. The three expectations items at lines 100–103 are not option-count dependent and remain accurate.

Codex Finding 2 — Multi-issue Refs routing: Resolved

create.md L146: The prose now explicitly routes each Refs #Y line into ## Related and never onto ${CLOSES_LINE}. The invariant "Refs never on the closing-keyword line" is now stated at every point it could be violated: §2.4.0 multi-issue prompt, §2.4.0 orphan-PR options (lines 148–153), §2.4.1 (line 202), and §2.4.2 rationale (line 214).

Stale `3-option` comment — Resolved

create.md L134: Updated from "3-option prompt" to "2-option prompt".

CHANGELOG

The [0.13.2] entry is extended to document the multi-issue routing fix and eval alignment. Both changes are appropriate patch additions to the same version entry (they are part of the same #630 fix sequence).

No Further Findings

All review items are addressed. The safety-subset invariant holds end-to-end, prose is internally consistent, evals match the current two-option contract, and no stale references remain.

Approved.

kyle-sexton · 2026-07-20T09:14:48Z

@claude re your line-134 finding:

#	Finding	Classification	Evidence	Reacted
1	Stale inline comment at branch-issue fallback reads `3-option prompt`; orphan prompt is now 2 options	VALID — fixed	Line 148 now reads "prompt with two options" (`Closes #N`, `No related issue:`), so the `# fall through to orphan-PR 3-option prompt below` comment at the `ISSUE_NUM=""` fallback was stale.	👍

Corrected to 2-option. Fixed in bf00083, alongside the two codex P2 findings (multi-issue Refs routing + orphan-PR eval alignment).

…ce (#642) (#666) ## Summary The babysit readiness gate blocks while source findings outnumber their per-finding classification rows. The shared classifier counted a self-authored classification pipe-row in ANY comment, including PR-level review-summary comments that are never thread-resolved. A review thread's findings are discounted when it resolves (the #465 lifetime-vs-open guard), but a PR-level comment can never be — so a stale classification posted outside a thread kept counting after its finding was discounted, inflating the classified count past a fresh, still-unclassified open-thread finding and emitting a fail-open `READINESS_OK`. This was the single live fail-open on `main` and the ratified backstop for the #476 gate-off flip: per the backstop clause it must be fixed before the flip. The dispatched fix shape ("non-thread PR-level comments never contribute to `classified`") turned out unsafe on verification: `review-discipline.md` §D5 *mandates* that issue/review-level (PR-level) findings be answered with a detached PR-level classification comment. Thread-only counting would count every such classification as zero and permanently block any PR whose findings come from review summaries — and break the Python↔bash convergence. Corrected in agreement with the tower. ## Fix - **Per-surface classification credit (`babysit_classify.py`).** `count_effective_classified` buckets comments by surface and caps credit within each bucket via `min(classified, findings)`, then sums — a classification can only offset a finding on its own surface, so a stale PR-level row can no longer spill over to cover an open-thread finding. `count_classified` and `count_findings` stay pure raw counters; the bucketing composes them. On unsignalled input every comment lands in one bucket and this collapses to `min(classified, findings)`, preserving existing behavior and the bash convergence property. - **Surface discriminator (`comment_surface`).** Three surfaces — review-thread, PR-level, and an isolated bucket for comments bearing no surface signal — resolved from two signals in order: the explicit `in_review_thread` stamp (authoritative when present), then the `fetch-all-pr-comments.sh` `type` tag on the `--comments-json` reuse path (`inline` → thread; `general`/`review` → PR-level). A comment with neither signal is isolated so its rows cannot offset — and its findings cannot be offset by — a known surface (fail-closed for unknown provenance), preserving the "no signal = PR-level lifetime" model `thread_is_open` documents. - **Surface stamping (`babysit_findings.py`).** `_comment()` records `in_review_thread` (true only when fetched from a review thread); issue-level and review-summary comments are stamped PR-level. The entrypoint emits the effective count. - **Bash degrade cap (`babysit-readiness-gate.sh`).** The thread-blind safe-tier degrade gains the thread-state-free analogue `classified = min(classified, findings)` so a row over-count can't mask a finding and the degrade stays convergent with the Python `min` on unsignalled input. Per-surface bucketing is inherently surface-aware and remains Python-only, exactly like the #465 discount. ## Behavior flip (documented, fail-closed) A PR whose inline-thread findings are answered only by detached PR-level classification replies now reports `READINESS_BLOCKED` where it previously passed — a PR-level row no longer offsets an inline-thread finding. This mechanically enforces §D5's already-ratified reply routing (inline findings MUST reply threaded, "NEVER a detached `pr comment`"). Runs already following §D5 are unaffected; only runs relying on the previously-tolerated detached-reply shape change verdict. ## Residual (documented, not closable here) An orphaned PR-level classification covering a fresh *PR-level* finding is irreducible: GitHub's flat issue comments carry no finding↔classification linkage, so a stale PR-level row is data-identical to a live one. It is reachable only via a §D5 routing violation (an inline finding answered with a detached PR comment) or a reviewer editing/deleting a finding; a §D5-compliant, no-edit run never hits it. The per-surface fix closes the entire linkable (thread) side, on both the live and `--comments-json` reuse paths. ## Verification - `python -m unittest discover -s tests` — 269 tests OK, incl. new `EffectiveClassifiedTests` (per-surface credit, reuse-path `type` inference, explicit-stamp precedence, isolated-unknown), `SurfaceStampingTests`, and `Main642FailOpenTests`. - `babysit-readiness-gate.test.sh` — 60 pass, incl. the live-path #642 scenario, the reuse-path inline-`type` scenario, the unsignalled-provenance isolation scenario (all Python-gated like #465), and all five convergence cases including the over-classified cap. - `engine.test.sh` — pass (unittest + `ruff check`); shellcheck + shfmt clean; `markdownlint-cli2` + `validate-plugins.sh` clean. ## Versioning - `plugins/source-control/.claude-plugin/plugin.json` → **0.13.3**; CHANGELOG entry under `[0.13.3]` documenting the fail-open closure and the §D5 PASS→BLOCK behavior flip. `origin/main` merged in (0.13.2 base from #651). ## Related Closes #642 - #534 / #634 — shared babysit classifier this hardens (found during the #634 digest). - #465 — the lifetime-vs-open discount this mirrors for classifications. - #476 — gate-off flip this fail-open blocks. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01FM1RfM3jHkgenpdbMv4o64 --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

main advanced source-control to 0.13.3 (#651, #666) while this branch bumped from an older 0.13.1 base; resolve the version and CHANGELOG conflict by re-homing this PR's entry as [0.13.4] on top of main's [0.13.3], keeping all main entries intact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FM1RfM3jHkgenpdbMv4o64

kyle-sexton added the automated Opened by automation. label Jul 20, 2026

kyle-sexton marked this pull request as ready for review July 20, 2026 08:48

chatgpt-codex-connector Bot reviewed Jul 20, 2026

View reviewed changes

Comment thread plugins/source-control/skills/pull-request/reference/create.md

Comment thread plugins/source-control/skills/pull-request/reference/create.md

kyle-sexton mentioned this pull request Jul 20, 2026

ops: loop-lane telemetry — running per-lane status (babysit / triage / work) #502

Open

kyle-sexton merged commit 81544d2 into main Jul 20, 2026
17 checks passed

kyle-sexton deleted the fix/630-source-control-pr-optout-marker-asymmetry branch July 20, 2026 09:18

kyle-sexton mentioned this pull request Jul 20, 2026

fix(source-control): scope babysit classification credit to its surface (#642) #666

Merged

kyle-sexton mentioned this pull request Jul 20, 2026

program: fresh-eyes checkpoint audit — tag skill actions for same-context bias and retrofit delegation #304

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(source-control): drop Refs #N from pull-request create opt-out gate (#630) - #651

fix(source-control): drop Refs #N from pull-request create opt-out gate (#630)#651
kyle-sexton merged 2 commits into
mainfrom
fix/630-source-control-pr-optout-marker-asymmetry

kyle-sexton commented Jul 20, 2026

Uh oh!

claude Bot commented Jul 20, 2026 •

edited

Loading

Uh oh!

claude Bot commented Jul 20, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

claude Bot commented Jul 20, 2026 •

edited

Loading

Uh oh!

kyle-sexton commented Jul 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

kyle-sexton commented Jul 20, 2026

Summary

Fix

Verification

Related

Uh oh!

claude Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review

Summary

Correctness

Prose Consistency

CHANGELOG and Version Bump

No Findings

Uh oh!

claude Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Gate Logic — Correct

One Stale Inline Comment

Prose Consistency — Clean

CHANGELOG and Version Bump

Verdict

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

claude Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Codex Finding 1 — evals.json eval 8: Resolved

Codex Finding 2 — Multi-issue Refs routing: Resolved

Stale 3-option comment — Resolved

CHANGELOG

No Further Findings

Uh oh!

kyle-sexton commented Jul 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

claude Bot commented Jul 20, 2026 •

edited

Loading

claude Bot commented Jul 20, 2026 •

edited

Loading

claude Bot commented Jul 20, 2026 •

edited

Loading

Stale `3-option` comment — Resolved