feat(release): mechanise vNEXT resolution, enforce the heading rule, gate release scope - #702
Merged
Merged
Conversation
…gate release scope Three release-process gaps, all found by running the v0.91.0 release. Each was a rule that existed only as prose plus a hand-run command, and each failed the same way: a release is exactly when parallel branches converge, so any step of the form 'run this grep when releasing' eventually loses a merge race. 1. make vnext-resolve VERSION=X.Y.Z Checklist step 4 resolved 54 placeholders by hand off the checker's own output. The scanner already separates a live gate from prose with perfect precision, so --resolve reuses it and rewrites only outside inline-code spans -- including on a line carrying both a quoted mention and a live gate, which a line-level sed corrupts. Refuses a VERSION disagreeing with pyproject.toml, because packaging parses 'v0.91' and '0.91' happily. 2. vNEXT in a markdown heading is now fatal on EVERY PR Resolving a placeholder in a heading rewrites its anchor slug and breaks inbound links. The rule was a release-time grep; in 0.91.0 it lost a race (#697 ran two minutes before #694 and #696 landed headings of their own) and three shipped. Checking at authoring time makes the race impossible. Numeric headings are deliberately not flagged -- a resolved tag never changes again, so its slug is stable. 3. make release-scope-check Proves the changelog entry covers every PR the tag will CONTAIN, not the scope collected when the release PR was opened. In 0.91.0, #625 merged nine minutes before the release PR and landed inside the tag with no release note; changelog-check cannot see this, since it proves every released version has an entry, never that an entry covers every commit under the tag. Armed in CI exactly like the vNEXT gate (version-raising PRs only), with the checkout deepened only then and a fail-open path so an ordinary PR can never go red on it. 39 new tests. CONTRIBUTING's release checklist grows to 18 steps.
Comment on lines
+188
to
+197
| for lineno, line in enumerate( | ||
| path.read_text(encoding="utf-8", errors="replace").splitlines(), start=1 | ||
| ): | ||
| if VNEXT_TOKEN not in line or not HEADING_RE.match(line): | ||
| continue | ||
| # Same quotation rule as the residue scan: a heading that merely | ||
| # names the token in backticks is prose about the placeholder. | ||
| if VNEXT_TOKEN in INLINE_CODE_RE.sub("", line): | ||
| flagged.append(VnextResidue(path=rel, line=lineno, text=line.strip())) | ||
| return flagged |
There was a problem hiding this comment.
🟡 Fenced command-list gates misread as headings
find_heading_placeholders matches any line starting with # plus a space and never tracks fenced code blocks. CLAUDE.md's ## All CLI Commands is one large fenced block whose gate comments begin with # and, in a feature PR, carry (since vNEXT). Such a line is reported as a fatal heading placeholder and fails make version-gate-check on every PR, though it is a code comment with no anchor and the residue scan (find_vnext_residue) deliberately treats the same fenced line as a live gate.
Prompt for agents
find_heading_placeholders in scripts/check_version_gates.py classifies any line matching HEADING_RE (^ {0,3}#{1,6} ) as a markdown heading, but it does not track fenced code blocks (``` / ~~~). CLAUDE.md's `## All CLI Commands` section is a single fenced code block whose version-gate comment lines start with `# ` (e.g. `# workspace load (since 0.91.0, #687)`), and in a feature PR those gates are written `(since vNEXT)`. The residue scan find_vnext_residue intentionally does NOT strip fences because those fenced comments are genuine live gates (see the module-level comment and test_gate_inside_a_fence_is_still_a_gate). So a legitimate fenced gate comment gets misclassified as a heading and, because the heading check is fatal in every mode, it fails `make version-gate-check` on the feature PR that writes it — with no valid fix, since a fenced code comment is not a heading and has no anchor slug. Make the heading scan fence-aware: skip lines that fall inside a fenced code block (toggle on ``` / ~~~ fence markers) before applying HEADING_RE, so only real ATX headings outside fences are flagged.
Was this helpful? React with 👍 or 👎 to provide feedback.
soustruh
added a commit
that referenced
this pull request
Aug 26, 2026
A "#" comment line inside a markdown code fence renders as content. It has no anchor slug, so the slug-breakage rationale behind the heading check (#702) does not apply to it. CLAUDE.md's command section is one giant fence full of such lines, and convention #17 requires feature PRs to tag new notes there with vNEXT -- the check therefore failed every PR that follows the documented process. This PR was the first to hit it. The residue scan is untouched: a fenced gate is still a live gate the release PR must resolve.
This was referenced Aug 26, 2026
soustruh
added a commit
that referenced
this pull request
Aug 27, 2026
A "#" comment line inside a markdown code fence renders as content. It has no anchor slug, so the slug-breakage rationale behind the heading check (#702) does not apply to it. CLAUDE.md's command section is one giant fence full of such lines. Convention #17 requires feature PRs to tag new notes there with vNEXT. The check therefore failed every PR that follows the documented process. This PR was the first to fail on it. The residue scan is untouched: a fenced gate is still a live gate the release PR must resolve.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three release-process gaps, all surfaced by actually running the v0.91.0 release. Each was a rule that existed as prose in CONTRIBUTING.md plus a command someone was supposed to remember — and each failed the same way.
1.
make vnext-resolve VERSION=X.Y.ZChecklist step 4 had a human resolve 54 placeholders by hand, working off the checker's own output. That is backwards:
check_version_gates.pyalready separates a live gate from prose with perfect precision — it is the same scanner that fails CI — but had no--fix.--resolvereuses it and substitutes only outside inline-code spans, so:-+ "--flag" +(since vNEXT)the placeholder **`vNEXT`** -- `(since vNEXT)`.`vNEXT` is the placeholder; (since vNEXT) is liveThat last row is the case a line-level
sedgets wrong, and it is not hypothetical —CLAUDE.md:170is exactly that shape.It also refuses any
VERSIONthat disagrees withpyproject.toml. Format validation alone cannot catch a typo here:packagingparsesv0.91and0.91as valid, so a slip would stamp a wrong-but-parseable version into every gate in the tree at once.2.
vNEXTin a markdown heading is now fatal on every PRResolving a placeholder inside a heading rewrites the heading, which rewrites its generated anchor slug, which breaks every inbound
#...link.The rule was a release-time
grep -rn '^##.*vNEXT' plugins/. In 0.91.0 it lost a race: #697 ran it at 13:20; #694 landed its heading at 13:18 and #696 at 13:21. All three shipped and had to be cleaned up after the tag (found by Devin on #699).Deliberately armed outside release mode, unlike the advisory
vNEXTresidue check. A placeholder in a heading is never correct at any point in the cycle, so it must fail the PR that writes it — which is what makes the race impossible rather than merely unlucky. Verified against the real regression: reintroducing thegotchas.md:530heading turns a plainmake version-gate-checkred.Numeric headings (
## … (since v0.80.0)) are not flagged: an already-resolved tag never changes again, so its slug is stable and there is no inbound link at risk. Only.mdfiles are scanned — a#insrc/**/*.pyopens a comment, which has no anchor.3.
make release-scope-checkProves the changelog entry covers every PR the tag will contain, rather than the scope collected when the release PR was opened.
In 0.91.0, #625 merged at 14:27 — nine minutes before the release PR did — landing inside the tag's tree with no release note of any kind. It was caught only because the tag happened to be deferred.
make changelog-checkstructurally cannot see this: it proves every released version has an entry, never that an entry covers every commit under the tag.Armed in CI exactly like the vNEXT gate (version-raising PRs only). Two deliberate constraints:
make release-scope-check, the cost of a false arming is the whole repo blocked.Run it in the release PR before merging and nothing needs ignoring — the release PR's own number is not in the log until its merge commit exists.
--ignore-pr Ncovers the after-the-fact case.Tests
39 new tests, all written before the implementation.
Two of them earned their keep by failing for the right reason and correcting my design:
test_rejects_a_malformed_version— I first assertedv0.91would be rejected. It is not;packagingaccepts it. The test premise was wrong, not the code — and finding that out is what produced the pyproject cross-check inmain(), which is a far stronger guard than format validation.test_live_repository_has_no_placeholder_headings— pins the real tree, so a reintroduction is caught even if no fixture covers the shape.make check: 6356 passed, 12 skipped. Everything green.Not included
Two things from the same diagnosis, deliberately left out so this PR stays reviewable:
check_command_sync.pyvalidates thatCLAUDE.md,context.pyandcommands-reference.mdagree with the live Typer tree, but all three are still hand-edited.SKILL.mdis already generated; doing the same for the other three would remove a whole class of drift and a large share of the per-release gate churn.