Skip to content

feat(release): mechanise vNEXT resolution, enforce the heading rule, gate release scope - #702

Merged
padak merged 1 commit into
mainfrom
claude/release-process-gates
Aug 25, 2026
Merged

feat(release): mechanise vNEXT resolution, enforce the heading rule, gate release scope#702
padak merged 1 commit into
mainfrom
claude/release-process-gates

Conversation

@padak

@padak padak commented Aug 25, 2026

Copy link
Copy Markdown
Member

Three release-process gaps, all surfaced by actually running the v0.91.0 release. Each was a rule that existed as prose in CONTRIBUTING.md plus a command someone was supposed to remember — and each failed the same way.

The shared root cause. A release is precisely the moment parallel branches converge. So any rule of the form "run this grep when you release" eventually loses a merge race, and the loss is silent. All three fixes move the rule from a remembered step to an enforced one.

1. make vnext-resolve VERSION=X.Y.Z

Checklist step 4 had a human resolve 54 placeholders by hand, working off the checker's own output. That is backwards: check_version_gates.py already separates a live gate from prose with perfect precision — it is the same scanner that fails CI — but had no --fix.

--resolve reuses it and substitutes only outside inline-code spans, so:

Line Result
- + "--flag" + (since vNEXT) rewritten
the placeholder **`vNEXT`** -- `(since vNEXT)`. untouched
`vNEXT` is the placeholder; (since vNEXT) is live only the second token rewritten

That last row is the case a line-level sed gets wrong, and it is not hypothetical — CLAUDE.md:170 is exactly that shape.

It also refuses any VERSION that disagrees with pyproject.toml. Format validation alone cannot catch a typo here: packaging parses v0.91 and 0.91 as valid, so a slip would stamp a wrong-but-parseable version into every gate in the tree at once.

2. vNEXT in a markdown heading is now fatal on every PR

Resolving a placeholder inside a heading rewrites the heading, which rewrites its generated anchor slug, which breaks every inbound #... link.

The rule was a release-time grep -rn '^##.*vNEXT' plugins/. In 0.91.0 it lost a race: #697 ran it at 13:20; #694 landed its heading at 13:18 and #696 at 13:21. All three shipped and had to be cleaned up after the tag (found by Devin on #699).

Deliberately armed outside release mode, unlike the advisory vNEXT residue check. A placeholder in a heading is never correct at any point in the cycle, so it must fail the PR that writes it — which is what makes the race impossible rather than merely unlucky. Verified against the real regression: reintroducing the gotchas.md:530 heading turns a plain make version-gate-check red.

Numeric headings (## … (since v0.80.0)) are not flagged: an already-resolved tag never changes again, so its slug is stable and there is no inbound link at risk. Only .md files are scanned — a # in src/**/*.py opens a comment, which has no anchor.

3. make release-scope-check

Proves the changelog entry covers every PR the tag will contain, rather than the scope collected when the release PR was opened.

In 0.91.0, #625 merged at 14:27 — nine minutes before the release PR did — landing inside the tag's tree with no release note of any kind. It was caught only because the tag happened to be deferred. make changelog-check structurally cannot see this: it proves every released version has an entry, never that an entry covers every commit under the tag.

Armed in CI exactly like the vNEXT gate (version-raising PRs only). Two deliberate constraints:

  • The checkout is deepened only when the version differs — the check needs tags and real history, and paying that on every PR to buy a check that only matters on release PRs is the wrong trade.
  • It fails open. An unreadable base or a shallow checkout without tags produces a warning and exit 0. An ordinary PR must never go red because of this; the cost of a missed arming is one local make release-scope-check, the cost of a false arming is the whole repo blocked.

Run it in the release PR before merging and nothing needs ignoring — the release PR's own number is not in the log until its merge commit exists. --ignore-pr N covers the after-the-fact case.

Tests

39 new tests, all written before the implementation.

Two of them earned their keep by failing for the right reason and correcting my design:

  • test_rejects_a_malformed_version — I first asserted v0.91 would be rejected. It is not; packaging accepts it. The test premise was wrong, not the code — and finding that out is what produced the pyproject cross-check in main(), which is a far stronger guard than format validation.
  • test_live_repository_has_no_placeholder_headings — pins the real tree, so a reintroduction is caught even if no fixture covers the shape.

make check: 6356 passed, 12 skipped. Everything green.

Not included

Two things from the same diagnosis, deliberately left out so this PR stays reviewable:

  • Retiring dead gates. 77 distinct versions are gated and 79% are older than 0.80.0 (the oldest is 0.21.0). Nobody runs those, but they cost prompt budget in a file with a hard 70 000 B cap. Needs a policy decision on the floor version, not just code.
  • Generating the command lists. check_command_sync.py validates that CLAUDE.md, context.py and commands-reference.md agree with the live Typer tree, but all three are still hand-edited. SKILL.md is already generated; doing the same for the other three would remove a whole class of drift and a large share of the per-release gate churn.

Open in Devin Review

…gate release scope

Three release-process gaps, all found by running the v0.91.0 release. Each was
a rule that existed only as prose plus a hand-run command, and each failed the
same way: a release is exactly when parallel branches converge, so any step of
the form 'run this grep when releasing' eventually loses a merge race.

1. make vnext-resolve VERSION=X.Y.Z
   Checklist step 4 resolved 54 placeholders by hand off the checker's own
   output. The scanner already separates a live gate from prose with perfect
   precision, so --resolve reuses it and rewrites only outside inline-code
   spans -- including on a line carrying both a quoted mention and a live gate,
   which a line-level sed corrupts. Refuses a VERSION disagreeing with
   pyproject.toml, because packaging parses 'v0.91' and '0.91' happily.

2. vNEXT in a markdown heading is now fatal on EVERY PR
   Resolving a placeholder in a heading rewrites its anchor slug and breaks
   inbound links. The rule was a release-time grep; in 0.91.0 it lost a race
   (#697 ran two minutes before #694 and #696 landed headings of their own) and
   three shipped. Checking at authoring time makes the race impossible.
   Numeric headings are deliberately not flagged -- a resolved tag never
   changes again, so its slug is stable.

3. make release-scope-check
   Proves the changelog entry covers every PR the tag will CONTAIN, not the
   scope collected when the release PR was opened. In 0.91.0, #625 merged nine
   minutes before the release PR and landed inside the tag with no release
   note; changelog-check cannot see this, since it proves every released
   version has an entry, never that an entry covers every commit under the tag.
   Armed in CI exactly like the vNEXT gate (version-raising PRs only), with the
   checkout deepened only then and a fail-open path so an ordinary PR can never
   go red on it.

39 new tests. CONTRIBUTING's release checklist grows to 18 steps.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

Comment on lines +188 to +197
for lineno, line in enumerate(
path.read_text(encoding="utf-8", errors="replace").splitlines(), start=1
):
if VNEXT_TOKEN not in line or not HEADING_RE.match(line):
continue
# Same quotation rule as the residue scan: a heading that merely
# names the token in backticks is prose about the placeholder.
if VNEXT_TOKEN in INLINE_CODE_RE.sub("", line):
flagged.append(VnextResidue(path=rel, line=lineno, text=line.strip()))
return flagged

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Fenced command-list gates misread as headings

find_heading_placeholders matches any line starting with # plus a space and never tracks fenced code blocks. CLAUDE.md's ## All CLI Commands is one large fenced block whose gate comments begin with # and, in a feature PR, carry (since vNEXT). Such a line is reported as a fatal heading placeholder and fails make version-gate-check on every PR, though it is a code comment with no anchor and the residue scan (find_vnext_residue) deliberately treats the same fenced line as a live gate.

Prompt for agents
find_heading_placeholders in scripts/check_version_gates.py classifies any line matching HEADING_RE (^ {0,3}#{1,6} ) as a markdown heading, but it does not track fenced code blocks (``` / ~~~). CLAUDE.md's `## All CLI Commands` section is a single fenced code block whose version-gate comment lines start with `# ` (e.g. `# workspace load (since 0.91.0, #687)`), and in a feature PR those gates are written `(since vNEXT)`. The residue scan find_vnext_residue intentionally does NOT strip fences because those fenced comments are genuine live gates (see the module-level comment and test_gate_inside_a_fence_is_still_a_gate). So a legitimate fenced gate comment gets misclassified as a heading and, because the heading check is fatal in every mode, it fails `make version-gate-check` on the feature PR that writes it — with no valid fix, since a fenced code comment is not a heading and has no anchor slug. Make the heading scan fence-aware: skip lines that fall inside a fenced code block (toggle on ``` / ~~~ fence markers) before applying HEADING_RE, so only real ATX headings outside fences are flagged.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@padak
padak merged commit 79eb227 into main Aug 25, 2026
4 checks passed
@padak
padak deleted the claude/release-process-gates branch August 25, 2026 21:11
soustruh added a commit that referenced this pull request Aug 26, 2026
A "#" comment line inside a markdown code fence renders as content.
It has no anchor slug, so the slug-breakage rationale behind the
heading check (#702) does not apply to it. CLAUDE.md's command
section is one giant fence full of such lines, and convention #17
requires feature PRs to tag new notes there with vNEXT -- the check
therefore failed every PR that follows the documented process. This
PR was the first to hit it. The residue scan is untouched: a fenced
gate is still a live gate the release PR must resolve.
soustruh added a commit that referenced this pull request Aug 27, 2026
A "#" comment line inside a markdown code fence renders as content.
It has no anchor slug, so the slug-breakage rationale behind the
heading check (#702) does not apply to it. CLAUDE.md's command
section is one giant fence full of such lines. Convention #17
requires feature PRs to tag new notes there with vNEXT. The check
therefore failed every PR that follows the documented process.
This PR was the first to fail on it. The residue scan is untouched:
a fenced gate is still a live gate the release PR must resolve.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant