ci: gate the two release-drift failures this release actually hit - #639
Merged
Conversation
padak
force-pushed
the
claude/release-drift-gates
branch
from
August 22, 2026 15:11
fab8f63 to
39b1af5
Compare
Preparing 0.88.0 surfaced two mistakes that no check in the repo could see. Both are mechanical, so both are now gated. 1. Version gates naming a version that never ships `make version-gate-check` (new, and wired into the CI `check` job) rejects any `(since vX.Y.Z)` / `X.Y.Z+` marker whose version has no CHANGELOG entry. `keboola-expert.md` turns those markers into a hard refusal rule: the agent compares the user's installed version and declines anything newer. A marker naming a version that never ships therefore makes the agent refuse a flag the user actually has. Collapsing 0.89.0 into 0.88.0 broke 20 markers across 8 files at once, and only a hand-grep caught them. Scope is the two explicit gate syntaxes, never a bare version-looking string -- prose carries `keboola-mcp-server v1.76.2`, `manifest v3`, `RFC 8628`, and flagging those would make the check unusable. Cost of that choice: a stale bare mention passes silently. It covers 418 markers across 71 versions today, all resolving. Offline, unlike changelog-check, so it runs per-PR in CI. The CI comment claiming these tags "need judgement" is split: whether a marker names the RIGHT version does and stays with /kbagent:review; that the version EXISTS does not. 2. Changelog entries that were never published `make changelog-check` gains the inverse of the audit it already ran. It verified every release has an entry; nothing verified every entry has a release. A version bump rides along with any PR touching pyproject.toml while publishing is a separate manual step, so a merge-train strands a whole block and the next tag absorbs it -- its notes written from one block, the other shipped unannounced. That is what happened here. It has happened 18 times against 161 published releases. Those are frozen in KNOWN_UNRELEASED as a baseline the check exists to stop growing; publishing a tag retroactively would only advertise a wheel nobody can install. The audit is bounded to the window `gh release list --limit` actually returned, so a truncated fetch cannot report early releases as missing -- the first draft of this check did exactly that, and the same truncation is why the orphan count was first reported as 5. 20 tests added. Both helpers are pure so they are unit-testable without network; the live-repo assertions guard the guards (a broken glob would make the version-gate check pass vacuously).
The gate job's comment claims it runs the same silent-drift gates ci.yml enforces so a tag pushed directly cannot bypass them. The version-gate check was added to ci.yml in the parent commit but not here, which left exactly the wrong hole: a renumbered release is a mistake that reaches a TAG rather than a PR, and it breaks every marker naming the dropped version at once.
padak
force-pushed
the
claude/release-drift-gates
branch
from
August 22, 2026 15:23
39b1af5 to
670da35
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #638 — review/merge that first.
Why
Preparing 0.88.0 surfaced two mistakes that no check in this repo could see. Both are mechanical, so both are now gated rather than left to the next person's diligence.
1.
make version-gate-check(new, in CI)Rejects any
(since vX.Y.Z)/X.Y.Z+marker whose version has noCHANGELOGentry.keboola-expert.mdturns those markers into a hard refusal rule — the agent compares the user's installedkbagent versionand declines anything newer. A marker naming a version that never ships makes the agent refuse a flag the user actually has. Collapsing the unreleased0.89.0into0.88.0broke 20 markers across 8 files at once, and only a hand-grep caught them.Verified it catches the real thing:
Scope is deliberately the two explicit gate syntaxes, never a bare version-looking string. Prose carries
keboola-mcp-server v1.76.2,manifest v3,RFC 8628— flagging those makes the check unusable. The cost is that a stale bare mention (# 0.89.0 MCP-parity flagsin a test comment) passes silently; those mislead a reader, not the agent's version gate. Documented in the script docstring.Offline (reads the tree only), so unlike
changelog-checkit runs per-PR in CI. That splits the existing CI comment's claim that these tags "need judgement": whether a marker names the right version does and stays with/kbagent:review; that the version exists does not.Covers 418 markers across 71 versions today, all resolving.
2.
make changelog-checkgains the inverse directionIt verified every release has an entry. Nothing verified every entry has a release.
A version bump rides along with any PR touching
pyproject.toml; publishing is a separate manual step. So a merge-train strands a whole block and the next tag absorbs it — notes written from one block, the other shipped unannounced. Exactly what happened here.It has happened 18 times against 161 published releases:
Frozen in
KNOWN_UNRELEASEDas a baseline the check stops from growing — publishing those tags retroactively would advertise a wheel nobody can install, and rewriting old notes reaches nobody who already read them.One correctness note worth reviewing: the audit is bounded to the window
gh release list --limitactually returned. Without that floor, a truncated fetch reports every early release as unpublished — the first draft did exactly that (--limit 50vs 173 changelog keys → 124 false positives). The same truncation is why the orphan count was first measured as 5 rather than 18; the limit is also raised 50 → 500 so the window covers the whole history.Tests
20 added, all offline. Both helpers are pure, so the audits are unit-testable without network; the live-repo assertions guard the guards —
test_the_scan_actually_finds_gatesfails if a broken glob would make the version-gate check pass vacuously.make checkgreen: 5853 passed, 12 skipped.