Skip to content

Refuse a tag that is not on main or whose CI did not pass #497

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MongLong0214 merged 2 commits into from
Aug 8, 2026
Merged

Refuse a tag that is not on main or whose CI did not pass #497

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3216a54
Qualify the release before it exists rather than after
MongLong0214 Aug 8, 2026
8ffb31c
Refuse a tag that is not on main or whose CI did not pass
MongLong0214 Aug 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
184 changes: 183 additions & 1 deletion .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ permissions:
contents: write
id-token: write
attestations: write
# `exact-head-ci` reads the check runs GitHub recorded for the tagged commit.
# Reading a branch's green badge would answer a different, weaker question.
checks: read

jobs:
# Refuses the whole release before a single binary is built. A release
Expand All @@ -37,13 +40,192 @@ jobs:
- name: Tag, package.json, and `commitlore --version` agree
run: node scripts/check-release-version.mjs "$GITHUB_REF_NAME"

# A version-consistent tag can still name a commit that exists only on dev.
# `merge-base` can answer that only from complete history: a shallow checkout
# is not a smaller proof, it is an incomplete graph that the script refuses.
# The accepted commit is an output so the API gate below queries the commit an
# annotated tag resolves to, rather than assuming the event SHA is one.
release-target:
runs-on: ubuntu-latest
outputs:
commit: ${{ steps.ancestry.outputs.sha }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: 22
- id: ancestry
name: The pushed tag is already contained in main
run: node scripts/check-release-target.mjs "$GITHUB_REF_NAME" main --github-output

# A tag push does not create a new CI result for the commit it names. The
# relevant evidence is the existing set of check runs at that exact commit,
# not the last green run on main and not a local suite this release happens
# to start. The script has a fixed required list and treats every missing or
# non-success result as a block; an empty API response is therefore a failure.
exact-head-ci:
needs: release-target
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 1
- uses: actions/setup-node@v4
with:
node-version: 22
- name: Every required CI check succeeded at the tagged commit
env:
GITHUB_TOKEN: ${{ github.token }}
run: node scripts/check-exact-head-ci.mjs "$GITHUB_REPOSITORY_OWNER" "${GITHUB_REPOSITORY#*/}" "${{ needs.release-target.outputs.commit }}"

# RELEASE-GATE §4 used to be a human checklist run after `gh release create`.
# That made a failed row a correction to something already published: the
# release existed, but its documented installation did not work. This job
# makes the six checks a prerequisite instead, so publication cannot get
# ahead of the artifact it claims to qualify.
install-gate:
runs-on: ubuntu-latest
steps:
# The default checkout follows the event SHA today, but qualifying the
# branch that happened to contain that SHA would weaken the claim when
# checkout's default changes. The pushed tag is the artifact users name.
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: 22

# A checkout can hide exactly the distribution failures this protects
# against: an untracked bundle, a path that reaches back to the source
# tree, or dependencies left over from CI. Clone the pushed tag anew,
# then run every check from that clone as an installer does.
- name: The tagged installation passes RELEASE-GATE section 4
run: |
set -euo pipefail
clone_dir="$(mktemp -d)/commitlore"
git clone --quiet --branch "$GITHUB_REF_NAME" --depth 1 \
"$GITHUB_SERVER_URL/$GITHUB_REPOSITORY.git" "$clone_dir"
cd "$clone_dir"

# The bundle is the shipped CLI; no build or npm install belongs in
# this job, because either would conceal a clone missing what ships.
clone_version="$(node dist/commitlore.mjs --version)"
printf 'fresh clone version: %s\n' "$clone_version"

if validation_output="$(printf 'Subject\n\nBlast: wide\n' | node dist/commitlore.mjs validate 2>&1)"; then
echo "expected the invalid message to be rejected"
exit 1
else
validation_exit=$?
fi
printf '%s\n' "$validation_output"
test "$validation_exit" -eq 1 || {
echo "validate exited $validation_exit, expected 1"
exit 1
}
case "$validation_output" in
*Blast*) ;;
*)
echo "validate rejected the message without reporting its Blast violation"
exit 1
;;
esac

# A fresh clone is allowed to warn about unfetched notes and setup it
# has not opted into. `doctor` exits zero exactly when none of those
# findings is a fail, which is the RELEASE-GATE row's condition.
node dist/commitlore.mjs doctor

# The installed stub records node's absolute path. This command
# removes node from PATH while retaining git, so a rejection proves
# the hook used that recorded interpreter rather than ambient PATH.
git config user.name "Release gate"
git config user.email "release-gate@example.invalid"
node dist/commitlore.mjs hooks install
if hook_output="$(env -i PATH=/usr/bin:/bin git commit --allow-empty \
-m 'Release gate invalid message' -m 'Blast: wide' 2>&1)"; then
echo "expected the PATH-less hook to reject the invalid commit"
exit 1
else
hook_exit=$?
fi
printf '%s\n' "$hook_output"
test "$hook_exit" -ne 0 || {
echo "the PATH-less hook accepted an invalid commit"
exit 1
}
case "$hook_output" in
*Blast*) ;;
*)
echo "the hook rejected the commit without running validation"
exit 1
;;
esac

# This is the earlier, marker-bearing stub body. Installing first
# records an otherwise healthy target; replacing only the body makes
# a real stale-stub fixture rather than merely testing a missing hook.
hook_path="$(git rev-parse --git-path hooks/commit-msg)"
printf '%s\n' '#!/bin/sh' '# commitlore:commit-msg:v1' \
'exec commitlore validate "$1"' > "$hook_path"
chmod +x "$hook_path"

# Doctor deliberately runs a hook probe with PATH=/usr/bin:/bin.
# This historical body therefore also produces the expected separate
# hook-runtime fail and doctor exits 1. The row being qualified is
# commit-msg-hook, whose required verdict is warn rather than ok, so
# read that row instead of mistaking the fixture's other finding for
# this row's outcome.
if stale_report="$(node dist/commitlore.mjs doctor --json)"; then
stale_doctor_exit=0
else
stale_doctor_exit=$?
fi
printf '%s\n' "$stale_report"
printf 'stale-hook doctor exit: %s\n' "$stale_doctor_exit"
printf '%s' "$stale_report" | node -e '
let input = "";
process.stdin.on("data", (chunk) => { input += chunk; });
process.stdin.on("end", () => {
const report = JSON.parse(input);
const hook = report.checks.find((check) => check.id === "commit-msg-hook");
if (hook?.status !== "warn") {
console.error("expected the stale commit-msg hook to warn, got:", hook);
process.exit(1);
}
console.log("stale commit-msg hook: warn");
});
'

# The runner intentionally prefers a CLI on PATH, so keep PATH narrow
# and include only the node directory needed for the clone fallback.
# Comparing versions catches a different installation that still
# answers --version successfully (#483).
node_dir="$(dirname "$(command -v node)")"
plugin_version="$(env PATH="/usr/bin:/bin:$node_dir" \
CLAUDE_PLUGIN_ROOT="$clone_dir" "$clone_dir/scripts/commitlore-run.sh" --version)"
printf 'plugin version: %s\n' "$plugin_version"
test "$plugin_version" = "$clone_version" || {
echo "plugin resolved $plugin_version, expected clone version $clone_version"
exit 1
}

# Nothing is compiled and nothing is attached. ADR-0026 makes the plugin the
# canonical install path and `install.sh`/`install.ps1` the secondary one, and
# both take a pinned source checkout -- so the release itself is the tag, and the
# tag is what those installers resolve. A release with no assets is the whole
# artifact set, not an incomplete one.
publish:
needs: version-consistency
# A skipped or failed prerequisite skips this job by GitHub's default needs
# semantics. Keep it free of `if:`: a condition such as `always()` would
# turn a failed qualification into a path that can still publish.
needs: [version-consistency, install-gate, release-target, exact-head-ci]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand Down
45 changes: 39 additions & 6 deletions docs/RELEASE-GATE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,11 @@ exclusion, so this table is a spot check of the rule, not the rule itself.

## 4. The installation the documentation describes actually works

These checks run as the `install-gate` job in the tag-triggered release workflow,
against a fresh clone of the pushed tag. `publish` depends on this job as well as
the version, ancestry, and exact-head-CI gates, so a GitHub Release cannot exist
until every automated prerequisite has passed.

| check | command | pass |
|---|---|---|
| fresh clone runs | `git clone`, then `dist/commitlore.mjs --version` | exit 0 |
Expand All @@ -64,23 +69,51 @@ whichever the CLI install is**, and the release check passed for two releases
while reporting the wrong version, because it only asked whether *something*
resolved (#483). Leaving `PATH` alone here would keep asking that question.

## 5. Every published claim is reproducible
## 5. The tag is a promoted, exactly green commit

A matching version and a working fresh clone say nothing about *where* a tag was
cut. A dev-only commit can satisfy both. Nor does a green `main` badge prove the
tagged SHA passed: it may be a failed, cancelled, skipped, timed-out, still
running, or entirely untested commit between two green ones.

The tag workflow therefore blocks `publish` on both jobs below.

| check | command | pass |
|---|---|---|
| release target | `scripts/check-release-target.mjs <tag> main` from a `fetch-depth: 0` checkout | the tag resolves to a commit contained in `main`; a shallow checkout fails rather than guessing |
| exact-head CI | `scripts/check-exact-head-ci.mjs <owner> <repo> <resolved-tag-sha>` | all six explicitly named check runs below are present at that SHA, `completed`, and concluded `success` |

The exact-head list is fixed rather than inferred from the API response:
`check (22)`, `check (24)`, `git-matrix (ubuntu-latest)`, `git-matrix
(macos-latest)`, `install-script`, and `install-ps1`. Any other conclusion —
including `failure`, `cancelled`, `timed_out`, `skipped`, `neutral`, `stale`, or
`action_required` — blocks publication. So do `queued` and `in_progress`, a
check reported for another SHA, and an absent check; an empty result is six
absent checks, not a clean result.

`release-target` exports the commit behind the tag only after the ancestry check
passes. `exact-head-ci` consumes that exact commit, so annotated tags do not
accidentally query CI using a tag-object SHA. Both are direct `publish`
dependencies and `publish` has no `if:` condition that can override a failed or
skipped dependency.

## 6. Every published claim is reproducible

- `scripts/check-readme-numbers.mjs` exits 0 — no number in any README is typed
by hand.
- No README asserts something a command in this file contradicts. The three that
did (green on `main`, a clone carries its dependencies, a clone carries the
whole memory) are the reason this section exists.
- CI is green **at the exact commit being released**, checked by looking at CI
rather than at a local test run. A local suite passed at every one of the three
commits where CI was red.
- CI is green **at the exact commit being released**, as enforced by section 5's
explicit check-run set rather than inferred from a local test run or a branch
badge. A local suite passed at every one of the three commits where CI was red.

## 6. The suite proves something
## 7. The suite proves something

- `npx vitest run` green, and the run reports `Test Files N passed` — a bare test
count is not evidence. A delegated task once reported 943 of a 1108 baseline
because another process was writing to the worktree during its run.
- Each fix in sections 1–4 has a test that fails when that one fix is reverted.
- Each fix in sections 1–5 has a test that fails when that one fix is reverted.
Reverting is the evidence; a passing test proves nothing on its own.

## What this gate deliberately does not require
Expand Down
Loading
Loading