fix: require release spine evidence before publish - #1205
Conversation
|
Warning Review limit reached
Next review available in: 48 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthrough릴리스 워크플로우에 Changes릴리스 스파인 증거 게이트
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c2a9142c39
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
📊 Benchmark Results✅ All benchmarks passed
Updated: 2026-07-04T05:58:51.833Z · Commit: b358b56 |
2c0fa12 to
154f359
Compare
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/release.yml:
- Around line 21-23: The release workflow timeout increase should be checked
against the script’s own timeout budget. Review the release job in release.yml
together with release-spine-evidence.mts and its defaultTotalTimeoutMs setting,
then align the workflow timeout to leave a clear safety margin over the internal
script timeout rather than using an unnecessarily large buffer. Keep the release
job timeout and the script-level timeout consistent so the job can still fail
promptly on hangs.
- Around line 157-163: The Upload release spine evidence step uses
actions/upload-artifact@v4 instead of a SHA-pinned reference, which violates the
repository’s hash-pin policy. Update this workflow step to use the exact commit
SHA for actions/upload-artifact, matching the required pinned-action pattern
used by the release pipeline.
In `@scripts/release-spine-evidence.mts`:
- Around line 148-312: The release manifest in
createReleaseSpineEvidenceManifest has a timeout-ordering issue: the summed
per-check timeouts exceed defaultTotalTimeoutMs, so later checks can be skipped
before collecting evidence. Move the required release-metadata EvidenceCommand
earlier in the array, or raise/rebalance the total budget, so mandatory evidence
is gathered before the timeout can be exhausted.
- Around line 962-1000: The argument parsing in the CLI option handling for
parseArgs() is order-dependent because the --root branch always overwrites
outputDir with join(rootDir, defaultOutputDirectory), which can erase a prior
--output-dir value. Update the logic so --output-dir is tracked independently
from the default-derived path, and only set outputDir from --root when no
explicit output directory was provided; use the existing parseArgs, rootDir,
outputDir, and defaultOutputDirectory symbols to keep the behavior consistent
regardless of argument order.
In `@scripts/tests/release-spine-evidence.spec.ts`:
- Around line 323-338: The parseArgs test only covers the case where --root
comes before --output-dir, so it misses the ordering bug when the flags are
reversed. Add a new regression test in release-spine-evidence.spec.ts that calls
parseArgs with --output-dir before --root and asserts the explicit output
directory is preserved, using createTempRepo, parseArgs, rootDir, and outputDir
to locate the relevant behavior.
In `@scripts/tests/release-workflow.spec.ts`:
- Around line 200-214: The release workflow test for the Release spine evidence
gate is missing a fail-closed assertion for its own if condition. Update
release-workflow.spec.ts in the test that checks the Release spine evidence step
to also verify the step’s if expression does not include always() and still
reflects the intended gate behavior around release_spine_evidence. Use the
existing readReleaseWorkflow() and the Release spine evidence /
release_spine_evidence markers to locate the assertion block, and add a check
that would fail if someone weakened this gate to always() while keeping the
current run/artifact/order assertions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: f8700a77-fec1-470c-9081-75a400490a3c
📒 Files selected for processing (5)
.github/workflows/release.ymlpackage.jsonscripts/release-spine-evidence.mtsscripts/tests/release-spine-evidence.spec.tsscripts/tests/release-workflow.spec.ts
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5a13be87c7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
1de79db to
864cebd
Compare
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
scripts/tests/release-spine-evidence.spec.ts (1)
1-429: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win테스트 파일 위치를
src/tests/[ClassName].spec.ts패턴으로 맞추세요.현재 파일은
scripts/tests/release-spine-evidence.spec.ts에 있어 mandatory test path rule과 다릅니다. 이동 시../release-spine-evidence.mtsimport 경로도 함께 조정해야 합니다.As per coding guidelines,
**/tests/**/*.spec.ts: “Place test files insrc/tests/[ClassName].spec.tspattern (mandatory rule)”.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/tests/release-spine-evidence.spec.ts` around lines 1 - 429, The release-spine-evidence spec is in the wrong test location and must be moved to the required src/tests/[ClassName].spec.ts pattern. Relocate the test file, then update the relative import from release-spine-evidence.mts so it still resolves correctly after the move; use the existing release-spine-evidence.spec.ts and runReleaseSpineEvidence references to find the affected test module.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/release.yml:
- Around line 21-23: The current job-level timeout in release workflow is too
tight because it applies to all steps, not just release:spine-evidence.
Recalculate the full duration across checkout, setup, install, audit, check,
summary/artifact handling, and dry-run publish, then update timeout-minutes on
the workflow job so the total budget safely covers every step; use the release
job definition in release.yml as the place to adjust it.
In `@scripts/release-spine-evidence.mts`:
- Around line 670-681: The failureReason function is prioritizing artifactReason
before checking the command result, which lets artifact-missing messages
overwrite timeout or non-zero exit failures. Update failureReason to derive the
command failure cause first using CommandRunResult fields like timedOut,
errorMessage, and status, and only fall back to artifactReason when the command
itself did not fail. Keep the logic in failureReason so timed_out cases report
the real command reason instead of “artifact not produced.”
- Around line 942-945: The parsePositiveInteger helper is allowing partial
numeric parses for flags like --total-timeout-ms, so update the
parsePositiveInteger function to validate the entire input string is a positive
integer before converting it. Replace the current Number.parseInt-based logic
with a full-string digit check followed by numeric conversion, and keep the
existing flag-specific error handling in parsePositiveInteger.
- Around line 519-531: The timeout handling in the spawn flow only kills the
immediate child process, so descendant workers from the command can survive;
update the process management in the command execution logic around
spawn/activeCommandProcess/timeoutTimer to terminate the whole process tree
instead. Use a process-group aware approach such as spawning detached and
signaling the negative pid, or a tree-kill style helper, and make sure the
SIGTERM/SIGKILL fallback path covers all spawned children.
In `@scripts/tests/release-spine-evidence.spec.ts`:
- Around line 14-26: The import in release-spine-evidence.spec.ts mixes runtime
symbols with inline type-only symbols, so split the type references out of the
existing import. Keep the value imports like createReleaseSpineEvidenceManifest,
defaultCommandRunner, markReportInterrupted, parseArgs, and
runReleaseSpineEvidence in the current import, and move Clock, CommandRunResult,
CommandRunner, EvidenceArtifactExpectation, EvidenceCommand, and
ReleaseSpineEvidenceReport into a separate import type statement.
In `@scripts/tests/release-workflow.spec.ts`:
- Around line 229-253: The current release workflow test in
release-workflow.spec.ts only checks for the always() condition on the whole
workflow text, so it cannot distinguish between the Publish release spine
evidence summary step and the Upload release spine evidence step. Update the
test around the release_spine_evidence / release-spine-evidence assertions to
slice each step separately and verify the if condition on each unique step
block, using the existing identifiers like "Publish release spine evidence
summary" and "Upload release spine evidence" to ensure both retain always()
independently.
---
Outside diff comments:
In `@scripts/tests/release-spine-evidence.spec.ts`:
- Around line 1-429: The release-spine-evidence spec is in the wrong test
location and must be moved to the required src/tests/[ClassName].spec.ts
pattern. Relocate the test file, then update the relative import from
release-spine-evidence.mts so it still resolves correctly after the move; use
the existing release-spine-evidence.spec.ts and runReleaseSpineEvidence
references to find the affected test module.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 4a5e813a-fbab-4540-8163-2e42e9da5310
📒 Files selected for processing (5)
.github/workflows/release.ymlpackage.jsonscripts/release-spine-evidence.mtsscripts/tests/release-spine-evidence.spec.tsscripts/tests/release-workflow.spec.ts
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e3ea4f64fa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e91252336
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6e8999446e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fdf4316941
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d76f07d0a3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/release.yml:
- Around line 21-23: The release workflow timeout budget is missing the time for
the Release gate maintenance self-check, since should_run_publish_gates and
should_verify_release_gate_maintenance can both be true in the same job. Update
the timeout plan in release.yml so the wrapper budget and timeout-minutes
reflect this extra serialized step, or split the self-check into its own
job/timeout; use the existing publish-gates and maintenance-gate flow to locate
the affected job definition.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 42bcaab5-455a-4a51-b6e9-7b6136ae9018
📒 Files selected for processing (4)
.github/workflows/release.ymlscripts/release-spine-evidence.mtsscripts/tests/release-spine-evidence.spec.tsscripts/tests/release-workflow.spec.ts
d76f07d to
973d21f
Compare
Fixes #1170.
Summary
pnpm release:spine-evidence, a consolidated publish-path gate that runs the release-critical Croco 1.0 spine evidence set and writes JSON/Markdown reports underci-reports/release/..github/workflows/release.ymlbehind the final spine evidence command before dry-run publish, while preserving audit, repository policy checks, provenance verification, and release summary/artifact upload.Verification
COREPACK_ENABLE_DOWNLOAD_PROMPT=0 corepack pnpm exec vitest run scripts/tests/release-spine-evidence.spec.ts scripts/tests/release-workflow.spec.ts --config vitest.config.ts- passed, 23 tests.COREPACK_ENABLE_DOWNLOAD_PROMPT=0 corepack pnpm exec oxfmt --check scripts/release-spine-evidence.mts scripts/tests/release-spine-evidence.spec.ts scripts/tests/release-workflow.spec.ts .github/workflows/release.yml package.json- passed.COREPACK_ENABLE_DOWNLOAD_PROMPT=0 corepack pnpm exec oxlint scripts/release-spine-evidence.mts scripts/tests/release-spine-evidence.spec.ts scripts/tests/release-workflow.spec.ts- passed.COREPACK_ENABLE_DOWNLOAD_PROMPT=0 corepack pnpm exec tsc --noEmit --allowImportingTsExtensions --module NodeNext --moduleResolution NodeNext --target ES2022 --types node,vitest/globals --skipLibCheck scripts/release-spine-evidence.mts scripts/tests/release-spine-evidence.spec.ts scripts/tests/release-workflow.spec.ts- passed.COREPACK_ENABLE_DOWNLOAD_PROMPT=0 corepack pnpm changeset-required:check -- --base origin/trunk --head HEAD- passed, no publishable package behavior changes detected.COREPACK_ENABLE_DOWNLOAD_PROMPT=0 corepack pnpm check- passed.git diff --checkandgit diff --cached --check- passed.Self-review gates
Review and QA
Notes
LEFTHOOK=0after Lefthook invoked an interactivepnpmreinstall prompt from another worktree. The equivalent Corepack verification above passed locally.Summary by CodeRabbit