fix: keep filtered smoke evidence consistent

[kang-heewon]

Outcome

Dependency-aware filtered generated-app smoke runs no longer fail after all selected cases pass merely because they omit the canonical full-spine journey bundle. Required matrix evidence remains blocking for filtered runs.

Full spine and scaffold runs continue to require and validate the canonical journey bundle. When journey validation fails, the diagnostic is attached to the journey artifact instead of an unrelated required matrix.

Fixes #1768

Verification

• pnpm exec vitest run scripts/tests/verification-manifest.spec.ts scripts/tests/release-spine-evidence.spec.ts — 63 passed
• affected Oxlint, Oxfmt, and git diff --check — passed
• pnpm changeset-required:check -- --base origin/trunk --head HEAD — no publishable package behavior changes
• pre-push workspace test — 234/234 tasks passed
• pre-push workspace typecheck — 233/233 tasks passed

Review gates

• Correctness and regression — PASS. Actual filtered and full manifest commands prove optional journey omission passes, required journey omission fails, required matrices remain blocking, and journey validation failures stay attached to the journey artifact.
• API, security, compatibility, and release — PASS. No package API, manifest, dependency, lockfile, secret, or changeset surface changed; full-spine evidence enforcement remains intact.
• Maintainability and minimality — PASS. One manifest requirement is selection-aware, and the collector validates only required or actually copied journey evidence.
• Independent adversarial review — PASS after fix. Review reproduced a mixed required/optional artifact failure; collector targeting and the exact manifest regression were corrected and re-reviewed with no remaining findings.

Residual risk

None identified within the issue scope.

Summary by CodeRabbit

• 버그 수정

  • 생성 앱 스모크 검증에서 선택적 여정 번들이 없어도 필수 산출물만으로 검증을 통과하도록 개선했습니다.
  • 실행 프로필에 따라 스모크 검증 산출물의 필수 여부를 정확히 적용합니다.
  • 번들 검증 실패가 해당 여정 참조에 올바르게 기록됩니다.

• 테스트

  • 필수·선택 산출물 조합과 다양한 스모크 실행 프로필에 대한 검증 범위를 확대했습니다.

[coderabbitai]

Warning

Review limit reached

@kang-heewon, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 58 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b9a68284-df80-4042-ba75-62e93fb2064c

📥 Commits

Reviewing files that changed from the base of the PR and between d125a6b and fd04368.

📒 Files selected for processing (2)

• scripts/tests/release-spine-evidence.spec.ts
• scripts/tests/verification-manifest.spec.ts

📝 Walkthrough

Walkthrough

생성 앱 스모크의 spine-blocking-journeys 아티팩트를 전체 실행에서만 필수로 변경했습니다. 필터링된 실행에서는 해당 아티팩트가 없어도 증거 검증이 통과합니다. 번들 검증은 필수이거나 복사된 참조에만 수행합니다.

Changes

생성 앱 스모크 증거

Layer / File(s)	Summary
여정 번들 필수 여부 계약 scripts/verification-manifest.mts, scripts/tests/verification-manifest.spec.ts	spine-blocking-journeys는 전체 스모크 티어에서 필수이고, 선택된 개별 스모크 실행에서는 선택 항목입니다. spine-blocking-matrix는 두 실행 유형에서 필수입니다.
조건부 여정 번들 검증 scripts/release-spine-evidence.mts, scripts/tests/release-spine-evidence.spec.ts	검증 대상은 spine-blocking-journeys 디렉터리 또는 report.json 참조입니다. 필수이거나 복사된 경우에만 번들을 검증합니다. 필터링된 실행에서 선택적 번들이 없어도 통과하는 테스트를 추가했습니다.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SmokeRun as 생성 앱 스모크 실행
  participant Manifest as verification-manifest.mts
  participant Evidence as release-spine-evidence.mts
  participant Journey as spine-blocking-journeys
  SmokeRun->>Manifest: 실행 프로필 전달
  Manifest-->>SmokeRun: 아티팩트 필수 여부 반환
  SmokeRun->>Evidence: 생성된 증거 전달
  Evidence->>Journey: 여정 번들 참조 탐색
  Evidence->>Journey: 조건부 번들 검증
  Journey-->>Evidence: 검증 결과 또는 copyError

Loading

Possibly related PRs

• croco-dev/framework#1411: spine-blocking-journeys를 조건부 필수로 처리하는 검증 규칙과 직접 연결됩니다.
• croco-dev/framework#1671: release-spine-evidence.mts와 관련 테스트의 증거 검증 동작을 직접 수정합니다.
• croco-dev/framework#1566: 생성 앱 브라우저 여정 번들의 증거 처리와 직접 연결됩니다.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	제목은 필터링된 smoke 증거의 일관성 수정이라는 주요 변경 사항을 간결하고 명확하게 설명합니다.
Linked Issues check	✅ Passed	변경 사항은 필터링 및 전체 실행의 journey bundle 요구 조건과 matrix 증거 요구 조건을 모두 충족합니다 [#1768].
Out of Scope Changes check	✅ Passed	모든 코드와 테스트 변경은 필터링된 smoke 증거 처리 및 관련 검증 계약이라는 이슈 범위에 포함됩니다.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch issue-1768-filtered-smoke-evidence

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

📊 Benchmark Results

✅ All benchmarks passed

Benchmark	p75	Threshold	Baseline	vs Baseline	Status	Notes
CrocoApp constructor	22.1μs	30.0ms	8.2μs	+170.5%	❌	-
CrocoApp lambdaHandler (10 controllers)	863.0μs	50.0ms	258.4μs	+233.9%	❌	-
Lambda cold-start simulation	913.7μs	80.0ms	418.1μs	+118.5%	❌	-
Lambda cold-start with headers	866.5μs	80.0ms	369.7μs	+134.4%	❌	-
Lambda cold-start with binary body	820.8μs	80.0ms	339.1μs	+142.0%	❌	-
Lambda cold-start with query params	775.1μs	80.0ms	301.3μs	+157.2%	❌	-
Lambda cold-start with authorizer context	765.4μs	80.0ms	299.8μs	+155.3%	❌	-
Lambda cold-start realistic scenario	731.0μs	80.0ms	299.2μs	+144.3%	❌	-
EventBusConfig.start (10 handlers)	1.1μs	10.0ms	1.4μs	-23.9%	✅	-
EventPublisher.publishNow single event	1.3μs	2.0ms	1.7μs	-24.4%	✅	-
DefaultHandlerResolver.resolve × 10	0.1μs	5.0ms	0.1μs	-37.5%	✅	-
Container.get singleton (cold)	34.8μs	5.0ms	70.3μs	-50.5%	✅	-
Container.register × 50 components	1.8ms	10.0ms	3.2ms	-44.8%	✅	-
Container.validate (50 components)	2.1ms	20.0ms	3.4ms	-39.4%	✅	-
Container.get singleton (warm)	1.0μs	500.0μs	1.6μs	-41.1%	✅	-
TelemetryRuntime.init (lambda preset)	1.2μs	200.0ms	1.1ms	-99.9%	✅	-
lambdaPreset config creation	0.7μs	2.0ms	1.4μs	-49.3%	✅	-

_{Updated: 2026-08-07T02:55:41.335Z · Commit: 20fdfd6}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/tests/release-spine-evidence.spec.ts`:
- Around line 264-267: Update the assertion in the report checks test to select
the optional journey bundle artifact by its sourcePath before matching exists
and required. Do not use find(({ required }) => !required), so changes to other
optional artifacts cannot satisfy the assertion.

In `@scripts/tests/verification-manifest.spec.ts`:
- Around line 363-368: Update the assertion around generatedSmoke.artifacts to
first verify that exactly two spine-blocking-matrix artifacts are present, then
assert that every matched artifact has required set to true. Preserve the
existing packagePath context and avoid relying on every alone, since it passes
for an empty match.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b2e88ddb-e09f-4c46-ae61-7e4b8e241c23

📥 Commits

Reviewing files that changed from the base of the PR and between cfe0d14 and d125a6b.

📒 Files selected for processing (4)

• scripts/release-spine-evidence.mts
• scripts/tests/release-spine-evidence.spec.ts
• scripts/tests/verification-manifest.spec.ts
• scripts/verification-manifest.mts

[kang-heewon]

@coderabbitai resolve

[coderabbitai]

✅ Action performed

Comments resolved and changes approved.