Skip to content

feat: generate real-browser frontend test defaults - #1566

Merged
kang-heewon merged 5 commits into
trunkfrom
issue-1485-frontend-testing
Jul 27, 2026
Merged

feat: generate real-browser frontend test defaults#1566
kang-heewon merged 5 commits into
trunkfrom
issue-1485-frontend-testing

Conversation

@kang-heewon

@kang-heewon kang-heewon commented Jul 26, 2026

Copy link
Copy Markdown
Member

Outcome

  • Production and admin application starters now generate real-Chromium Vitest component tests, fail-closed MSW API fixtures, and Playwright success and Problem journeys.
  • Generated browser CI runs two journey shards, retains failure evidence, merges blob reports, and uses immutable action revisions.
  • Generated-app smoke executes the supported browser configuration for both frontend-bearing presets, while the browser tooling remains independent of @croco/testing core APIs.

Verification

  • pnpm test --filter=create-croco-app
  • pnpm typecheck --filter=create-croco-app
  • pnpm check
  • pnpm create-croco-app:smoke -- production-app-starter
  • pnpm create-croco-app:smoke -- admin-console-starter
  • pnpm changeset-required:check -- --base origin/trunk --head HEAD
  • pre-push full test and typecheck hooks

Review gates

  • Acceptance: every issue criterion is covered by generated contracts and executable browser evidence.
  • Structure: browser tooling stays inside generated presentation templates and does not change @croco/testing public APIs.
  • Quality: fail-closed requests report method and URL, failure artifacts cover trace, screenshot, console, and network evidence, and CI actions are SHA-pinned.
  • Adversarial review: admin journey execution, immutable actions, and explicit MSW escape evidence were challenged, corrected, and re-reviewed with no remaining actionable findings.

Residual risk

Browser dependency and action revisions are intentionally pinned and will require explicit upgrades as upstream releases change.

Fixes #1485

Summary by CodeRabbit

  • 새로운 기능

    • 생성되는 React 앱에 Chromium 기반 컴포넌트 테스트와 브라우저 여정 테스트를 추가했습니다.
    • 브라우저 테스트용 CI 실행, 샤딩, 통합 리포트 및 실패 아티팩트를 지원합니다.
    • API 문제 정보를 코드·상태·상세·복구 안내와 함께 표시하고 재시도할 수 있습니다.
    • MSW 기반의 엄격한 API 모킹과 기본 API 프록시 설정을 제공합니다.
  • 문서

    • 생성된 앱의 브라우저 테스트 설치·실행·검증 절차를 README에 반영했습니다.
  • 버그 수정

    • GitHub Actions 표현식이 템플릿 렌더링 중 손상되지 않도록 개선했습니다.

@coderabbitai

coderabbitai Bot commented Jul 26, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@kang-heewon, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 20 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 7ed2da37-effb-4a1f-b8d5-2aed5233baf0

📥 Commits

Reviewing files that changed from the base of the PR and between 614d2d1 and 9512e3a.

⛔ Files ignored due to path filters (1)
  • packages/problems-core/src/generated/problem-code-registry.ts is excluded by !**/generated/**
📒 Files selected for processing (33)
  • .changeset/brave-browsers-test.md
  • .oxfmtignore
  • docs/problem-code-registry.json
  • packages/create-croco-app/src/helpers/fs.ts
  • packages/create-croco-app/src/tests/e2e-generation.spec.ts
  • packages/create-croco-app/src/tests/templates-build.spec.ts
  • packages/create-croco-app/templates/admin-console/README.md.hbs
  • packages/create-croco-app/templates/admin-console/apps/console-web/package.json.hbs
  • packages/create-croco-app/templates/admin-console/package.json.hbs
  • packages/create-croco-app/templates/blank/.gitignore
  • packages/create-croco-app/templates/spa-be-split/.github/workflows/browser-tests.yml
  • packages/create-croco-app/templates/spa-be-split/README.md.hbs
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/package.json.hbs
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/public/mockServiceWorker.js
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/App.tsx.hbs
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/ProblemNotice.tsx
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/api/client.ts
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/api/useUsers.ts
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/test/browser.ts
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/test/server.ts
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/tests/ProblemNotice.spec.tsx
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/tsconfig.json.hbs
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/vite.config.ts.hbs
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/vitest.config.ts
  • packages/create-croco-app/templates/spa-be-split/package.json.hbs
  • packages/create-croco-app/templates/spa-be-split/playwright.config.ts
  • packages/create-croco-app/templates/spa-be-split/pnpm-workspace.yaml.hbs
  • packages/create-croco-app/templates/spa-be-split/tests/journeys/browser-evidence.ts
  • packages/create-croco-app/templates/spa-be-split/tests/journeys/create-user.spec.ts
  • packages/create-croco-app/templates/spa-be-split/tests/journeys/problem-rendering.spec.ts
  • packages/docs/src/content/docs/api/problems-core/src/variables/CROCO_PROBLEM_CODE_REGISTRY.md
  • packages/docs/src/content/docs/en/reference/problem-recovery-cookbook.md
  • scripts/create-croco-app-generated-smoke.mts
📝 Walkthrough

Walkthrough

생성 앱에 Chromium 기반 Vitest 컴포넌트 테스트, fail-closed MSW 픽스처, Playwright 사용자 여정, sharded 브라우저 CI 리포트, Problem 표시 UI와 생성 결과 검증이 추가되었습니다.

Changes

생성 앱 브라우저 테스트 기본값

Layer / File(s) Summary
컴포넌트 테스트와 Problem 계약
packages/create-croco-app/templates/spa-be-split/apps/console-web/...
Vitest Browser Mode, MSW 테스트 픽스처, 구조화된 FrontendProblem, ProblemNotice, API 경로 및 컴포넌트 테스트를 추가합니다.
MSW 서비스 워커와 빌드 경계
packages/create-croco-app/templates/spa-be-split/apps/console-web/public/mockServiceWorker.js, packages/create-croco-app/templates/spa-be-split/pnpm-workspace.yaml.hbs, .oxfmtignore
서비스 워커의 요청 직렬화·클라이언트 통신·모킹 응답 처리를 추가하고 MSW 빌드 및 포맷 예외를 설정합니다.
Playwright 여정과 브라우저 CI
packages/create-croco-app/templates/spa-be-split/playwright.config.ts, tests/journeys/*, .github/workflows/browser-tests.yml, README.md.hbs
사용자 생성과 API Problem 여정을 구성하고, 브라우저 오류 증거·shard 리포트·HTML 리포트 병합을 CI에 연결합니다.
관리 콘솔 테스트 통합
packages/create-croco-app/templates/admin-console/...
관리 콘솔 preset에 브라우저 설치·컴포넌트·여정·CI 스크립트와 관련 의존성을 추가하고 문서를 갱신합니다.
생성 결과와 렌더링 계약 검증
packages/create-croco-app/src/..., scripts/create-croco-app-generated-smoke.mts, .changeset/brave-browsers-test.md
GitHub 표현식을 Handlebars 처리에서 보존하고, 생성 파일·스크립트·워크플로우·MSW 설정·Chromium 설치·여정 실행을 검증합니다.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

Possibly related PRs

  • croco-dev/framework#838 — 생성 smoke 검증에 특정 실행 계약을 추가하는 점에서 브라우저 CI·여정 계약 검증과 코드 수준으로 연결됩니다.

Sequence Diagram(s)

sequenceDiagram
  participant GeneratedApp
  participant MSW
  participant Vitest
  participant Playwright
  participant BrowserCI
  GeneratedApp->>MSW: API 요청 전달
  MSW-->>Vitest: Problem 픽스처 또는 미처리 요청 오류
  Vitest-->>GeneratedApp: 컴포넌트 렌더링 검증
  Playwright->>GeneratedApp: 사용자 여정 실행
  BrowserCI->>Playwright: Chromium shard 실행
  Playwright-->>BrowserCI: 브라우저 증거와 blob 리포트 업로드
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 28.57% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목이 실제 변경 사항인 실브라우저 프런트엔드 테스트 기본 생성과 정확히 맞습니다.
Linked Issues check ✅ Passed #1485의 핵심 요구사항인 Chromium 기반 컴포넌트 테스트, MSW, Playwright 여정, CI 아티팩트, 스모크 검증이 모두 반영되었습니다.
Out of Scope Changes check ✅ Passed 변경들은 템플릿, 테스트, 워크플로우, 문서 범위에 한정되어 있어 요구사항과 무관한 확장으로 보이지 않습니다.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-1485-frontend-testing

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 26, 2026

Copy link
Copy Markdown

📊 Benchmark Results

✅ All benchmarks passed

Benchmark p75 Threshold Baseline vs Baseline Status Notes
CrocoApp constructor 10.1μs 30.0ms 8.2μs +23.7% -
CrocoApp lambdaHandler (10 controllers) 313.8μs 50.0ms 258.4μs +21.4% -
Lambda cold-start simulation 425.3μs 80.0ms 418.1μs +1.7% -
Lambda cold-start with headers 378.6μs 80.0ms 369.7μs +2.4% -
Lambda cold-start with binary body 418.3μs 80.0ms 339.1μs +23.4% -
Lambda cold-start with query params 296.2μs 80.0ms 301.3μs -1.7% -
Lambda cold-start with authorizer context 291.9μs 80.0ms 299.8μs -2.6% -
Lambda cold-start realistic scenario 287.2μs 80.0ms 299.2μs -4.0% -
EventBusConfig.start (10 handlers) 1.9μs 10.0ms 1.4μs +30.0% -
EventPublisher.publishNow single event 2.1μs 2.0ms 1.7μs +26.7% -
DefaultHandlerResolver.resolve × 10 0.1μs 5.0ms 0.1μs +0.0% -
Container.get singleton (cold) 62.1μs 5.0ms 70.3μs -11.7% -
Container.register × 50 components 3.2ms 10.0ms 3.2ms -1.4% -
Container.validate (50 components) 3.7ms 20.0ms 3.4ms +9.2% -
Container.get singleton (warm) 1.6μs 500.0μs 1.6μs -1.9% -
TelemetryRuntime.init (lambda preset) 2.3μs 200.0ms 1.1ms -99.8% -
lambdaPreset config creation 1.4μs 2.0ms 1.4μs -2.1% -

Updated: 2026-07-27T11:02:00.390Z · Commit: c538543

@kang-heewon
kang-heewon force-pushed the issue-1485-frontend-testing branch from 1016894 to 614d2d1 Compare July 27, 2026 07:39

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/create-croco-app/src/helpers/fs.ts`:
- Around line 55-68: Replace the regex-based GitHub expression protection in
packages/create-croco-app/src/helpers/fs.ts#L55-L68 with a shared stateful
parser that recognizes quoted strings and locates the actual closing delimiter,
then use that same boundarying helper during rendering and in
packages/create-croco-app/src/tests/e2e-generation.spec.ts#L81-L86. Ensure
validation checks only leftover Handlebars placeholders, while valid GitHub
expressions containing }} inside quoted strings remain protected and restored
correctly.

In `@packages/create-croco-app/src/tests/e2e-generation.spec.ts`:
- Around line 334-340: Update assertBrowserWorkflowUsesImmutableActions to
extract actual uses: lines from workflow and validate every remote action
reference ends with exactly a 40-character hexadecimal commit SHA. Do not rely
on searching expected SHA strings or only rejecting version tags; exclude local
or Docker references as appropriate while rejecting branches, tags, and other
mutable refs.

In
`@packages/create-croco-app/templates/spa-be-split/apps/console-web/src/ProblemNotice.component.spec.tsx`:
- Line 4: Update the imports in ProblemNotice.component.spec.tsx to keep the
runtime ProblemNotice import separate from the type-only FrontendProblem import,
using a standalone import type declaration for FrontendProblem.

In
`@packages/create-croco-app/templates/spa-be-split/apps/console-web/src/test/browser.ts`:
- Around line 8-12: Update the onUnhandledRequest handler to throw a dedicated
Problem subclass such as UnhandledApiRequestProblem instead of a generic Error.
Define or reuse the project-standard Problem subtype with appropriate code and
category fields, while preserving the existing fail-closed message containing
the request method and URL.

In
`@packages/create-croco-app/templates/spa-be-split/apps/console-web/src/test/server.ts`:
- Around line 4-11: Rename the immutable problem fixture to
SCREAMING_SNAKE_CASE: update problemFixture to PROBLEM_FIXTURE in
packages/create-croco-app/templates/spa-be-split/apps/console-web/src/test/server.ts
(lines 4-11), and update its import and usages in
packages/create-croco-app/templates/spa-be-split/apps/console-web/src/ProblemNotice.component.spec.tsx
(line 7). Rename the problem fixture to PROBLEM in
packages/create-croco-app/templates/spa-be-split/tests/journeys/problem-rendering.spec.ts
(lines 3-10) and update all references there.

In
`@packages/create-croco-app/templates/spa-be-split/apps/console-web/vitest.config.ts`:
- Around line 8-9: Align the Vitest test discovery and template test location:
update the include pattern in vitest.config.ts to target src/tests/**/*.spec.ts,
and move/rename the ProblemNotice component spec from
src/ProblemNotice.component.spec.tsx to src/tests/ProblemNotice.spec.ts while
preserving its test behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: fd723abc-0ff8-473f-ba98-3e4b38ae53fb

📥 Commits

Reviewing files that changed from the base of the PR and between 3d9e585 and 614d2d1.

📒 Files selected for processing (30)
  • .changeset/brave-browsers-test.md
  • .oxfmtignore
  • packages/create-croco-app/src/helpers/fs.ts
  • packages/create-croco-app/src/tests/e2e-generation.spec.ts
  • packages/create-croco-app/src/tests/templates-build.spec.ts
  • packages/create-croco-app/templates/admin-console/README.md.hbs
  • packages/create-croco-app/templates/admin-console/apps/console-web/package.json.hbs
  • packages/create-croco-app/templates/admin-console/package.json.hbs
  • packages/create-croco-app/templates/blank/.gitignore
  • packages/create-croco-app/templates/spa-be-split/.github/workflows/browser-tests.yml
  • packages/create-croco-app/templates/spa-be-split/README.md.hbs
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/package.json.hbs
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/public/mockServiceWorker.js
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/App.tsx.hbs
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/ProblemNotice.component.spec.tsx
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/ProblemNotice.tsx
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/api/client.ts
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/api/useUsers.ts
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/test/browser.ts
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/src/test/server.ts
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/tsconfig.json.hbs
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/vite.config.ts.hbs
  • packages/create-croco-app/templates/spa-be-split/apps/console-web/vitest.config.ts
  • packages/create-croco-app/templates/spa-be-split/package.json.hbs
  • packages/create-croco-app/templates/spa-be-split/playwright.config.ts
  • packages/create-croco-app/templates/spa-be-split/pnpm-workspace.yaml.hbs
  • packages/create-croco-app/templates/spa-be-split/tests/journeys/browser-evidence.ts
  • packages/create-croco-app/templates/spa-be-split/tests/journeys/create-user.spec.ts
  • packages/create-croco-app/templates/spa-be-split/tests/journeys/problem-rendering.spec.ts
  • scripts/create-croco-app-generated-smoke.mts

Comment thread packages/create-croco-app/src/helpers/fs.ts
Comment thread packages/create-croco-app/src/tests/e2e-generation.spec.ts Outdated
@kang-heewon

Copy link
Copy Markdown
Member Author

@coderabbitai resolve

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
✅ Action performed

Comments resolved and changes approved.

@kang-heewon
kang-heewon force-pushed the issue-1485-frontend-testing branch from cc215a8 to 3cdca5a Compare July 27, 2026 08:57
@kang-heewon
kang-heewon merged commit b07ae3a into trunk Jul 27, 2026
10 checks passed
@kang-heewon
kang-heewon deleted the issue-1485-frontend-testing branch July 27, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[testing] Generate frontend component and browser journey testing defaults

1 participant