feat: @game-ci/runtime-test-framework - test the built player, not the Editor

[frostebite]

Summary

Fully implemented, not a draft anymore. game-ci test-runtime <buildPath> launches the actual built player — not the Editor, and not Unity's own Test Framework's specialized test player (a genuinely different artifact) — and reports on whatever tests its in-game harness ran, via a small results-file contract this plugin defines.

Why this is distinct from game-ci test

game-ci test's -runTests path (Docker or local) runs Unity's own official Test Framework — editmode/playmode/standalone-test-mode assemblies, executed by a specialized test player Unity's own tooling builds. It never exercises a project's real shipped build. test-runtime targets the actual player executable a build step produced instead — real assertions against real runtime behavior, on the real artifact a player would download and run.

What's implemented

• resolvePlayerExecutable — per-platform executable discovery: single .exe on Windows, single .app bundle on macOS (resolving to Contents/MacOS/<bundle name>), single executable-bit file on Linux. Errors clearly on zero or multiple candidates rather than guessing.
• launchAndCollectResults — spawns the player with GAME_CI_RUNTIME_TEST_MODE=1 and GAME_CI_RUNTIME_TEST_RESULTS_PATH set, deletes any stale results file from a previous run first, kills the process and fails loudly on timeout, and treats the results file — not the exit code — as authoritative.
• parseRuntimeTestResults / summarizeRuntimeTestResults — schema validation and pass/fail summarization for the results contract (schemaVersion: 1, { tests: [{ name, passed, durationMs?, message? }] }).
• RuntimeTestCommand — ties it together: --timeout/--resultsPath/--args options, a pass/fail summary printed to CI, fails the step on any failed test or a results file that never appeared.

Core wiring (matching the precedent from #123 exactly)

• CommandFactory: test-runtime added alongside deploy to the engine-detection bypass list — a built player carries no Unity/Godot/Unreal project markers of its own.
• CliCommands: registered test-runtime [buildPath] (no target sub-dispatch needed, unlike deploy).
• cli.ts: loaded via PluginLoader.load('@game-ci/runtime-test-framework'), never a static import — now in the default load list alongside orchestrator and steam-deploy, since it's real.

Verification

• tsc --noEmit: 737 errors, matching baseline exactly (git stash -u comparison).
• plugins/runtime-test-framework's own tsc --noEmit: clean.
• bun test ./src: 203 pass, 0 fail (was 202 before this PR), including a new integration test confirming the plugin loads and test-runtime resolves without engine detection.
• plugins/runtime-test-framework's own vitest: 17 pass, 1 skipped (the Linux executable-bit test is skipped on Windows, where NTFS doesn't represent POSIX mode bits — the logic is platform-independent; only that one assertion needs a real POSIX filesystem, which CI's Linux runners provide).
• bun run build: succeeds, new code confirmed present in the bundle.
• Real functional smoke test end-to-end: ran test-runtime against both a nonexistent path and a real empty directory, both producing exactly the expected error — option parsing, command dispatch, and executable-resolution logic all confirmed wired correctly, not just type-checked.
• oxfmt --check: clean.

Test plan

• New unit tests: player-executable resolution (Windows/.exe, macOS/.app, Linux/executable-bit — one skipped on non-POSIX CI hosts)
• New unit tests: launch/collect (results read on success, missing-results error, timeout+kill, stale-file cleanup)
• New unit tests: results parsing/validation and pass/fail summarization
• New integration test: plugin loads via PluginLoader, test-runtime resolves without engine detection
• Real end-to-end functional smoke test (see above)
• tsc --noEmit / oxfmt --check clean
• bun test ./src passes
• bun run build succeeds

[coderabbitai]

Warning

Review limit reached

Next included review available in 9 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7762bf79-481c-411e-b5f4-4546211cd376

📥 Commits

Reviewing files that changed from the base of the PR and between efc0055 and 106cbd5.

⛔ Files ignored due to path filters (1)

• bun.lock is excluded by !**/*.lock

📒 Files selected for processing (16)

• package.json
• plugins/runtime-test-framework/README.md
• plugins/runtime-test-framework/package.json
• plugins/runtime-test-framework/src/index.ts
• plugins/runtime-test-framework/src/launch-and-collect-results.test.ts
• plugins/runtime-test-framework/src/launch-and-collect-results.ts
• plugins/runtime-test-framework/src/resolve-player-executable.test.ts
• plugins/runtime-test-framework/src/resolve-player-executable.ts
• plugins/runtime-test-framework/src/runtime-test-command.ts
• plugins/runtime-test-framework/src/runtime-test-results.test.ts
• plugins/runtime-test-framework/src/runtime-test-results.ts
• plugins/runtime-test-framework/tsconfig.json
• src/cli-commands.ts
• src/cli.test.ts
• src/cli.ts
• src/command/command-factory.ts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}