feat(skills): add skill accuracy evaluation pipeline

[akiojin]

Summary

• Added a two-layer skill accuracy evaluation pipeline so skill/payload mismatches are detected before merge and routing quality can be measured continuously.
• Corrected unity-cli skill command examples to match current tool parameter contracts so LLM-generated commands execute reliably.
• Added benchmark/history artifacts and CI workflow coverage so regressions in skill accuracy can be tracked over time.

Changes

• .claude-plugin/plugins/unity-cli/skills/*/SKILL.md: fixed payload keys and command examples (for example path vs filePath, prefabPath vs assetPath, gameObjectName vs gameObjectPath) to align with implemented tool contracts.
• scripts/skill-eval/static-skill-contract-check.sh: added static validator that scans SKILL command samples and checks tool existence, required keys, and unknown keys.
• scripts/skill-eval/llm-routing-eval.sh: added routing evaluator supporting predictions file or external runner command, threshold checks, summary output, and append-only history.
• specs/perf/skill-routing-benchmark.jsonl: added 120 routing benchmark cases (including ambiguous multi-skill cases) with expected skill/tool/payload keys.
• .github/workflows/lint.yml: added required Skill Contract Check job for PR/push gating.
• .github/workflows/skill-routing-eval.yml: added daily/manual workflow for static check + optional nightly LLM routing evaluation.
• docs/development.md: documented the new CI jobs, evaluation commands, artifacts, and thresholds (English/Japanese sections).
• .gitignore: ignored generated evaluation report files (specs/perf/skill-static-report.json, specs/perf/skill-routing-summary.json).

Testing

• scripts/skill-eval/static-skill-contract-check.sh — passes with 128 commands scanned and 0 errors.
• bash -n scripts/skill-eval/static-skill-contract-check.sh — shell syntax check passes.
• bash -n scripts/skill-eval/llm-routing-eval.sh — shell syntax check passes.
• cargo fmt --all -- --check — formatting check passes.
• cargo clippy --all-targets -- -D warnings — lint check passes.

Related Issues / Links

• None

Checklist

• Tests added/updated
• Lint/format passed (cargo clippy, cargo fmt, svelte-check) — cargo clippy and cargo fmt passed; svelte-check is N/A for this repository.
• Documentation updated (if user-facing change)
• Migration/backfill plan included (if schema/data change) — N/A: no schema/data change.
• CHANGELOG impact considered (breaking change flagged in commit) — no breaking change.

Context

• Skill docs had drifted from actual tool contracts, which causes avoidable LLM misfires at execution time even when routing is correct.
• This PR standardizes skill examples against current CLI/Unity handler contracts and adds automated contract checks to keep them aligned.
• The new benchmark/eval flow provides measurable routing quality (top1, top2, tool_correct, payload_valid) with append-only history for trend tracking.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feature/skill-accuracy-enhancement

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}