chore(evals): author skill evals for docs-hygiene (5 skills)

[kyle-sexton]

What

Adds rich-form evals/evals.json for all five docs-hygiene skills and bumps the plugin's plugin.json version so marketplace consumers pick up the new shipped eval component.

Skill	Cases	Focus
compress	6	default-vs-audit routing, content preserved over flavor, separate semantic-diff pass, sub-3% revert, refuse code file, SSOT-relocation-not-flavor
declutter	6	shape/tier classification, read-only when asked to fix, slot-variable not ghost-ref, exempt sections, flavor is /compress, Diataxis preamble
encapsulation-audit	6	detect routing, single-violation not gated by Rule of Three, filter-taxonomy legal hits, Path B missing-action files+stops, scripts/ entry-surface legal, dedup routes to /extract-ssot
extract-ssot	6	identify on duplication smell, refuse below Rule of Three, Tier 0 over recall, code cluster out of scope, DESCRIBE/USE/EXPOSE, one-level-deep citations
rename-references	6	audit sweep, never trust single-pattern grep, English-verb ambiguous confirm, re-sweep until zero, refuse framework migration, plan-doc auto-excluded

Two markdown fixtures back the compress and declutter happy-path cases. Each case carries id, kebab-case name, prompt, expected_output, optional files, and an expectations array of objectively-verifiable checks — modeled on the bug-report rich-form exemplar.

Warrant

All five skills carry judgment-bearing behavioral contracts (action routing + guardrail/refusal rules + guarded anti-patterns) — none is pure-reference, so no skip verdicts.

Decision called out

Version-bump level was unspecified by the issue. docs-hygiene is a single multi-skill plugin, so one manifest bump covers all five: chose a minor bump (0.1.0 -> 0.2.0) — evals are an additive, backward-compatible shipped component. Matches the sibling evals PR precedent; reversible to patch if a maintainer prefers.

Validation

• check-jsonschema against the bundled plugins/skill-quality/reference/evals.schema.json — all 5 conform
• check-skill.sh presence gate — all 5 PASS, 0 errors (remaining WARNs are pre-existing skill-body Gotchas-surface items, out of scope for an evals-only PR)
• scripts/validate-plugins.sh (manifests + catalog, --strict) — pass
• LF endings, final newline, typos clean

Doc drift noted

The issue cites docs/MIGRATION-PLAYBOOK.md "Evals — warrant policy and consumer-verify recipe" and docs/evals-coverage.md; neither exists on current main (they live on the unmerged evals-backfill branch). Did not block this work; the coverage doc is now stale re docs-hygiene and will need reconciling when that branch lands.

Refs melodic-software/medley#1450

---

Note

Low Risk
Additive eval JSON and fixtures only; no skill runtime or application logic changes. Version bump is backward-compatible packaging.

Overview
Ships rich-form evals/evals.json for all five docs-hygiene skills (compress, declutter, encapsulation-audit, extract-ssot, rename-references), with six cases each covering action routing, guardrails, refusals, and sibling-skill boundaries. Cases use prompt, expected_output, and verifiable expectations (plus optional files).

Adds markdown fixtures for the compress and declutter fixture-driven cases (verbose-onboarding-snippet.md, noisy-rule-snippet.md).

Bumps docs-hygiene plugin manifest version 0.1.0 → 0.2.0 so marketplace installs pick up the new eval component.

^{Reviewed by Cursor Bugbot for commit ef0ab0b. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_d6a1f196-7ea7-4b3d-96d8-e10de25b304c)

[claude]

Claude finished @kyle-sexton's task in 3m 24s —— View job

---

PR Review: chore(evals): author skill evals for docs-hygiene (5 skills)

Tasks

• Gather context and understand the request
• Read changed files and existing evals exemplar for comparison
• Review eval JSON structure and content correctness
• Check fixture files
• Check plugin.json version bump
• Post final review

---

Summary

Well-structured eval set with strong behavioral grounding. All 30 cases are schema-compliant and traced to specific hard rules in the corresponding SKILL.md files. A few minor gaps worth addressing, none of which block merge.

---

Schema / Structure

All five evals.json files conform to the evals.schema.json rich form — id (integer), name (kebab-case), prompt, expected_output, files, expectations. additionalProperties: false is satisfied; no unknown fields present. Consistent with the bug-report exemplar pattern.

---

Behavioral Contract Fidelity

Each case checked against the corresponding SKILL.md:

compress (6/6) — All cases trace cleanly:

• Case 1 (default-compress-preserves-content): fixture contains the exact words the expectations test for (MAX_UPLOAD_MB=25, "warnings are treated as errors", filler terms). ✓
• Case 3 (semantic-diff-is-separate-not-self-audit): anchored in the SKILL.md Gotcha: "The semantic-diff dispatch must run as a SEPARATE fresh-context audit, never a self-audit." ✓
• Case 6 (ssot-relocation-not-flavor): anchored in the SKILL.md "What this skill is NOT" paragraph on content relocation vs. flavor removal. ✓

declutter (6/6) — Cases accurate, one gap noted below.

encapsulation-audit (6/6) — Case 3 is particularly apt: the plugin-cache hit uses ~/.claude/plugins/cache/docs-hygiene/... — self-referential but correctly classified as legal. ✓

extract-ssot (6/6) — All cases map to documented hard rules. Case 5 (describe-use-expose-classification) correctly resolves the README restatement as EXPOSE (may restate with a cross-reference) vs. the rule body as USE (must cite). ✓

rename-references (6/6) — Case 6 (plan-doc-auto-excluded) matches the SKILL.md auto-exclusion rule AND the override-flag caveat: "apply mode rejects --include-plan-docs with explicit error." ✓

---

Findings

1. declutter case 1: fixture preamble not covered in expected_output (minor)

declutter/evals/evals.json L8–16

The fixture noisy-rule-snippet.md contains a ## Why this file exists preamble. The expected_output lists four findings (citation, ghost-ref, scope-meta, enum-list) but never mentions the preamble. Meanwhile, expectation #1 asserts output uses the shape vocabulary (citation, ghost-ref, **preamble**, enum-list, scope-meta) — implying preamble appears in the output. This creates a tension: the expected_output doesn't include a preamble finding, but expectation #1 implies the vocabulary (including preamble) should appear.

The preamble on this Explanation-quadrant file (a retry policy) would likely be Tier 2 KEEP — but it's still a reportable finding. A grader comparing the expected_output against a real run could be confused by the omission.

Suggested fix: either add the preamble finding to expected_output, or narrow expectation #1 to (citation, ghost-ref, enum-list, scope-meta) if the intent was to test only the four shapes the fixture actually triggers. Fix this →

---

2. compress case 4: CLAUDE.md as a concrete path in the prompt (informational)

compress/evals/evals.json L43–51

The prompt says "Compress CLAUDE.md". This tests the behavioral route (sub-3% revert) without a files entry pointing to an actual fixture. The eval grader would simulate the scenario, but a literal runner wouldn't find a predictably-tight CLAUDE.md to exercise the revert path. This is fine for behavioral testing but worth noting — the eval is testing routing logic, not file content. Consider an expected_output note clarifying this is a scenario-based (not fixture-based) case, or adding a minimal fixture that represents a tight instruction file. Low priority since the current structure matches how several other evals in the repo handle scenario-only cases.

---

3. encapsulation-audit case 1: action name misalignment in name field (micro-nit)

encapsulation-audit/evals/evals.json L6

The name is detect-routes-and-tables-violations but the case tests that raw grep hits are classified via the filter taxonomy (not all routed as violations). The name suggests "routes" and "tables violations" when the point of the case is "detect → classify → violations table + filtered-legal table". A clearer name would be detect-classifies-via-filter-taxonomy or just detect-and-classify. Ultra-minor; doesn't affect eval correctness.

---

Fixtures

Both fixtures are correct:

• verbose-onboarding-snippet.md: Contains every term the expectations test — basically, really, just, honestly, perhaps, MAX_UPLOAD_MB=25, "25 MB", "warnings are treated as errors", "block the merge". ✓
• noisy-rule-snippet.md: Contains all four expected noise items — dated citation, ghost-ref (.work/net-hardening-slice/PLAN.md), scope-meta, enum-list — plus an exempt ## Cross-references section. ✓

---

Version Bump

plugin.json: 0.1.0 → 0.2.0 (minor). Appropriate for an additive, backward-compatible shipped component. Consistent with sibling PR precedent.

---

Verdict

Approve with minor note. Finding #1 (preamble gap in declutter case 1) is the only item worth addressing before this is used as a grading fixture — it creates an observable contradiction between expected_output and expectation #1. Findings #2 and #3 are informational and don't affect correctness.

[cursor]

Risk: low. Did not approve: Cursor Bugbot completed with status skipping (usage limit reached) and did not produce a clean review signal. Human review was needed, but this PR is already merged; no reviewers assigned post-merge.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}