Skip to content

chore(implementation): consume ecosystem-commands contract — ladder resolution, setup, v0.2.0#100

Merged
kyle-sexton merged 3 commits into
mainfrom
chore/retrofit-implementation
Jul 12, 2026
Merged

chore(implementation): consume ecosystem-commands contract — ladder resolution, setup, v0.2.0#100
kyle-sexton merged 3 commits into
mainfrom
chore/retrofit-implementation

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Retrofit the implementation plugin to consume the ecosystem-commands contract (docs/conventions/ecosystem-commands/README.md, ecosystem.schema.json).

What changed

  • One bundled default set. The two divergent reference tables (skills/build/reference/ecosystem-config.md, skills/lint/reference/ecosystem-config.md) are replaced by schema-conformant reference/ecosystems/<eco>.yaml (8 files, validated against the contract schema). Unified command keys: build-cmd / test-cmd / check-cmd / fix-cmd — the old build-table lint-cmd and lint-table check-cmd collapse to check-cmd.
  • Ladder resolution in /build and /lint via a shared reference/resolution-ladder.md: consumer .claude/ecosystems/<eco>.yaml (additive over ~/.claude/ecosystems/ user-global + .local.yaml overlay) → inference → ask → bundled fallback. Malformed consumer file warns + degrades to inference, never hard-stops.
  • /implementation:setup — re-runnable ladder writer (interview + infer + write .claude/ecosystems/*.yaml); codebase-audit skills/setup is the shape precedent.
  • Delivery: plugin.json → 0.2.0; new CHANGELOG.md; README documents the seam and cites the contract by path.

Decisions (pre-baked from the design gate; recorded, not reopened)

  • Data unified, scope preserved. Unifying into one 8-ecosystem set would pull the lint-only yaml and cross-cutting surfaces into /build — and cross-cutting's ** glob matches every change. Per the contract's canonical-verb-vs-context-binding split, the data unifies while each skill keeps its scope: /build covers dotnet/python/typescript/bash/powershell/markdown; yaml + cross-cutting stay /lint-only.
  • Reconciled divergence: dropped the lint-table's $REPO_ROOT/-prefixed dotnet command for the contract example's <solution-or-project-file> form (the running skill resolves absolute paths).
  • Config home concern-named .claude/ecosystems/; bundled defaults rung-4 only; no new userConfig knob; task-runner deferred (opaque command strings).

Verification

  • check-jsonschema against ecosystem.schema.json: all 8 files ok.
  • claude plugin validate ./plugins/implementation and claude plugin validate --strict . (catalog): both pass.
  • --plugin-dir smoke in a clean non-source repo (only .git): all 11 skills load, incl. new implementation:setup.
  • markdownlint-cli2 on changed markdown: 0 errors.

Gate gap (documented per non-interactive policy): deep ${CLAUDE_PLUGIN_ROOT} runtime asset-read could not be exercised headlessly — claude -p sandboxes reads to CWD and does not export the var to a raw tool call. The ${CLAUDE_PLUGIN_ROOT}/reference/... idiom matches the established codebase-audit setup precedent; plugin validate and clean-repo skill discovery pass.

Refs melodic-software/medley#1420


Note

Medium Risk
Changes how every repo resolves build/test/lint commands for /build and /lint; repos without .claude/ecosystems/ may see different defaults or inference paths than the old duplicated tables, though malformed config degrades softly.

Overview
Bumps the implementation plugin to 0.2.0 and replaces the split build/lint markdown command tables with a single ecosystem-commands contract: eight schema-conformant reference/ecosystems/*.yaml defaults plus shared reference/resolution-ladder.md.

/implementation:build and /implementation:lint now resolve each ecosystem through a four-rung ladder (consumer .claude/ecosystems/*.yaml with user-global and .local.yaml overlays → infer → ask → bundled defaults), use unified keys build-cmd / test-cmd / check-cmd / fix-cmd, and treat malformed consumer YAML as a warn-and-infer fallback. /build still scopes to six ecosystems; yaml and cross-cutting stay lint-only so cross-cutting's ** glob does not affect build detection.

Adds re-runnable /implementation:setup to interview, infer, and write tracked .claude/ecosystems/*.yaml. README and new CHANGELOG document the seam; plugin.json gains a setup keyword.

Reviewed by Cursor Bugbot for commit 0588f75. Bugbot is set up for automated code reviews on this repo. Configure here.

Replace the two divergent per-ecosystem reference tables with one bundled,
schema-conformant set of portable defaults at reference/ecosystems/*.yaml, and
resolve each ecosystem's build/test/lint commands through the contract's
four-rung ladder (consumer .claude/ecosystems/<eco>.yaml authoritative →
inference → ask → bundled fallback). Collapse the old build-table lint-cmd and
lint-table check-cmd into the contract's check-cmd.

Add /implementation:setup — the ladder's re-runnable writer that interviews,
infers, and writes the consuming repo's tracked .claude/ecosystems/*.yaml.

Data is unified while each skill keeps its scope: /build covers
dotnet/python/typescript/bash/powershell/markdown; the lint-only yaml and
cross-cutting surfaces (cross-cutting's ** glob) stay /lint-only, per the
contract's canonical-verb-vs-context-binding split.

Bump plugin.json to 0.2.0; add CHANGELOG.md; document the seam in README.

Refs melodic-software/medley#1420

Co-Authored-By: Claude <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_aa01b2ae-ab25-45d4-8895-9a7b1024f285)

@claude

claude Bot commented Jul 12, 2026

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 4m 10s —— View job


PR Review: chore(implementation): consume ecosystem-commands contract — ladder resolution, setup, v0.2.0

  • Gather context and understand the request
  • Read changed files and analyze the PR
  • Review plugin.json and CHANGELOG.md
  • Review new ecosystem YAML files
  • Review resolution-ladder.md
  • Review updated SKILL.md files (build + lint)
  • Review new setup/SKILL.md
  • Check alignment with CLAUDE.md guidelines
  • Post review findings

Summary

The architecture is sound. Unifying the two divergent reference tables into 8 schema-conformant YAML files, sharing one ladder document between /build and /lint, and preserving scope-by-surface (yaml/cross-cutting staying /lint-only) are all the right calls. The check-jsonschema validation pass and clean plugin validate output give good confidence in structural correctness. Three issues below warrant attention before merge.


Findings

1. typescript.yaml test-cmd is unexecutable prose — rung-4 fallback will fail at runtime [Medium]

reference/ecosystems/typescript.yaml line 7:

test-cmd: "the package.json test script (npm test / npx vitest run / npx jest — use what the project configures)"

This is the only command field across all 8 ecosystem files whose value is documentation text rather than a runnable shell string. The resolution-ladder contract says command values are opaque shell strings and null means "no such phase." When a consumer has no .claude/ecosystems/typescript.yaml and inference fails, the skill reaches rung 4, receives this prose string, and any attempt to execute it literally (the package.json test script …) will produce a shell "command not found" error.

The build-cmd and check-cmd for TypeScript are valid commands with inline shell comments (# ...) after the closing quote — those comments are stripped by YAML and the remainder executes correctly. Only test-cmd has this problem.

The intent is clearly "read package.json scripts and use whatever the project configures." Two options:

  • Set test-cmd: null (honest — signals no universal default) and let the build skill fall to rung 2 (infer from package.json), consistent with how Python and Bash express "no test framework without inference."
  • Use a concrete command like npm test as the least-common-denominator that works without config for most Node/TS projects.

Fix this →


2. $EC_BIN used in cross-cutting.yaml but not documented in resolution-ladder.md as a known placeholder [Low]

reference/ecosystems/cross-cutting.yaml line 14:

check-cmd: |
  typos
  gitleaks detect --source . --redact --no-banner --no-git
  "$EC_BIN"

The resolution-ladder.md documents four known placeholders substituted by the running skill (<files>, <solution-or-project-file>, <project-dir>, $REPO_ROOT). $EC_BIN is not among them. The lint SKILL.md does define and export $EC_BIN before running check-cmd, so it works in practice — but a future skill consuming cross-cutting via the shared ladder document has no indication that $EC_BIN must be resolved before execution.

The shared ladder doc should list $EC_BIN alongside the other substitution variables, with a note that it's cross-cutting-specific. This is especially relevant given the contract's intent as a multi-plugin seam — another plugin consuming this YAML would silently get an empty string for $EC_BIN if it doesn't know to set it.

Fix this →


3. *.psd1 absent from powershell.yaml globs, but mentioned in opt-in [Low]

reference/ecosystems/powershell.yaml lines 4 and 9:

globs: ["*.ps1", "*.psm1"]
...
opt-in: "PSScriptAnalyzerSettings.psd1 (or any PowerShell files with pwsh available)"

The opt-in field cites PSScriptAnalyzerSettings.psd1 as the trigger condition, but .psd1 is not in globs. A change limited to a .psd1 file (module manifest or PSScriptAnalyzer settings update) wouldn't trigger ecosystem detection, so PSScriptAnalyzer would never run to check its own config changes. The check-cmd uses -Recurse so it would analyze .psd1 files once triggered — the gap is detection, not execution.

Suggestion: add "*.psd1" to the globs array. The correction is trivial and makes the opt-in condition self-consistent with detection.

Fix this →


Observations (not blockers)

  • disable-model-invocation: false in setup/SKILL.md is consistent with the repo's existing pattern (used in codebase-audit/skills/setup/SKILL.md and dozens of other skills) — not a concern.
  • cross-cutting check-cmd comment-only lines (zizmor, schema checks) don't add executable commands — they're prose hints for the AI reading the YAML. This is the established pattern for opt-in-gated tools where the running skill is responsible for skipping when the config is absent.
  • User-global-only config silently falls to rung 2 — if only ~/.claude/ecosystems/<eco>.yaml exists with no team file, the ladder falls to inference instead of honoring the user-global. This is the design as specified in the contract. Worth a sentence in the resolution-ladder noting this behavior explicitly, but not a blocker.
  • CHANGELOG format — correctly follows Keep a Changelog with recorded-decision entries; the "Design decisions (not reopened)" subsection is a useful precedent for this category of change.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: low. Did not approve because Cursor Bugbot completed with skip status (usage limit reached), so the required automated review did not complete. Human review is needed before merge; no reviewers were assigned because the PR author is the sole repository contributor.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 72bb3b7a02

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/implementation/reference/resolution-ladder.md Outdated
Skip any ecosystem whose resolved `enabled` is false in /build and /lint
detection (excluded even under `all`), per the contract schema's `enabled`
key. Document it in the resolution ladder. Addresses Codex review P2.

Refs melodic-software/medley#1420
@cursor

cursor Bot commented Jul 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_58955375-fd6f-4b44-9e64-842356db5d62)

@claude

claude Bot commented Jul 12, 2026

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 2m 44s —— View job


PR Review: chore(implementation): consume ecosystem-commands contract — ladder resolution, setup, v0.2.0

  • Gather context and understand the request
  • Read changed files and diff
  • Review ecosystem YAML files
  • Review resolution-ladder.md
  • Review updated SKILL.md files (build, lint, setup)
  • Check plugin.json and CHANGELOG.md
  • Validate against CLAUDE.md guidelines
  • Post review findings

Summary

Two commits on this branch. The main commit (72bb3b7) lands the contract migration. The follow-up (6ae94b3) addressed the Codex P2 (enabled: false not honored) — both skills and resolution-ladder.md now explicitly skip disabled ecosystems. The architecture is sound; the shared ladder, unified YAML set, and scope-preserving split (yaml/cross-cutting staying /lint-only) are the right calls.

Three issues from the previous review pass remain unaddressed in HEAD. No new blockers found.


Findings

1. typescript.yaml test-cmd is unexecutable prose — rung-4 fallback crashes at runtime [Medium, not yet fixed]

reference/ecosystems/typescript.yaml line 7:

test-cmd: "the package.json test script (npm test / npx vitest run / npx jest — use what the project configures)"

This is the only command field across all 8 ecosystem files whose value is documentation prose, not a runnable shell string. Every other file either uses null (correct — signals no default) or a concrete command. When a consumer has no .claude/ecosystems/typescript.yaml and inference fails, the skill hits rung 4, receives this prose, and any attempt to run it literally errors on the. The previous review suggested either null (let rung 2 infer from package.json) or a concrete fallback like npm test. The setup/SKILL.md step 2 already describes how to infer TypeScript test-cmd from the scripts block — so null here is the cleanest choice.

Fix this →


2. $EC_BIN not listed in resolution-ladder.md's placeholder table [Low, not yet fixed]

reference/resolution-ladder.md lines 22–24:

Placeholders substituted by the running skill:
`<files>`, `<solution-or-project-file>`, `<project-dir>`, `$REPO_ROOT`

$EC_BIN is used in cross-cutting.yaml's check-cmd but is missing from this list. The lint SKILL.md does correctly resolve it via command -v probing before substitution (lines 82–90), so the skill itself works. The gap is the shared contract doc: another plugin consuming cross-cutting.yaml through the ladder has no indication that $EC_BIN must be resolved before execution, and an unset $EC_BIN in the quoted position ("$EC_BIN") produces a "command not found" shell error.

Fix this →


3. *.psd1 absent from powershell.yaml globs [Low, not yet fixed]

reference/ecosystems/powershell.yaml line 4:

globs: ["*.ps1", "*.psm1"]
opt-in: "PSScriptAnalyzerSettings.psd1 (or any PowerShell files with pwsh available)"

The opt-in cites PSScriptAnalyzerSettings.psd1 as a trigger condition, but .psd1 is absent from globs. A change limited to a .psd1 file (module manifest or PSScriptAnalyzer config update) won't trigger ecosystem detection, so the linter never runs on its own config changes. Adding "*.psd1" to the array is a one-word fix and makes the detection self-consistent with what the opt-in promises.

Fix this →


What's been addressed since the prior review pass

  • Codex P2 (enabled: false not honored) — Fixed in 6ae94b3. resolution-ladder.md, /build detection (step 1), and /lint detection (step 1) all now explicitly skip ecosystems whose resolved enabled is false, excluded even under all. The bundled defaults correctly omit the key (default true). The consumer overlay can flip it back. ✅

Observations (not blockers)

  • $EC_BIN runtime safety in /lint is already correct. lint/SKILL.md lines 82–90 probe command -v ec, editorconfig-checker, ec-windows-amd64 in order before substituting into the cross-cutting check-cmd. The gap is documentation in the shared ladder doc only, not the skill's runtime behavior.
  • yaml.yaml comment-only lines (# zizmor …, # plus any schema checks …) are shell comments — the effective executed command is just actionlint. This is the established advisory-hint pattern; not an execution issue.
  • User-global-only config silently falls to rung 2. If only ~/.claude/ecosystems/<eco>.yaml exists with no team file, rung 1 never triggers (rung 1 is conditional on the team file being present; user-global layers within rung 1 additively). The behavior is correct per the design, but a brief note in the resolution-ladder would help consumers who expect user-global to be authoritative on its own.
  • setup/SKILL.md idempotency and seed design are well-structured: step 1 loads existing files as a baseline, step 2 seeds from bundled defaults then specializes, and step 4 emits conformant files. The disable-model-invocation: false matches the codebase-audit/skills/setup precedent.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: medium. Did not approve because Cursor Bugbot completed with skip status, so the required automated review did not complete successfully. Human review is needed before merge; no reviewers were assigned because the PR author is the sole human contributor.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

@cursor

cursor Bot commented Jul 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_123ee594-1f78-4db0-b3b5-2085127a593c)

@claude

claude Bot commented Jul 12, 2026

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 1m 49s —— View job


PR Review: chore(implementation): consume ecosystem-commands contract — ladder resolution, setup, v0.2.0

  • Gather context and understand the request
  • Read changed files and check HEAD for previous findings
  • Review typescript.yaml test-cmd (previously flagged: unexecutable prose)
  • Review resolution-ladder.md for $EC_BIN placeholder documentation
  • Review powershell.yaml globs for *.psd1
  • Check for any new issues in the sync commit
  • Post review findings

Summary

The HEAD sync commit (0588f75) is a straight merge from main — no implementation plugin files changed. All three findings from prior passes remain open in HEAD. No new issues found in the merge.


Findings

1. typescript.yaml test-cmd is unexecutable prose — rung-4 fallback will crash [Medium, still open]

reference/ecosystems/typescript.yaml line 7:

test-cmd: "the package.json test script (npm test / npx vitest run / npx jest — use what the project configures)"

This remains the only command field across all 8 ecosystem files whose value is documentation prose rather than a runnable shell string (or null). Every other file correctly uses either a concrete command or null. When a consumer has no .claude/ecosystems/typescript.yaml and inference fails, the skill falls to rung 4, receives this prose, and any attempt to execute it literally produces a shell command not found error on the word the.

The setup/SKILL.md step 2 already describes how to infer TypeScript test-cmd from the scripts block — null is the cleanest choice, consistent with how Python and Bash express "no universally applicable default."

Fix this →


2. $EC_BIN not listed in resolution-ladder.md's placeholder table [Low, still open]

reference/resolution-ladder.md lines 21–24:

Placeholders substituted by the running skill:
`<files>`, `<solution-or-project-file>`, `<project-dir>`, `$REPO_ROOT`
(`git rev-parse --show-toplevel`).

$EC_BIN is used in cross-cutting.yaml's check-cmd ("$EC_BIN" on line 14) but is absent from this list. The /lint SKILL.md correctly resolves it (lines 82–90: probes ec, editorconfig-checker, ec-windows-amd64 in order), so the runtime is fine. The gap is the shared contract doc: any plugin consuming cross-cutting.yaml through the ladder has no indication that $EC_BIN must be resolved before execution. An unset $EC_BIN in the quoted position silently runs an empty command, which may produce a "command not found" shell error depending on the shell.

Fix this →


3. *.psd1 absent from powershell.yaml globs [Low, still open]

reference/ecosystems/powershell.yaml line 4:

globs: ["*.ps1", "*.psm1"]
opt-in: "PSScriptAnalyzerSettings.psd1 (or any PowerShell files with pwsh available)"

The opt-in cites PSScriptAnalyzerSettings.psd1 as a trigger condition, but .psd1 is absent from globs. A change limited to a .psd1 file (module manifest or PSScriptAnalyzer config update) won't trigger ecosystem detection, so PSScriptAnalyzer never runs on its own config changes. The check-cmd uses -Recurse so it would analyze .psd1 files once triggered — the gap is detection only. Adding "*.psd1" to the array is a trivial, self-consistent fix.

Fix this →


Status of previous review passes

  • Codex P2 (enabled: false not honored) — Fixed in 6ae94b3. Both skills and resolution-ladder.md correctly document and implement the skip. ✅
  • Findings 1–3 above: not yet addressed across any of the three commits on this branch.

Observations (not blockers)

  • The $EC_BIN resolution in /lint SKILL.md (lines 82–90) is correct and handles all three binary-name variants; the gap is documentation only.
  • cross-cutting.yaml comment-only lines (# zizmor …, # plus any schema checks …) are shell comments — the effective executed command is just typos, gitleaks, and $EC_BIN. This is the established advisory-hint pattern.
  • Architecture, CHANGELOG format, setup/SKILL.md idempotency design, and plugin version bump are all sound — no concerns there.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: medium. Did not approve because Cursor Bugbot completed with skip status (usage limit reached), so the required automated review did not complete successfully. Human review is needed before merge; no reviewers were assigned because the PR author is the sole repository contributor.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

@kyle-sexton
kyle-sexton merged commit 6ad0148 into main Jul 12, 2026
38 of 40 checks passed
@kyle-sexton
kyle-sexton deleted the chore/retrofit-implementation branch July 12, 2026 16:27
kyle-sexton added a commit that referenced this pull request Jul 12, 2026
…lls) (#123)

## What

Author rich-form `evals/evals.json` for five judgment-bearing skills,
one file per skill under `plugins/<plugin>/skills/<skill>/evals/`. Each
ships 6 cases covering trigger/routing, the happy path, at least one
refusal/guardrail, and at least one must-not-do anti-pattern, with
objectively-verifiable `expectations`.

- `discovery/explore`
- `discovery/explore-deep`
- `discovery/research`
- `discovery/research-deep`
- `claude-config-audit/memory-health`

Variant skills exercise their **distinctive** contract rather than
re-testing the parent:
- **explore-deep** — fork isolation, read-only boundary (must not edit
source), summary+artifact-path return (not the full 7-section report),
fallback when `CLAUDE_CODE_FORK_SUBAGENT` unset, empty-scope handling,
sidecar when EXPLORE.md exists.
- **research-deep** — dispatcher behavior: multi-topic → N parallel
agents (not one blob on the engine), tier selection,
does-not-run-inline, full discipline even on a small task.

## Per-skill warrant verdicts (re-checked against live SKILL.md)

| Skill | Verdict | Basis |
|-------|---------|-------|
| discovery/explore | **WARRANTED** | Judgment-bearing: mode routing,
6-dimension coverage, outcome gate, no-external-research boundary,
does-not-execute |
| discovery/explore-deep | **WARRANTED** | Fork-isolation + read-only
boundary + return-contract are behavioral contracts a model can violate
|
| discovery/research | **WARRANTED** | 3-phase discipline, source-tier
ratio, mandatory falsification, recency + outcome gates |
| discovery/research-deep | **WARRANTED** | Dispatcher decisions
(multi-topic split, tier selection, no-inline-deep-pass) are judgment
calls |
| claude-config-audit/memory-health | **WARRANTED** | Scope boundary,
contributor-local persistence, fix-gating, consumer-exemption handling
are behavioral |

No skill in this batch was pure-reference; no skip verdicts.

## Version bumps

Both touched plugins bumped `0.1.0` → `0.2.0` (minor — evals are an
additive shipped component; a version bump is the only delivery
vehicle). Matches the precedent of a minor bump for adding a new shipped
component (implementation plugin → v0.2.0 for a setup skill, #100).

## Validation

- All 5 files pass `check-jsonschema` against
`plugins/skill-quality/reference/evals.schema.json`.
- All 5 skills pass `check-skill.sh` (0 errors); no evals-missing WARN,
confirming presence is detected. Residual WARNs are pre-existing
skill-body items (Gotchas surface / `Use when:` phrasing), out of scope
for this evals-authoring task.

## Note on stale issue-body references

The claimed issue body cites two paths that do not exist on `main`:
`docs/MIGRATION-PLAYBOOK.md` "Evals — warrant policy and consumer-verify
recipe" and `docs/evals-coverage.md`. The warrant guidance actually
lives in `plugins/skill-quality/skills/skill-quality/SKILL.md`
(validate-evals action) — used as the warrant authority here. Flagging
for tracker hygiene; not a blocker.

Refs melodic-software/medley#1448

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Additive eval JSON and minor plugin version bumps only; no changes to
skill runtime logic or application code paths.
> 
> **Overview**
> Adds **rich-form** `evals/evals.json` (six cases each, with `prompt`,
`expected_output`, and verifiable `expectations`) for five
judgment-bearing skills: **discovery** `explore`, `explore-deep`,
`research`, `research-deep`, and **claude-config-audit**
`memory-health`.
> 
> The new cases encode each skill’s behavioral contract—happy paths,
scope/routing, guardrails (read-only, no auto-fix, no training-recall
claims), and variant-specific rules (e.g. fork isolation and sidecar
artifacts for `explore-deep`; dispatcher tiering and parallel topic
agents for `research-deep`; contributor-local reports and fix gating for
`memory-health`).
> 
> **Plugin versions** are bumped as the delivery vehicle for the new
shipped eval assets: `claude-config-audit` **0.1.0 → 0.2.0**,
`discovery` **0.2.0 → 0.3.0**.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
9406015. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant