chore(review-toolkit): ecosystem-specialist consumes ecosystem-commands contract#98
Conversation
…ds contract Resolve each ecosystem's build/test/lint command truth from the consumer's .claude/ecosystems/<ecosystem>.yaml files (authoritative when present) per the marketplace-wide ecosystem-commands contract, then the project's documented conventions, then the agent's bundled generic defaults as an explicit last resort. Detection may use the contract's globs when config exists. Report format, MISSING-tool handling, and detection behavior unchanged. Cite the contract by path in the agent body and README; add CHANGELOG; bump plugin.json to 0.2.0 (minor). Co-Authored-By: Claude <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_550c3e9f-f0b1-43f4-b4f4-8a6d1e19b25e) |
|
Claude finished @kyle-sexton's task in 2m 28s —— View job PR Review:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c1ae382dde
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…esolution - Distinguish an explicit `null` command (skip the phase, no fall-through) from an absent key (fall through the ladder). - Reorder "Before running": identify change set → detect ecosystems (re-classify by contract globs) → resolve command truth, removing the prerequisite circularity. - Record that this read-only agent intentionally omits the contract's infer-and-persist and ask-user rungs. - Reword the README bullet so the contract citation is a parenthetical aside, not a second fallback rung. Co-Authored-By: Claude <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_ca9716d1-7901-4f4e-9002-349a46712e08) |
|
Claude finished @kyle-sexton's task in 1m 58s —— View job PR Review:
|
| Finding | Status |
|---|---|
null vs. absent command |
✅ Fixed — explicit null = skip, no fall-through |
| Step ordering circularity | ✅ Fixed — reordered to: identify → detect → resolve |
| Ladder omits contract rungs 2/3 silently | ✅ Fixed — read-only note added |
| README em-dash structure | ✅ Fixed — contract citation is now a parenthetical |
Remaining issues
1. enabled: false not honored during glob re-classification (correctness)
agents/ecosystem-specialist.md:15
The schema defines enabled: boolean, default: true with the description: "False disables detection without deleting the file." Step 2 re-classifies changed files using contract globs for any ecosystem that has a YAML file — but never checks enabled. A consumer who sets enabled: false to temporarily suppress an ecosystem (e.g., a shell-heavy repo mid-migration that isn't ready for shellcheck) will still trigger detection and command resolution from that file, which is exactly the behavior enabled: false is meant to prevent.
Add a check: when the YAML file is present but enabled: false, skip that ecosystem entirely (no re-classification, no command resolution). Fix this →
2. Contract placeholders not bound before execution (correctness)
agents/ecosystem-specialist.md:22
The contract README documents four placeholders that consuming surfaces are responsible for binding: <files> (ecosystem-scoped changed-file list), <solution-or-project-file> (resolved via anchor), <project-dir> (from project-discovery), and $REPO_ROOT (repo root). The instruction says to use commands "verbatim" but never tells the agent to substitute those placeholders before running. A valid consumer command like shellcheck <files> or dotnet build <solution-or-project-file> would be handed to the shell literally — <files> parses as input redirection, causing a crash or unexpected behavior even though the ecosystem config is correct.
The contract README is explicit that context binding is the consuming surface's concern (not the schema's), so the binding instructions belong here in the agent. Add a note in the command-truth resolution section explaining how to bind each placeholder before execution. Fix this →
3. gates entries silently skipped (coverage)
agents/ecosystem-specialist.md:28-35
The schema defines a gates array for CI-parity checks beyond build/test/lint (lockfile drift, generated-artifact freshness, schema regeneration). Each gate has name, cmd, optional trigger-globs, and remediation. The agent's Verification workflow only runs three steps; any declared gates are silently skipped. A consumer who adds gates to .claude/ecosystems/dotnet.yaml expecting CI-parity will never see them run. Add a step 4 in the Verification workflow for gates. Fix this →
4. notes field not surfaced (minor)
agents/ecosystem-specialist.md:36
The schema's notes field is "Free-form caveats a resolver should surface (trap flags, ordering constraints)." The agent never reads or reports it, so a consumer notes: "run dotnet build before tests or test discovery fails" is silently ignored. Mention that when a notes value is present, it should appear at the top of that ecosystem's section in the report. Small fix; low risk.
5. CHANGELOG missing dates (style)
Both [0.2.0] and [0.1.0] entries are missing the YYYY-MM-DD date required by Keep a Changelog 1.1.0 (## [x.y.z] - YYYY-MM-DD). The file's own header claims to follow that format.
Confirmed correct (no action needed)
null/absent distinction — fixed correctly;nullblocks fall-through, absent key permits it.- Step ordering — identify → detect (with two-pass glob re-classification) → resolve is logically correct.
- Read-only rung note — accurately records the intentional deviation from the contract's 4-rung ladder.
- Field names (
build-cmd,test-cmd,check-cmd,install-hint,globs) match the schema exactly. - Overlay order (user-global → team →
.local.) matches the contract. - Version bump
0.1.0 → 0.2.0is correct semver. - Plugin-form safety — no hardcoded paths, no
../reach-outs. - README bullet is clear and correctly scoped after the em-dash rewrite.
Overall: The 86f04e6 fixes are all correct. Issues 1 and 2 (above) are correctness gaps worth addressing before merge; 3–5 are lower priority but straightforward to fix.
|
Addressed the bot-review findings in
Deferred (out of this issue's scope fence — #1421 keeps report format / MISSING handling / detection behavior, changing only command-truth sourcing):
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 86f04e6663
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…alse - Instruct the agent to expand the contract placeholders (<files>, <solution-or-project-file>, <project-dir>, $REPO_ROOT) before running a configured command, instead of passing it to the shell literally. - Drop any ecosystem whose config sets enabled:false during glob-based detection, honoring the schema's disable-without-delete semantics. Co-Authored-By: Claude <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_f85bc798-3dea-4750-a38d-6dd1a9064ec4) |
|
Claude finished @kyle-sexton's task in 1m 43s —— View job PR Review:
|
| Finding | Status |
|---|---|
enabled: false not honored during glob re-classification |
✅ Fixed — step 2 now explicitly drops any ecosystem whose YAML sets enabled: false, even when its globs match |
| Contract placeholders handed to shell literally | ✅ Fixed — rung 1 now names all four placeholders and gives a concrete expansion example |
Confirmed correct (full PR)
nullvs absent command —nullskips the phase with no fall-through; absent key falls through the ladder. Both cases are now explicit. (L36)- Step ordering — identify → detect (two-pass glob re-classification) → resolve is logically sound. (L14-L16)
- Read-only rung note — intentional omission of infer-and-persist and ask-user rungs is documented. (L26)
- Field names (
build-cmd,test-cmd,check-cmd,install-hint,globs) match the schema exactly. - Overlay order (user-global → team →
.local.) matches the contract's documented resolution. enabled: false— checked during detection (step 2), before resolution (step 3). No late-check gap.- Placeholder list — all four contract placeholders (
<files>,<solution-or-project-file>,<project-dir>,$REPO_ROOT) are named with theshellcheck -x <files>expansion example illustrating why literal hand-off fails. - Version bump
0.1.0 → 0.2.0is correct semver for a behavioral change. - Plugin-form safety — no hardcoded paths, no
../reach-outs, nouserConfigadded. - README bullet is clear and correctly scoped after the em-dash rewrite.
One minor note
Placeholder resolution methods not specified (L22)
The placeholder-binding note says "bind…to this run's values" but doesn't say how <solution-or-project-file> resolves (via the ecosystem file's anchor) or how <project-dir> resolves (via project-discovery). The contract README is cited in the same sentence, so a motivated agent will follow it — not a blocker, but explicitly naming the resolution methods would make the instruction self-contained. Easily addressed in a follow-up.
Deferred items — rationale accepted
gatesandnotes— the author's scope fence is coherent: those fields add new verification behavior (running CI-parity gates; surfacing trap caveats in the report) rather than changing command-truth sourcing. Expanding here would exceed the pre-baked scope and create a larger-than-intended diff. Both are worth a follow-up issue.- CHANGELOG dates — dateless format matches the existing sibling
plugins/songwriting/CHANGELOG.md; repo convention takes precedence over the format declaration.
Overall: All correctness gaps identified across three review passes have been resolved. The PR is ready to merge. The one minor note above (placeholder resolution method specificity) and the two deferred items (gates, notes) are reasonable follow-up candidates.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e0eff0d4e2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
An omitted build-cmd/test-cmd/check-cmd is undeclared (fall through the ladder for that one command), distinct from an explicit null (skip the phase). A file present only for classification no longer suppresses valid build/test/lint. Consolidate the null semantics into the resolution ladder to avoid duplication. Co-Authored-By: Claude <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_c029f8d0-f5ba-4712-837a-64ca7dbfa64e) |
|
Claude finished @kyle-sexton's task in 1m 49s —— View job PR Review:
|
| Condition | Behavior |
|---|---|
| Key present + non-null | Authoritative — use verbatim (with placeholder binding) |
Key set to null |
Phase does not apply — skip, no fall-through |
| Key omitted | Undeclared at rung 1 — fall through to rung 2, then rung 3, for that one command |
This is the correct semantics per the contract. A .claude/ecosystems/shell.yaml with only globs now correctly lets build-cmd, test-cmd, and check-cmd each fall through independently — it no longer suppresses valid verification just because the file exists for classification purposes.
agents/ecosystem-specialist.md:22
Full four-commit review — confirmed correct
| Concern | Status | Commit |
|---|---|---|
| Step ordering (identify → detect → resolve) | ✅ | 86f04e6 |
null vs absent command distinction |
✅ | 86f04e6 |
| Read-only rung deviation documented | ✅ | 86f04e6 |
| README em-dash structure | ✅ | 86f04e6 |
enabled: false honored before re-classification |
✅ | e0eff0d |
| Contract placeholders bound before execution | ✅ | e0eff0d |
| Per-command-key fallthrough for omitted keys | ✅ | 54b7271 |
Field names (build-cmd, test-cmd, check-cmd, install-hint, globs) match the schema exactly. Overlay order (user-global → team → .local.) matches the contract's documented resolution. Version bump 0.1.0 → 0.2.0 is correct semver. Plugin-form safety: no hardcoded paths, no ../ reach-outs, no userConfig added.
Deferred items — rationale accepted
gatesandnotes— these add new verification behavior (running CI-parity gates; surfacing trap caveats in the report) rather than changing command-truth sourcing, which was the defined scope. Reasonable follow-up candidates.- CHANGELOG dates — dateless format matches
plugins/songwriting/CHANGELOG.md; repo convention takes precedence over the format declaration.
Overall: All correctness gaps identified across four review passes have been resolved. The PR is ready to merge.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 54b7271fb1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…chain at detection Detection now merges globs/enabled across user-global -> team -> .local. before classifying, so a .local. overlay that disables an ecosystem or overrides its globs takes effect at detection, not only at command resolution. Co-Authored-By: Claude <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_b109ec94-f63b-4069-95d2-28b0a246f3a4) |
|
Claude finished @kyle-sexton's task in 1m 57s —— View job PR Review:
|
| Concern | Status | Commit |
|---|---|---|
| Step ordering (identify → detect → resolve) | ✅ | 86f04e6 |
null vs absent command distinction |
✅ | 86f04e6 |
| Read-only rung deviation documented | ✅ | 86f04e6 |
| README em-dash structure | ✅ | 86f04e6 |
enabled: false honored before re-classification |
✅ | e0eff0d |
| Contract placeholders bound before execution | ✅ | e0eff0d |
| Per-command-key fallthrough for omitted keys | ✅ | 54b7271 |
globs + enabled resolved through full overlay chain at detection |
✅ | 53bc956 |
Field names (build-cmd, test-cmd, check-cmd, install-hint, globs) match the schema exactly. Overlay order (user-global → team → .local.) is consistent across step 2 (detection) and rung 1 (command resolution). Version bump 0.1.0 → 0.2.0 is correct semver. Plugin-form safety: no hardcoded paths, no ../ reach-outs, no userConfig added.
Deferred items — rationale accepted
gatesandnotes— add new verification/report behavior beyond command-truth sourcing; clearly outside the pre-baked scope fence.- CHANGELOG dates — dateless format matches
plugins/songwriting/CHANGELOG.md; repo convention takes precedence over the format declaration. - Placeholder resolution method specificity (minor) — the contract README is cited inline; the resolution methods for
<solution-or-project-file>(viaanchor) and<project-dir>(viaproject-discovery) are specified there. Not a correctness risk given the citation, and a reasonable follow-up candidate.
Overall: All correctness gaps identified across five review passes have been resolved. The PR is ready to merge.


Retrofits the
review-toolkitplugin'secosystem-specialistagent to consume the marketplace-wide ecosystem-commands contract (docs/conventions/ecosystem-commands/README.md) instead of carrying its own inline command defaults.Changes
agents/ecosystem-specialist.md— new Command-truth resolution ladder: (1) consumer.claude/ecosystems/<ecosystem>.yaml(authoritative forbuild-cmd/test-cmd/check-cmd,globs,install-hint, with user-global +.local.overlay layering); (2) the project's documented conventions; (3) the agent's generic defaults, demoted to explicit last-resort ("when neither exists"). Ecosystem detection may use the contract'sglobswhen config exists. MISSING-tool handling ties to the ecosystem file'sinstall-hintwhen present. Report format, MISSING handling, and detection behavior otherwise unchanged — only the command-truth sourcing moved.README.md— new "Command truth from.claude/ecosystems/" bullet citing the contract by path.CHANGELOG.md— added (new file);[0.2.0]entry.plugin.json— version0.1.0→0.2.0(minor).Pre-baked decisions honored (from #1390)
.claude/ecosystems/— no reference to theimplementationplugin's paths.userConfigpath knob added; location is conventional per the contract.Verification
scripts/validate-plugins.sh→ all manifests + catalog pass.markdownlint-cli2on the three changed markdown files → 0 errors.Refs melodic-software/medley#1421
Note
Low Risk
Changes are limited to plugin prompts, README, changelog, and version metadata—no executable code paths or auth/data handling.
Overview
Bumps review-toolkit to 0.2.0 and documents the change in a new CHANGELOG and README.
ecosystem-specialistnow follows the marketplace ecosystem-commands contract (docs/conventions/ecosystem-commands/README.md) for build/test/lint truth instead of treating inline defaults as peers. A Command-truth resolution ladder applies: consumer.claude/ecosystems/<ecosystem>.yaml(with user-global and.local.overlays, per-keynull/omit rules, and placeholder binding) → project docs/CI/scripts → generic defaults only as last resort. Ecosystem detection can override path heuristics with resolvedglobsand honorenabled: false. MISSING-tool reports prefer the file’sinstall-hint. The agent explicitly skips infer-and-persist / ask-user rungs because it stays read-only; report shape and overall behavior are otherwise unchanged.Reviewed by Cursor Bugbot for commit 53bc956. Bugbot is set up for automated code reviews on this repo. Configure here.