Skip to content

docs(mcp): clarify ambiguous loopover-mcp stdio tool descriptions for LLM selection - #6270

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
nghetienhiep:fix/issue-6245
Jul 16, 2026
Merged

docs(mcp): clarify ambiguous loopover-mcp stdio tool descriptions for LLM selection#6270
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
nghetienhiep:fix/issue-6245

Conversation

@nghetienhiep

Copy link
Copy Markdown
Contributor

What

Audit of the loopover-mcp stdio tool descriptions (packages/loopover-mcp/bin/loopover-mcp.js,
STDIO_TOOL_DESCRIPTORS) for LLM tool-selectability, per #6245. An MCP/agent client picks a tool
almost entirely from its registered description, so a vague, jargon-heavy, or parameter-silent
description leads to the wrong tool being called — or a fitting tool never being discovered.

I reviewed all 42 stdio tool descriptions against the issue's three flags and rewrote the 7 that
failed one of them. The rest were already clear enough to disambiguate and left untouched.

Flagged and rewritten

Tool Problem Fix
loopover_get_repo_context "canonical repo intelligence bundle" is opaque — doesn't say what's in it or which repo Spells out the contents (registration, lane, queue health, collisions, config quality) and names the owner/repo inputs
loopover_get_registry_changes "registry change report" undefined; no hint what a change is Explains it reports repos added/removed/re-registered upstream; notes it takes no parameters
loopover_get_decision_pack "decision pack" is internal jargon; no contents; "GitHub login" unexplained Describes the ranked repos/issues + go/raise/avoid guidance and clarifies login is the contributor's username
loopover_explain_repo_decision circular ("decision pack" again); not disambiguated from get_decision_pack; missing owner/repo States it's the per-repo slice, cross-references the broader tool, names all three inputs
loopover_agent_plan_next_work "base-agent planner" jargon; output unspecified Says it returns the single recommended next unit of work; planning-only; names login + optional inputs
loopover_agent_start_run "copilot-only base-agent run" jargon; required inputs unstated Explains copilot mode (no GitHub writes) and names objective/actorLogin + return value
loopover_agent_get_run required runId not mentioned — an agent can't construct a valid call Names runId and points to loopover_agent_start_run as its source

Every new description names its real input parameters (verified against each tool's inputSchema)
and stays inside the existing public-safe boundary — no scores, rewards, or wallet/key terms.

Scope

This covers the loopover-mcp stdio package. The hosted src/mcp/server.ts surface is the sibling
tool set and is a natural separate PR, per the issue's "split into a few PRs by tool category"
guidance.

Tests

No behavior change (descriptions are metadata). Existing tool-registration tests don't hardcode the
old text, so none break. Added two invariant checks to test/unit/mcp-discovery.test.ts that list
the live tools over the MCP protocol and assert every description is non-empty and unique enough to
disambiguate (no two tools share identical wording) and that no tool description leaks a forbidden
public term — the same safety property already enforced for resources and prompts, now extended to
tools.

Verified locally: typecheck, targeted vitest (mcp-discovery, mcp-cli-tools, mcp-cli-basics,
mcp-tool-rename-aliases, mcp-cli-packets), build:mcp, test:mcp-pack, and the
command-reference/docs/manifest drift checks all pass.

Closes #6245

… LLM selection

An MCP client selects a tool almost entirely from its registered description,
so vague or jargon-heavy text causes wrong-tool selection or non-discovery.

Audit all 42 stdio tool descriptions in STDIO_TOOL_DESCRIPTORS and rewrite the
7 that were too short to disambiguate, used internal jargon ("intelligence
bundle", "decision pack", "base-agent run") without explanation, or omitted a
required parameter (agent_get_run's runId). Each rewrite names the tool's real
inputs, verified against its inputSchema, and stays inside the public-safe
boundary.

Extend mcp-discovery to assert tool descriptions are non-empty, unique enough
to disambiguate, and free of forbidden public terms -- the safety property
already enforced for resources and prompts.

Closes JSONbored#6245
@nghetienhiep
nghetienhiep requested a review from JSONbored as a code owner July 15, 2026 23:49
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.35%. Comparing base (e9977ad) to head (a409184).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6270   +/-   ##
=======================================
  Coverage   95.35%   95.35%           
=======================================
  Files         598      598           
  Lines       47127    47127           
  Branches    14994    14994           
=======================================
  Hits        44940    44940           
  Misses       1463     1463           
  Partials      724      724           
Flag Coverage Δ
shard-1 44.02% <ø> (-0.03%) ⬇️
shard-2 36.66% <ø> (+0.06%) ⬆️
shard-3 32.35% <ø> (+0.01%) ⬆️
shard-4 34.36% <ø> (+<0.01%) ⬆️
shard-5 31.79% <ø> (ø)
shard-6 44.91% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 16, 2026
@loopover-orb

loopover-orb Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-16 00:04:01 UTC

2 files · 1 AI reviewer · no blockers · readiness 80/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This is a docs-only change that rewrites 7 of 42 MCP stdio tool descriptions to remove opaque internal jargon ("canonical repo intelligence bundle", "decision pack", "base-agent") and explicitly name the tool's parameters, directly per issue #6245's LLM tool-selectability flags. It also adds two new discovery tests asserting all tool descriptions are non-empty/unique and free of forbidden public terms, giving this change regression coverage it previously lacked. The rewritten descriptions are more concrete and correctly cross-reference sibling tools (e.g. explain_repo_decision vs get_decision_pack) without altering any runtime behavior.

Nits — 5 non-blocking
  • test/unit/mcp-discovery.test.ts:105 uses FORBIDDEN_PUBLIC_TERMS — confirm this regex is already imported/defined elsewhere in the file since it isn't shown in the diff.
  • The new description for loopover_agent_start_run (packages/loopover-mcp/bin/loopover-mcp.js:576) names the parameter as actorLogin — worth double-checking that matches the tool's actual input schema key rather than login used elsewhere in the file.
  • The duplicate-description test in mcp-discovery.test.ts:88 only guards against exact-string collisions, not near-duplicate wording that could still confuse an LLM client — fine for now but worth noting as a soft check.
  • Consider running the new mcp-discovery tests against the full 42-tool list in CI output to confirm none of the untouched 35 descriptions collide with the 7 rewritten ones.
  • If actorLogin/login naming is inconsistent across tool schemas, a follow-up normalizing parameter names across descriptions would further help LLM disambiguation.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #6245
Related work ⚠️ 2 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 99 registered-repo PR(s), 53 merged, 0 issue(s).
Contributor context ✅ Confirmed Gittensor contributor nghetienhiep; Gittensor profile; 99 PR(s), 0 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Partially addressed
The PR only audits and rewrites descriptions in packages/loopover-mcp/bin/loopover-mcp.js (7 of 42 stdio tools), but the issue explicitly requires reviewing both that file and src/mcp/server.ts (~150 tools combined), and the PR description itself scopes to only the stdio tools.

Review context
Contributor next steps
  • Start here: Review top overlaps.
  • Then work through the remaining 2 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopOver approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 2fbfd52 into JSONbored:main Jul 16, 2026
16 checks passed
@github-actions github-actions Bot mentioned this pull request Jul 16, 2026
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

research: audit all MCP tool descriptions for LLM-selectability/clarity

1 participant