Skip to content

feat(mcp): add gittensory_check_test_evidence tool (deterministic coverage-gap self-check) - #4244

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
jeffrey701:feat/mcp-check-test-evidence
Jul 8, 2026
Merged

feat(mcp): add gittensory_check_test_evidence tool (deterministic coverage-gap self-check)#4244
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
jeffrey701:feat/mcp-check-test-evidence

Conversation

@jeffrey701

Copy link
Copy Markdown
Contributor

What

The engine already has a deterministic changed-path→test-evidence classifier (classifyTestCoverage in src/signals/test-evidence.ts, which the slop detector uses to flag missing coverage), but there's no MCP tool an agent can call to ask "do my changed files carry enough test evidence?" before opening a PR. This surfaces the classifier as a metadata-only tool gittensory_check_test_evidence (paths in → coverage band + guidance out), mirroring the existing gittensory_check_slop_risk source-free self-checks.

Deliverables

  • Zod inputSchema (changedPaths + optional testFiles) in src/mcp/server.ts.
  • Handler calls classifyTestCoverage and returns the coverage-gap classification (strong/adequate/weak/absent) + actionable guidance strings — no source content required.
  • Zod outputSchema for the classification result.
  • Tests: code-only-no-tests (absent), code+tests (strong), both threshold branches (adequate, weak), and docs-only (not-applicable) — covering every classifier band and guidance branch.

Closes #2235

…erage-gap self-check)

Surface the engine's changed-path->test-evidence classifier (classifyTestCoverage,
src/signals/test-evidence.ts) as a metadata-only MCP tool so an agent can ask
whether its changed files carry enough test evidence before opening a PR — paths
in, coverage band (strong/adequate/weak/absent) + guidance out, no source uploaded.
Mirrors the existing gittensory_check_slop_risk source-free self-checks.

Closes JSONbored#2235
@jeffrey701
jeffrey701 requested a review from JSONbored as a code owner July 8, 2026 18:24
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 8, 2026
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.78%. Comparing base (a4e50c6) to head (c2be438).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4244   +/-   ##
=======================================
  Coverage   93.78%   93.78%           
=======================================
  Files         394      394           
  Lines       36619    36637   +18     
  Branches    13402    13406    +4     
=======================================
+ Hits        34342    34360   +18     
  Misses       1622     1622           
  Partials      655      655           
Files with missing lines Coverage Δ
src/mcp/server.ts 95.23% <100.00%> (+0.11%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb

loopover-orb Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-08 18:35:25 UTC

2 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
Adds a new metadata-only MCP tool, gittensory_check_test_evidence, that wraps the existing classifyTestCoverage classifier (src/signals/test-evidence.ts) behind a Zod input/output schema, mirroring the established checkSlopRisk pattern (rate-limited, no source upload). The handler combines changedPaths and optional testFiles into one array, classifies it, and derives guidance text per band; the five added tests exercise all four classifier bands (absent/strong/adequate/weak) plus the docs-only not-applicable path, matching the PR's stated deliverables. This is a narrow, additive change with no schema/migration surface, and it closes the linked issue #2235 directly.

Nits — 5 non-blocking
  • src/mcp/server.ts: testFileCount is computed from `allPaths` (changedPaths + testFiles) via isTestPath, so if a caller passes the same test file in both changedPaths and testFiles it gets double-counted — worth flooring to a de-duplicated set (e.g. `new Set(allPaths)`) since agents may pass overlapping lists.
  • src/mcp/server.ts:858-859: the 400/2000 length caps are unexplained magic numbers; confirm they match checkSlopRiskShape's existing limits and consider extracting a shared constant if so, per the external brief's size-smell note.
  • src/mcp/server.ts is now ~2672 lines per the size-smell check — not something to fix in this PR, but another tool registration is another data point that this file may be due for splitting (e.g. per-tool modules).
  • Consider deduplicating changedPaths/testFiles before computing testFileCount and passing to classifyTestCoverage, to make the tool robust against agents that don't perfectly partition the two lists.
  • If checkSlopRiskShape already defines the same 400/2000 limits, factor them into a shared constant (e.g. MAX_PATH_LEN / MAX_PATHS) referenced by both tools.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #2235
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 113 registered-repo PR(s), 43 merged, 0 issue(s).
Contributor context ✅ Confirmed Gittensor contributor jeffrey701; Gittensor profile; 113 PR(s), 0 issue(s).
Gate result ✅ Passing No configured blocker found.
Linked issue satisfaction

Addressed
The diff adds the gittensory_check_test_evidence MCP tool in src/mcp/server.ts with a zod inputSchema (changedPaths + optional testFiles), a handler that calls classifyTestCoverage and returns the coverage band plus guidance strings, a zod outputSchema, and a new test file covering absent, strong, adequate, weak, and docs-only (not-applicable) cases as required by the issue's deliverables.

Review context
  • Author: jeffrey701
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: TypeScript, Clojure, JavaScript
  • Official Gittensor activity: 113 PR(s), 0 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit a622947 into JSONbored:main Jul 8, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(mcp): add gittensory_check_test_evidence tool (deterministic coverage-gap self-check)

1 participant