Skip to content

feat(signals): wire test coverage classification across contributor surfaces - #2680

Closed
luciferlive112116 wants to merge 1 commit into
JSONbored:mainfrom
luciferlive112116:feat/signals-test-coverage-classification
Closed

feat(signals): wire test coverage classification across contributor surfaces#2680
luciferlive112116 wants to merge 1 commit into
JSONbored:mainfrom
luciferlive112116:feat/signals-test-coverage-classification

Conversation

@luciferlive112116

Copy link
Copy Markdown
Contributor

Summary

  • Extend test-evidence with fixture/mock path detection and a shared buildTestCoverageSummary() (strong / adequate / weak / absent).
  • Surface testCoverage on local workspace intelligence and add a new weak_test_coverage slop signal (info-level, weight 10 — does not block alone).
  • Classify open PRs with disproportionately weak test coverage separately from fully missing tests in the contributor open-PR monitor.
  • Update OpenAPI schema and generated openapi.json.

Scope

  • Conventional Commit feature title (feat(signals): …).
  • Focused signals-layer feature with regression tests only.
  • No linked issue — repo linkedIssuePolicy is preferred; self-evident contributor-intelligence feature.

Validation

  • git diff --check
  • npm run typecheck
  • Targeted unit tests (203 passing across signals/local-branch/openapi suites)
  • npm run ui:openapi (OpenAPI contract updated)
  • Full npm run test:ci — CI on GitHub Actions

Safety

  • Deterministic, public-safe signals only; no auth/API/wallet changes.
  • Empty PR file lists still treated as unknown (no false missing_tests).

…urfaces

Extend test-evidence with fixture-path detection and a shared coverage summary, then surface weak/adequate/strong classification in local workspace intelligence, slop assessments, the contributor open-PR monitor, and the OpenAPI contract.

Co-authored-by: Cursor <cursoragent@cursor.com>
@codecov

codecov Bot commented Jul 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.82759% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.10%. Comparing base (b3244d1) to head (3b626d7).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
src/signals/slop.ts 87.50% 0 Missing and 2 partials ⚠️
src/signals/test-evidence.ts 96.55% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2680      +/-   ##
==========================================
- Coverage   96.10%   96.10%   -0.01%     
==========================================
  Files         237      237              
  Lines       26542    26585      +43     
  Branches     9625     9640      +15     
==========================================
+ Hits        25509    25549      +40     
  Misses        424      424              
- Partials      609      612       +3     
Files with missing lines Coverage Δ
src/openapi/schemas.ts 100.00% <ø> (ø)
src/signals/contributor-open-pr-monitor.ts 96.36% <100.00%> (+0.13%) ⬆️
src/signals/local-workspace-intelligence.ts 100.00% <100.00%> (ø)
src/signals/test-evidence.ts 96.87% <96.55%> (-3.13%) ⬇️
src/signals/slop.ts 97.07% <87.50%> (-1.01%) ⬇️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb

loopover-orb Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - reject/close recommended

Review updated: 2026-07-03 05:20:10 UTC

10 files · 1 AI reviewer · 1 blocker · readiness 80/100 · CI failing · blocked

🛑 Suggested Action - Reject/Close

  • AI reviewers agree on a likely critical defect: src/signals/test-evidence.ts:45 computes `testToChangedRatio` as evidence paths divided by all changed paths, so a reachable PR with one source file, one test file, and many docs/config files is incorrectly classified as `weak` instead of measuring coverage against the source/evidence surface. — Resolve the flagged defect, or override if the AI reviewers are mistaken, then re-run the gate.

Review summary
The change correctly centralizes test/fixture evidence classification and wires the new weak coverage category through slop, local intelligence, contributor monitoring, and OpenAPI. The main correctness issue is that the shared ratio uses every changed path as the denominator, so unrelated non-source files can incorrectly downgrade a source change with matching tests into weak coverage. The fixtures-as-evidence behavior and empty file-list handling are otherwise coherent in the visible diff.

Blockers

  • src/signals/test-evidence.ts:45 computes `testToChangedRatio` as evidence paths divided by all changed paths, so a reachable PR with one source file, one test file, and many docs/config files is incorrectly classified as `weak` instead of measuring coverage against the source/evidence surface.
Nits — 5 non-blocking
  • src/signals/contributor-open-pr-monitor.ts:90 calls `missingTestsFromFiles(files)` twice for the same PR; compute the value once next to `weakTestCoverageFromFiles(files)` to avoid drift and repeated classification work.
  • test/unit/test-evidence.test.ts should add a mixed source+test+docs/config case so the intended denominator is locked down and this regression cannot recur.
  • In `src/signals/test-evidence.ts`, base the ratio on `sourcePathCount + evidencePathCount` rather than `changedPathCount`, while still reporting `changedPathCount` for diagnostics.
  • Add contributor monitor coverage for a PR that includes source, matching tests, and unrelated non-code files to prove the monitor does not misclassify it as `weak_test_coverage`.
  • Reuse the computed missing/weak coverage booleans in `buildContributorOpenPrMonitor` before calling both `mapPendingClassToWorkClassification` and `buildNextStepPacket`.

Why this is blocked

  • src/signals/test-evidence.ts:45 computes `testToChangedRatio` as evidence paths divided by all changed paths, so a reachable PR with one source file, one test file, and many docs/config files is incorrectly classified as `weak` instead of measuring coverage against the source/evidence surface.

CI checks failing

  • codecov/patch — 94.82% of diff hit (target 99.00%)
Signal Result Evidence
Code review ❌ 1 blocker 1 reviewer
Linked issue ✅ No-issue rationale PR body explains why no issue is linked.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (size label size:L; no linked issue context).
Validation posture ❌ 5/25 Preflight is holding this PR: the review lane is unavailable, so it is not ready for automated review.
Contributor workload ✅ 10/10 Author activity: 43 registered-repo PR(s), 19 merged, 7 issue(s).
Contributor context ✅ Confirmed Gittensor contributor luciferlive112116; Gittensor profile; 43 PR(s), 7 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: luciferlive112116
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 43 PR(s), 7 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Await review-lane availability.
  • Refresh registry data or choose a registered active repo.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added gittensor gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. labels Jul 3, 2026

@JSONbored JSONbored left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge conflicts + review blockers.

This branch has conflicts that must be resolved
Use the [web editor](https://github.com/JSONbored/gittensory/pull/2680/conflicts) or the command line to resolve conflicts before continuing.

test/unit/test-evidence.test.ts

@JSONbored JSONbored closed this Jul 3, 2026
statxc pushed a commit to statxc/gittensory that referenced this pull request Jul 3, 2026
…ubmissions (JSONbored#2586)

* fix(review): stop the surface lane from auto-closing clean registry submissions

Two bugs in the registry surface-review lane were closing structurally
clean registry submissions:

1. classifyRegistryPrScope already approves a genuine debut-provider
   companion file riding alongside an entry submission (isAllowed
   matches providerFilePattern), but runSurfaceReview threw that away
   and routed any companion file straight to manual review. It now
   validates the companion via the spec's assessProviderEntry and
   combines it with the entry's own assessment: merge only when both
   sides are clean, close if either is invalid. artifactPattern
   companions (generated build output) are still allowed as-is with no
   validation attempted. The entry/base/companion fetches run
   concurrently instead of sequentially.

2. A duplicate_pr_risk finding (severity "warning") escalated into a
   hard blocker by duplicatePrGateMode: "block" was able to
   singlehandedly override a clean, deterministic surface-lane merge
   and force the whole PR closed via applySurfaceGate's unconditional
   union. It now downgrades to a neutral hold instead, mirroring the
   existing AI-judgment-only carve-out, and the held check-run's
   title/summary name the actual reason. This is scoped to exactly
   duplicate_pr_risk (not every warning-severity finding), since
   missing_linked_issue / self_authored_linked_issue /
   manifest_linked_issue_required / manifest_missing_tests are also
   warning-severity but block-mode-escalatable via their own
   independent maintainer-configured gate and must still close
   outright when a maintainer opts into that.

Both were confirmed live against JSONbored/metagraphed: PR JSONbored#2654 (an
entry + debut-provider companion) and PR JSONbored#2680 (a provider-only
resubmission sharing JSONbored#2654's linked issue) now resolve to merge and a
held-for-review neutral hold respectively, instead of reject/close.

* fix(review): confirm a provider companion is actually a debut before merging it

classifyRegistryPrScope identifies a companion by file path alone
(providerFilePattern), which only proves the file is shaped like a
provider submission, not that it's a genuine debut (a brand-new
provider, not an edit to one already registered). runSurfaceReview now
also fetches the companion's base content and only runs it through the
debut-provider merge/close flow when base is absent; a companion that
already exists at base routes to manual instead, since editing an
existing, unrelated provider record alongside an entry submission is a
more sensitive shape that needs a human. The entry and both companion
refs are still fetched in one concurrent round-trip.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants