feat(services): miner-vs-human cohort split for the gate-precision dashboard card - #4623
Conversation
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
❌ 1 Tests Failed:
View the full list of 1 ❄️ flaky test(s)
To view more test analytics, go to the Test Analytics Dashboard |
|
Caution 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 🛑 Gittensory review result - fixes requiredReview updated: 2026-07-10 07:56:39 UTC
🛑 Suggested Action - Manual Review
Review summary Nits — 7 non-blocking
CI checks failing
Linked issue satisfactionPartially addressed Review context
Contributor next steps
Signal definitions
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
…shboard card (#4520) Adds an additive, opt-in miner-vs-human split to loadGatePrecisionReport (#2191) -- no new migration, no identity added to any privacy-scoped table. pull_requests already carries authorLogin (unlike the identity-free review_audit), so the split is computed entirely at read time: fetch the full confirmed-miner login set once (fetchOfficialGittensorMinerLogins, new), then classify each gate-block outcome's PR author against it. ?includeCohorts=true opts a maintainer-dashboard request into the extra Gittensor API call; every existing caller (no param) stays byte-identical. An unresolvable/unconfirmed author always falls to "human" -- never over-classified as a miner. Scope note: the issue also names reversal-rate and findings-by-category as stretch targets ("at minimum" gate-precision). reversal-rate's dashboard- card wiring isn't as concretely defined as gate-precision's (its clearest existing implementation, computeAgentHealth, is consumed by the internal ops-alerting system, not a confirmed dashboard endpoint), and findings-by-category has no repo-wide aggregate to extend yet (only per-PR MCP counts from #4519). Deferred rather than guessed at. Fixes #4520
a76409c to
2e15a6b
Compare
…igest (#4521) Extends MaintainerRecapRepoInput/RecapReport with an additive, opt-in miner-vs-human split, sourced entirely from the upstream GatePrecisionReport cohorts field (#4520) -- no independent identity read, no new privacy surface. A repo's recap entry gets a `cohorts` field whenever its injected GatePrecisionReport carried one; the aggregate totals.cohorts sums only across repos that did, so a window mixing cohort-aware and legacy call sites still degrades gracefully instead of half-reporting zeros. runMaintainerRecapJob now opts loadGatePrecisionReport into includeCohorts by default -- a periodic digest is exactly the "occasional aggregate view" that option was designed for, unlike a hot webhook path. formatMaintainerRecap gains an additive "## Cohorts" section, rendered only when totals.cohorts is present, sitting between Totals and Per-repo. Stacks on #4520 (branch feat/gate-precision-cohort-split-4520, PR #4623) -- GatePrecisionReport.cohorts doesn't exist without it. Do not merge before #4623. Fixes #4521
…igest (#4521) (#4625) Extends MaintainerRecapRepoInput/RecapReport with an additive, opt-in miner-vs-human split, sourced entirely from the upstream GatePrecisionReport cohorts field (#4520) -- no independent identity read, no new privacy surface. A repo's recap entry gets a `cohorts` field whenever its injected GatePrecisionReport carried one; the aggregate totals.cohorts sums only across repos that did, so a window mixing cohort-aware and legacy call sites still degrades gracefully instead of half-reporting zeros. runMaintainerRecapJob now opts loadGatePrecisionReport into includeCohorts by default -- a periodic digest is exactly the "occasional aggregate view" that option was designed for, unlike a hot webhook path. formatMaintainerRecap gains an additive "## Cohorts" section, rendered only when totals.cohorts is present, sitting between Totals and Per-repo. Stacks on #4520 (branch feat/gate-precision-cohort-split-4520, PR #4623) -- GatePrecisionReport.cohorts doesn't exist without it. Do not merge before #4623. Fixes #4521
Summary
loadGatePrecisionReport(the feat(ui): gate-precision analytics card (precision + confusion matrix) #2191 gate-precisioncard) — the existing blended
perGateType/overalloutput is completely unchanged; a newcohortsfieldis only present when explicitly requested.
pull_requestsalready carriesauthorLogindirectly (unlikereview_audit, which is deliberately identity-free by design) — the splitis computed entirely at read time by cross-referencing each gate-block outcome's PR author against a
confirmed-miner login set.
fetchOfficialGittensorMinerLogins(new,src/gittensor/api.ts) fetches the full miner list in ONE call,for callers that need to classify many distinct submitters at once — the existing
fetchOfficialGittensorMiner(login)re-fetches per login (fine for a single webhook-triggered lookup, notfor a dashboard read touching many PRs' authors). Fail-safe: any fetch failure degrades to an empty set
(every author reads as human), never throws.
GET /v1/repos/:owner/:repo/gate-precision?includeCohorts=trueopts a maintainer-dashboard request intothe extra Gittensor API call; every existing caller (no param) stays byte-identical, including zero extra
network calls.
human— never over-classified as a miner.Scope note: the issue names reversal-rate and findings-by-category as additional targets ("at minimum"
gate-precision). I looked at both:
computeAgentHealthinsrc/review/ops.ts) is consumedby the internal ops-alerting system (
src/review/alerts.ts), not a confirmed maintainer-dashboardendpoint — extending it risked building against the wrong surface.
Rather than guess at unconfirmed wiring, I'm shipping the concretely-defined, explicitly-named floor
("at minimum... gate-precision") as its own focused PR. Happy to pick up the other two as follow-ups once
their actual dashboard surface is confirmed.
Fixes #4520
Test plan
test/unit/gate-precision.test.ts: 9 new cases — cohort split absent by default (byte-identical),miner/human split with MIN_SAMPLE applied independently per cohort, unresolvable-author fail-safe to
human, case-insensitive login matching, no-identity-leak privacy check,
loadGatePrecisionReport'sincludeCohortsopt-in (fetches the miner list exactly once, zero calls when omitted), and aGittensor-API-failure fail-safe (empty set, not a thrown error)
test/unit/gittensor-api.test.ts: 4 new cases forfetchOfficialGittensorMinerLogins— one-callbatch fetch, missing-
githubUsernameentries skipped, and fail-safe-to-empty-set on both an HTTPerror and a network-level rejection
test/integration/api.test.ts: extended the existing/gate-precisionroute coverage with?includeCohorts=true(reuses the test's own already-stubbed/minersendpoint)gate-precision.ts,gittensor/api.ts,api/routes.ts): zero uncovered lines or branches in this diffnpx tsc --noEmitclean;npm run ui:openapi:checkclean (this route isn't part of the generatedOpenAPI surface, confirmed via a clean check with no diff);
npm run db:migrations:checkclean (nomigrations touched)
origin/mainat push time;git diff --checkclean