feat(review): add a global cross-repo blended contributor trust score - #7711
Merged
Conversation
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
…r true created_at backfillContributorGateHistory bound nowIso() instead of the source review_audit row's own created_at, so a reconstructed row silently claimed the decision happened the moment the backfill ran. computeContributorGateEval's rolling-window filter and any day-bucketed view both need the real historical date to age rows out correctly.
computeContributorGateEval and contributorFairnessFlags only ever scored a contributor per-(login, project) -- there was no single figure summarizing a login's accuracy across every repo in the shared self-hosted install. Adds computeBlendedContributorGateEval and contributorGlobalFairnessFlags, folding the same contributor_gate_history/review_audit cells by login alone and pooling raw prediction/outcome counts across every fairness-analytics- eligible project before computing one volume-weighted precision figure -- not an average of each project's own accuracy, so a login mostly active on one high-volume repo isn't distorted by a thin-sample row elsewhere. Reuses the exact REVERSAL_DISCOUNT_WEIGHT credit semantics and per-repo opt-out. Wires the blend into getContributorTrustProfile (blendedGateAccuracy), both internal fairness routes, and a new operator-dashboard tile -- all counts- only, same bearer-gated/internal-only posture as the per-project data. Adds a structural import-reachability test asserting the per-login module family (including the new blended functions) is never reachable from a public- facing entry point.
JSONbored
force-pushed
the
fix/global-contributor-trust-score
branch
from
July 21, 2026 10:26
e6c9893 to
635cf54
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7711 +/- ##
==========================================
+ Coverage 88.56% 89.36% +0.79%
==========================================
Files 83 88 +5
Lines 20664 23323 +2659
Branches 3648 4506 +858
==========================================
+ Hits 18302 20842 +2540
- Misses 2184 2211 +27
- Partials 178 270 +92
Flags with carried forward coverage won't be shown. Click here to find out more.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
computeContributorGateEval/contributorFairnessFlagsonly ever scored a contributor per-(login, project); this addscomputeBlendedContributorGateEval/contributorGlobalFairnessFlags, which pool the same underlying prediction/outcome counts across every repo a login has touched into one volume-weighted accuracy figure (not an average of each project's own precision, so a login mostly active on one high-volume repo isn't distorted by a thin-sample row elsewhere).getContributorTrustProfile(blendedGateAccuracy), both internal/v1/internal/fairness/contributors*routes, and a new operator-dashboard tile — counts-only, same bearer-gated/internal-only posture as the existing per-project data.backfillContributorGateHistory: it boundnowIso()instead of the sourcereview_auditrow's owncreated_at, so a backfilled row silently claimed the decision happened the moment the backfill ran instead of when it actually happened.public-stats.ts,orb-collector.ts'sexportOrbBatch).Backfill data note
Verified against the live self-hosted DB:
contributor_gate_historypredictions are only reconstructable back to ~2026-07-03 (whengate_decisionwrites were fixed for self-host in a prior commit); ground truth (pr_outcome) goes back to 2026-06-20. The ~13-day window before 2026-07-03 has no predicted-decision data recorded anywhere and is not recoverable. Once this PR is deployed, I'll re-run the (now-fixed) backfill route to pull in the reconstructable ~2026-07-03→07-08 window.Test plan
npx tsc --noEmitcleannpm audit --audit-level=moderateclean