Skip to content

calibration: implicit-confirmation override signals for non-AI-judgment rules (merge-without-reversal => confirmed) #8761

Description

@JSONbored

Context

Parent: #8757; feeds #8082. Today only recordAiJudgmentHoldConfirmations (src/review/outcomes-wire.ts:561-573, #8123) ever writes a "confirmed" HumanOverrideEvent, and only for ai_consensus_defect / ai_review_split. Every other rule's overrides are reversal-only (recordConfiguredGateBlockerOverrides :504-523, recordLinkedIssueScopeMismatchOverride :536-549), so computeRulePrecision (confirmed/decided) trends toward 0 for those rules as data accumulates — corrupting the backtest_regression gate, knob ladders, and public-rule-precision surface regardless of the rules' real accuracy.

Requirements

  • Extend the calibration: record a 'confirmed' HumanOverrideEvent when an owner closes an aiReviewLowConfidenceHold PR #8123 implicit-confirmation pattern: when a terminal human disposition consistent with the rule's verdict occurs and the fired signal was never reversed, record verdict: "confirmed" for that (ruleId, targetKey). Concretely: owner CLOSES (not merges) a PR a blocker fired against => confirmation for that blocker, mirroring the AI-judgment case; define and document the per-rule terminal-event mapping explicitly in code comments.
  • Never double-record: a (ruleId, targetKey) that already has a reversal or confirmation stays single-verdict (mirror the existing idempotence approach in outcomes-wire.ts).
  • Weak-evidence honesty: tag implicit confirmations metadata.basis = "implicit_terminal_disposition" so downstream consumers can weight or exclude them.

Deliverables

  • The confirmation writer + per-rule terminal-event mapping.
  • Idempotence + basis tagging.
  • Tests: confirmation recorded on terminal disposition; no double-record after reversal; basis tag present; AI-judgment codes unaffected (still handled by their existing writer).

Test Coverage Requirements

99%+ patch coverage, branch-counted.

Boundaries

Additive event writes only. No changes to how precision is COMPUTED in this issue — consumers learn to read basis tags in their own follow-ups if weighting is wanted.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions