Skip to content

calibration: record rule-fired signals for advisory/off gate modes too, tagged with the resolved mode #8760

Description

@JSONbored

Context

Parent: #8757; feeds everything in #8082. recordConfiguredGateBlockerSignals (src/rules/advisory.ts:1125-1172) and isConfiguredGateBlocker (:1019-1092) only record a RuleFiredEvent when the finding's own gate mode resolved to "block" — but every gate defaults to advisory or off (linkedIssue :1023, aiReview :1034, manifestPolicy :1057, linkedIssueSatisfaction :1075, contentLaneDeliverable :1080, backtestRegression :1083, lockfileIntegrity :1087, cla :1090). Result: the majority of real traffic (advisory findings a human actually saw) records NOTHING, so every backtest corpus, reliability curve, and knob-loosening decision is built from the biased block-mode-only minority. The pattern to generalize already exists: recordGateScoreSignals records quality_gate_score whenever mode !== "off" (:1216), not only "block".

Requirements

  • Record a RuleFiredEvent for every configured-blocker-class finding whenever its gate mode !== "off", with metadata.gateMode = the resolved mode ("block" | "advisory") so downstream consumers can segment.
  • secret_leak stays permanently excluded from raw-context capture (calibration: capture bounded raw context for the remaining isConfiguredGateBlocker codes, excluding secret_leak #8130's boundary) — mode tagging applies, raw context does not.
  • Downstream consumers that assume corpus == block-mode-only (backtest corpus builder, precision computations, knob evaluators) must either segment by gateMode or be explicitly audited as mode-agnostic; document the decision per consumer in the PR.
  • Additive only: no rewriting/migrating existing recorded events.

Deliverables

  • The recording change with mode tagging.
  • The per-consumer segmentation audit (in-PR write-up) + any needed consumer filters.
  • Tests: advisory-mode finding records with gateMode tag; off-mode records nothing; block-mode unchanged.

Test Coverage Requirements

99%+ patch coverage, branch-counted.

Boundaries

No gate-behavior change — this is observation only. Volume note: audit_events write rate rises; keep the existing best-effort .catch(() => undefined) write posture.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions