From 3768d156282394ba4896f2290e20e92227902494 Mon Sep 17 00:00:00 2001
From: davion-knight <298846663+davion-knight@users.noreply.github.com>
Date: Thu, 9 Jul 2026 15:25:47 -0500
Subject: [PATCH] =?UTF-8?q?docs(miner-selfimprove):=20self-improve=20phase?=
 =?UTF-8?q?=20design=20note=20=E2=80=94=20calibration,=20value-weighting,?=
 =?UTF-8?q?=20anti-farming=20boundaries=20(#4265)?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add packages/gittensory-miner/docs/miner-selfimprove-calibration.md documenting,
for a contributor who only has Phase 7's contributor-safe tools, WHERE the
read-only-measurement / maintainer-only-enforcement line is and why:

- What calibration measures: the Phase7CalibrationLoopResult contract combining
  historical_replay + pr_outcome against the documented 0.62 baseline, and
  computeGateEval's confusion matrix (GateEvalRow, mergeFalse = the dangerous
  error) as the ground truth.
- Value-weighting: why weighted precision (REVERSAL_DISCOUNT_WEIGHT, currently 0)
  measures DURABLE correctness, not volume — a reverted merge earns no credit.
- The maintainer-only line: personalized tuning (#2349), fleet anti-farming
  (#2350), and calibration-gated circuit-breaking (auto-tune.ts hold-only breaker,
  #2352) — and why every contributor-safe tool is read-only/local-only BECAUSE a
  write-capable version would be a farming vector.
- An explicit 'what this phase does NOT do' section (no autonomy/threshold changes,
  no writes to maintainer-owned calibration/circuit-breaker state).

Prose design note under docs/ (matching the coding-agent-driver.md / cross-repo-
discovery-phase1.md convention); cross-links #2348/#2349/#2350/#2352 for context
without reopening them. Every referenced symbol verified against the source.

Closes #4265
---
 .../docs/miner-selfimprove-calibration.md     | 84 +++++++++++++++++++
 1 file changed, 84 insertions(+)
 create mode 100644 packages/gittensory-miner/docs/miner-selfimprove-calibration.md

diff --git a/packages/gittensory-miner/docs/miner-selfimprove-calibration.md b/packages/gittensory-miner/docs/miner-selfimprove-calibration.md
new file mode 100644
index 0000000000..289c120009
--- /dev/null
+++ b/packages/gittensory-miner/docs/miner-selfimprove-calibration.md
@@ -0,0 +1,84 @@
+# Miner self-improve — calibration, value-weighting, and the anti-farming boundary
+
+Phase 7's **contributor-safe** half — the drift detector, the calibration dashboard, the prediction ledger, the
+metrics exporter, the extension panel, and the confusion-matrix regression tests — is deliberately scoped to
+**read-only measurement and detection**. The half that actually *acts* on calibration data (bumps autonomy levels,
+tightens gate thresholds, adjusts per-contributor trust) is **maintainer-only** and lives behind a separate
+milestone.
+
+This note is for a contributor building against the contributor-safe tools. The single most important thing to
+understand is **where the line is and why** — because the natural instinct ("just wire the dashboard's accuracy
+number into an autonomy-level bump") is precisely the maintainer-only side of the phase, and a contributor-safe tool
+that could do it would defeat the safety it exists to provide.
+
+## What calibration measures
+
+The engine contract is `Phase7CalibrationLoopResult`
+([`packages/gittensory-engine/src/phase7-calibration-loop.ts`](../../gittensory-engine/src/phase7-calibration-loop.ts)):
+it combines the **historical-replay** composite score with the passive **`pr_outcome`** signal and tracks accuracy
+against the documented **`DOCUMENTED_CALIBRATION_BASELINE = 0.62`** (the 62% self-review baseline from the Phase 7
+roadmap). The engine owns the deterministic combine / freshness / threshold / hold-reason logic; the miner runtime
+owns scheduling replay runs and persisting ledger rows.
+
+The ground truth underneath is `computeGateEval`'s confusion matrix (`src/review/parity.ts`, `GateEvalRow`): per
+project, how many predictions were `wouldMerge` vs `wouldClose`, and how each resolved against the human decision —
+`mergeConfirmed` (predicted merge, human merged) vs **`mergeFalse` (predicted merge, human *closed* — the dangerous
+error)**, and symmetrically for close. `mergePrecision` / `closePrecision` are the headline accuracy numbers a
+dashboard renders.
+
+## Value-weighting: durable correctness, not volume
+
+Raw merge/close precision is not the real objective, and a contributor reading a dashboard number should understand
+why. `GateEvalRow` also carries `weightedMergeConfirmed` / `weightedCloseConfirmed`: a prediction's credit is
+**discounted by `REVERSAL_DISCOUNT_WEIGHT` (currently `0` — full discount)** when its outcome was later reversed (a
+merge a human undid, or a bot-closed PR a contributor reopened). The denominator (`wouldMerge`) is unchanged — only
+the *credit* for a call that didn't hold up is removed. So the weighted precision measures **durable** correctness:
+a prediction that "won" on the day but was reverted a week later earns nothing. This is maintainer-owned prior work
+(#2348) that the contributor-safe tools *display*; they never recompute or re-weight it.
+
+## The maintainer-only line, and why it's drawn there
+
+Three maintainer-only concepts sit just past the boundary. A contributor-safe tool must never read or write any of
+them:
+
+- **Personalized tuning (#2349).** Per-actor confidence adjustment based on calibration history. A contributor-facing
+  surface that could *see or influence its own personalized-trust score* would defeat the entire point of it — so the
+  contributor-safe tools are, by construction, blind to it.
+- **Fleet anti-farming (#2350).** The whole phase is designed around gaming vectors from day one: inflating
+  merge-precision by only ever opening trivially-safe PRs, or farming duplicate-issue-claim elections. **Every
+  contributor-safe tool in this batch is read-only / local-only *because* a write-capable version of any of them
+  would be a farming vector** — the restriction is the design, not an arbitrary limitation. (The anonymization posture
+  for any telemetry that does leave an instance mirrors `src/selfhost/orb-collector.ts`'s HMAC-with-local-secret
+  approach.)
+- **Calibration-gated circuit-breaking (#2352).** `src/review/auto-tune.ts` already holds a self-tightening safety
+  breaker: when merge precision drops below a floor over a real sample, it **disables its own auto-merge (hold-only)
+  and alerts a human — it only ever makes itself MORE cautious, never loosens**, and a human clears it once fixed.
+  That breaker is the eventual consumer of miner-sourced calibration signal (via `computeGateEval`'s `source`
+  scoping), but the *enforcement* is maintainer-only. The contributor-safe tools only ever **produce read-only signal**
+  for that maintainer-owned system to act on.
+
+## What this phase does NOT do
+
+From any contributor-safe tool in this batch:
+
+- **No autonomy-level changes** — measuring accuracy never bumps an autonomy level (that gate is fail-closed and
+  maintainer-owned).
+- **No gate-threshold changes** — the dashboard/exporter/ledger surface numbers; they never tune the gate.
+- **No writes to maintainer-owned calibration or circuit-breaker state** — no touching personalized-trust scores, the
+  auto-tune hold-only flag, or any autonomy configuration.
+- **No de-anonymizing or write-capable telemetry** — local-only or HMAC-anonymized, read-only.
+
+## The neighborhood
+
+- Contributor-safe (this batch): drift detector, calibration dashboard + extension panel, prediction ledger, metrics
+  exporter, confusion-matrix regression tests — all read-only measurement.
+- Maintainer-only (cited for context, **not** to be reopened or reimplemented here): #2348 (value-weighted
+  calibration, merged), #2349 (personalized tuning), #2350 (fleet anti-farming), #2352 (wiring calibration accuracy
+  into the live auto-tune breaker).
+
+## References
+
+- [`packages/gittensory-engine/src/phase7-calibration-loop.ts`](../../gittensory-engine/src/phase7-calibration-loop.ts) — the calibration-loop contract + `DOCUMENTED_CALIBRATION_BASELINE`.
+- `src/review/parity.ts` — `GateEvalRow` (the confusion matrix) and `REVERSAL_DISCOUNT_WEIGHT` (value-weighting).
+- `src/review/auto-tune.ts` — the existing hold-only accuracy circuit-breaker (#2352's eventual consumer).
+- `src/selfhost/orb-collector.ts` — the anonymized HMAC-with-local-secret telemetry pattern.