[P3] Confidence intervals for behavioral metrics

[cursor]

Summary

Do not report tiny score differences as meaningful without uncertainty. Support confidence intervals where trial counts permit (e.g. Baseline 89% ± 3% vs Candidate 87% ± 4%).

Priority

priority:p3 — Research quality; prevents false regressions in scorecards.

Suggested labels

enhancement, priority:p3, research, area:probes, area:docs

Acceptance criteria

• Interval estimation for accuracy / pass@k when n trials sufficient
• Docs warn against calling small overlapping intervals a regression
• Integrated optionally into eval/scorecard JSON
• Unit tests for interval helpers

Related

• [P1] First-class pass@k / pass^k reliability metrics #106 pass@k, Add compatibility scorecard and human-readable change explanations #77 scorecard, [P2] Behavioral Compatibility Horizon metric #110 horizon