[P2] Efficiency regression metrics (calls, retries, latency, tokens)

[cursor]

Summary

Two interfaces can both succeed with very different operational cost. Track tool-call count, failed calls, retries, tokens, latency, optional model cost, wall-clock duration—and flag efficiency regressions relative to baseline.

Priority

priority:p2 — Complements correctness; never replaces it. Distinct from scale/caching work (#100).

Suggested labels

enhancement, priority:p2, area:probes, research

Acceptance criteria

• Efficiency metrics never replace correctness metrics in pass/fail
• Relative regressions easy to read (e.g. tool calls 8→31, latency +94%)
• Cost tracking optional when provider metadata present
• Eval/report EFFICIENCY section when data available
• Tests with fixture traces/metrics

Related

• Parallel probes, result caching, cost tracking, and scale benchmarks #100 parallel/cache/cost, Add unified tool-semantics eval command and report #76 eval, [P1] First-class pass@k / pass^k reliability metrics #106 reliability