Reduce default-mode primer overhead and avoid discovery rounds

[justrach]

Problem

After fixing the licensed-CodeDB exact-key route, default-mode Graff now matches Pi's minimal lookup trajectory (2 model calls / 1 tool call), so the remaining fixed cost is much easier to see.

On GPT-5.6 Luna at high effort, with fresh matched fixtures and no --lean:

• Graff primer: 4,108 provider-exact input tokens
• Pi 0.84.1 primer: 1,311 input tokens
• OpenCode 1.15.12 primer: 7,736 input tokens
• Graff composed system prompt: 10,907 characters
• Graff serialized primer request: 20.4 KB

A live ablation replaced only the built-in base policy while retaining normal tools, MCP, skills, presence, and high effort. It remained correct and reduced:

• Provider-exact input: 4,108 → 2,456 (-40.2%)
• Serialized request: 20.4 KB → 12.9 KB (-36.8%)

This proves meaningful headroom exists, but the minimal ablation is not shippable by itself: the production prompt must retain edit safety, verification, worktree ownership, user constraints, release/commit rules, and autonomous completion behavior.

There is a second round-cost issue: deferred-tool guidance still says unloaded tools cannot be called and should be loaded first, while dispatch now auto-loads a confident provider-legal call inline. That stale contract can induce an avoidable load_tool_schemas model/tool round.

Why this approach

Optimize measured context and rounds rather than startup or local tool execution:

• Graff reaches ready in about 0.10s.
• The exact lookup tool takes roughly 1–6ms.
• Provider calls dominate wall time.
• Prompt/schema bytes are paid again on every uncached model call.

The work should proceed as controlled prompt/schema variants with correctness gates, not ad-hoc deletion. Broad repo maps and explicit safety policy have demonstrated value on long tasks, so the goal is task-sensitive or compact representation—not removing capability.

Proposed work

1. Distill duplicated base-prompt prose while retaining every behavioral invariant.
2. Compact heavyweight built-in tool descriptions/schema property prose, especially delegation/workflow metadata.
3. Align deferred-tool guidance with inline auto-load for known names/arguments; retain explicit loading for discovery.
4. Evaluate a task-sensitive repo map (named-file tasks vs broad multi-file work).
5. Compress presence for unrelated repositories while preserving a polished, visible same-worktree ownership callout.
6. Integrate provider-exact cumulative terminal usage into the release branch and SDKs; keep context_tokens clearly separate as a window meter.
7. Add debug-only prompt-component/request-size and redirect-target telemetry.

Acceptance criteria

• Normal/default mode; no requirement to pass --lean.
• Full Tier 1 green, including the 600-line ceiling, reachability, build, unit suite, invariants, and SDK drift.
• Full scripted Tier 2 green.
• Exact-key lookup remains 5/5 correct and exactly 2 model / 1 tool call.
• Existing expensive hidden-correctness tasks do not regress.
• Primer provider-exact input improves by at least 25% from the 4,108-token baseline, measured over matched repetitions.
• No increase in median model/tool rounds on representative code-reading and editing tasks.
• Same-worktree collision ownership remains visible as a polished callout.
• Published v0.0.248 is not mutated; changes ship in a follow-up release.

Measurement

Use a four-arm comparison:

1. Current normal mode
2. Distilled base prompt
3. Compact tool descriptions/schemas
4. Both

For each arm capture provider-exact trajectory input (uncached_tokens + cache_read_tokens), serialized request bytes, latency, model calls, tool calls, and exact correctness. Run primer, exact lookup, broad multi-file implementation, and the complete Tier 2 behavior set.

[justrach]

Implemented and measured the first behavior-safe prompt-distillation tranche on fix/oob-targeted-lookup (default mode, no --lean). No commit or release yet.

What changed

• Distilled src/prompt_text.zig while preserving the pinned safety/behavior invariants and byte-exact snapshot coverage in src/prompt_snapshot_tests.zig.
• Updated the standalone comparison dashboard with the production-prompt candidate results.
• Kept the exact-key native→CodeDB Pro redirect behavior unchanged: one model-facing read_file({path, contains}) still becomes one path-scoped literal Pro search.

Provider-exact results (GPT-5.6 Luna, high effort)

Gate	Before	Distilled candidate	Result
One-call primer input	4,108	3,080	−25.024%
Full-capability prompt snapshot	7,406 chars	2,735 chars	−63.1%
Primer correctness / rounds	5/5 · 1 model / 0 tools	5/5 · 1 / 0	no regression
Primer median wall	—	3.137s	informational
Lookup correctness / rounds	5/5 · 2 model / 1 tool	5/5 · 2 / 1	no regression
Lookup median input	8,428	6,377	−24.3%
Lookup cumulative input (5 runs)	42,133	31,890	−24.3%
Lookup median wall	10.43s	7.141s	noisy but favorable

Final raw runs: /tmp/graff-pi-opencode-oob-v248/distilled-final-results/

Correctness and repository gates

• bash scripts/eval-tier1.sh: green, 1,207 tests.
• atomic_inventory expensive task: 9/9 (4/4 visible + 5/5 hidden), tests unchanged, 84.95s.
• Exact-key scripted Tier 2 case: passes.
• Candidate full scripted Tier 2: 46 pass / 5 fail.
• Untouched HEAD archive full scripted Tier 2: the same 46 pass / same 5 fail.

The five pre-existing failures are:

1. budget-final-call-lands-the-answer
2. uncapped-run-never-sees-landing-note
3. landing-note-fires-exactly-once
4. budget-exhausted-past-the-landing-dies-fatal
5. peer-message-all-broadcasts-device-wide

The first four assertions still expect folded todo_write schemas (and the one-shot case expects a tool outside its deliberately reduced catalog). The peer case expects the older device-wide session:"all" behavior while current peer_channel.zig rejects model broadcasts. These are baseline suite/behavior drift, not new prompt-distillation failures; I did not hide them by weakening assertions in this tranche.

Why this shape

The one-line minimal-policy ablation reached 2,456 tokens but removed safety semantics. This candidate instead clears the issue's 25% primer gate while retaining the explicit rails around read-before-edit, external paths, /rewind, destructive Git, user identity, verification in the target environment, constraints, and autonomous completion. Tool-schema compaction remains a separate arm so we can attribute gains cleanly.

Updated dashboard: benchmarks/graff-pi-opencode-oob-v248-report.html.

[justrach]

Final-prompt correction: I reran atomic_inventory after the last 4-token primer trim, from a fresh fixture. It remains 9/9 (4/4 visible + 5/5 hidden), tests unchanged, in 87.72s. Final artifact: /tmp/graff-pi-opencode-oob-v248/distilled-final-expensive/results/atomic_inventory.grade.json. The earlier 84.95s number was the penultimate prompt candidate; correctness is unchanged.

[justrach]

Already on release/v0.0.258 via bccd9cc (perf(prompt): avoid named-target discovery rounds (#489)). The named-target primer change is an ancestor of 258; no further port needed.