Ultracode: learn orchestration and tier policy, not only prompt variants

[justrach]

What changed / requested

Extend Ultracode's experiment loop so it can learn or select the orchestration policy—workflow shape and role/tier assignment—instead of evolving only system-prompt/persona overrides inside a statically selected shape.

At minimum, make the boundary explicit in user-facing diagnostics: report which decisions came from the fixed shape catalog, which were authored by the root model, which came from fleet fitness, and why each worker model/tier was selected.

Why

Problem / observed behavior

In Graff 0.0.233, invoking ultracode injects the fixed shape catalog from src/shapes.zig and tells the root model to pick one shape based on task intent. The root then authors the concrete workflow JSON, and the workflow engine executes it.

The experiment/fleet scoring path fingerprints the system-prompt override and attributes judge fitness to that prompt genome plus niche metadata. Shape slot and provider class help partition scoring, but neither the orchestration graph nor the model/tier policy is the learned genome.

That makes the behavior look more learned than it currently is: Ultracode performs a scored multi-agent experiment, but the high-level topology and model routing are prescribed or inherited rather than selected from measured fitness.

A real v0.0.233 research run showed this distinction:

• Ultracode statically mapped the request to sweep -> synthesize.
• The root authored four researcher tasks.
• All researchers, judges, and synthesis ran on the inherited gpt-5.6-sol route.
• The four researchers shared the same prompt fingerprint, and judges scored prompt/persona fitness—not whether Sol, Terra, Luna, or a different workflow shape was the better quality/cost policy.
• An explicit standalone subagent(model: "gpt-5.6-luna") smoke test succeeded, confirming Luna was available but was not selected by the Ultracode experiment.

The stale worker route observed in that session is separately covered by #371. This issue is about the broader learning boundary, not /model recomputation.

Expected behavior

One of these should be explicit:

1. Learned orchestration: treat task class, workflow shape, role, and tier/model policy as experiment dimensions; record quality/cost/latency and allow the fleet to select policies from prior fitness, while retaining safe bootstrap defaults.
2. Narrower contract: clearly describe Ultracode as fixed-shape orchestration with learned prompt/persona variants, and trace every static/root-authored/inherited decision so users do not infer that the fleet selected the shape or model.

A useful trace entry would include, per worker:

shape=sweep
role=researcher
tier=mid
resolved_model=gpt-5.6-terra
source=persona | workflow override | learned policy | session default | ladder
policy_or_genome_id=...

Reason for this approach

Prompt fitness alone cannot answer whether a different topology or cheaper model tier would have produced a better result. Keeping shape/tier as contextual niche metadata preserves fair prompt comparisons, but a separate policy layer can learn routing without incorrectly attributing model effects to the prompt genome (the integrity concern raised in #290).

Constraints / trade-offs

• Do not mix prompt and model effects into one fitness value without preserving attribution.
• Quality should be balanced against token cost and latency, or frontier-heavy policies will dominate quality-only judging.
• Sparse shape × role × tier cells need bootstrap defaults and hierarchical fallback.
• Explicit user model/provider pins must always outrank learned policy.
• Cross-provider routing must preserve existing consent boundaries.

Related issues

• fleet: model tier and shape slot belong in the MAP-Elites niche, not the genome #290 — tier and shape slot as niche metadata rather than prompt genome
• subagents: per-persona and per-spawn model pin (tier in agent frontmatter) #292 — persona/per-spawn model pins
• ultracode: fixed catalog of five default orchestration shapes #293 — fixed Ultracode shape catalog and static tier guidance
• /model leaves the #291 worker default frozen at startup: workers stay on the OLD provider's rung (or never descend at all) #371 — stale automatically derived worker route after /model changes (separate bug)

[justrach]

Landed on main (5e13526, CI run 30819556505 green). Both resolutions the issue asked for are now in place — the boundary is explicit everywhere, and the first real learning increment is live.

Routing trace (every spawn path). subagent tool, workflow phase tasks, and pipeline stages each emit shape= role= tier= resolved_model= source= policy_or_genome_id= to three sinks: tracer notes ({"ev":"route",...} in .graff/traces), the local DGM archive (kind:"route" rows), and an additive agent_route --json event carrying the task label. source distinguishes explicit-pin | persona | learned-policy | workflow-override | session-default | ladder — exactly the static/root-authored/inherited/learned attribution requested. Live-verified twice: the agent's smoke reproduced this issue's original scenario (sweep→synthesize on an inherited route now shows source=session-default per worker), and an independent smoke on a codex-sub machine showed workers seating on gpt-5.6-luna flat-rate with the root-authored genome fingerprint in policy_or_genome_id.

Learned tier policy. Score rows now self-describe (shape, role, tier, model); route_policy.foldCells pools them into (shape, role) cells, blends with the bench sheet (sheet_weight pseudo-counts, cost preserved, spd re-derived), and tier resolution consults exact cell → shape pool → ladder. Guardrails per this issue's constraints: a learned answer may only replace the ladder's if it dominates it (≥ quality, strictly cheaper effective cost — frontier-heavy policies cannot win on quality alone, and the policy can never trade up, by test); explicit model pins bypass the policy entirely (by test); rungAffordable and provider-consent boundaries re-clear. Prompt attribution stays clean (#290): routing is keyed by (shape, role, tier), never attributed to a prompt genome. Deterministic demonstration in-tree: synthetic observations re-seat a dominated mid rung to luna while unobserved cells fall back to today's ladder answer; a round-trip test proves the capture writer and fold reader agree on the wire format.

Held boundary: workflow phase workers are traced and contribute observations but are not yet routed by learned policy — providerClass buckets sol/terra/luna identically, so cross-run prompt-genome comparisons can't yet stratify by rung. That increment (variantProviderClass + phase-uniform routing) is filed as #376. Closing this one.