Skip to content

ultracode: fixed catalog of five default orchestration shapes #293

Description

@justrach

Problem

ultracode leaves the orchestration shape entirely to the model: it authors whatever workflow JSON it feels like. Results vary run to run, the fleet cannot tell which structure produced a score, and there is no house style for "how do we review" or "how do we research".

Proposal: a fixed catalog of five shapes, selected by task intent

A - Review / audit (default when there is a diff)

phase "find"       -> N reviewers, one per dimension (correctness, security, perf, tests), parallel
phase "verify"     -> skeptic per finding, prompted to REFUTE; when: "FINDING"
phase "synthesize" -> 1 task, ranked output

B - Research / understand (default for a question)

phase "sweep"      -> researchers, each searching a DIFFERENT way (by symbol, by call-graph,
                      by git history, by tests) - blind to each other
phase "synthesize" -> one map

C - Design / solve (already exists via scoreVariants)

phase "variants" -> N implementers with different system_prompt (MVP-first, risk-first, perf-first)
                    on the SAME task -> auto-judged -> DGM scoring round
phase "build"    -> synthesize from winner, graft runners-up

D - Migration / mechanical (pipeline, never phases)

pipeline { items: discovered sites, stages: [transform, verify], isolation: worktree }

E - Build a feature

phase "scope"     -> 1 researcher
phase "implement" -> implementers in worktrees, parallel by component
phase "review"    -> reviewer + skeptic on the diff

Tier assignment

One rule across every shape: frontier for judgment (scope, judge, verify, synthesize), mid for production (search, implement, transform), small for mechanical. Depends on the tier ladder and per-persona pin issues.

Scale knob

"find bugs" -> 3 finders, 1 verify. "thoroughly audit" -> 6 finders, 3-vote adversarial, synthesis phase.

Relationship to the fleet

Each shape slot (find / verify / synthesize / transform / ...) is emitted as the third component of the composite niche key from the archive-integrity issue. The catalog is the scaffold evolution runs inside; the slot is a coordinate of the space it illuminates.

Shapes themselves are deliberately not evolved yet - a shape is a program, not a prompt, and the sample rate would be far too low to learn anything. Once there is volume, shapes get their own coarser evolution keyed on eval_set_hash (already computed per task), reusing the same score stream one level up.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions