Let a judge model answer a colony's questions when nobody is there - #145
Merged
Merged
Conversation
Rebuilt on main rather than merged: this branch predated the tree-wide format (#142) and the sessions.rs split (#132), so its diff collided with both — the agent-link code it edits now lives in events.rs, and every hunk fought the reflow. The feature is unchanged; only where it attaches moved. - `autonomy.rs` is taken as it was written. - The open question carries its questions now (`Runtime::open_question` returns `(id, questions)`), so the judge can answer among the labels the agent offered — and the dispatch that fills it is the typed `AgentEvent::Question` arm in events.rs, which already had them. - `Activity::judged`, the `autonomy` module kind and its settings, the config entry, the docs and the web bits are as the branch had them. Verified on this tree: cargo test --workspace 246 passed (240 + this branch's 6), clippy -D warnings clean, cargo fmt --all --check clean, and the web suite 67 passed with tsc clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ParallelEntrepreneur
force-pushed
the
feat/autonomous-judge
branch
from
September 18, 2026 10:24
9b48c59 to
914bcea
Compare
Collaborator
Author
|
Rebuilt on main and force-pushed. The branch predated the tree-wide format (#142) and the sessions.rs split (#132), so it no longer merged: the agent-link code this edits now lives in The feature is unchanged — Verified on the rebuilt tree: |
ParallelEntrepreneur
added a commit
that referenced
this pull request
Sep 23, 2026
Let a judge model answer a colony's questions when nobody is there
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #143.
What
A colony that asks a question stops until a person answers. That is right when someone is watching, and it is the whole cost of the thing when nobody is.
Settings → Modules → Autonomy.
offby default. Set it tojudge, pick a model, and a question nobody has answered goes to that model with the task and the options the agent offered. The colony carries on.Any model, and why the field says what it says
The judge field takes any model, because the right one depends on what you run colonies for. It also says plainly that a frontier model judges best — it is deciding for you, on less context than you have, and the difference shows.
fable,opus,sonnet) goes to Anthropic with the saved Claude credential.provider/modelgoes to a provider you added, on its own API key.Worth knowing before merging: when the saved credential is a subscription token from
claude setup-token, it was issued for Claude Code to use inside a colony. Judging from the Mothership is a use it was not issued for. That is a deliberate choice (asked and confirmed), not an oversight, and it is written into the code comment and the commit so it stays visible.The line it is built around
The judge chooses only among the labels the agent offered, and the reply is checked against them before anything is sent. A colony reads repository content, content can carry instructions, and a judge reading the same content can be steered by it.
free_textis offmax_answersNothing pretends a judged answer is a person's: the colony is told so in the answer itself, and the session log records the model and its reason.
Test plan
cargo test: 178 pass, 6 new — the prompt carries every option; an unoffered label is refused; a question left out is refused; free text refused unless switched on; several labels only where several were invited; prose around the JSON tolerated, prose instead of it not; off until it has a modelmain(12 warnings, none inautonomy.rs);tsc --noEmitandvite buildclean"tests/smoke.rs"→ colony carried on and volunteered that it had been "noted as an automatic stub/judge response in autonomous mode, not an actual maintainer". Status returned toidlewith no attention flag.429, not401— but this account is rate-limited, so no completion came back through it. The provider path is fully exercised; the plain-id path is not.Not in this PR