Refuse out-of-scope launches and submits instead of ending the run - #442
Conversation
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Round 1 — reviewed head 620dfccd — reviewer summarizer:hermes/gpt-5.6-terra over coverage+credentials+deployment+general+lifecycle+prose.
terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.
Verdict: 3 blocking, 0 advisory.
2 findings attached to the lines below.
A first scope refusal still ends the run when the author ends its session. [deployment] After delivering the first refusal, the no-request path reaches this unconditional scope check and returns scope-violation even though no second out-of-scope request was accepted; the new end-retry tests fail with scope-violation instead of no-improvement. (src/outerloop/orchestrator.py:1911; high confidence)
Three blocking issues remain: refusal-delivery outages can publish a rejected out-of-scope tree; combined invalid-syscall and out-of-scope requests can evade the consecutive-refusal bound; and ending immediately after one refusal is still treated as a scope violation. Rejected findings: none; the credentials, lifecycle, and prose lenses reported no findings.
…llowed scope; rejected trees never sealed Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Round 1 — reviewed head 2630a39d — reviewer hermes/gpt-5.6-terra.
terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.
Verdict: 1 blocking, 0 advisory.
1 finding attached to the lines below.
A refusal-delivery error can still seal a rejected tree.
|
Merging over the one deferred round-2 finding, with the code owner's approval. The residual path (an exception at the start of |
An author that leaves a stray file in its tree (a report, a log,
train.py.orig) or touches a protected path had its run ended asscope-violationthe moment it launched or submitted, discarding every GPU-hour the run had spent and never reaching the judges. On one deployment this ended about 6% of runs, almost all of them harmless leftovers.What changes
measure_and_decidestays terminal as a backstop, so an out-of-scope tree is still never executed or measured.Compatibility (RELEASING.md)
Scope refusals use existing kernel-note payloads and refusal keys; no persisted fields or formats change. Legacy v1/v2 inbox coverage verifies repeated reads and interrupted-append retries. Existing messages and ended runs are unchanged; in-flight runs use the new admission behaviour on their next request. No backfill. Rollback is safe and restores terminal scope checks.
Tests
Launch, submit and stale-checkpoint violations refuse and resume with the paths and allowed scope listed; repeated violations keep refusing (mutation-checked) and a clean retry proceeds; nothing snapshotted, launched, measured or charged on refusal (mutation-checked); abandoned, timeout, outage and error endings never seal a rejected tree; the measurement backstop stays terminal; endpoint loss while delivering a refusal does not seal the rejected tree. Gate: 2611 passed, 6 skipped; ruff, format, mypy clean.
Built by codex from my brief; my cross-review before this PR.
🤖 Generated with Claude Code