Skip to content

Refuse out-of-scope launches and submits instead of ending the run - #442

Merged
renmengye merged 2 commits into
mainfrom
fix/scope-refuse
Sep 30, 2026
Merged

renmengye merged 2 commits into
mainfrom
fix/scope-refuse

Conversation

@renmengye

@renmengye renmengye commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

An author that leaves a stray file in its tree (a report, a log, train.py.orig) or touches a protected path had its run ended as scope-violation the moment it launched or submitted, discarding every GPU-hour the run had spent and never reaching the judges. On one deployment this ended about 6% of runs, almost all of them harmless leftovers.

What changes

  • One admission check runs before every snapshot, launch, submit and stale-submit checkpoint. Out-of-scope paths refuse the request, every time: nothing is snapshotted, launched, measured or charged, and the author is resumed with a plain kernel note, e.g. "Refused: this request at submit changes paths the contract does not allow authors to change: . Nothing ran and nothing was charged. Authors may change: <the contract's allowed paths>." Repeats say "Refused again:". Refusals never end the run; loops are bounded by the session's walltime and the contract's sleep, launch and GPU-hour budgets.
  • Scope is checked even when the request has another problem (budget, malformed), so a combined request is refused for scope too.
  • A rejected tree is never sealed, snapshotted or pushed on any path (stop, timeout, outage, error); an author that abandons a refused tree ends as a normal no-improvement.
  • Plain submits are now checked at admission too; before, only the measurement-time check (terminal) covered them.
  • The measurement-time check in measure_and_decide stays terminal as a backstop, so an out-of-scope tree is still never executed or measured.
  • Refusals are recorded in the run's inbox like other kernel refusals, so attempts to touch protected paths remain auditable.

Compatibility (RELEASING.md)

Scope refusals use existing kernel-note payloads and refusal keys; no persisted fields or formats change. Legacy v1/v2 inbox coverage verifies repeated reads and interrupted-append retries. Existing messages and ended runs are unchanged; in-flight runs use the new admission behaviour on their next request. No backfill. Rollback is safe and restores terminal scope checks.

Tests

Launch, submit and stale-checkpoint violations refuse and resume with the paths and allowed scope listed; repeated violations keep refusing (mutation-checked) and a clean retry proceeds; nothing snapshotted, launched, measured or charged on refusal (mutation-checked); abandoned, timeout, outage and error endings never seal a rejected tree; the measurement backstop stays terminal; endpoint loss while delivering a refusal does not seal the rejected tree. Gate: 2611 passed, 6 skipped; ruff, format, mypy clean.

Built by codex from my brief; my cross-review before this PR.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 1 — reviewed head 620dfccd — reviewer summarizer:hermes/gpt-5.6-terra over coverage+credentials+deployment+general+lifecycle+prose.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: 3 blocking, 0 advisory.

2 findings attached to the lines below.

A first scope refusal still ends the run when the author ends its session. [deployment] After delivering the first refusal, the no-request path reaches this unconditional scope check and returns scope-violation even though no second out-of-scope request was accepted; the new end-retry tests fail with scope-violation instead of no-improvement. (src/outerloop/orchestrator.py:1911; high confidence)

Three blocking issues remain: refusal-delivery outages can publish a rejected out-of-scope tree; combined invalid-syscall and out-of-scope requests can evade the consecutive-refusal bound; and ending immediately after one refusal is still treated as a scope violation. Rejected findings: none; the credentials, lifecycle, and prose lenses reported no findings.

Comment thread src/outerloop/orchestrator.py Outdated
Comment thread src/outerloop/orchestrator.py
…llowed scope; rejected trees never sealed

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@renmengye renmengye added the autoresearch:review Request a fresh advisory review of this PR's current state label Sep 30, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 1 — reviewed head 2630a39d — reviewer hermes/gpt-5.6-terra.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: 1 blocking, 0 advisory.

1 finding attached to the lines below.

A refusal-delivery error can still seal a rejected tree.

Comment thread src/outerloop/orchestrator.py
@renmengye

Copy link
Copy Markdown
Member Author

Merging over the one deferred round-2 finding, with the code owner's approval. The residual path (an exception at the start of _resume right after a scope refusal) can at worst put a rejected tree on the agent's own line branch; nothing runs or merges from there, and the next admission refuses the same paths again. Closing it needs containment around the whole attempt loop; kept as a follow-up rather than part of this change.

@renmengye
renmengye merged commit 7830baa into main Sep 30, 2026
5 checks passed
@renmengye
renmengye deleted the fix/scope-refuse branch September 30, 2026 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autoresearch:review Request a fresh advisory review of this PR's current state

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant