A capacity refusal parks the run instead of ending it - #453
Conversation
A launch refused for capacity after the in-session retry now parks the run in the capacity wait, keeping its session, candidate and meters; the next wake resumes the session with the refusal and a note that capacity may now be free. Other refusals and stale submits are unchanged.
There was a problem hiding this comment.
Round 1 — reviewed head 9c3fbd9b — reviewer summarizer:hermes/gpt-5.6-terra over coverage+credentials+deployment+general+lifecycle+prose.
terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.
Verdict: 1 blocking, 0 advisory.
Capacity waits from non-resumable authors end on their first wake. [coverage+lifecycle+deployment] The capacity-refusal path parks when _can_resume() is false, but the scheduled wake rejects the same harness when supports_resume is false and ends the run as session-error instead of retrying capacity. (src/outerloop/attempt.py:1583; high confidence)
Merged one blocking finding: coverage, lifecycle, and deployment agree that a capacity refusal from a non-resumable author is parked but becomes session-error at its first wake. Rejected findings: none; the three reports were duplicates of the same issue, merged with combined lens attribution.
There was a problem hiding this comment.
Round 1 — reviewed head e1ab3dfb — reviewer hermes/gpt-5.6-terra.
terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.
Verdict: 1 blocking, 0 advisory.
1 finding attached to the lines below.
Capacity refusals can still end runs for authors without session resume support.
There was a problem hiding this comment.
Round 2 — reviewed head 8567a2e5 — reviewer hermes/gpt-5.6-terra.
terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.
Verdict: 1 blocking, 0 advisory.
1 finding attached to the lines below.
Non-resumable authors cannot use the new capacity-wait flow.
|
Compatibility statement (added for the 0.3.0rc1 release audit, per RELEASING.md).
|
A run ended because its launch was refused for capacity. Live case: a parked run woke and its author asked for a 2-GPU sweep, which the operator GPU limit refused (the target's 4 GPUs were held by other slots). The kernel resumed the session once with the refusal. The author re-staged the sweep at 1 GPU and slept; that was refused too, so the request, including the sleep, was dropped ("measuring as-is"). The leg ended "ended without a submit" and the run ended as negative-result. The author did nothing wrong, and the project's rule (#442) is that admission refusals never end a run.
CapacityErrorand the kernel will not resume the session again (after the in-session refusal, or when it cannot resume), the run now parks in the existing capacity wait (RunParked(..., capacity_wait=True)). It keeps its session, sealed candidate and meters, and nothing is charged.Compatibility: no new persisted state.
capacity_waitalready exists in the stage and the inbox note uses the existing message format, so no backfill or fixture is needed; a rolled-back kernel handles a capacity-wait park as it does today.Review: built by Codex; I reviewed it and moved the capacity park after the stale-submit check so that case keeps its outcome. Tests: two capacity refusals in one leg park with the same session and meters, and the next wake delivers the note; repeated budget refusals keep the existing ending; capacity-wait wakes don't count toward stuck. Gate: pytest, ruff check, ruff format --check, mypy.