fix(agent): pre-send compaction gate + in-turn recovery for context-window overflow (#193) - #195
Merged
Merged
Conversation
…indow overflow (#193) Auto-compaction was gated on last_context_tokens, which only updates from the server's returned usage, so it lags tool output appended *within* a turn. A burst of large parallel tool results pushes this turn's input past the model's window before the between-turns 80% meter (mainloop.zig) ever sees it — codex / gpt-5.x then rejects with "exceeds the context window" and the turn dies. graff already had a local estimator (fullInputEstimateTokens, #174) but only used it to correct the meter after a response, never as a pre-send gate. Two layers, mirroring codex (run_pre_sampling_compact) and opencode (isOverflow-before-each-turn / compactAfterOverflow): 1. Pre-send gate (agent.zig runTurn): estimate the full input locally before every request and compact when it is already >= compactAt(), so we never ship an over-cap request. The gate brackets compactOrRecover with closeCodexWs() on both sides — mid-turn the codex WS holds a prev_id / codex_sent_upto watermark keyed to the pre-trim history, so (a) the first reset lets compact()'s own summary request run against a clean session (a stale prev_id makes the server re-prepend the full pre-trim context and defeat/overflow the summary), and (b) the second re-anchors the next in-turn request() to the trimmed full input instead of a delta referencing dropped messages. This matches how every other compactOrRecover call site is bookended by runTurn's closeCodexWs. 2. In-turn recovery (agent_request.zig): the local estimate is a byte/4 lower bound, so the backend can still reject. On "exceeds the context window", emergency-trim and retry the same request once (guarded to a single retry), then fall through to the original error — the turn recovers invisibly instead of failing. closeCodexWs re-anchors so the retry sends the trimmed full input, not a stale WS delta. Verified: zig build test 176/176 (adds an inputOverCompactThreshold unit test), zig fmt clean; adversarial multi-agent review of the diff (which caught and drove the closeCodexWs-bracket fix above); end-to-end drive against a local mock — a would-be-overflow turn now emits [compacting] -> [history compacted] -> completes instead of dying with "exceeds the context window". Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
justrach
added a commit
that referenced
this pull request
Aug 6, 2026
CI's codex PTY scenario caught the note turn. Investigating it turned up a real gap and corrected a wrong assumption about where the boundary is. THE GAP, now closed. #391 is about a PLANNED rollover: context is filling, so reserve a buffer and spend one call before the window turns over. It is not about a rescue. Once the provider has rejected a request for exceeding the window (last_request_context_overflow), or the meter is at the destructive- recovery boundary where compactOrRecover may drop real history (Provider.nearContextLimit, 95%), the session has demonstrably run out of room and a note is the last thing it can afford. decideRoom becomes decideContext and refuses those outright, BEFORE the token-buffer question — which would have said yes, because at 95% of a 200k window there are still 10k tokens of nominal room. contextOf reads the same two signals compactOrRecover uses to authorize destructive trimming, so the note fires exactly when compaction is scheduled and never when it is damage control. Two unit tests, one pure and one through maybeWrite with a live provider. THE ASSUMPTION, corrected with evidence. run_midturn_compaction_scenario is NOT the recovery path. test-pty-codex-ws.py sets the server meter to 90% of the window and codex_ws_test.py says why: "Cross compact@ (80%) but stay below the destructive recovery boundary (95%)." runTurn's mid-turn gate is inputOverCompactThreshold (80%), and trim_on_fail is false there. So it is the planned rollover, and the single most representative instance of #391 in the tree — a long tool-loop turn crossing the threshold mid-flight, the failure mode the issue was filed over. Suppressing the note there would suppress it on the main path. So the scenario is taught the new shape rather than the note suppressed, and the #195 WS invariant is CHECKED rather than argued: the note turn is a quiet tool-less SSE request that opens no WebSocket (connection_id is None), runs inside runTurn's existing closeCodexWs bracket against a throwaway clone of history, and leaves the choreography untouched — ws_connections stays 2, the post-compaction turn still re-anchors on a fresh socket, and no request carries previous_response_id. Verified by running it, not by reasoning. Both compaction fixtures are re-keyed on the request's last USER TURN instead of its ordinal. An ordinal-keyed fixture re-targets silently when compaction gains a step: the summary reply landed on the note turn, the note reply became the handoff summary, and midturn still went green while proving something else. The transactional scenario failed outright for the same reason. And the scenario now pins #391 end to end on the wire, which no unit test can reach: the note is absent from `instructions` before it is written, present VERBATIM in every request after it, and absent from `input` on the post-compaction turn — state, not conversation, with nothing in the history for the next compaction to paraphrase away. Confirmed load-bearing by disabling the injection and watching it fail. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fixes #193 — a single turn's tool-output burst could push the request input past the model's context window before auto-compaction ever fired, so codex / gpt-5.x rejected with
exceeds the context windowand the turn died. That makes graff a poor harness exactly when it's working hardest (many large parallel tool reads in one turn).Root cause
Auto-compaction was gated on
last_context_tokens, which only updates from the server's returned usage — so it lags tool output appended within a turn. The between-turns 80% meter (mainloop.zig) never saw the mid-turn spike. graff already had a local estimator (fullInputEstimateTokens, #174) but only used it to correct the meter after a response, never as a pre-send gate.Fix (two layers, mirroring codex
run_pre_sampling_compactand opencodeisOverflow/compactAfterOverflow)Pre-send gate (
agent.zigrunTurn): estimate the full input locally before every request and compact when it's already ≥compactAt(), so we never ship an over-cap request. The gate bracketscompactOrRecoverwithcloseCodexWs()on both sides — mid-turn the codex WS holds aprev_id/codex_sent_uptowatermark keyed to the pre-trim history, so the first reset letscompact()'s own summary request run against a clean session (a staleprev_idmakes the server re-prepend the full pre-trim context and defeat/overflow the summary), and the second re-anchors the next in-turnrequest()to the trimmed full input instead of a delta referencing dropped messages. This matches how every othercompactOrRecovercall site is bookended byrunTurn'scloseCodexWs.In-turn recovery (
agent_request.zig): the local estimate is abytes/4lower bound, so the backend can still reject. Onexceeds the context window, emergency-trim and retry the same request once (single-retry guard), then fall through to the original error — the turn recovers invisibly instead of failing.Verification
zig build test→ 176/176 (adds aninputOverCompactThresholdunit test),zig fmtclean.closeCodexWsbracket above and drove that fix, then confirmed the fixed version correct.[compacting]→[history compacted]→ completes, instead of dying withexceeds the context window.Known follow-ups (documented on #193, out of scope here)
TruncationPolicy/ opencodemax_bytes) — the in-turn recovery is codex/.responses-only; a dense burst on anthropic/openai that both undercounts and overflows still dies as before (same as pre-fix baseline).[compacting ~N tokens…]line reads ~0 when the gate fires off the local estimate (the server meter is still 0). Cosmetic.