fix(agent): per-tool-output cap + anthropic/openai overflow recovery (#193 follow-up) - #196
Merged
Conversation
…193 follow-up) #195 closed the codex/gpt-5.x context-overflow gap but left anthropic and openai exposed: they had neither a per-tool-output cap (only .responses caps function_call_output, via normalizeResponsesHistory) nor the in-turn recovery that reclaims room and retries when the backend rejects an over-window request (that lived only in the codex .err branch). A dense tool-output burst — or one uncapped result (a runaway MCP tool, a big fetch on a small-window model) — that slips past the byte/4 pre-send estimate still died on those providers. Two provider-agnostic layers, mirroring the #195 split (bound + recover): 1. Per-output cap (agent_compact.capOversizedToolOutputs): at send time bound any single tool output to Provider.perOutputCap() = context*2 bytes (~50% of the window in est tokens) across all three wire shapes, reusing the existing truncateToolOutput. Window-proportional, so large-context models (>=128k window: every existing tool cap already fits) keep full results untouched; only a result big enough to threaten the window on its own is trimmed, with a marker. Guarantees no single output can alone overflow past what emergencyTrim can reclaim (it keeps the most-recent outputs verbatim). Runs in the same pre-loop slot as normalizeResponsesHistory, so the codex WS delta stays consistent (buildBody re-reads the trimmed history after). 2. In-turn recovery for anthropic/openai (agent_request.recoverContextOverflow): isContextOverflow() detects the rejection across wire formats, pins the meter to the window, emergency-trims and retries the request once (shared context_retried guard so a second overflow falls through and never loops). Wired into all three error branches an overflow can surface through — streamed error event, non-streamed {"type":"error"} envelope, and the generic apiErrorMessage path. These formats resend full input each rebuild, so no closeCodexWs re-anchor is needed. The codex branch keeps its own inline copy (it must closeCodexWs) and now shares isContextOverflow for detection. Verified: zig build test 179/179 (adds isContextOverflow, recoverContextOverflow, capOversizedToolOutputs unit tests), zig build + zig fmt clean; a 5-lens adversarial review of the diff (control-flow, provider-routing, codex-WS watermark, regression, tests) returned zero findings. Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
This was referenced Jul 13, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #195 (issue #193). #195 gave codex/gpt-5.x a pre-send compaction gate + in-turn overflow recovery, but left anthropic and openai without two things codex already had:
.responsescappedfunction_call_output, vianormalizeResponsesHistory), and.errbranch).So a dense tool-output burst — or a single uncapped result (a runaway MCP tool, a big fetch on a small-window model) — that slipped past the
bytes/4pre-send estimate still died on those providers.Two provider-agnostic layers (mirroring the #195 bound + recover split)
1. Per-output cap —
agent_compact.capOversizedToolOutputsAt send time, bound any single tool output to
Provider.perOutputCap()=context * 2bytes (~50% of the window in estimated tokens) across all three wire shapes, reusing the existingtruncateToolOutput. Window-proportional, so large-context models (any ≥128k window — every existing tool cap, up to webfetch's 256KB, already fits) keep full results untouched; only a result big enough to threaten the window on its own is trimmed, with a marker. Runs in the same pre-loop slot asnormalizeResponsesHistory, so the codex WS delta stays consistent (buildBodyre-reads the trimmed history after).2. In-turn recovery for anthropic/openai —
agent_request.recoverContextOverflowisContextOverflow()detects the rejection across wire formats, pins the meter to the window, emergency-trims and retries the request once (sharedcontext_retriedguard so a second overflow falls through and never loops). Wired into all three error branches an overflow can surface through — streamed error event, non-streamed{"type":"error"}envelope, and the genericapiErrorMessagepath. These formats resend full input each rebuild, so nocloseCodexWsre-anchor is needed. The codex branch keeps its own inline copy (it mustcloseCodexWs) and now sharesisContextOverflowfor detection.Verification
zig build test179/179 (addsisContextOverflow,recoverContextOverflow,capOversizedToolOutputsunit tests),zig build+zig fmtclean.normalizeResponsesHistory.Closes the anthropic/openai half of #193.
Co-Authored-By: blackfloofie 265516171+blackfloofie@users.noreply.github.com