Skip to content

fix(agent): per-tool-output cap + anthropic/openai overflow recovery (#193 follow-up) - #196

Merged
justrach merged 1 commit into
mainfrom
fix/193-tool-output-cap
Jul 15, 2026
Merged

fix(agent): per-tool-output cap + anthropic/openai overflow recovery (#193 follow-up)#196
justrach merged 1 commit into
mainfrom
fix/193-tool-output-cap

Conversation

@justrach

Copy link
Copy Markdown
Owner

Follow-up to #195 (issue #193). #195 gave codex/gpt-5.x a pre-send compaction gate + in-turn overflow recovery, but left anthropic and openai without two things codex already had:

  • a per-tool-output cap (only .responses capped function_call_output, via normalizeResponsesHistory), and
  • the in-turn recovery that reclaims room and retries when the backend rejects an over-window request (that lived only in the codex .err branch).

So a dense tool-output burst — or a single uncapped result (a runaway MCP tool, a big fetch on a small-window model) — that slipped past the bytes/4 pre-send estimate still died on those providers.

Two provider-agnostic layers (mirroring the #195 bound + recover split)

1. Per-output cap — agent_compact.capOversizedToolOutputs
At send time, bound any single tool output to Provider.perOutputCap() = context * 2 bytes (~50% of the window in estimated tokens) across all three wire shapes, reusing the existing truncateToolOutput. Window-proportional, so large-context models (any ≥128k window — every existing tool cap, up to webfetch's 256KB, already fits) keep full results untouched; only a result big enough to threaten the window on its own is trimmed, with a marker. Runs in the same pre-loop slot as normalizeResponsesHistory, so the codex WS delta stays consistent (buildBody re-reads the trimmed history after).

2. In-turn recovery for anthropic/openai — agent_request.recoverContextOverflow
isContextOverflow() detects the rejection across wire formats, pins the meter to the window, emergency-trims and retries the request once (shared context_retried guard so a second overflow falls through and never loops). Wired into all three error branches an overflow can surface through — streamed error event, non-streamed {"type":"error"} envelope, and the generic apiErrorMessage path. These formats resend full input each rebuild, so no closeCodexWs re-anchor is needed. The codex branch keeps its own inline copy (it must closeCodexWs) and now shares isContextOverflow for detection.

Verification

  • zig build test 179/179 (adds isContextOverflow, recoverContextOverflow, capOversizedToolOutputs unit tests), zig build + zig fmt clean.
  • 5-lens adversarial review of the diff (control-flow, provider-routing, codex-WS watermark, regression, test-adequacy) → 0 findings. The codex-WS lens (the fix(agent): pre-send compaction gate + in-turn recovery for context-window overflow (#193) #195 bug class) confirmed the in-place cap is safe-by-precedent with normalizeResponsesHistory.

Closes the anthropic/openai half of #193.

Co-Authored-By: blackfloofie 265516171+blackfloofie@users.noreply.github.com

…193 follow-up)

#195 closed the codex/gpt-5.x context-overflow gap but left anthropic and openai
exposed: they had neither a per-tool-output cap (only .responses caps
function_call_output, via normalizeResponsesHistory) nor the in-turn recovery
that reclaims room and retries when the backend rejects an over-window request
(that lived only in the codex .err branch). A dense tool-output burst — or one
uncapped result (a runaway MCP tool, a big fetch on a small-window model) — that
slips past the byte/4 pre-send estimate still died on those providers.

Two provider-agnostic layers, mirroring the #195 split (bound + recover):

1. Per-output cap (agent_compact.capOversizedToolOutputs): at send time bound any
   single tool output to Provider.perOutputCap() = context*2 bytes (~50% of the
   window in est tokens) across all three wire shapes, reusing the existing
   truncateToolOutput. Window-proportional, so large-context models (>=128k
   window: every existing tool cap already fits) keep full results untouched;
   only a result big enough to threaten the window on its own is trimmed, with a
   marker. Guarantees no single output can alone overflow past what emergencyTrim
   can reclaim (it keeps the most-recent outputs verbatim). Runs in the same
   pre-loop slot as normalizeResponsesHistory, so the codex WS delta stays
   consistent (buildBody re-reads the trimmed history after).

2. In-turn recovery for anthropic/openai (agent_request.recoverContextOverflow):
   isContextOverflow() detects the rejection across wire formats, pins the meter
   to the window, emergency-trims and retries the request once (shared
   context_retried guard so a second overflow falls through and never loops).
   Wired into all three error branches an overflow can surface through — streamed
   error event, non-streamed {"type":"error"} envelope, and the generic
   apiErrorMessage path. These formats resend full input each rebuild, so no
   closeCodexWs re-anchor is needed. The codex branch keeps its own inline copy
   (it must closeCodexWs) and now shares isContextOverflow for detection.

Verified: zig build test 179/179 (adds isContextOverflow, recoverContextOverflow,
capOversizedToolOutputs unit tests), zig build + zig fmt clean; a 5-lens
adversarial review of the diff (control-flow, provider-routing, codex-WS
watermark, regression, tests) returned zero findings.

Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant