fix(agent): context-overflow follow-up — recovery-path parity + local-model wedge (#201-#204) - #207
fix(agent): context-overflow follow-up — recovery-path parity + local-model wedge (#201-#204)#207justrach wants to merge 3 commits into
Conversation
…202) The #193 follow-up (b227ef8) added the per-output cap + in-turn overflow recovery for anthropic/openai but left four holes the codex/.responses path had already closed (#174/#175/#195). This brings the other providers to parity and makes truncation visible everywhere. G1 (#201) — recent-output wedge. perOutputCap() was context*2 bytes (~50% of the window each), but trimOldestToolOutputs keeps keep_recent=4 outputs verbatim, so four recent large results pinned ~2x the window past what recovery could reclaim -> hard wedge. Cap is now ~1/8 of the window (context/2 bytes) with a 256 KB absolute ceiling, so 4 recent outputs occupy ~50% of the window and stay reclaimable; the ceiling also bounds one pathological result on huge windows (codex uses a fixed cap with no recency exemption). G3 (#202) — recordUsage froze the meter when the provider omitted usage (early `orelse return`), so the between-turns compaction gate could never fire. It now floors last_context_tokens at max(fullInputEstimateTokens, req_body_len/4), the same fallback recordUsageResponses already applies (#174). recordUsage now takes req_body_len; both call sites pass body.len. G6 (#202) — capOversizedToolOutputs truncated silently (call site discarded the count). It now emits a tracer note + an interactive status line when it fires; the model already saw the inline marker. G10 (#202) — every in-place trim zeroed the meter, opening a transient blind window and clobbering an overflow recover-pin. Both trim sites now re-estimate (fullInputEstimateTokens) instead of setting 0. fullInputEstimateTokens is now member-aliased on Agent so agent_compact can call it. Tests: +perOutputCap (#201), +recordUsage-fallback (#202); the trimOldest / capOversized tests now assert the re-measure contract. zig build test 181/181, zig fmt clean. Closes #201 Closes #202 Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
…red overflow detection + metering parity (#203, #204) Follows ea230cc. Closes the local/unknown-model wedge cluster (#203) and the two metering-parity items (#204). Together with #201/#202 this breaks the compound wedge chain end to end (bad window guess -> undercount -> gate never fires -> needle miss -> no recovery). G8 (#203) — unknown/local models defaulted to a 200k guess, mis-sizing every cap and threshold. GRAFF_CONTEXT / GRAFF_CONTEXT_WINDOW now declares the real window; contextWindowFor applies it ONLY when the model isn't catalogued (new pricing.isKnownModel), so a global override can't shrink a known model that happens to be 200k. Root fix — cascades to compactAt, perOutputCap, and the gate. G4 (#203) — the pre-send gate estimate omitted the system prompt + tool schemas and under-fired. inputOverCompactThreshold now adds a baseline for that fixed prefill, clamped to 1/8 of the window so it can't dominate a small local window. Kept OUT of fullInputEstimateTokens, which must stay pure over self.messages for the unit tests. G2 (#203) — overflow was detected only by ~6 English substrings. isContextOverflow now matches the structured error.code (context_length_exceeded / context_window_exceeded) first, so a local or non-English provider whose message differs still recovers; substrings remain the fallback. error.code is extracted at the two envelope branches (streamed + non-streamed); codex/.responses and the generic path pass null (their English messages match). HTTP-status detection was intentionally not added — codex itself reads the code from the stream, not a 400, and the code check already covers the local-provider case. G5 (#204) — a same-format provider switch kept the previous model's absolute token count against the new window. applyProviderInner now re-estimates the meter from the (kept/translated) history after the switch. G9 (#204) — the 80% compaction threshold was hardcoded. GRAFF_COMPACT_PCT overrides it (1..100), and unlike codex's one-directional clamp it may lower OR raise. Tests: +contextWindowFor (#203), +compactAt-override (#204); isContextOverflow gains structured-code cases; recoverContextOverflow / isContextOverflow call sites updated for the new code param. zig build test 183/183, zig fmt clean. Closes #203 Closes #204 Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
… test (#203) Extends the G2 work in 3a603d0. The structured-code detection covered the anthropic-style {"type":"error"} envelopes, but the GENERIC apiErrorMessage path — where openai-compatible errors ({"error":{...}}, no top-level "type") actually land — still passed null for the code. So a local provider (LM Studio, mlx, any openai-compat base_url) whose message text we don't match on would NOT be recognized as an overflow. New errorCode(root) pulls root.error.code (or a top-level root.code) and threads it into recoverContextOverflow there. E2E test (scripts/test-pty-overflow.py): points graff at a mock lmstudio backend (127.0.0.1:1234) that injects an error and drives a real terminal turn. A. code=context_length_exceeded with a Dutch message (matches no English substring) -> graff pins the ctx meter to the window (200k/200k, 100%), proving detection went through the structured code, and stays responsive (runs /help, exits cleanly) instead of wedging. B. code=rate_limit_exceeded, non-overflow message -> meter must NOT pin, proving detection is precise. Deterministic across repeated runs; skips only if 127.0.0.1:1234 is already held. Tests: zig build test 184/184 (+errorCode unit test), zig fmt clean, +E2E PTY test. Refs #201 #202 #203
|
Added an end-to-end overflow PTY test ( Writing it caught a real gap in the G2 fix: the structured- The test proves both directions:
Deterministic across repeated runs ( |
Follow-up to #196, closing the ranked backlog from the context-overflow audit.
Together these break the compound local-model wedge chain end to end: bad window
guess -> undercount -> gate never fires -> needle miss -> no recovery -> hard wedge
(previously escapable only by a manual
/compact).Stacked on #196 (base =
fix/193-tool-output-cap); GitHub will retarget this tomainwhen #196 merges, at which point theCloseskeywords fire.Commits
ea230cc— G1/G3/G6/G10 (context-overflow: perOutputCap × keep_recent=4 lets recent tool outputs wedge the window (hole in #193 per-output cap) #201, context-overflow recovery hardening: usage char/4 fallback + visible truncation + meter re-estimate (anthropic/openai parity) #202): the recovery-path holes3a603d0— G8/G4/G2/G5/G9 (local/unknown models wedge on context overflow: 200k default window + system-prompt-blind estimate + substring-only detection #203, context metering parity: same-format provider switch carries stale token count; compaction threshold hardcoded at 80% #204): local-model window + metering parityWhat changed
perOutputCapwas ~50% of the window each; withkeep_recent=4verbatim, four recent outputs pinned ~2x the window past recovery. Now ~1/8 window + 256 KB ceiling, so 4 recent stay reclaimable.recordUsagefroze the meter on missing usage; now floors atmax(fullInputEstimateTokens, body/4)(the fallback the codex path already had).capOversizedToolOutputstruncated silently; now emits a tracer note + status line.GRAFF_CONTEXTdeclares the real window, applied only to non-catalogued models (pricing.isKnownModel).fullInputEstimateTokens, which stays pure).error.codefirst (codex parity), substrings as fallback.GRAFF_COMPACT_PCToverrides it (both directions).Design note: the openai/codex research behind the local-model decisions (window
override vs probe vs catalog, structured detection) is captured in #203.
Tests
zig build test183/183 (was 179; +perOutputCap, +recordUsage-fallback,+contextWindowFor, +compactAt-override; overflow-detection tests gain structured-code
cases),
zig fmtclean.Closes #201
Closes #202
Closes #203
Closes #204