emrg: implement tool result sliding window - #936
Merged
Conversation
argszero
commented
Aug 22, 2026
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle. Re-verified head ecad963 against design doc (tool-window-design.md):
_apply_tool_windowis a pure fold (no session/disk access); atomic groups = assistant-with-tool_calls + following tool messages, so OpenAI pairing (session._validate_tool_messages) always holds — verified by the new validation test.- In-place placement preserves message order; only the most recent keep_rounds groups stay complete; keep_rounds<=0 = identity (old behavior).
- Called at the top of the tool-loop while True before the round-budget check and the auto-compact token estimate — both history and same-request accumulation are covered; disk history.jsonl stays complete.
- Placeholder carries executed counts + tool_call_ids + absolute history path + grep hint for backtracking; role=assistant without tool_calls → API-valid.
- system.j2 renders the notice only when tool_window_rounds > 0 and is only rendered via _build_system_prompt (which always sets the ctx key) — no undefined-variable path.
- Full suite: 1007 passed, 1 skipped; CI test + test-windows green (run 32561317425).
…doc-count conflict: pytest 1008 + GUI 20/129 from #935)
argszero
commented
Aug 22, 2026
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle (2nd). Re-verified merged head 0c2f5de (merge of master d56cc2d):
- Conflict with #935 (Agent.md doc counts) resolved: pytest count kept at 1008 (my +5 tests, verified: 1007 passed + 1 skipped collected), GUI counts updated to master's 20 conn-manager / 129 renderer smoke from #935.
- Local verification on merged head: 1007 passed, 1 skipped; import + CLI OK.
- CI re-run on merged head: test + test-windows both pass (run 32562164734); mergeStateStatus now MERGEABLE.
- Functional review unchanged from cycle 1259: _apply_tool_window pure fold, atomic group = assistant-with-tool_calls + following tool messages, keep_rounds<=0 identity, placeholder carries counts + tool_call_ids + history path, system.j2 gated on tool_window_rounds>0 with key always set.
argszero
commented
Aug 22, 2026
argszero
left a comment
Owner
Author
There was a problem hiding this comment.
✅ LGTM — cycle (3rd). Fresh-eyes re-verification of head 0c2f5de:
- Diff vs master: 5 files +246/−2 — Agent.md (test count 1008), config.py (tool_window_rounds=7, parsed from llm config, 0=off), daemon.py (+106: constructor wiring, ctx in _build_system_prompt always-set, _apply_tool_window/_fold_tool_group pure functions, call as first statement of tool-loop while True), system.j2 (fold notice gated on tool_window_rounds>0), tests (+5: 10-group fold, zero identity, within-window no-op, count aggregation, pairing validation).
- Folding preserves OpenAI message pairing: atomic group = assistant-with-tool_calls + following tool messages; placeholder is role=assistant without tool_calls + carries executed counts + tool_call_ids + absolute history.jsonl path + grep hint for backtracking; disk history stays complete.
- keep_rounds<=0 = identity (old behavior); in-place order preserved.
- Local: 1007 passed + 1 skipped (=1008 collected); import + CLI OK. CI: test + test-windows green (run 32562164734); mergeStateStatus CLEAN.
argszero
pushed a commit
that referenced
this pull request
Aug 22, 2026
argszero
added a commit
that referenced
this pull request
Aug 22, 2026
argszero
added a commit
that referenced
this pull request
Aug 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tool Result Sliding Window (rant 2026-08-22T11:33:54)
Problem: every LLM request re-sends the full session history including all complete tool results (bash/read/grep outputs). Long sessions (dozens of tool rounds, up to
max_tool_rounds=270) grow token usage linearly and hit auto-compact (lossy summarization) early. Early tool results are rarely useful but dominate the token budget.Solution: fold tool results older than the most recent N rounds (default 7, config
tool_window_rounds,0= off = old behavior) into a placeholder assistant message before each LLM request.What changed
emrg/config.py— newLlmConfig.tool_window_rounds: int = 7(reads[llm] tool_window_roundsfrom config.toml;0disables).emrg/server/daemon.py:_apply_tool_window(messages, keep_rounds, history_path)— pure function (no session/disk access). Atomic group = assistant-with-tool_calls+ its followingtoolmessages (OpenAI pairing constraint). Keeps the most recent N groups in full; each older group is replaced in-place by a placeholder assistant message. Never folds system/user/plain-text-assistant/summary messages._fold_tool_group()— builds the placeholder:[Tool results omitted — older than recent N rounds]+ executed tool counts +tool_call_ids+ absolutehistory.jsonlpath + grep backtrack hint.while True:(before round-budget checks and the auto-compact token estimate), so both the session history and same-request accumulated rounds are covered. On-diskhistory.jsonlstays complete — nothing is lost, only the sent payload is folded._build_system_promptinjectstool_window_roundsinto the render context.emrg/server/prompts/system.j2— fold notice + "summarize valuable info proactively" advice block, rendered only whentool_window_rounds > 0(zero-config behavior identical to before). The gitignored packaged copydist/runtime/source/emrg/server/prompts/system.j2was synced on disk as well (build artifact, regenerated by packaging).tests/test_daemon.py— +5 tests: 10-group fold (3 oldest → placeholders, 7 complete, order preserved),keep_rounds=0identity, within-window no-op, multi-tool count aggregation, non-foldable preservation + OpenAI-pairing validation via_validate_tool_messages.Agent.md— test count 1003 → 1008 (doc-count guard).Verification
emrg --helpOK.<cwd>/.emrg/sessions/<session_id>/history.jsonl(verified againstSession._history_path).