Skip to content

emrg: implement tool result sliding window - #936

Merged
argszero merged 2 commits into
masterfrom
feature/tool-result-sliding-window
Aug 22, 2026
Merged

emrg: implement tool result sliding window#936
argszero merged 2 commits into
masterfrom
feature/tool-result-sliding-window

Conversation

@argszero

Copy link
Copy Markdown
Owner

Tool Result Sliding Window (rant 2026-08-22T11:33:54)

Problem: every LLM request re-sends the full session history including all complete tool results (bash/read/grep outputs). Long sessions (dozens of tool rounds, up to max_tool_rounds=270) grow token usage linearly and hit auto-compact (lossy summarization) early. Early tool results are rarely useful but dominate the token budget.

Solution: fold tool results older than the most recent N rounds (default 7, config tool_window_rounds, 0 = off = old behavior) into a placeholder assistant message before each LLM request.

What changed

  • emrg/config.py — new LlmConfig.tool_window_rounds: int = 7 (reads [llm] tool_window_rounds from config.toml; 0 disables).
  • emrg/server/daemon.py:
    • _apply_tool_window(messages, keep_rounds, history_path)pure function (no session/disk access). Atomic group = assistant-with-tool_calls + its following tool messages (OpenAI pairing constraint). Keeps the most recent N groups in full; each older group is replaced in-place by a placeholder assistant message. Never folds system/user/plain-text-assistant/summary messages.
    • _fold_tool_group() — builds the placeholder: [Tool results omitted — older than recent N rounds] + executed tool counts + tool_call_ids + absolute history.jsonl path + grep backtrack hint.
    • Called at the top of the tool-loop while True: (before round-budget checks and the auto-compact token estimate), so both the session history and same-request accumulated rounds are covered. On-disk history.jsonl stays complete — nothing is lost, only the sent payload is folded.
    • _build_system_prompt injects tool_window_rounds into the render context.
  • emrg/server/prompts/system.j2 — fold notice + "summarize valuable info proactively" advice block, rendered only when tool_window_rounds > 0 (zero-config behavior identical to before). The gitignored packaged copy dist/runtime/source/emrg/server/prompts/system.j2 was synced on disk as well (build artifact, regenerated by packaging).
  • tests/test_daemon.py — +5 tests: 10-group fold (3 oldest → placeholders, 7 complete, order preserved), keep_rounds=0 identity, within-window no-op, multi-tool count aggregation, non-foldable preservation + OpenAI-pairing validation via _validate_tool_messages.
  • Agent.md — test count 1003 → 1008 (doc-count guard).

Verification

  • Full suite: 1007 passed, 1 skipped (was 1002; +5 new).
  • Import check + emrg --help OK.
  • Placeholder backtrack path = <cwd>/.emrg/sessions/<session_id>/history.jsonl (verified against Session._history_path).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle. Re-verified head ecad963 against design doc (tool-window-design.md):

  • _apply_tool_window is a pure fold (no session/disk access); atomic groups = assistant-with-tool_calls + following tool messages, so OpenAI pairing (session._validate_tool_messages) always holds — verified by the new validation test.
  • In-place placement preserves message order; only the most recent keep_rounds groups stay complete; keep_rounds<=0 = identity (old behavior).
  • Called at the top of the tool-loop while True before the round-budget check and the auto-compact token estimate — both history and same-request accumulation are covered; disk history.jsonl stays complete.
  • Placeholder carries executed counts + tool_call_ids + absolute history path + grep hint for backtracking; role=assistant without tool_calls → API-valid.
  • system.j2 renders the notice only when tool_window_rounds > 0 and is only rendered via _build_system_prompt (which always sets the ctx key) — no undefined-variable path.
  • Full suite: 1007 passed, 1 skipped; CI test + test-windows green (run 32561317425).

…doc-count conflict: pytest 1008 + GUI 20/129 from #935)

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle (2nd). Re-verified merged head 0c2f5de (merge of master d56cc2d):

  • Conflict with #935 (Agent.md doc counts) resolved: pytest count kept at 1008 (my +5 tests, verified: 1007 passed + 1 skipped collected), GUI counts updated to master's 20 conn-manager / 129 renderer smoke from #935.
  • Local verification on merged head: 1007 passed, 1 skipped; import + CLI OK.
  • CI re-run on merged head: test + test-windows both pass (run 32562164734); mergeStateStatus now MERGEABLE.
  • Functional review unchanged from cycle 1259: _apply_tool_window pure fold, atomic group = assistant-with-tool_calls + following tool messages, keep_rounds<=0 identity, placeholder carries counts + tool_call_ids + history path, system.j2 gated on tool_window_rounds>0 with key always set.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle (3rd). Fresh-eyes re-verification of head 0c2f5de:

  • Diff vs master: 5 files +246/−2 — Agent.md (test count 1008), config.py (tool_window_rounds=7, parsed from llm config, 0=off), daemon.py (+106: constructor wiring, ctx in _build_system_prompt always-set, _apply_tool_window/_fold_tool_group pure functions, call as first statement of tool-loop while True), system.j2 (fold notice gated on tool_window_rounds>0), tests (+5: 10-group fold, zero identity, within-window no-op, count aggregation, pairing validation).
  • Folding preserves OpenAI message pairing: atomic group = assistant-with-tool_calls + following tool messages; placeholder is role=assistant without tool_calls + carries executed counts + tool_call_ids + absolute history.jsonl path + grep hint for backtracking; disk history stays complete.
  • keep_rounds<=0 = identity (old behavior); in-place order preserved.
  • Local: 1007 passed + 1 skipped (=1008 collected); import + CLI OK. CI: test + test-windows green (run 32562164734); mergeStateStatus CLEAN.

@argszero
argszero merged commit 6d4f3bc into master Aug 22, 2026
2 checks passed
argszero pushed a commit that referenced this pull request Aug 22, 2026
argszero added a commit that referenced this pull request Aug 22, 2026
#936 #937) (#939)

* emrg: revert DeepSeek thinking-mode reasoning pass-back (rant 2026-08-22T21:12:46)

* emrg: revert tool result sliding window (#936) (rant 2026-08-22T21:23:18)
@argszero
argszero deleted the feature/tool-result-sliding-window branch August 23, 2026 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant