feat(runtime): mid-turn capacity compaction with a single final-payload verdict owner by Astro-Han · Pull Request #996 · apache/maka

Astro-Han · 2026-07-14T21:08:38Z

Summary

Refs #882 (PR 1 of 3, per the split in #882 (comment)).

Today the runtime can only compact history at turn boundaries, so a long-running turn that approaches the context window has no recourse: the next provider request either overflows or the turn is cut off with no explicit outcome. This PR extends the standard historyCompact protocol to phase='mid_turn' so the runtime can compact the active turn's durable ledger before exhaustion — the proactive first line of defense (the reactive compact-and-retry on provider overflow is PR 2; runtime-owned defaults across surfaces are PR 3).

Design points, in dependency order:

Protocol (history-compact-checkpoint.ts): checkpoints gain optional phase: 'pre_turn' | 'mid_turn' and headAnchor. Both are hashed into the checkpoint id only when set, so existing pre_turn checkpoint ids stay byte-stable. A mid_turn replay projects [compact block, verbatim head anchor, tail] — the current turn's user message is re-rendered verbatim, never summarized. Builder and matcher fail closed unless the anchor is the coverage's through turn's role='user'/author='user' event.
Engine (mid-turn-capacity-compact.ts, pure shaper): safe-boundary selection over the durable turn ledger — retreats before the first partial event, treats an unmatched function_call as an open span (no cut past it), never splits a call/response pair. It only returns compacted | skip | fail_open; it issues no window verdict.
Verdict owner (ai-sdk-backend.ts): every prepareStep hook only shapes (tool availability → capacity compact → active tool-result prune → the feat(runtime): add attention-first semantic compaction #986 experimental hooks, which keep their yield precedence). One owner at the end of the pipeline measures the final outgoing payload — serialized messages plus active tool schemas, the bytes the provider will actually see — and issues the single safety-critical verdict: estimate = last step's real usage + signed char/4 delta against the previous request's measured payload. A trigger miss forces one bounded capacity re-entry before terminating; only a request that still exceeds the window after all shapers becomes stopReason='context_budget_exhausted' with typed detail (no_safe_completed_span | summarizer_failed | head_anchor_exceeds_capacity).
Durability boundary (async-queue.ts + agent-run.ts): the coverage pool is the durable run ledger, read through an injected AgentRun.loadTurnRuntimeEvents seam. Because a replacement projection replaces the whole message list, a lagging ledger read would be silent content loss, not a conservative under-count — so the read is gated by a seq-ack boundary: the producer stamps a monotonic sequence at enqueue, the consumer acks each event after fully processing it (the generator pull is the ack), and the capacity hook waits, condition-driven, until the pump has flushed every completed step and the consumer has caught up, then reads once (the read itself re-awaits the run's serialized write queue). Checkpoint is persisted before the projection is replaced, same order as pre_turn.
Scope cuts: feature is default OFF behind HistoryCompactMidTurnPolicy (env MAKA_CONTEXT_HISTORY_COMPACT_MID_TURN; defaults sink in PR 3). Child sessions deliberately do not get the ledger seam — a child-built checkpoint would poison the session-global checkpoint CAS for the parent projection; full support waits for lineage-partitioned checkpoint streams. The feat(runtime): add attention-first semantic compaction #986 experiment files (semantic-compact.ts, active-full-compact.ts) are untouched.

Verification

npm --workspace @maka/runtime test: 1887 tests, 0 fail (7 pre-existing skips). New coverage: engine unit tests (boundary selection incl. open tool span), checkpoint protocol tests (anchor pinning fails closed on both build and match), policy tests, seq-ack queue unit tests, a kernel test locking the child-seam exclusion, and a streaming integration suite that runs twice — immediate and slow-consumer ledger scheduling — including rolling second compaction, same-step load_tools schema growth, prune-rescue-before-exhausted, runaway-summary refusal, and a slow-consumer text-loss regression. Negative controls: with the seq-ack boundary stashed, the slow-consumer suite fails; with the verdict-owner fixes stashed, the four round-3 repro tests fail 8/8.
npm run typecheck and npm run build: clean repo-wide.
External review: 7 codex review rounds against the full diff. Round 3's systemic diagnosis (no single estimate owner over the real next projection) drove the verdict-owner refactor rather than local patches; rounds 4–5 converged the remaining owner-internal defects (usage baseline input-only with cold-start fallback, turn-tail decoration reuse, validate → persist → apply lifecycle, replay-admissibility through the recovery path's own gate, system prompt in the payload measure); round 6 closed the feature with no open P0–P2; round 7 focused on the post-merge reconciliation below.
Not run: Playwright E2E (no renderer/main surface change; desktop and CLI each add a one-line seam passthrough).

Reconciliation with #972

This branch merged main after #972 made recordLlmCall fail-closed on usage evidence. The mid-turn exhaust aborts in prepareStep before the SDK's totalUsage resolves, which would have silently dropped the terminal record that carries the capacity diagnostics. The reconciliation accumulates each completed step's normalized usage at the finish-step boundary and uses the sum as the aborted send's usage — only when every completed step produced a usable sample (one unusable sample fails the whole record closed; a partial sum has no partial marker and would violate #972's no-fabrication invariant). Side benefit: user-stop / stream-error aborts of multi-step sends now record the real cost of the steps that ran instead of losing it. The terminal outcome never depends on this record — stopReason and the exhausted detail are durable on the CompleteEvent either way.

Review focus

Two invariants carry the design:

The verdict owner is the only place that may terminate a turn for capacity, and it judges only the final post-shaping payload. Hooks report shaping failures into state; they never abort. If a future hook reshapes messages, it composes inside the pipeline and the verdict stays correct by construction.
The ledger read under a replacement projection must be complete, not merely recent. The seq-ack boundary counts the event stream itself instead of enumerating event kinds, so it cannot drift when new event kinds appear. If a second feature ever needs read-your-durable-writes, reuse this boundary — do not add a predicate.

… phase Add a phase (pre_turn|mid_turn) and head-anchor reference to the V2 HistoryCompactCheckpoint so a checkpoint can fold a contiguous prefix that reaches into the current turn's completed steps while re-rendering the covered head anchor (the current turn's user message) verbatim on replay. Coverage stays a contiguous event prefix so the digest math is unchanged; pre_turn checkpoint ids stay byte-stable. projectHistoryCompactCheckpointReplay centralises the deterministic [block, head anchor, tail] projection.

…ry engine Turn-agnostic, side-effect-free helpers for the active-turn context invariant: estimateNextRequestTokens anchors on the last step's real provider usage plus a char/4 tail delta (whole-projection char/4 on cold start); exceedsHighWater and exceedsContextWindow gate the two failure tiers; selectMidTurnSafeBoundary picks the largest covered prefix that ends on an immutable non-partial event and never straddles a tool call/result pair, reporting no_safe_completed_span otherwise.

HistoryCompactPolicy.midTurn carries enabled + reserveTokens + reserveTailEvents. MAKA_CONTEXT_HISTORY_COMPACT_MID_TURN opts in (default off, PR 3 sinks it on), reusing the shared MAKA_CONTEXT_HISTORY_COMPACT_RESERVE_TOKENS (16384) high-water reserve. A standalone revert leaves every surface's behavior unchanged.

A first-class CompleteEvent.stopReason for when the runtime cannot produce a provider-safe request even after mid-turn compaction, with a detail field (no_safe_completed_span | summarizer_failed | head_anchor_exceeds_capacity). failureClassFromCompleteStopReason maps it to a distinct failure class so the turn is recorded as an explicit budget outcome rather than a provider error.

planMidTurnCapacityCompaction ties the measurement engine, safe-boundary selection, and the V2 checkpoint protocol into one deterministic decision: skip below the high-water; fold a safe completed prefix into a mid_turn checkpoint (re-rendering the head anchor verbatim and continuing with the preserved tail) via the injected summarizeHistoryCompact seam; roll forward from a matching previous checkpoint. Two failure tiers per the design: below the window a failure fails open, above the window it returns an explicit context_budget_exhausted outcome (no_safe_completed_span / summarizer_failed / head_anchor_exceeds_capacity). Recovery re-projection replay-validates against the same ledger prefix.

CompactionDecisionDiagnostic and the runtime CompactionDecision carry an optional phase ('pre_turn' | 'mid_turn'); absent on legacy data means pre_turn. Mid-turn capacity compaction records its trigger, replacement, fail-open, and exhausted decisions on the existing compactionDecisions channel with this dimension.

…rojection A mid_turn checkpoint's coverage reaches into the compacted turn's own completed steps, so replay matches it against the full compactable-event projection before the turn-granular guards (tail selection would otherwise retain the covered span and miss the prefix, and a single giant turn must not be rejected as insufficient_turns). Replay stays the deterministic [block, verbatim head anchor, uncovered tail] and the pre_turn path is unchanged. Exports isHistoryCompactContentEvent as the shared predicate for the backend's mid-turn projection.

…ackend Completes the issue #882 PR 1 invariant end to end. AiSdkFlow forwards ctx.branch and the persisted head anchor through BackendSendInput; the backend taps its send() queue to accumulate the current turn's content RuntimeEvents with exact ledger identity (same mapper, ids, and branch as the flow), tracks each finished step's real provider usage, and composes a mid-turn prepareStep hook (gated on historyCompact.midTurn.enabled, default off). Between steps it measures the next request as last-step usage plus a char/4 tool-result delta against contextWindow - reserve; over the high water it plans a safe-boundary fold, durably records the mid_turn checkpoint BEFORE replacing the projection, and continues the same turn on the materialized [compact block, verbatim head anchor, preserved tail] without re-executing completed tool calls. Failures under the window fail open with a mid_turn diagnostic; over the window the turn ends with the explicit complete stopReason context_budget_exhausted (no_safe_completed_span / summarizer_failed / head_anchor_exceeds_capacity), aborting the stream and handling AI SDK's graceful abort wind-down. Streaming integration tests cover trigger, persist-before-replace ordering, prompt replacement, branch recovery re-projection, all three exhausted details, fail-open, and the flow plumbing.

…l review Four verified findings, fixed at their owners: - Full-request re-estimate (F1): after folding, the plan re-estimates the complete next request (usage-anchored estimate minus the covered span's share plus the [block, anchor, tail] projection) instead of comparing only the replacement events to the window, so a huge fixed overhead with a tiny foldable span is exhausted (head_anchor_exceeds_capacity), and a replacement that would GROW past a window the raw request fits fails open (replacement_exceeds_window) rather than replacing. - Partial-free coverage (F5): the safe boundary retreats strictly before the first partial anywhere in the prefix (not just at the cut), and buildHistoryCompactCheckpoint rejects any coverage containing a partial snapshot — a digest over a replaced/deleted snapshot can never replay. - Anchor integrity fail-closed (F6): the builder requires the head anchor to be the covered turn's user event, and matchHistoryCompactCheckpointPrefix fails a mid_turn match as coverage_miss when the anchor reference is corrupted (uncovered id, wrong turn, or non-user role) instead of silently replaying without the user message. - Replay before the high-water skip (F3): an accepted mid_turn checkpoint is a correctness invariant, not a capacity optimization, so its replay match now precedes the below-high-water early return; recovery tests run on normal thresholds instead of a degenerate highWaterRatio.

…minal state The complete-event mapping dropped contextBudgetExhaustedDetail, so the persisted RuntimeEvent could not distinguish no_safe_completed_span / summarizer_failed / head_anchor_exceeds_capacity. completeRuntimeEvent now maps the full CompleteEvent and records the typed detail alongside stopReason/failureClass in the terminal stateDelta, locked by a flow-mapping round-trip test.

…sed first Root fix for the two review P1s about the backend integration (F4, F2): one composed provider-visible projection, coverage only from events already confirmed on disk, and no mirrored state carrying the hard capacity invariant. Durable-read seam (F4): AgentRun exposes loadTurnRuntimeEvents() — it waits for every ledger write enqueued so far, then reads the store — and the kernel injects it into backends next to the checkpoint loader/recorder (BackendFactoryContext, cli and desktop factories pass it through). The mid-turn trigger reads the current turn's persisted RuntimeEvents as its coverage pool, so a checkpoint can never be recorded before its covered source events are durable (the crash window is gone) and byte-identity with recovery replay holds by construction, including under ctx.branch. Last-step real usage now comes synchronously from the SDK's own step results (the same numbers as the finish-step chunk), so the wall-clock waitForSteps synchronization is deleted along with TappedAsyncEventQueue, the SessionEvent mirror mapping, and the reconstructed InvocationContext; the BackendSendInput.branch plumb that existed only for that mirror is removed. A lagging ledger read only shrinks the tail delta of the usage-anchored estimate, and every failure-driven skip (ledger_read_failed, head_anchor_not_durable) records an explicit failedOpen decision — no silent skips. Composition order (F2): composePrepareStep now runs the capacity hook before activeToolResultPrune and semantic/active-full compaction, so prune re-archives large tool results in the rebuilt tail instead of having its placeholders undone, and on the exact step the capacity hook replaced, semantic/active-full compaction yields with a recorded mid_turn_capacity_precedence decision — one step never runs two summarizers. Integration tests now drive the durable-read fixture (consumer persists mapped events exactly like AgentRun before the seam serves them) and add the review-named combinations: ledger-read fail-open diagnostics, midTurn x activeToolResultPrune tail re-convergence, and midTurn x semanticCompact precedence.

…unting the tail Two engine findings from the second external review: - Open tool span (N5): straddlesToolPair skipped spans missing one side, so with a zero tail reserve an unmatched function_call could be folded and its later response would arrive as an orphan. A call without a response is now an open span — any cut past the call is unsafe; a response without a call stays inert (its call precedes the pool). - Tail double-count (N2): the post-fold re-estimate added back the whole [block, anchor, tail] replacement although the usage-anchored estimate already contains the retained tail, misreporting rescuable turns as head_anchor_exceeds_capacity (repro: covered 505, tail ~400, estimate 700, window 500). The formula now adds back only the covered span's substitute [block, anchor]; the repro is a regression test.

A self-consistent anchor (role user, matching self-reported turnId) could resolve to ANOTHER covered turn's user event — e.g. a prior turn's prompt — and both build and match accepted it, so the replay silently dropped the real current prompt. The compacted turn is the coverage's through turn: builder and matcher now require anchor.turnId to equal it and the event to be an author='user' user event, failing closed (build error / coverage_miss) otherwise; locked by prior-turn-anchor tests on both paths.

…rmark Root fix for the second recurrence of the sync seam (N1), plus truthful write diagnostics (N6): Durable watermark: the SDK's step results are the source of truth for which tool calls completed, so the trigger derives an explicit watermark from options.steps and, before measuring or selecting coverage, loops until the durable turn ledger contains the FINAL function_call/function_response for every one of them. Each iteration re-reads through the seam — which re-awaits the run's serialized write queue and re-checks store availability after the wait — and the only exits are the watermark itself, an abort (failedOpen ledger_wait_aborted), or a read failure (failedOpen ledger_read_failed): condition-driven, no wall clock. This closes both halves of the repro: a lagging ledger can no longer under-count the tail delta (letting an over-window request out) nor re-count the same results as a fresh delta at the next boundary. The review's consumer-scheduling perturbation is a real fixture mode now: the full integration suite runs twice (immediate + slow consumer, 11 tests each) and a negative control without the watermark fails 10 of the slow-mode tests. Truthful write diagnostics: historyCompactWritesAttempted/WriteFailures are recorded only on the tiers where the recorder actually ran — pre-recorder fail-opens no longer claim a write, a write failure under the window records failedOpen write_failed with the counters, and over the window the exhaust path now carries a separate diagnostic reason so write_failed lands in the durable diagnostics (via the terminal LLM-call record) even though the terminal enum keeps summarizer_failed. A post-write materialization skip records the successful write it performed. The head-anchor gate also requires author='user', matching the checkpoint protocol.

A child run has no top-level prior context, so a mid-turn checkpoint built from its child-only ledger would claim to cover a session-scoped projection prefix and — through the session-global checkpoint cache/CAS, which compares coverage only by size — replace the parent's checkpoint and coverage_miss the parent projection. ensureChildActive no longer injects loadTurnRuntimeEvents (the backend requires the seam, so child mid-turn capacity compaction cannot arm), with the lineage-partitioning follow-up documented at the seam. A kernel test locks both sides: the parent backend reads its durable turn ledger through the seam; the child factory context has no seam and performs no read.

…stimate owner Review round 3 (findings A, C, D): capacity estimation had no single owner — the trigger counted durable response chars, the engine issued a post-fold window verdict against the raw ledger span, and the verdict ran before the active tool-result prune could rescue the step. Now every prepareStep hook only shapes; one owner at the end of the pipeline measures the final (messages + active tool schema) payload and issues the pass/terminate verdict: - estimate = last step's real usage + SIGNED char/4 delta against the previous request's measured payload, so a rolling second compaction is judged by the real replacement projection (A), and same-turn load_tools schema growth counts like any other payload growth (D); - the verdict runs after pruning, and a trigger miss forces one bounded capacity re-entry before context_budget_exhausted (C); - the engine loses its post-fold window claim entirely; the hook refuses a materialized replacement that does not shrink the real payload (runaway summary) as a shaping decision, keeping the raw projection.

…urability boundary Review round 3 (finding B): the watermark waited only for the FINAL tool call/response pair, but a step's thinking/text completion events are enqueued later, at the pump's finish-step flush — under a slow consumer the ledger could satisfy the watermark while the step's already-emitted assistant text was still missing, and because the replacement projection replaces the whole message list, that text was silently dropped from the next request. The old 'a lagging read only shrinks the delta' claim was wrong and is corrected. No event-kind predicate can close this class of gap, so the wait now counts the event stream itself: the producer stamps a monotonic sequence at enqueue (AsyncEventQueue.pushedCount), the consumer acks after fully PROCESSING each event (the generator pull in drain() is the ack, so deliberately-unpersisted events can never deadlock it), and the capacity hook reads the ledger exactly once, after the pump has flushed every completed step boundary and consumedCount has caught pushedCount. Exits: boundary, abort, detached consumer, or read failure — the polling watermark predicate is deleted.

…cycle truthful Review round 4 — four findings inside the verdict owner's implementation, architecture unchanged: - Estimate baseline is now the last request's INPUT tokens only: the signed payload delta already carries the step's freshly generated output and tool results, so an input+output baseline double-counted them (~500-token requests estimated as ~900, falsely exhausting rescuable turns). A usage sample without a positive input count is unusable, not zero — the estimate falls back to the whole-payload cold start instead of '0 + delta', so a huge request with a tiny delta can no longer slip past the window. - The head anchor in a replacement projection now renders through the same decoration owner (appendTurnTailPrompt) as the raw projection's user message, so the volatile turn tail (cwd, shell context, task state) is never silently dropped by compaction — or counted as shrinkage. - Lifecycle is validate → persist → apply: the replacement is materialized and shrink-checked BEFORE the checkpoint is recorded, so a rejected checkpoint never becomes the session's latest (replay applies checkpoints ahead of any high-water check and would have kept re-selecting it). Persistence still precedes application; validation failures attach no write counters because the recorder was never reached. - A non-shrinking fold terminates as summarizer_failed (the summarizer's output is unusable), not head_anchor_exceeds_capacity, keeping the replacement_not_smaller diagnostic reason.

…ount the system prompt in the payload measure Review round 5 (2 P1 + 1 P3): - validate = materializable AND smaller AND replay-admissible: before persisting, reuse evaluateHistoryCompactCheckpointReplay (the same single gate the recovery path runs) so an accepted checkpoint can never be rejected at the next replay and re-inject the covered span - midTurnRequestPayloadChars now includes the system prompt chars sent through the separate system field; constant between adjacent requests so signed deltas are unchanged, but the cold-start whole-payload estimate no longer under-counts by the system prompt - fix stale priorUsageTokens doc: input-only, never input+output

…-persist lifecycle and full payload measure

…-capacity-compact

… send has no total usage #972 made the terminal LLM-call record fail-closed on usage evidence, but an aborted send (mid-turn exhaust, user stop, stream error) never resolves the SDK totalUsage promise, so the record carrying the capacity verdict diagnostics was skipped entirely. Every COMPLETED step reports real usage at its finish-step boundary; accumulate those samples per send and fall back to the sum at terminal record time. No completed step means no evidence and the record is still skipped, preserving the #972 no-fabrication invariant.

…able step sample An unusable completed-step sample (normalizeAiSdkUsage returns undefined, #972) made the accumulated sum a PARTIAL cost, and LlmCallRecord has no partial marker — downstream would read it as the whole call. Track sample completeness per send and use the sum only when every completed step reported usable usage; otherwise keep the fail-closed no-record behavior. The terminal outcome never depended on this record: stopReason and the exhausted detail are durable on the CompleteEvent, now asserted explicitly. Also rewrite the stale pre-#972 'missing tokens normalize to 0' comment at the capacity hook's usage read.

* fix(headless): harden real-provider smoke reliability (#972) * fix(headless): fail closed on missing usage * fix(headless): count model steps accurately * fix(headless): retry OpenCode apt setup * fix(headless): persist failures with missing usage * fix(runtime): preserve missing usage semantics * fix(headless): preserve unavailable cell metrics * fix(runtime): normalize AI SDK detail usage * fix(headless): count runtime steps per turn * fix(headless): preserve unknown TSV usage * test(headless): align continuation step counts * fix: preserve unmetered request telemetry * fix(storage): avoid atomic temp file collisions * test(desktop): clean up failed E2E launches * fix(storage): serialize settings initialization * fix(headless): stop when provider cost is unknown * fix(runtime): enforce per-turn step budgets * fix(headless): version persisted usage semantics * test(runtime): align model step budget contract * fix: preserve incomplete provider usage semantics * fix: fail closed on incomplete usage evidence * fix(headless): propagate unknown cost through optimization * fix: close usage evidence replay gaps * fix: close final cost observation gaps * fix: invalidate incomplete usage checkpoints * fix(storage): preserve legacy usage history * fix(headless): require usage evidence for A/B gates * fix: preserve usage across processes and views * fix: preserve authoritative usage aggregation * Revert "fix: preserve authoritative usage aggregation" This reverts commit 7320705. * Revert "fix: preserve usage across processes and views" This reverts commit 0dc3e76. * Revert "fix(storage): preserve legacy usage history" This reverts commit 4a2ab0c. * refactor: narrow usage reliability scope * refactor: restore headless smoke scope * fix(runtime): reject incomplete provider usage * fix(headless): exclude unmetered attested runs (cherry picked from commit 4b736dc) (reland after #1005 squash revert) * feat(runtime): mid-turn capacity compaction with a single final-payload verdict owner (#996) * feat(runtime): extend history compact checkpoint protocol to mid_turn phase Add a phase (pre_turn|mid_turn) and head-anchor reference to the V2 HistoryCompactCheckpoint so a checkpoint can fold a contiguous prefix that reaches into the current turn's completed steps while re-rendering the covered head anchor (the current turn's user message) verbatim on replay. Coverage stays a contiguous event prefix so the digest math is unchanged; pre_turn checkpoint ids stay byte-stable. projectHistoryCompactCheckpointReplay centralises the deterministic [block, head anchor, tail] projection. * feat(runtime): add pure mid-turn capacity measurement and safe-boundary engine Turn-agnostic, side-effect-free helpers for the active-turn context invariant: estimateNextRequestTokens anchors on the last step's real provider usage plus a char/4 tail delta (whole-projection char/4 on cold start); exceedsHighWater and exceedsContextWindow gate the two failure tiers; selectMidTurnSafeBoundary picks the largest covered prefix that ends on an immutable non-partial event and never straddles a tool call/result pair, reporting no_safe_completed_span otherwise. * feat(runtime): add mid-turn history compact policy surface (default off) HistoryCompactPolicy.midTurn carries enabled + reserveTokens + reserveTailEvents. MAKA_CONTEXT_HISTORY_COMPACT_MID_TURN opts in (default off, PR 3 sinks it on), reusing the shared MAKA_CONTEXT_HISTORY_COMPACT_RESERVE_TOKENS (16384) high-water reserve. A standalone revert leaves every surface's behavior unchanged. * feat(core): add context_budget_exhausted complete outcome A first-class CompleteEvent.stopReason for when the runtime cannot produce a provider-safe request even after mid-turn compaction, with a detail field (no_safe_completed_span | summarizer_failed | head_anchor_exceeds_capacity). failureClassFromCompleteStopReason maps it to a distinct failure class so the turn is recorded as an explicit budget outcome rather than a provider error. * feat(runtime): add mid-turn capacity compaction orchestration planMidTurnCapacityCompaction ties the measurement engine, safe-boundary selection, and the V2 checkpoint protocol into one deterministic decision: skip below the high-water; fold a safe completed prefix into a mid_turn checkpoint (re-rendering the head anchor verbatim and continuing with the preserved tail) via the injected summarizeHistoryCompact seam; roll forward from a matching previous checkpoint. Two failure tiers per the design: below the window a failure fails open, above the window it returns an explicit context_budget_exhausted outcome (no_safe_completed_span / summarizer_failed / head_anchor_exceeds_capacity). Recovery re-projection replay-validates against the same ledger prefix. * feat(core): add phase dimension to compaction decision diagnostics CompactionDecisionDiagnostic and the runtime CompactionDecision carry an optional phase ('pre_turn' | 'mid_turn'); absent on legacy data means pre_turn. Mid-turn capacity compaction records its trigger, replacement, fail-open, and exhausted decisions on the existing compactionDecisions channel with this dimension. * feat(runtime): replay mid_turn checkpoints against the full content projection A mid_turn checkpoint's coverage reaches into the compacted turn's own completed steps, so replay matches it against the full compactable-event projection before the turn-granular guards (tail selection would otherwise retain the covered span and miss the prefix, and a single giant turn must not be rejected as insufficient_turns). Replay stays the deterministic [block, verbatim head anchor, uncovered tail] and the pre_turn path is unchanged. Exports isHistoryCompactContentEvent as the shared predicate for the backend's mid-turn projection. * feat(runtime): wire mid-turn capacity compaction into the streaming backend Completes the issue #882 PR 1 invariant end to end. AiSdkFlow forwards ctx.branch and the persisted head anchor through BackendSendInput; the backend taps its send() queue to accumulate the current turn's content RuntimeEvents with exact ledger identity (same mapper, ids, and branch as the flow), tracks each finished step's real provider usage, and composes a mid-turn prepareStep hook (gated on historyCompact.midTurn.enabled, default off). Between steps it measures the next request as last-step usage plus a char/4 tool-result delta against contextWindow - reserve; over the high water it plans a safe-boundary fold, durably records the mid_turn checkpoint BEFORE replacing the projection, and continues the same turn on the materialized [compact block, verbatim head anchor, preserved tail] without re-executing completed tool calls. Failures under the window fail open with a mid_turn diagnostic; over the window the turn ends with the explicit complete stopReason context_budget_exhausted (no_safe_completed_span / summarizer_failed / head_anchor_exceeds_capacity), aborting the stream and handling AI SDK's graceful abort wind-down. Streaming integration tests cover trigger, persist-before-replace ordering, prompt replacement, branch recovery re-projection, all three exhausted details, fail-open, and the flow plumbing. * fix(runtime): close mid-turn compaction correctness gaps from external review Four verified findings, fixed at their owners: - Full-request re-estimate (F1): after folding, the plan re-estimates the complete next request (usage-anchored estimate minus the covered span's share plus the [block, anchor, tail] projection) instead of comparing only the replacement events to the window, so a huge fixed overhead with a tiny foldable span is exhausted (head_anchor_exceeds_capacity), and a replacement that would GROW past a window the raw request fits fails open (replacement_exceeds_window) rather than replacing. - Partial-free coverage (F5): the safe boundary retreats strictly before the first partial anywhere in the prefix (not just at the cut), and buildHistoryCompactCheckpoint rejects any coverage containing a partial snapshot — a digest over a replaced/deleted snapshot can never replay. - Anchor integrity fail-closed (F6): the builder requires the head anchor to be the covered turn's user event, and matchHistoryCompactCheckpointPrefix fails a mid_turn match as coverage_miss when the anchor reference is corrupted (uncovered id, wrong turn, or non-user role) instead of silently replaying without the user message. - Replay before the high-water skip (F3): an accepted mid_turn checkpoint is a correctness invariant, not a capacity optimization, so its replay match now precedes the below-high-water early return; recovery tests run on normal thresholds instead of a degenerate highWaterRatio. * fix(runtime): keep context_budget_exhausted detail in the durable terminal state The complete-event mapping dropped contextBudgetExhaustedDetail, so the persisted RuntimeEvent could not distinguish no_safe_completed_span / summarizer_failed / head_anchor_exceeds_capacity. completeRuntimeEvent now maps the full CompleteEvent and records the typed detail alongside stopReason/failureClass in the terminal stateDelta, locked by a flow-mapping round-trip test. * refactor: source mid-turn coverage from the durable run ledger, composed first Root fix for the two review P1s about the backend integration (F4, F2): one composed provider-visible projection, coverage only from events already confirmed on disk, and no mirrored state carrying the hard capacity invariant. Durable-read seam (F4): AgentRun exposes loadTurnRuntimeEvents() — it waits for every ledger write enqueued so far, then reads the store — and the kernel injects it into backends next to the checkpoint loader/recorder (BackendFactoryContext, cli and desktop factories pass it through). The mid-turn trigger reads the current turn's persisted RuntimeEvents as its coverage pool, so a checkpoint can never be recorded before its covered source events are durable (the crash window is gone) and byte-identity with recovery replay holds by construction, including under ctx.branch. Last-step real usage now comes synchronously from the SDK's own step results (the same numbers as the finish-step chunk), so the wall-clock waitForSteps synchronization is deleted along with TappedAsyncEventQueue, the SessionEvent mirror mapping, and the reconstructed InvocationContext; the BackendSendInput.branch plumb that existed only for that mirror is removed. A lagging ledger read only shrinks the tail delta of the usage-anchored estimate, and every failure-driven skip (ledger_read_failed, head_anchor_not_durable) records an explicit failedOpen decision — no silent skips. Composition order (F2): composePrepareStep now runs the capacity hook before activeToolResultPrune and semantic/active-full compaction, so prune re-archives large tool results in the rebuilt tail instead of having its placeholders undone, and on the exact step the capacity hook replaced, semantic/active-full compaction yields with a recorded mid_turn_capacity_precedence decision — one step never runs two summarizers. Integration tests now drive the durable-read fixture (consumer persists mapped events exactly like AgentRun before the seam serves them) and add the review-named combinations: ledger-read fail-open diagnostics, midTurn x activeToolResultPrune tail re-convergence, and midTurn x semanticCompact precedence. * fix(runtime): keep open tool calls out of coverage and stop double-counting the tail Two engine findings from the second external review: - Open tool span (N5): straddlesToolPair skipped spans missing one side, so with a zero tail reserve an unmatched function_call could be folded and its later response would arrive as an orphan. A call without a response is now an open span — any cut past the call is unsafe; a response without a call stays inert (its call precedes the pool). - Tail double-count (N2): the post-fold re-estimate added back the whole [block, anchor, tail] replacement although the usage-anchored estimate already contains the retained tail, misreporting rescuable turns as head_anchor_exceeds_capacity (repro: covered 505, tail ~400, estimate 700, window 500). The formula now adds back only the covered span's substitute [block, anchor]; the repro is a regression test. * fix(runtime): pin the mid-turn head anchor to the compacted turn A self-consistent anchor (role user, matching self-reported turnId) could resolve to ANOTHER covered turn's user event — e.g. a prior turn's prompt — and both build and match accepted it, so the replay silently dropped the real current prompt. The compacted turn is the coverage's through turn: builder and matcher now require anchor.turnId to equal it and the event to be an author='user' user event, failing closed (build error / coverage_miss) otherwise; locked by prior-turn-anchor tests on both paths. * fix(runtime): gate the mid-turn trigger on a durable tool-result watermark Root fix for the second recurrence of the sync seam (N1), plus truthful write diagnostics (N6): Durable watermark: the SDK's step results are the source of truth for which tool calls completed, so the trigger derives an explicit watermark from options.steps and, before measuring or selecting coverage, loops until the durable turn ledger contains the FINAL function_call/function_response for every one of them. Each iteration re-reads through the seam — which re-awaits the run's serialized write queue and re-checks store availability after the wait — and the only exits are the watermark itself, an abort (failedOpen ledger_wait_aborted), or a read failure (failedOpen ledger_read_failed): condition-driven, no wall clock. This closes both halves of the repro: a lagging ledger can no longer under-count the tail delta (letting an over-window request out) nor re-count the same results as a fresh delta at the next boundary. The review's consumer-scheduling perturbation is a real fixture mode now: the full integration suite runs twice (immediate + slow consumer, 11 tests each) and a negative control without the watermark fails 10 of the slow-mode tests. Truthful write diagnostics: historyCompactWritesAttempted/WriteFailures are recorded only on the tiers where the recorder actually ran — pre-recorder fail-opens no longer claim a write, a write failure under the window records failedOpen write_failed with the counters, and over the window the exhaust path now carries a separate diagnostic reason so write_failed lands in the durable diagnostics (via the terminal LLM-call record) even though the terminal enum keeps summarizer_failed. A post-write materialization skip records the successful write it performed. The head-anchor gate also requires author='user', matching the checkpoint protocol. * fix(runtime): withhold the turn-ledger seam from child sessions A child run has no top-level prior context, so a mid-turn checkpoint built from its child-only ledger would claim to cover a session-scoped projection prefix and — through the session-global checkpoint cache/CAS, which compares coverage only by size — replace the parent's checkpoint and coverage_miss the parent projection. ensureChildActive no longer injects loadTurnRuntimeEvents (the backend requires the seam, so child mid-turn capacity compaction cannot arm), with the lineage-partitioning follow-up documented at the seam. A kernel test locks both sides: the parent backend reads its durable turn ledger through the seam; the child factory context has no seam and performs no read. * fix(runtime): move the mid-turn capacity verdict to a final-payload estimate owner Review round 3 (findings A, C, D): capacity estimation had no single owner — the trigger counted durable response chars, the engine issued a post-fold window verdict against the raw ledger span, and the verdict ran before the active tool-result prune could rescue the step. Now every prepareStep hook only shapes; one owner at the end of the pipeline measures the final (messages + active tool schema) payload and issues the pass/terminate verdict: - estimate = last step's real usage + SIGNED char/4 delta against the previous request's measured payload, so a rolling second compaction is judged by the real replacement projection (A), and same-turn load_tools schema growth counts like any other payload growth (D); - the verdict runs after pruning, and a trigger miss forces one bounded capacity re-entry before context_budget_exhausted (C); - the engine loses its post-fold window claim entirely; the hook refuses a materialized replacement that does not shrink the real payload (runaway summary) as a shaping decision, keeping the raw projection. * fix(runtime): replace the mid-turn durable watermark with a seq-ack durability boundary Review round 3 (finding B): the watermark waited only for the FINAL tool call/response pair, but a step's thinking/text completion events are enqueued later, at the pump's finish-step flush — under a slow consumer the ledger could satisfy the watermark while the step's already-emitted assistant text was still missing, and because the replacement projection replaces the whole message list, that text was silently dropped from the next request. The old 'a lagging read only shrinks the delta' claim was wrong and is corrected. No event-kind predicate can close this class of gap, so the wait now counts the event stream itself: the producer stamps a monotonic sequence at enqueue (AsyncEventQueue.pushedCount), the consumer acks after fully PROCESSING each event (the generator pull in drain() is the ack, so deliberately-unpersisted events can never deadlock it), and the capacity hook reads the ledger exactly once, after the pump has flushed every completed step boundary and consumedCount has caught pushedCount. Exits: boundary, abort, detached consumer, or read failure — the polling watermark predicate is deleted. * fix(runtime): make the capacity estimate baseline and checkpoint lifecycle truthful Review round 4 — four findings inside the verdict owner's implementation, architecture unchanged: - Estimate baseline is now the last request's INPUT tokens only: the signed payload delta already carries the step's freshly generated output and tool results, so an input+output baseline double-counted them (~500-token requests estimated as ~900, falsely exhausting rescuable turns). A usage sample without a positive input count is unusable, not zero — the estimate falls back to the whole-payload cold start instead of '0 + delta', so a huge request with a tiny delta can no longer slip past the window. - The head anchor in a replacement projection now renders through the same decoration owner (appendTurnTailPrompt) as the raw projection's user message, so the volatile turn tail (cwd, shell context, task state) is never silently dropped by compaction — or counted as shrinkage. - Lifecycle is validate → persist → apply: the replacement is materialized and shrink-checked BEFORE the checkpoint is recorded, so a rejected checkpoint never becomes the session's latest (replay applies checkpoints ahead of any high-water check and would have kept re-selecting it). Persistence still precedes application; validation failures attach no write counters because the recorder was never reached. - A non-shrinking fold terminates as summarizer_failed (the summarizer's output is unusable), not head_anchor_exceeds_capacity, keeping the replacement_not_smaller diagnostic reason. * fix(runtime): gate mid-turn checkpoints on replay admissibility and count the system prompt in the payload measure Review round 5 (2 P1 + 1 P3): - validate = materializable AND smaller AND replay-admissible: before persisting, reuse evaluateHistoryCompactCheckpointReplay (the same single gate the recovery path runs) so an accepted checkpoint can never be rejected at the next replay and re-inject the covered span - midTurnRequestPayloadChars now includes the system prompt chars sent through the separate system field; constant between adjacent requests so signed deltas are unchanged, but the cold-start whole-payload estimate no longer under-counts by the system prompt - fix stale priorUsageTokens doc: input-only, never input+output * docs(runtime): align stale mid-turn comments with the validate-before-persist lifecycle and full payload measure * fix(runtime): record accumulated completed-step usage when an aborted send has no total usage #972 made the terminal LLM-call record fail-closed on usage evidence, but an aborted send (mid-turn exhaust, user stop, stream error) never resolves the SDK totalUsage promise, so the record carrying the capacity verdict diagnostics was skipped entirely. Every COMPLETED step reports real usage at its finish-step boundary; accumulate those samples per send and fall back to the sum at terminal record time. No completed step means no evidence and the record is still skipped, preserving the #972 no-fabrication invariant. * fix(runtime): fail the aborted-send usage fallback closed on any unusable step sample An unusable completed-step sample (normalizeAiSdkUsage returns undefined, #972) made the accumulated sum a PARTIAL cost, and LlmCallRecord has no partial marker — downstream would read it as the whole call. Track sample completeness per send and use the sum only when every completed step reported usable usage; otherwise keep the fail-closed no-record behavior. The terminal outcome never depended on this record: stopReason and the exhausted detail are durable on the CompleteEvent, now asserted explicitly. Also rewrite the stale pre-#972 'missing tokens normalize to 0' comment at the capacity hook's usage read. (cherry picked from commit 8ef9373) (reland after #1005 squash revert) * fix(ui): restore quiet composer picker triggers (#999) (cherry picked from commit ecf515d) (reland after #1005 squash revert) * fix(ui): keep in-flight live turn armed when persisted history covers all steps (#1000) Symptom: the desktop composer's "in progress" indicator flickers off during a running turn. At every step-to-step lull, when all tool/thinking evidence is already covered by the persisted transcript, the busy state drops to idle until the next event recreates the projection. Cause: reconcileTerminalLiveTurn deleted the whole live-turn projection (returning undefined) whenever the filtered steps array became empty, even for a NON-terminal projection. app-shell calls it on every messages/activeLiveTurn change mid-turn, so the projection vanished and turnInFlight (projection exists && !terminal) went false. Fix: an empty result only deletes the projection when current.terminal, mirroring the existing precedent in settleLiveTurnStep. A non-terminal projection survives as { ...current, steps: [] } with its arm preserved. (cherry picked from commit 8153519) (reland after #1005 squash revert)

Astro-Han added 23 commits July 15, 2026 00:21

docs(runtime): align stale mid-turn comments with the validate-before…

d042864

…-persist lifecycle and full payload measure

Merge remote-tracking branch 'origin/main' into feat/runtime-mid-turn…

4f31f57

…-capacity-compact

Astro-Han merged commit 8ef9373 into main Jul 15, 2026
3 checks passed

Astro-Han deleted the feat/runtime-mid-turn-capacity-compact branch July 15, 2026 00:13

Astro-Han mentioned this pull request Jul 15, 2026

fix: reland merges reverted by #1005's snapshot squash #1014

Merged

This was referenced Jul 15, 2026

feat: runtime-owned context compaction defaults across surfaces #1018

Merged

feat(runtime): compact active turns before context exhaustion #882

Closed

Navigation Menu

Uh oh!

feat(runtime): mid-turn capacity compaction with a single final-payload verdict owner - #996

feat(runtime): mid-turn capacity compaction with a single final-payload verdict owner#996
Astro-Han merged 23 commits into
mainfrom
feat/runtime-mid-turn-capacity-compact

Astro-Han commented Jul 14, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

Astro-Han commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Verification

Reconciliation with #972

Review focus

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Astro-Han commented Jul 14, 2026 •

edited

Loading