feat(runtime): implement mid-turn guidance injection with continuation pass - #629
feat(runtime): implement mid-turn guidance injection with continuation pass#629sunheyi6 wants to merge 5 commits into
Conversation
4b2b73e to
6aa0a9c
Compare
Astro-Han
left a comment
There was a problem hiding this comment.
Approved. I like the shape of the mid-turn guidance path. I found a few follow-ups that I think are worth fixing, but I’m leaving this as an approval rather than a blocker.
- P2: The desktop renderer exposes
sessions.injectGuidance, butAppShelldoes not passonInjectGuidanceintoComposer. Because of that, the “Now” action in the queued composer path falls back to a normal send instead of steering the running turn. WiringonInjectGuidancetowindow.maka.sessions.injectGuidance(activeId, text)should close the loop. - P2:
shellRunContextSummaryno longer gets merged into the current user turn tail. The helper and input contract are still there, butsend()now only usesresolveTurnTailPrompt(). Restoring thejoinPromptFragments([turnTail, shellRunContextSummary])path would keep live shell context visible to the model. - P3: Guidance continuation calls are not counted in
token_usageorrecordLlmCall. That means the new follow-up generation path can underreport usage and cost. This can be handled either by accumulating continuation usage into the final usage event or by emitting separate usage for each continuation.
|
Also rebase is needed! |
a57e495 to
860243f
Compare
|
CI failed and need repair. |
…rializer PR apache#629 regressed the per-step persistence and RuntimeEvent replay materializer introduced by apache#633. Restore flushStep/currentStepMessageId (one AssistantMessage per AI SDK step, signed thinking merged with its tool call, orphan tool_results dropped) while keeping the guidance continuation pass and standing-steer prepareStep.
jackwener
left a comment
There was a problem hiding this comment.
Deep review done. The feature is worth having and the abort interplay + sessionSendChain serialization are solid, but there's one bug that blocks merge outright:
BLOCKER — guidance injected during a tool window corrupts next-turn replay (provider 400). The guidance event is appended to the ledger in stream order, so injecting while a tool is executing (or parked on a permission prompt — the single most natural moment to steer) lands it between function_call and function_response. On the next turn, materializeRuntimeReplayPlan hits the user-role guidance, flushes the pending calls via pushToolResults before the result has been iterated, and emits a tool_use with no matching tool_result → Anthropic 400, session wedged. Simplest fix: buffer guidance and append to the ledger only at the next step boundary (never between a call and its result); alternatively defer the user message in the replay materializer until buffered calls' results flush.
Also required:
RuntimeKernel.injectGuidancefans out to every active child subagent sharing the sessionId — one steer click injects into the parent AND all running subagents (duplicate 引导已注入 rows, stray user messages in subagent transcripts). Target the top-level run only.- The injection UI is unreachable exactly when steering matters most: during a permission park the status is
waiting_for_user,isComposerResponseBusysays not-busy, so sends bypass injection and queue as a new turn. Treatwaiting_for_useras busy for injection (after the blocker is fixed). - Continuation passes drop token usage entirely (the :1252 comment concedes it) — every pass re-sends the full history uncounted. Accumulate totalUsage across passes into the turn's token_usage.
guidanceConsumedCountis racy: a steer arriving during the last prepareStep's async body can be counted consumed without being delivered (and never triggers continuation). Track a monotonic delivered-index instead of a count both sides mutate.- Test coverage doesn't touch any hard path: the only runtime test uses a fake backend yielding a canned guidance event. Need real-AiSdkBackend (mock model) tests for: mid-tool-step injection → well-formed next-turn replay (no orphan tool_use); text-only turn + late steer → exactly one continuation; loop termination under repeated injection; abort during continuation.
Minor: hasSteer keys on user-row count alone, so a steer consumed inline (no continuation) still fragments a normal multi-step answer into 引导跟进 blocks — distinguish steer-with-continuation from steer-consumed-inline. And please drop the cosmetic quote-style comment churn + BrowserPanelFallback block move from the diff.
The core idea and most of the plumbing are right — fix the ledger-ordering blocker and the fan-out, add the replay tests, and this merges.
2fa7cb6 to
273c387
Compare
…n pass - Add mid-turn guidance injection in runtime (ai-sdk-backend, model-adapter, runtime-kernel, session-manager) - Inject guidance persists across steps; continuation pass triggers a follow-up model call when guidance arrives after a text-only reply - Expose injectGuidance() on RuntimeKernel and SessionManager; add injectGuidance to AgentBackend contract in @maka/core/backend-types - Add guidance event type to events.ts; record guidance events to session store via agent-run.ts - UI: materializeTurns surfaces mid-turn guidance as a 'steer' entry in the turn timeline (renders a '引导已注入' marker); chat-view renders it via TurnTimelineEntry - Composer supports sending guidance mid-turn; desktop IPC exposes injectGuidance channel - Tests for composer-helpers, materialize, session-manager
Blocker + required fixes (@jackwener, @Astro-Han): - Buffer guidance events to step boundaries (finish-step / turn-end) so a steer injected during a tool window or permission park never lands between a tool_call and its tool_result — next-turn replay stays well-formed (no orphan tool_use). injectGuidance now only buffers; flushPendingGuidanceEvents emits at safe boundaries. - Replace racy guidanceConsumedCount with a monotonic deliveredToModelIndex (advanced only when the slice is read) and a separate pendingGuidanceEventIndex. - RuntimeKernel.injectGuidance targets the top-level run only — no fan-out to child subagents (was duplicating steers across subagent transcripts). - Accumulate continuation-pass token usage into the turn's token_usage via sumNormalizedAiSdkUsage so follow-up passes no longer underreport usage/cost. - isComposerResponseBusy treats waiting_for_user as busy so a send during a permission park steers the running turn instead of queueing a new one. - AppShell wires onInjectGuidance to window.maka.sessions.injectGuidance(activeId, text). - Restore shellRunContextSummary merge into the current user turn tail (joinPromptFragments([turnTail, shellRunContextSummary])). UI: surface mid-turn guidance as a 'steer' entry in the turn timeline (rendered via TurnTimelineEntry) instead of separate steers/assistantFollowups arrays — a steer consumed inline no longer fragments a normal multi-step answer; a steer followed by a text entry is a guidance continuation. Tests: real-AiSdkBackend (mock model) tests for mid-tool-step injection (no orphan tool_use), text-only turn + late steer (exactly one continuation), repeated injection loop termination, and abort during continuation. Update materialize + composer-helpers tests for the new model.
273c387 to
3cab623
Compare
- ui/chat-view: guard timeline item .items access behind item.kind === 'tools' (union now includes 'steer', which has no .items member) — was a hard TS2339/TS7006 error. - desktop/app-shell-session-events: fix misplaced brace so the 'guidance' and 'default' cases are not swallowed inside the 'complete' block (latent syntax error once ui build unblocks). - desktop/streaming-handoff.test: drop the obsolete 'assistant streaming handoff' describe block. It exercised drainAssistantStreamSlot / clearSettledAssistantStreamSlot / AssistantStreamSlot, all of which were removed from the codebase, so the suite no longer compiles.
…stability)
- ui: drop per-callsite strokeWidth on the guidance icons
(MessageCircleQuestion in chat-view, GripVertical/ArrowUp/Pencil/
Trash2 in composer). The icon + typography governance contract
forbids fragmenting lucide's governed stroke.
- desktop test: the effect-stability harness bundles app-shell-effects
with esbuild; the PR's new import of preservePendingOptimistic pulls
@maka/ui -> lucide-react (CJS) into the bundle, whose
require('react') breaks esbuild's shim. Externalize all node_modules
(packages: 'external') so react/react-dom/lucide-react resolve at
runtime instead.
关闭原因 / Why this PR is being closedThis PR is superseded and closed in favour of #2222 — the same fix rebuilt on top of what upstream shipped since this PR was created. Timeline:
Why not rebase this PR: its runtime ( #2222 reuses the upstream steering queue and adds only what was missing:
|
Summary
This feature allows users to inject new steering instructions while the AI is still generating a response — the model will persistently perceive the guidance across subsequent steps, and auto-trigger a follow-up generation if the guidance arrives after a text-only reply.
Key Changes
Runtime Core
pendingGuidance(one-shot) →injectedGuidance(persistent across steps)injectGuidance()onRuntimeKernelandSessionManageronThinkingCompletecallback toModelAdapteragent-run.tsEvents
guidanceevent type toevents.tsUI/Materialize
materializeTurnssupports multiple user/assistant messages per turn: first = prompt/answer, subsequent =steers/assistantFollowupsMessageCircleQuestionicon for guidance indicatorsDesktop Integration
injectGuidancechannel via main/preloadTests
composer-helpers,materialize,session-managermodel-adapterandstreaming-handofftest fixturesFiles Changed
25 files modified across 4 packages (
core,runtime,ui,desktop)