feat(threads): a silent provider stall no longer renders as an honest-looking working timer - #9
Merged
Conversation
…l tool output The omp provider rendered a bare timeline: thinking was discarded, tool rows flattened to generic labels with one-line 84-char output summaries, inline diffs were dropped on the wire, and subagents never reached the Agents panel because omp's subagent frames are RPC-mode-only. Server: - OmpAdapter buffers agent_thought_chunk segments and settles them as reasoning items; ingestion appends them as reasoning.completed activities. - The wire projection keeps a capped multi-line rawOutput.fullText and the ACP diff content blocks on tool.completed rows (tool.updated stays slim), plus a bounded toolInfo (device/tool name, action, scalar args, eval code) derived from rawInput — retroactive for already-persisted threads. - omp's task tool calls synthesize the full subagent lifecycle from the TaskToolDetails streamed in rawOutput: task.started per subtask, deduped task.progress with intent/tool/usage/model, and task.completed driven by per-subtask progress status — the async-job mode acks the call as completed while agents still run, and the runtime evicts completed calls from its merge map, so the roster is cached per toolCallId and agents outliving the turn are marked backgrounded instead of spinning forever. - AcpSessionRuntime gains an opt-in emission predicate so omp task progress in rawOutput survives the title/detail update-suppression gate. - Informative provider titles (omp per-call intents) win the row heading over flattened action labels, and update frames no longer overwrite them with the bare "Tool" fallback. Web: - Thinking rows render italic/dimmed and expand to markdown. - Expanded tool rows show the full output, per-file FileDiff blocks built from the preserved diff contents, structured tool args, syntax-highlighted eval code, and markdown LSP results; icons key on the tool identity. Verified live against a real omp session: thinking, intent titles, inline diffs, and the Agents panel roster with per-agent status, intent, model, token and tool counts. Built with Claude Fable 5 on Oh My Pi.
…chat # Conflicts: # apps/web/src/components/chat/MessagesTimeline.tsx
…-looking working timer A thread sat at 'Working for 10m' with zero output while the provider (omp) retried upstream 529s internally forever without sending a single ACP frame. Nothing in the UI distinguished that from real work. Server: - AcpSessionRuntime stamps every inbound native frame (session/update before any filtering, permission requests, elicitations) and exposes a liveness snapshot with the in-flight tool-call count. - OmpAdapter runs a 15s watchdog per session: when a running turn has had no frame for 75s — observed retry cadence was 60-105s, normal streaming gaps are seconds, worst-case first-token latency stays under ~60s — it emits a transition-only session.state.changed carrying providerQuietSince, and a clearing transition when frames resume. An open tool call, a pending approval, or a pending user-input question suppresses the flag: those wait on the tool or the human, not the provider. Every sendTurn resets the silence baseline. - OrchestrationSession gains optional providerQuietSince, persisted through a new projection column so a client opening the thread mid-stall sees the flag in the snapshot — exactly the incident scenario. getSnapshot also had a latent inline session mapping that silently dropped unknown fields; it now goes through mapSessionRow. Clients: - Web swaps the working row for 'No output from the provider for Xs · last activity HH:MM:SS — still waiting; you can stop the turn' (same self-ticking text-node pattern, one commit per transition). Stop stays in the composer. - Mobile renders the same truthful label, and — audit finding — now shows session.lastError in the chat at all: a compact error card above the composer. Web already surfaced it via ThreadErrorBanner. Deterministic tests: quiet-transition resolver matrix, ingestion mark/clear/ settle round-trip through the real SQL projection, and native-frame stamping via the mock ACP agent. Verified live by SIGSTOPping the provider mid-turn (flag appeared at +85s with the freeze timestamp) and resuming (flag cleared, turn settled). Built with Claude Fable 5 on Oh My Pi.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem (real incident on this machine)
A thread sat at
Working for 10m 21swith zero output, zero tool calls, zero error. The upstream model API was returning 529s and the omp harness retried internally forever without sending a single ACP frame. T3 dispatched correctly and then rendered an honest-looking timer over dead air; the user had to read SQLite and process state to find out.Fix
Detection —
AcpSessionRuntimestamps every inbound native frame (session/update before any filtering, permission requests, elicitations) and exposesgetActivitySnapshotwith the in-flight tool-call count. The omp adapter runs a 15s watchdog: a running turn with no frame for 75s (observed retry cadence was 60–105s; normal streaming gaps are seconds; worst-case first-token latency stays under ~60s) emits a transition-onlysession.state.changedcarryingproviderQuietSince, and a clearing transition when frames resume. An open tool call, a pending approval, or a pending user-input question suppresses the flag — those wait on the tool or the human, not the provider. Every sendTurn resets the baseline, so steers never false-positive.Wire —
OrchestrationSession.providerQuietSince?(additive optional; old clients drop it, old servers omit it), persisted through a new projection column (migration 043) so a client opening the thread mid-stall sees the flag in the snapshot — exactly the incident scenario. Also fixes a latent bug found on the way:getSnapshotbuilt sessions inline instead of viamapSessionRow, silently dropping any new session field.Clients — web swaps the working row for a warning row with the same self-ticking text-node timer (one React commit per transition, no repaint loop); mobile renders the equivalent two-line label with its existing timer pattern. Stop stays reachable in both composers. Audit finding fixed along the way: mobile never rendered
session.lastErrorat all — it now shows a compact error card above the composer (web already hadThreadErrorBanner).Verification
The stall, rendered truthfully (provider frozen mid-turn)
Frames resume → label clears, turn completes
Known limits: watchdog is omp-only (the incident's transport — Claude/Codex need equivalent stamps in their own adapters; wire and UI are already provider-agnostic). A tool that itself hangs silently doesn't flag, by design. Separate concern, not bundled: provider probing leaks orphaned
fake-claude.mjschildren — real, observed, worth its own change.Built with Claude Fable 5 on Oh My Pi.