[bug] Terminal has no scrollback — every snapshot is fetched then discarded except the last, and none are written while the engine is busy

[serge-ivo]

The terminal has no scrollback: every snapshot is fetched, all but the last are thrown away, and none are written while the engine is busy

Reported: the Terminal tab on a live coding agent sits at (waiting for output...), and the owner asked why output is not served from the DB and paged like chat history.

Partly it already is — the caching exists and works better than assumed. What is missing is when it writes and how much it shows.

Where (waiting for output...) comes from

agents/coder/web/src/CodingTab.tsx:352:

const newText = live || savedTerminalRef.current || "(waiting for output...)";

The placeholder means both the live capture and the DB fallback were empty. Three compounding reasons that happens:

1. Snapshots are only written while the engine is IDLE. workers/api/src/routes/coding.ts:254-258:

if (pane.trim() && runState === "idle") {
  const last = await lastTerminal(c.env, sessionId);
  if (pane.trim() !== (last ?? "").trim()) {
    await appendTimeline(c.env, { …, type: "terminal", content: pane.slice(-8000) });
  }
}

The gate is deliberate and its reasoning is sound (:250-251 — /capture runs every 3s per open session, and a read+write per poll is a lot of D1 for a pane that is not moving). But the consequence is that a session which has been busy since it started has no snapshot at all — which is exactly a Loop run working through 40 steps. The output the owner most wants to see is the output never written.

2. Nothing is written unless someone is watching. The write lives inside the /capture handler, and /capture only runs while a session view is open and polling. Work done while the tab is closed persists nothing, so reopening shows the placeholder.

3. The fallback is cleared before it is loaded. CodingTab.tsx:417 does setSavedTerminal(""), then :420 awaits /start, then :429 awaits /timeline?full=1. Between the clear and the load, savedTerminal is "" — so the placeholder flashes on every session open, even when a snapshot exists.

The bigger finding: the history is downloaded and then discarded

?full=1 returns the entire typed timeline — every terminal row, seq-ordered (coding.ts:411-413). The client then does this (agents/coder/web/src/timeline-chat.ts:69-73):

export function lastTerminalSnapshot(payload: TimelinePayload): string {
  const terminals = (payload.timeline || []).filter((e) => e.type === "terminal");
  const last = terminals[terminals.length - 1];
  return (last?.content || last?.text || "").trim();
}

Every snapshot but the most recent is fetched over the network and thrown away in the client. So this is not a "download older on demand" gap — the data is already in memory and dropped. There is no scrollback model for the terminal at all: no pagination, no "load older", no concatenation. One pane, most recent, or nothing.

That is the direct answer to "does it show only the last few and download previous on demand like chats?" — no. It shows exactly one, and there is nothing to page.

What to do — rework, in dependency order

1. Write snapshots during busy runs, not only at idle. Keep the dedup, replace the runState === "idle" gate with a throttle: persist when the pane has changed AND at most once every N seconds (N ≈ 15-30 keeps D1 volume near today's while covering long runs). The current gate optimises the case where nothing is happening and fails the case where everything is.

2. Do not clear the fallback before the reload. Keep the previous savedTerminal until the timeline resolves, or render a skeleton. One line, removes the guaranteed flash.

3. Give the terminal a real scrollback. The rows exist, are seq-ordered, and are already being fetched:

• render the snapshots as a scrollable history (newest at the bottom), not a single pane;
• add a Load older control paging backwards by seq;
• add real pagination params to /timeline (before, limit) so ?full=1 stops shipping an entire session's snapshots — a long run at 8000 chars each is a large payload for one visible pane.

This is the same defect shape as #428 (chat pagination), and the same fix pattern: a seq/key-ordered cursor with an explicit hasMore. Worth doing together so one cursor convention covers both.

4. Say why it is empty. (waiting for output...) is shown for four different states: runner offline, session starting, engine busy with nothing persisted, and genuinely no output. The route already returns runnerConnected, alive, ready and runState (coding.ts:212, 218) — the console has everything needed to say "runner offline — run pags up", "engine is working, no output captured yet", or "no output". A single placeholder for four causes is why this reads as broken rather than as waiting.

Alternatives considered and rejected

• Persist on every 3s poll. Rejected for the reason already in the code comment: a read+write per poll per open session is a lot of D1 for a static pane. The throttle in (1) gets the coverage without the write amplification.
• Stream the pane over the relay WebSocket instead of polling. Attractive, and it would make "straight away" literal — but the relay is a request/response command channel today (callRunner), so this is a transport change, not a fix. Worth its own ticket if the polling latency is the real complaint; it does not address history, which is the substance here.
• Keep only the last snapshot and drop the rest server-side. Rejected — it would make the current UI correct by deleting the capability being asked for.
• Render the whole timeline without pagination. Rejected — 8000 chars per snapshot over a long session is exactly the payload problem that motivates (3).

Acceptance criteria

• A session busy for several minutes has terminal snapshots in coding_timeline before it goes idle.
• Reopening a session shows the last known output immediately, with no (waiting for output...) flash.
• The Terminal view scrolls back through earlier snapshots, loading older ones on demand.
• /timeline accepts a cursor and does not return an entire session's snapshots in one payload.
• An empty terminal states which of the four causes applies.

Regression risk

• Raising write frequency increases D1 volume on the hottest poll in the product; the throttle constant should be measured against a real Loop run, not guessed.
• coding_timeline has no retention policy that I found; a scrollback that encourages long sessions makes that gap matter. Worth checking before (1) lands, or the table grows without bound.
• lastTerminalSnapshot is used by the Co-pilot's context too — changing it to a paged model must not change what the Co-pilot reads (it wants the most recent pane, which stays correct).

Related: #428 (identical pagination-cursor gap in chat — share the convention), #247 (the tmux→child-process move that produced this pane model), #348 (why a terminal-connector pane is unmetered — the same "a pane is rendered text" constraint).

[serge-ivo]

Shipped in f8df8a0 — all four items, verified live in production

Verified against production, just now

GET https://api.proagentstore.online/v1/instances/12ebf1f0-…/coding/sessions/csess_7e1ac098-…/timeline?terminal=1&limit=2
→ keys: ["terminal", "hasMore", "oldestSeq"]
  hasMore: true   oldestSeq: 8366   rows: 2   seqs: [8366, 8367]

The route that used to ship an entire session's snapshots so the client could keep one now answers a
bounded page with a cursor.

The four items

1. Snapshots are written while the engine is busy. lib/terminal-snapshot.ts (new, pure, tested)
   replaces the runState === "idle" gate with changed AND (idle OR 20s throttle). The idle path is
   byte-identical, so the case the old comment optimised for is unchanged; the case it failed — a Loop
   run busy through 40 steps, whose output was the output most wanted and never written — now records.
2. The fallback is no longer cleared before the reload. A loadingHistory state keeps the previous
   pane when reopening the same session. Switching to a different session still clears, deliberately:
   showing repo A's output under repo B's header is worse than the flash.
3. Real scrollback. ?terminal=1&before=<seq>&limit=<n> → {terminal, hasMore, oldestSeq}, an
   exclusive keyset cursor on seq — not an offset, because /capture appends while you scroll and an
   offset page shifts under you. hasMore comes from limit+1, no COUNT. ?full=1 is retained unpaged
   for the copy-as-JSON button only. terminal-history.ts stitches overlapping panes; without it a paged
   scrollback renders the same output five times over and is worse than the single pane.
4. The placeholder names which of the four causes applies — runner offline, session starting, engine
   busy with nothing persisted, genuinely no output.

lastTerminalSnapshot is deleted (superseded). The Co-pilot still reads the most recent pane, which was
the named constraint.

Two things stated as arithmetic, not dressed up as measurement

• The 20s constant is a bound, not a measurement. The issue asked for it to be measured against a
  real Loop run; that was not run here, so it is justified as poll cadence × cap (~3 rows/min, ~180/hour
  at ≤8000 chars) and the arithmetic is written next to the constant.
• coding_timeline has no retention policy — checked because the issue asked. Rows are deleted only
  when a repo's sessions are deleted (coding-store.ts:280) or the chat is cleared; neither touches
  terminal rows. Raising write frequency makes that matter sooner. A retention policy is a migration
  and separable work — flagging it rather than filing it, since issues here are the analyst's to write.

#428 is not implemented — only the cursor convention was designed to be shareable with it.

Full bar green, including pnpm test:e2e chromium+webkit (349 passed). The #431/#454 header-height
assertions still pass; CodingTab.tsx:989 untouched. CI, API-Worker and Host-Worker deploys all green.