Docs: ai-bot streaming modes, event schema, and throttle defaults

[lukemelia]

Updates the ai-bot README to match the code and document the streaming knobs.

Changes

• Corrected throttle defaults: AI_BOT_STREAM_THROTTLE_MS is 250 ms (was documented as 600) and AI_BOT_STREAM_MIN_DELTA is 0 (was documented as 300).
• Streaming modes: new section documenting AI_BOT_STREAMING_MODE — room-edits (default), off, and to-device — what each delivers and the to-device device-targeting fallback.
• Event schema: documents the app.boxel.response-stream to-device event payload (AppBoxelResponseStreamContent) field by field, including its cumulative-state / last-writer-wins-by-sequence semantics.
• Rationale: explains the homeserver-load trade-off between the three modes.

Base

Stacked on cs-12261-response-stream-event-schema, which implements the streaming modes and event type this documents.

Test plan

Docs-only; verified every value and field against packages/ai-bot/lib/responder.ts, packages/ai-bot/main.ts, and packages/runtime-common/matrix-constants.ts.

🤖 Generated with Claude Code

[habdelra]

[Claude Code 🤖] Reviewed as a fidelity check: since this is docs-only, I traced every documented value, field, and behavior back to the code it claims to describe (responder.ts, response-publisher.ts, main.ts, matrix-utils.ts, matrix-constants.ts) on the head branch.

Bottom line: the doc is accurate on nearly everything, with two claims that contradict the code and one that overstates a knob's reach. No blocking issues; all three are quick doc edits.

What lands right (verified, not skimmed):

• Throttle defaults match: AI_BOT_STREAM_THROTTLE_MS = 250 (responder.ts:143) and AI_BOT_STREAM_MIN_DELTA = 0 (responder.ts:193) — the old 600/300 values were indeed stale.
• The MIN_DELTA exception list ("new reasoning, a changed tool call, the first content, and the final send always go through") maps one-to-one onto the shouldSend disjunction at responder.ts:207-212.
• Streaming modes and their delivery: room-edits uses m.replace (matrix-utils.ts:42-47), off sends nothing mid-turn, to-device previews go over sendToDevice and only the final consolidated state lands as a room event (responder.ts:133-141) — all correct.
• Event schema: every AppBoxelResponseStreamContent field matches (matrix-constants.ts:146-161), sequence is per-turn monotonic from 0 (responder.ts:157), and arguments: {} until the streamed JSON parses is exactly toCommandRequest's catch branch (response-publisher.ts:33-41).

Recommendations:

1. Fix the tool-requests wire key: app.boxel.tool-requests → app.boxel.toolRequests — see the thread on the schema table (should-fix).
2. Rework the to-device fallback paragraph — a tool/code-patch continuation keeps streaming via the turn-history fallback in main.ts, rather than dropping to off; see the thread on that paragraph (should-fix).
3. Scope the throttle-tuning intro so MIN_DELTA isn't credited with gating to-device previews — see the thread on that section (non-blocking).

Adjacent, out of scope: the body character-delta gate (contentDelta >= minDelta) keys off body length only; reasoning-only growth is covered by the separate reasoningDelta > 0 term. That's consistent and not worth documenting — noting it only so the "body characters" wording in the table isn't later "fixed" to include reasoning.

---

Generated by Claude Code