[bug] Every chat reply is capped at 1024 output tokens and truncation is never reported — stop_reason is read by nothing

[serge-ivo]

Every Anthropic-backed chat reply is capped at 1024 output tokens, and nothing says when it hits

user-ai.ts:108:

max_tokens: body.maxTokens ?? 1024,

The chat loop never passes one. grep maxTokens workers/api/src/agent-think.ts → nothing; both
runUserWorkersAi calls in the chat path send { messages, tools } and take the default.

Every other call site sets its own, deliberately: loop-orchestrator 300, coding-copilot 160/600,
coding-brains 700, resume-parse 2000, the pipeline ai_generate step 500. Chat — the surface
that produces by far the longest text, and the only one a human reads end to end — is the single
caller that never chose, and it inherits ~1024 tokens ≈ 4,000 characters.

And a truncated reply is indistinguishable from a finished one

runAnthropic reads content and usage off the response (:183-197). stop_reason is never
looked at — grep -rn stop_reason workers/api/src finds it only in the loop-run store and
instance-work, both about a different thing entirely (a run's outcome).

So when a reply is cut at the cap the platform has the fact, in the response body, and discards it.
Nothing marks the message, nothing warns the user, nothing logs it.

Seen in the wild, in the transcript that prompted #395

The Chess coder message of 2026-08-07 23:24:31 ends, mid-file, mid-import:

"use client"

import { useState } from "react"
import { useRouter } from "next/navigation"
import

That is not a formatting artefact. The message stops on import because the generation stopped
there. It is a long reply, it hit the cap, and it was stored and displayed as though it were the
whole answer.

Why it matters more on this product than on a chatbot

The agents most likely to exceed 4,000 characters are the technical ones — a Repo Coder explaining
a file, an agent listing issues with bodies, anything quoting code — and guardrails.responseStyle: "technical" explicitly asks them to cite files and functions. The cap bites hardest exactly where
the product is trying to be most useful.

The codebase already learned this lesson once, one directory away
(routes/instances-translation.ts:280): "Explicit max_tokens: the Workers AI default is small
enough to TRUNCATE". Chat never got the same treatment.

Suggested fix

1. Set an explicit cap for chat, sized for a reply a person reads — 4k is a reasonable starting
   point for claude-sonnet-4-6 — rather than inheriting a default that was chosen for nothing in
   particular.
2. Read stop_reason. When it is max_tokens, say so: mark the message, and log it, so
   "the agent stopped mid-sentence" is a fact on the record instead of a user's impression. This is
   the same rule the voice stack now follows for a turn it could not transcribe ([bug] A voice turn rejected as noise is erased along with the live capture — nothing survives, nothing is logged #377) and the loop
   for a cancel it accepted ([bug] Stop on a loop looks like it does nothing — the server records the cancel and no UI ever reads cancelRequested #376) — the platform knows, so the user should.
3. Consider making the cap a per-agent setting: a terse operator agent and a code-explaining Repo
   Coder do not want the same ceiling, and settingsSchema already exists for exactly this kind of
   per-agent number.

Files: workers/api/src/lib/user-ai.ts:108,183-197, workers/api/src/agent-think.ts:700-706.
Related: #395, #397 (the same loop's tool-result protocol).

[serge-ivo]

Shipped in a761a4f. Deployed (API worker green).

Chat was indeed the only caller of runUserWorkersAi that never named a maxTokens. Rather than adding a fifth literal, the four hand-written argument lists in the chat path (tool-less, per-round, #395's correction round, final answer) collapsed into one chatComplete wrapper — the file now has fewer provider calls than before it was fixed. It sets CHAT_MAX_TOKENS = 4096, reads the new stopReason, and writes a durable chat.truncated row the first time a turn hits the cap.

The user-facing half rides #395's notice channel: a leading ⚠️ **platform** line naming the cap and suggesting you ask it to continue. So "the agent stopped mid-sentence" is now a fact on the record rather than an impression — the rule #377 and #376 already follow.

runAnthropic surfaces stopReason on both returns, not just the text one: a round that stops mid-tool_use is the same loss with worse consequences.

Item 3 deliberately not done — a per-agent cap via settingsSchema is a new subscriber-facing field, not this fix. Say the word and I'll have it filed.

One thing left as-is on purpose: the ?? 1024 fallback in user-ai.ts. It is what a new call site inherits by forgetting, and raising it would silently raise every existing caller's bill.

Verified: typecheck, full suite (+46 tests across the three issues in this lane), biome, design tokens, file-size ratchet. Not verified live — a real long reply on a technical agent is the check: it should now run past ~4,000 characters, and a genuinely capped one should carry the platform notice.

[serge-ivo]

Audit (picked this up for implementation and found it already delivered): shipped in a761a4f, which is on main and deployed. No further code was needed.

Verified against the acceptance rather than the commit message:

• CHAT_MAX_TOKENS = 4096 in workers/api/src/lib/reply-truncation.ts, applied through the single chatComplete wrapper at agent-think.ts:687 — and it really is single: all four provider calls in the chat path (:716 tool-less, :846 per-round, :816 the [bug] The model writes its own tool RESULTS and the platform believes them — invented GitHub issues and file contents reported as fetched, raw markup left in the message #395 correction round, :1030 final answer) route through it, so there is no fifth argument list left inheriting ?? 1024.
• stopReason is read (hitOutputCap), a durable chat.truncated row is written once per turn, and the user-facing notice rides [bug] The model writes its own tool RESULTS and the platform believes them — invented GitHub issues and file contents reported as fetched, raw markup left in the message #395’s channel.
• The ?? 1024 fallback in user-ai.ts is still there on purpose, which I agree with — it is what a new forgetful call site inherits, and raising it would silently raise every existing caller’s bill.

Item 3 (per-agent cap via settingsSchema) was deliberately not done and is a genuine new subscriber-facing field, not part of this fix.

Closing as delivered. Still not verified live — the check named in the shipping comment stands: a long reply on a technical agent should now run past ~4,000 characters, and a genuinely capped one should carry the ⚠️ platform notice. Worth doing, but it does not block the ticket.