[bug] A caught fabrication is only caught once — the invented turns stay in history and the model re-reads them as fact

[serge-ivo]

A fabrication that was caught yesterday is still being read as fact today

#395's guard works — verified live on Chess coder (26f71cd8), 2026-08-08 01:36 and 01:38:

⚠️ platform  The agent wrote tool results that no tool produced. They were removed, and it
             answered again from the results the platform actually has.
⚠️ platform  Named but never run this turn: repo_read_file.

It protects the turn it is on. It does nothing about the turns that got through before it shipped,
and those are still in the transcript.

The specific rows

GET /v1/instances/26f71cd8…/messages still returns, as ordinary assistant messages:

• 2026-08-07 08:31:20 — the message containing hand-written <tool_call> / <tool_response>
  pairs asserting repo_remote → pas-platform/chess-academy (the real call returned no remote)
  and github_list_issues → three issues (the call never ran).
• 2026-08-07 23:23:36 — "Yes, three open tickets as I just fetched", listing them again.
  That turn ran no tools at all. It is the model reading its own earlier invention out of
  history and re-asserting it as a fresh fetch, fifteen hours later.

Both rows are still there, still shaped like assistant answers, and still eligible to enter the
context window of the next turn. The 23:23 message is the proof that this is not theoretical: it is
what a stored fabrication does on its second pass.

Why the current guard cannot reach them

The guard is a check on generation — it compares what the model claims against what the platform
ran, for the turn in flight. A message written before it existed carries no marker, and nothing
re-examines stored history. So the invention has become durable context, and it now has better
standing than a real tool result would (a tool result is prose in an assistant turn too — #398).

Do

• Mark, don't delete. Stamp a stored assistant message that the guard rejected, and stamp
  the ones it can identify retroactively, so a reader and the context builder can both tell.
  Deleting rewrites the record; the platform's own habit elsewhere (a rejected voice turn, [bug] A voice turn rejected as noise is erased along with the live capture — nothing survives, nothing is logged #377)
  is to keep the evidence and label it.
• Exclude a marked message from the context sent to the model. This is the part that stops
  the compounding. A fabrication the user can still scroll back to is a transparency feature; one
  the model can still read is a lie with a memory.
• Show it in the console — the transcript should not render an invented answer identically
  to a real one.
• Decide about the existing rows. A one-off backfill can find the obvious ones (an assistant
  message containing <tool_response> or </parameter>), but only those. Anything subtler — the
  23:23 message contains no markup at all, just the invented content — cannot be found
  mechanically. Options: leave them and rely on exclusion-by-marker going forward, or offer the
  owner a "clear this conversation" as the honest reset. This one is a decision, not a task.

Related: #395 (the guard), #398 (why a result and a claim are indistinguishable in the first
place), #405 (why this agent is under constant pressure to invent).

[serge-ivo]

Three of the four items shipped in fb1ae10. CI green (typecheck × 11, 6,393 tests, biome, tokens, ratchet, migrations, isolation, docs drift, e2e); API + Host workers deployed successfully. Leaving this open — item 4 is yours, and nothing here is verified live yet. Both are spelled out at the bottom.

What landed

The quarantine is computed at READ time, not backfilled. lib/fabricated-history.ts → isFabricatedRecord is evaluated from a message's own text every time history is read. That is the design decision worth arguing with, so: a one-off backfill could only ever run once, over the instances awake at the time, and would then be a second record of the same fact — free to drift from stripToolMarkup, which is what actually decides. Read-time reaches a row in a DO that has been asleep since August, and it reaches it correctly if the stripper is ever improved.

The standard is #395's, unchanged: the platform never writes tool-result markup into an assistant message, so a result block inside one is not evidence of a fabrication, it is one. Two consequences that bound it, both tested — it is scoped to legacy rows by construction (since #395 a reply carrying this markup cannot be stored, only stripped/corrected/replaced), and a user message is never quarantined whatever it contains, because pasting tool markup into the chat is a legitimate thing for a person debugging an agent to do and it claims to be nobody's record.

Item 1 — mark, don't delete. /messages stamps fabricated: true on the offending assistant rows and serves the text exactly as written. Nothing is deleted and nothing is edited: the user's next message chose what to build from three issues that do not exist, so that row is the only thing that shows them what they acted on.

Item 2 — excluded from the model's context, at both readers. The obvious one is the recent-message window in agent-think.ts. The one that matters more, and which the ticket does not name, is agent-storage/summaries.ts: generateSummary builds its transcript from raw stored messages, so a fabrication reaching it is distilled into fact:* memory entries that are injected into every future prompt and outlive the conversation. The chat window ages an invented turn out after ten messages; the summariser turns it into a belief. Both now redact first, and fabricated-history.test.ts asserts it over the source of both — because the failure mode here is a third reader being added beside them.

Withheld, not dropped. A hole leaves the user's question apparently unanswered, which invites the model to answer it again from whatever it can reach, and it is the silent strip #395 rejected pointed at the model instead of the user. WITHHELD_TURN states the one fact that matters — nothing in that turn was fetched — and repeats none of the invention.

Item 3 — the console. A stamped bubble gets a red edge and a line above its text: "Not fetched. This reply wrote its own tool results — nothing in it was read from a tool. It is kept as a record and is withheld from the agent's context." Red rather than the platform's yellow: yellow means "the platform is reporting" (SystemMessage, #336) and this is a stronger claim — the content directly below it is false.

Item 4 — half of it is now made by construction, half is still yours

Because the marker is retroactive rather than going-forward, the 08:31 row is already covered: it is stamped for you and withheld from the agent today, with no migration and no script. That was the better half of the option you framed as "leave them and rely on exclusion-by-marker going forward".

What is genuinely left is the 23:23 row and anything like it — invented content, no markup, nothing mechanical finds it. I did not add a heuristic for it, deliberately: a false positive there silently deletes a good answer from the agent's memory, and you had already ruled it unfindable. There is a test asserting that limit rather than leaving it implied, so nobody later reads the quarantine as total. Removing the cause is what was available: with the 08:31 row withheld, the 23:23 answer has nothing to read back. Your call between leaving it and clearing that conversation (which already exists, and now costs you less because the obvious row is handled).

Not verified live — the check I could not run

I have no production session from here, so the two observable facts are unconfirmed:

GET /v1/instances/26f71cd8…/messages?limit=200   → the 2026-08-07T08:31:20 row carries "fabricated": true
console → Chess coder → Assistant, scroll to 08:31 → red edge + the "Not fetched" line

The public trial-chat probe I could run returns 402 (needs a BYOK key), so it proves the API deployed and routes, and nothing about this. Also not checked: WebKit at 320/390px — the state is not reachable from the e2e fixtures, which have no fabricated row, so the notice's wrap on a narrow screen is untested. If either looks wrong I will fix it forward.

Related: #395 (the guard, closed), #398 (why a result and a claim were the same shape, closed), #405 (why this agent was under pressure to invent, shipped in 5527cea).

[serge-ivo]

Items 1-3 delivered. Item 4 was a decision, and it no longer has a subject. Closing.

fb1ae10 is on main and in the deployed API (ancestor of b6f6c9f6, deployed 13:02 UTC today).

The three tasks

• Mark, don't delete. agent-do.ts:815 stamps fabricated: true on every read of /messages.
  Computed at READ time rather than backfilled, which is what lets it reach rows written long
  before it existed — including in a DO that has been asleep since. The row itself is untouched,
  so the user can still scroll back to the text they acted on.
• Excluded from the context sent to the model. redactFabricatedHistory is applied by both
  readers that hand stored history to a model: the recent-message window (agent-think.ts:27,668)
  and — the one that actually mattered — the summarizer (agent-storage/summaries.ts:8). The chat
  window ages an invented turn out after ten messages; generateSummary distils it into fact:*
  memory that is injected into every future prompt and outlives the conversation. A fabrication
  that reached that function stopped being a message and became a belief.
  fabricated-history.test.ts asserts both, because the failure mode here is a third reader
  being added beside them.
• Shown in the console. components/FabricatedNotice.tsx + a red edge on the bubble
  (pages/InstanceDetail.tsx:1136, 1166) — red rather than the platform's yellow, deliberately.

Withheld rather than holed: WITHHELD_TURN (lib/fabricated-history.ts:95) states the one fact that
matters and repeats none of the invention, so there is nothing left to read back as a result.

Item 4 — "decide about the existing rows"

Measured before recommending anything. Across all 28 instances on the account, 1,317 stored
messages, zero carry the stamp. And the instance the ticket is about — 26f71cd8…, which held both
named rows (the 08:31 markup row and the 23:23 restatement) — is no longer on the account; it does
not appear in /v1/instances/my/instances.

So the decision the ticket parked — backfill the findable ones, or offer a "clear this conversation"
as the honest reset — has nothing left to decide between. The rows it was about are gone, and no
surviving message anywhere trips the detector. The mechanism stands for anything written from here on.
Scope of that claim, stated plainly: one account, which is every account PAGS has today.

The limit stands, on purpose, and is pinned

The 23:23-style row — invented content, no markup — is not reachable and never will be by this
method. lib/fabricated-history.ts:56-59 says so, and fabricated-history.test.ts:66 pins it
("cannot reach the follow-up turn that has the invention but no markup"). That test is worth keeping
exactly as it is: the standard this quarantine stands on is that the platform never writes tool-result
markup into an assistant message, so the markup is the forgery. Loosen it to catch content and a
false positive silently deletes a good answer from the model's view, which is a worse failure than the
one being fixed. Removing the cause is what this module can do — with the 08:31 row withheld, the
23:23 answer has nothing to read.

Closed.