search_knowledge / read_knowledge / read_file return the same corpus the RAG block fences — the two paths are 40 lines apart in retrieval.ts and only one is fenced

[serge-ivo]

Part of the ingress enumeration that #725 and #746 come from. This one is the sharpest of the set, because the fenced path and the unfenced path are 40 lines apart in the same module, over the identical corpus.

The problem, from the owner's position

Repo Chat indexes a GitHub repo into the instance's vector store. The owner asks a question. Two things can happen:

• The chat turn's automatic RAG block retrieves the same chunks → they arrive fenced, and the model is told never to obey what is inside them.
• The model calls search_knowledge or read_knowledge for the same chunks → they arrive bare.

Same bytes, same source, same turn, opposite treatment. Which one the owner gets depends on whether the model decided to call a tool.

Where it is — VERIFIED

grep -c fenceUntrusted workers/api/src/lib/storage-tools.ts → 0.

lib/retrieval.ts holds both halves:

• Fenced — lib/retrieval.ts:136 (ragContextOrNotice, the automatic block):

  return context ? fenceUntrusted(context, "documents/URLs/repos/webhooks") : "";

• Not fenced — lib/retrieval.ts:99-113 (searchKnowledgeFor, the tool path) returns { ok: true, results } raw. Its one caller, lib/storage-tools.ts:273-279, stringifies them straight into the tool result:

  `${JSON.stringify(found.results, null, 2)}\n\nTo read around a file match and quote it exactly: read_file with id=sourceId and offset ≈ …`

  VectorSearchResult.text (agent-storage-types.ts:43) is the chunk text itself.

The other doc readers are bare too:

• lib/storage-tools.ts:293 — read_knowledge

  return ok(call.name, JSON.stringify({ id: doc.id, title: doc.title, content: doc.content }, null, 2));

• lib/storage-tools.ts:282-287 — list_knowledge (titles).
• lib/storage-tools.ts:349-382 — read_file (extracted document text, windowed).

The corpus is untrusted by the platform's own account. agent-think.ts:296 names it — "documents, ingested URLs, repo files and public webhook payloads, any of which an attacker can author" — and routes/public.ts:260-262 says of the unauthenticated ingest route:

it ingests third-party content (Zapier/Make/n8n payloads) straight into an instance's vector store, which is the untrusted-content path (#263).

Nothing re-fences downstream: agent-think.ts:994-997 (record) only caps length.

Reachability — MEASURED on production, 2026-08-23

GET /v1/instances/267286d5-6877-4f2a-8b5c-3c40b3e3be85/tools (Repo Chat), read-only:

search_knowledge  read  allowed=True  ok
read_knowledge    read  allowed=True  ok
list_knowledge    read  allowed=True  ok
fetch_url         read  allowed=True  ok

That is the exact chain agent-think.ts:296 says the fence exists to stop — "chain read-tools + fetch_url into an exfiltration of the owner's private data" — with the read-tool half unfenced. search_knowledge/read_knowledge are declared on 2 live instances; read_file on 1; fetch_url is a BASE tool and is on essentially all of them.

Mechanism

searchKnowledgeFor and ragContextOrNotice were split so the tool path could report a retrieval outage honestly (#628). The split is right. What went with it, silently, is that the fence lived on only one branch: buildRAGContext returns a prose block that ragContextOrNotice wraps, while searchKnowledgeFor returns a typed array that its caller stringifies — so there was no single "the retrieved text" string for the fence to sit on when the second path was written.

Compounding it: storage-tools.ts is not a connector, so it is outside the vocabulary of both #263 and #308, and outside security-invariants.test.ts:400's four-entry FENCES_REMOTE_TEXT map. It was never a candidate for the guard, because the guard's list is of connector modules.

What to do — cheapest first

1. Fence the three doc readers at the tool result. Origin strings should say what the source actually is, since the corpus is mixed:

// search_knowledge
return ok(call.name, fenceUntrusted(JSON.stringify(found.results, null, 2), "documents/URLs/repos/webhooks indexed in this agent's knowledge base") + "\n\nTo read around a file match…");

Keep the platform's own "To read around a file match…" hint outside the block — it is our instruction and the model must obey it. Same treatment for read_knowledge (:293) and read_file (:349-382); list_knowledge carries titles only and is the weakest case — include it or say why not.

2. Do NOT fence add_knowledge/update_knowledge results. Those are the platform reporting an outcome; fencing a confirmation teaches the model a fence marks nothing in particular, which is the failure lib/connectors/gmail.ts:60-70 warns about.

3. Add lib/storage-tools.ts to the guard. See the enumeration/ADR issue filed alongside; do not block this fix on it.

Alternatives considered and rejected

• Fence inside searchKnowledgeFor by returning a pre-fenced string. Rejected: it returns a typed VectorSearchResult[], which VectorsSection-style console callers and future non-model readers want structured. Fence at the tool-result seam, where the reader is known to be a model.
• Rely on the automatic RAG block and drop search_knowledge. Rejected: it is Repo Chat's declared toolset (migration 0050) and the whole point of that agent being "built the creator way". Removing it would regress feat(agents): declarative tool catalog + per-agent tool allowlist (#51) #59/Epic: agents live outside the monorepo — enforce the platform-only boundary #50.
• Fence only read_knowledge, on the grounds that search returns short chunks. Rejected: chunk size is ~512 chars and top_k goes to 20 — 10KB of attacker prose is not a short chunk, and a single chunk is more than enough for one instruction.
• Fence at agent-think.ts:994 for all tool results uniformly. Rejected here for the same reason as in GitHub issue and PR bodies reach the model unfenced — text any stranger can author on a public repo, on 22 of 42 live instances, one of which also holds a consented shell #746: it covers one of the four surfaces a tool answers, and it would fence the platform's own refusal strings. Belongs in the structural issue, not this one.

Acceptance criteria

• search_knowledge, read_knowledge and read_file results contain exactly one <untrusted_reference_material …> block, around the retrieved text only.
• A doc whose content contains </untrusted_reference_material> does not close the block early (assert exactly one closing marker) — mirroring untrusted-fence.test.ts:23-27.
• The platform's own trailing hint on search_knowledge is outside the block.
• RETRIEVAL_EMPTY_MESSAGE and the fail(...) paths are NOT fenced (they are platform statements).

Regression risk

• The MCP surface exposes search_instance_knowledge / read_knowledge-equivalents to an external client (Claude Desktop). Fencing at the source means those clients now see the marker. That is correct — the same remote text is untrusted there too — but it is a visible change to workers/mcp output; check platform-docs/mcp.md does not promise a bare payload.
• Prompt size: ~40 tokens per call.
• Test to catch a regression: a case in lib/storage-tools.test.ts asserting the fence on all three readers, plus the marker-neutralisation case.

Verified vs inferred

Verified: the grep (0 in storage-tools.ts), every file:line above, that VectorSearchResult.text is the chunk body, that the same vectorSearch backs both paths (agent-storage/context.ts:77 for the fenced one, retrieval.ts:105 for the bare one), the codebase's own labelling of the ingest route as untrusted, and the live tool verdicts on Repo Chat.
Inferred: nothing load-bearing. No injected document was planted and no agent was driven — read-only sweep.

[serge-ivo]

Parent / structural fix: #752 (ADR 0006 + replacing the four-name pin-list guard with a per-tool untrustedOutput declaration). Siblings from the same enumeration sweep: #746 #747 #748 #749 #750 #751, plus #725 (Gmail) and the in-flight find_confirmation_link fix. Each is independently shippable — do not block on #752.

[serge-ivo]

Shipped in 4264497d (CI green, API Worker deployed).

What changed. New workers/api/src/lib/knowledge-result.ts holds four pure builders and the placement rule; lib/storage-tools.ts calls them for search_knowledge, list_knowledge, read_knowledge and read_file. A module rather than four inline fence calls because storage-tools.ts was three lines under its size pin and the repo rule is split first, then raise — it stays at 823/826 and no pin moved.

Inside the block: the chunk JSON, a document’s title and content, a file’s NAME and its text window. Outside: the platform’s own instructions the model must obey — "To read around a file match…" and "call read_file again with offset=…".

list_knowledge IS included (the body left it open). A document title is authored by whoever supplied the document, including the unauthenticated webhook-ingest route.

One correction to the ticket, in your favour. The stated regression risk — "the MCP surface exposes search_instance_knowledge … those clients now see the marker" — does not arise. executeStorageTool has exactly one caller, agent-think.ts:1093 (the chat tool loop). These are not registry connector tools, so no pipeline step, no POST /v1/instances/:id/tools/:name and no MCP proxy path reaches them. That same fact is what makes it safe to leave the search_knowledge hint OUTSIDE the fence: trailing prose defeats unfenceUntrusted’s anchored regex, but nothing downstream $refs these results. platform-docs/mcp.md needed no change, and pnpm docs:drift is green.

Verified

• grep -c fenceUntrusted workers/api/src/lib/storage-tools.ts → 0 before the change, at 1f0d3a33.
• Both halves in lib/retrieval.ts at the coordinates given: fenceUntrusted in ragContextOrNotice, the bare {ok:true, results} from searchKnowledgeFor, and its single caller.
• Acceptance criteria as tests: exactly one block per reader; the hint outside; a document containing </untrusted_reference_material> produces exactly one closing marker; RETRIEVAL_EMPTY_MESSAGE, "The knowledge base is empty.", the add/update confirmations and every fail(...) stay unfenced. One test drives the REAL storage engine, not the pure builders — the regression that matters is a call site forgetting to use them.
• Not vacuous: neutering the fence turns 7 red. Full suite 10,454 passed; biome, all typechecks, file-size, docs-drift and the rest of ci.yml green.

Not verified

• No injected document was planted against a live agent and no agent was driven — same read-only posture as the sweep that filed this. The finding was that the platform’s own defence was absent on one of two paths over the same bytes; that is what was fixed and what is asserted.
• Prompt-size effect (~40 tokens per call) is computed from the fence preamble, not measured on a live turn.

Step 3 of the body — adding lib/storage-tools.ts to the guard — was deliberately NOT done. Making fence coverage countable is #752’s; a fifth entry in a four-name pin-list is the thing #752 exists to delete.