list_instance_tools says "a tool absent from the allowed set cannot be invoked, by chat or by call_instance_tool" — 11 tools every agent runs in chat are absent, and 2 of them ran on the audited instance

[serge-ivo]

The claim

list_instance_tools describes itself, verbatim (workers/mcp/src/instance-tools/base.ts:29):

"Returns EVERY registry tool with this instance's verdict … Use this to verify an agent is read-only before trusting it with sensitive data — a tool absent from the allowed set cannot be invoked, by chat or by call_instance_tool."

The second half is a security claim, and it is half wrong. It is true for call_instance_tool. It is false for chat.

Measured

Instance bd43f4de-ef35-4051-bdec-43f8571414a1 (Repo Coder, capabilities.tools declared), 2026-08-12:

• GET /v1/instances/:id/tools returns 70 entries; ?allowed=true returns 22.

• toolNamesFor() grants every agent the 19-name BASE set (workers/api/src/agent-do-tools.ts:15-…, and toolNamesFor at :203-205 starts from new Set(BASE) whenever an allowlist is declared).

• 11 of those 19 appear nowhere in the listing, allowed or otherwise:

  read_memory · write_memory · delete_memory · get_tasks · create_task · update_task · fetch_url · get_activity · get_user_context · set_user_preference · configure_board

Six of the eleven write (memory create/delete, task create/update, user preference, board layout). One — fetch_url — makes outbound HTTP.

Two of them were invoked on this very instance within the audit window, both visible in agent_trace:

• 2026-08-11 21:45:52 — ❌ **fetch_url** HTTP 401 Unauthorized: … (it ran; the remote refused)
• 2026-08-12 03:22:21 — ✅ **write_memory** Stored memory: fact:Staging environment:decision. All memory keys: …

So an auditor who did exactly what the tool's description tells them to do — call list_instance_tools, confirm nothing that writes is allowed — would conclude this agent cannot write to its own memory. It wrote to its own memory nineteen hours later.

Mechanism

Two layers, each correct in isolation:

• The listing enumerates the registry only. resolveToolPolicy (workers/api/src/lib/instance-tool-policy.ts:118-143) maps over registryTools() — [...REGISTRY.values()] (lib/tool-registry.ts:467). The DO's own catalog (lib/tools.ts AGENT_TOOLS, lib/storage-tools.ts STORAGE_TOOLS) is a different collection and is never walked. That is why start_work, check_work, get_behaviour, run_pipeline, record_feedback and get_stats DO appear — they happen to be registry tools — while write_memory and fetch_url do not.
• The gate the description describes is real, and it is only on one route. POST /v1/instances/:id/tools/:name calls getRegistryTool(name) and 404s a non-registry name (routes/tools.ts:141-146), so call_instance_tool genuinely cannot reach them. The chat runtime uses toolNamesFor + buildAgentToolDefinitions, which include them.

instance-tool-policy.ts's own header states the invariant this breaks: "'This agent is read-only' has to be a property of the INSTANCE, not of one surface's prompt-building." The policy module honours that for the registry; the listing built on it silently reports on one collection and is described as reporting on the agent.

What to do, cheapest first

1. Fix the sentence. One-line docs change, ships today, removes the false assurance:

   "Returns every connector/registry tool… A tool absent from the allowed set cannot be invoked through call_instance_tool. This listing does not cover the agent's built-in facilities (memory, tasks, board, fetch_url), which every agent has; use it to audit an agent's reach into external systems, not to establish that it cannot write at all."
   Do this even if (2) is taken, because (2) is a behaviour change and this is a correctness fix to a security claim.

2. Make the listing exhaustive. Include the BASE/storage tools with a tier field (base vs connector) so a caller can still ask the narrow question. ToolDef already carries tier: "connector" (see repo-local.ts:362), so the vocabulary exists. resolveToolPolicy takes its tool list as a parameter — the change is at the call site in instanceToolPolicy, not in the pure function.
3. Decide whether disabledTools should cover them. PUT /v1/instances/:id/tools/:name also gates on getRegistryTool, so the owner's veto — described as "the one control a creator's declaration must not outrank" — cannot currently switch off write_memory or fetch_url. If (2) ships without this, the listing will show a tool the owner is shown no way to disable.

Open question for the owner

Is (2) wanted, or is the honest scoping in (1) enough? I would take (1) now and (2) next. The listing's audience is an auditor asking "what can this thing reach", and fetch_url — arbitrary outbound HTTP, SSRF-guarded but not consent-gated — is squarely inside that question. But (2) drags in (3), which is a real permissions decision, and the false sentence should not wait for it.

Alternatives rejected

• Leave it; the base tools are "universal so they don't count." The description does not say that, it says the opposite, and it says it in the context of trusting an agent with sensitive data. A universal capability is still a capability an auditor has to know about.
• Remove the security sentence entirely. It is answering a real question and the answer is nearly right; scoping it is better than deleting it.

Acceptance

• The description no longer asserts that absence from the list implies the tool cannot be invoked by chat.
• If (2): GET /v1/instances/:id/tools for bd43f4de-…-shaped instance includes all 19 BASE names, and a test asserts toolNamesFor(caps) ⊆ {names in the listing} for a representative capability set — the invariant that would have caught this.

Regression risk

resolveToolPolicy is consumed by the agent runtime, the REST invoker and MCP (its header says all three share it). Adding entries changes what ?allowed=true returns, which the console's tool list and explainWriteConsent render — a base tool with no connector resolves to writeConsent: "n/a" by writeConsentOf's first branch, so it must not start rendering as ungated external access. contract.test.ts and tools.integration.test.ts are the guards.

Related: #351 (the last time this listing was confidently wrong in the permissive direction), #493 (the agent's own tool list never stated to be exhaustive — same failure, one surface over).

[serge-ivo]

Fixed in 7d11035 (on main, deployed — API/host/MCP workers all green).

What shipped — option (1) and (2) from the issue, plus (3), because (2) without (3) would have listed a tool the owner is shown no way to disable:

• lib/builtin-tool-policy.ts supplies the missing rows, derived from the same AGENT_TOOLS/STORAGE_TOOLS definitions the model is shown, through the same schema converter (legacyToolSchema, now shared with buildAgentToolDefinitions) — so the audit surface and the agent cannot describe different tools. scope cannot be derived from a ToolDef, so it is declared once in BUILTIN_TOOL_SCOPES; a test fails when a definition has no entry, and an unknown name resolves to write, since under-reporting a write is the direction of this bug.
• Every row now carries tier and invocableBy. invocableBy is the distinction the old sentence flattened: a built-in reports ["chat"] (the agent runs it; the invoker cannot reach it), a registry tool reports ["chat","call_instance_tool"]. So the narrow question — "what can this reach OUTSIDE the platform" — is still answerable, by filtering on connector/tier, instead of the listing silently answering it for you.
• (3) the owner’s off-switch now gates on the LISTING, not getRegistryTool. agent-think.ts:828 has always subtracted config.disabledTools from every resolved name, built-ins included — so the veto worked everywhere except at the route that sets it. write_memory and fetch_url are now switchable.
• POST …/tools/:name answers a built-in name with what it is rather than "Unknown tool" for a tool the same API just listed.
• The MCP description no longer claims what it cannot keep; the console’s "This agent has no tools that can change anything — it can only read" was the same false assurance rendered over a switch list, and now separates reaching outside the platform from changing the agent’s own data.

Verified in production on the audited instance bd43f4de-… and on e4d2d031-…:

GET /v1/instances/bd43f4de-…/tools  ->  104 entries (was 70), 35 allowed
  write_memory      allowed=True scope=write tier=base      invocableBy=["chat"]
  fetch_url         allowed=True scope=read  tier=base      invocableBy=["chat"]
  create_task       allowed=True scope=write tier=base      invocableBy=["chat"]
  configure_board   allowed=True scope=write tier=base      invocableBy=["chat"]
  github_create_issue allowed=True scope=write tier=connector invocableBy=["chat","call_instance_tool"]

POST /v1/instances/…/tools/write_memory -> 404
  "write_memory" is one of the agent's built-in tools: it runs in the agent's own chat loop and
  nowhere else. This route (and MCP call_instance_tool) dispatches connector tools — see
  `invocableBy` on GET /v1/instances/:id/tools.

All eleven names from the issue are present and allowed, with the six writes reported as writes.

The guard: builtin-tool-policy.test.ts asserts the containment the issue asks for — for each capability shape toolNamesFor branches on, every tool the chat resolves appears in the listing AND reads allowed:true. That is the assertion that would have failed the day the sentence was written. routes/tools.test.ts had "No write tool survives for a read-only agent — the assertion an auditor actually wants", and it passed only because the writes were hidden; it now asserts no write with a connector survives, with the built-in writes asserted present beside it.

One judgement call to flag: fetch_url stays scope:"read", for the same reason http_request is — the tool does not choose to write, the caller picks the verb — and its method/body schema rides on the same row. Classifying it differently from the registry tool with the identical shape would put two answers to one question in one listing.

Not done: find_confirmation_link is now listed (so an auditor asking "can this read my mail" gets a row instead of silence) but reports reason:"not_declared", because its gate is the permissions.email toggle in DO state, not the declaration — reading that would put a DO round-trip on the listing and on both gate routes. If you want allowed to be true when the permission is on, that is a small follow-up issue.

Not verified: the console Settings → Tools screen was not opened in a browser (the switch list now has ~19 more rows on every agent); its logic is unit-tested and the API is confirmed live.

[serge-ivo]

Closing: shipped in 7d11035, CI green, deployed, and confirmed live on the audited instance (70 → 104 rows; all eleven BASE names present, allowed and correctly scoped). The containment invariant is in the suite, so the sentence and the listing can no longer drift apart. The find_confirmation_link permission-state nuance is recorded in the comment above as a possible follow-up, not as unfinished work here.