Skip to content

Non-tool-capable default model silently disables ALL tools — a structured-data agent can't query its own collection #100

Description

@serge-ivo

Root cause found tracing the lead-finder chat (2026-08-02). A generic agent created without an explicit model gets DEFAULT_MODEL = @cf/meta/llama-3.2-3b-instruct (agent-do-prompt.ts:3, applied by ensureStateDefaults when state.model is empty). That model is not in TOOL_CAPABLE_MODELS, so:

agent-think.ts:261   const useTools = TOOL_CAPABLE_MODELS.has(state.model);   // → false for 3B

When useTools is false the chat is built with no tool definitions and no tool loop. The agent silently loses query_records / insert_record / list_collections / write_memory / fetch_url — everything. Its only grounding becomes the auto-injected RAG context (buildRAGContext → vector search over KB/files/summaries). Collection records are not vectorized, so they are invisible to it.

Real-world consequence (lead-finder instance 933ebec5…)

  • It has a populated leads collection (~48 records) but cannot read it — the chat can't call query_records.
  • To make leads answerable at all, they were duplicated as prose KB docs so RAG could surface them.
  • Result: fuzzy retrieval — "Richmond only" returned a top-k subset, counts are approximate, and a 3B model is listing real business names/phones (hallucination surface). See the agent-repo design issue.

The footgun

Nothing warns the creator. create_agent defaults the model to empty → 3B → a silently tool-less agent. Anyone building a collections/memory/fetch agent hits this and can't tell why their agent "ignores its tools."

Fix options (pick, or combine)

  • On create_agent/update_agent, if the agent declares tools (or capabilities that imply tools) and the model isn't tool-capable, warn (or reject) with a clear message.
  • Surface model tool-capability in the console (a badge: "this model can't use tools").
  • Consider a tool-capable default for agents that declare a non-empty capabilities.tools, or fall the platform-AI path to a tool-capable CF model when tools are needed.
  • At minimum, log/emit when a turn drops tools purely because the model isn't tool-capable (observability).

Surfaced by the lead-finder; related to the capability/tools work (#51) and instance-collection MCP access (#91).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions