Skip to content

subscribe_agent is not idempotent: a transient error that already subscribed tells the caller to retry, and the retry duplicates the instance #716

Description

@serge-ivo

What happened

Observed live, on the operator account, 2026-08-20:

  1. subscribe_agent("inbox-chat") returned Error: The service is updating — please try again in a moment.
  2. Taking that at its word, the caller retried.
  3. The retry succeeded.
  4. my_instances then showed two instances of agent_inbox_chat, created 7 seconds apart
    1ff12810-… at 23:38:41 and 44c68b4c-… at 23:38:48.

The first call had subscribed. It reported failure anyway — the error arrived after the row was
written, most likely because the deploy that was mid-flight cut the response rather than the work.
So the caller was told to retry an operation that had already happened, and doing as it was told
produced a duplicate.

The user noticed before we did: "wait, why are there 2 instances now? i only need one."

Why this one matters more than a stray row

Subscribing is not idempotent and nothing about it is guarded:

  • There is no natural key. Multi-instance is a FEATURE — PUT /v1/instances/:id/name exists
    precisely so a subscriber can run several instances of one agent and tell them apart — so the
    database cannot distinguish "the user wants a second one" from "the first attempt lied".
  • Every instance is a Durable Object with its own storage, its own KB, its own memory. A duplicate
    is not a cheap row; it is an empty parallel agent that looks exactly like the real one in the
    console and diverges from it the moment either is used.
  • The error text instructs the retry. Any client that believes it — a person, a script, an
    agent following the message — reproduces this.

Not verified

Which layer produced the error. The message is the platform's own "service is updating" string
rather than a D1 or DO error, and a deploy was in flight at the time (API Worker deploy for
c7c1ada completed 3m04s before). Whether the write completed and the response was cut, or the
write was retried internally and the second attempt won, is not established from the outside —
both fit the evidence. agent_trace/list_errors on the operator account around 2026-08-20
23:38:41Z is where that would be settled.

That distinction changes the fix, so it is worth settling first rather than guessing:

  • If the write completes and the RESPONSE is lost, the fix is an idempotency key on subscribe —
    the caller supplies one, a repeat returns the same instance instead of making another.
  • If the platform retries internally, the fix is that the retry is not idempotent and should be.

What a fix should probably NOT do

Refuse a second subscription to the same agent. That would break multi-instance, which is a
documented feature with a rename route built for it. The answer is an idempotency key, not a
uniqueness constraint.

Cleanup

The duplicate was cancelled by hand (cancel_instance on 1ff12810-…). Nothing was lost — it had
never been used — but that is luck, not design: had the retry been the one configured, the KB and
memory would have gone to the instance the console listed second.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guardsbackendBackend / Worker / API workbugSomething isn't workingmcpMCP server surface — the operator's instrument panel, not the product

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions