[storage] Seven defects: RAG bleed between agents, cross-collection data loss, and summarization that never stops

[serge-ivo]

From a read-only audit of the agent storage engine, the DO alarm state machine, repo ingestion, and the file/collection/vector routes.

1. A WebSocket chat on an un-initialised DO persisted the agent id as the literal "unknown" — which is the Vectorize partition key.
Every HTTP chat caller passes agentId; webSocketMessage synthesized its request without one, so the auto-init fell back to "unknown" and stored it. That value is the agentId written on every vector AND the only filter vectorSearch applies. First-party agents seeded by migration have no initialised template DO, so opening /v1/agents/<slug>/ws on two of them made both write vectors tagged agentId:"unknown" and then read each other's chunks — a RAG bleed across logically separate agents, and across owners. Their R2 file keys collapsed to agents/unknown/files/… too. Fixed by pinning the resolved agent id to the socket and refusing to auto-init without one.

2. Collection names were validated in exactly one write path, and a : collides with another collection's key prefix.
Records live at col:{name}:{id}, indexes at idx:{name}:{field}:…, and the name regex existed only in the create_collection agent tool — the DO handler, the HTTP route, the MCP tool (plain z.string()) and recordInsert's auto-create-on-first-insert all accepted any string. With leads and leads:2026 both present: querying leads returns the other collection's records and inflates total, and deleting leads destroys every record and index of leads:2026 while its schema survives with a false recordCount. Silent and irreversible, reachable by an LLM simply putting a colon in a name. Validation moved into collectionCreate, the one choke point every path goes through.

3. Summaries were keyed by a random UUID, so "the last summary" was picked at random and summarization never stopped.
DO list orders lexicographically, so reverse:true limit:1 returned the highest UUID, not the newest. The resume point jumped backwards: at message 60 it could return the summary whose boundary was message 20, producing a 40-message window that generated another summary fully overlapping the existing one — and the max-UUID key never advances, so every 20 further messages produced an ever-larger overlapping summary, re-extracted the same fact:* memories, and spent platform AI on all of it, forever. getSummaries had the same defect, so buildRAGContext injected an arbitrary 5 summaries under "## Conversation History" and labelled them with dates. Now time-ordered keys, and selection by createdAt so the legacy rows still resolve correctly.

4. The repo-ingest "retry on a later tick" collapsed to a 50ms retry.
Failed embeds didn't consume the per-tick chunk budget, so during a Workers-AI outage every file returned -1, chunks stayed 0, the budget never tripped, and one tick drained a 300-file queue (past the 1,000-subrequest cap, which embed's catch swallows). All 300 went to retry, the DO rescheduled at +50ms, the next tick repeated immediately, attempts exceeded the cap, and the repo reached status:"done" with zero vectors about 100ms after the outage began. A failure now costs budget, and a tick that indexed nothing asks for a real pause.

5. The 2GB multipart cap was decorative. It was checked against a client-declared size that is never used again — {"size":1} then 10,000 × 10MiB parts produced a ~100GB object, completed and registered. Part numbers are now bounded by the cap and the real size is re-checked after complete.

6. filetext:{id} survived file deletion. Up to 100KB of extracted document text is written on upload and fileDelete never removed it, so deleting a 5MB PDF left its extracted résumé/contract text in the DO forever — unreadable (fileGetText needs file:{id}), unreclaimable, and growing on every upload/delete cycle.

7. Voice recordings survived "clear chat". The DO clears messages, summaries, facts and vectors, but the audio lives in R2 and nothing deleted it — every recording stayed in the bucket, still served by GET /voice-audio/:turnId. The Coder timeline path already did this; the Assistant path never did. Contradicts the documented behaviour and is a data-retention defect.

Checked and clean: validateRecord no longer drops fields; index/unique cleanup is symmetric across insert/update/delete; vectorDelete/clearRepoVectors scope correctly; R2 key schemes agree between the upload paths; recordQuery pagination is stable; the tar reader and gunzip ceiling are applied before the write.