Skip to content

feat: Extended Memory module (P0-P2)#47

Open
jkyberneees wants to merge 20 commits into
mainfrom
proposal/extended-memory
Open

feat: Extended Memory module (P0-P2)#47
jkyberneees wants to merge 20 commits into
mainfrom
proposal/extended-memory

Conversation

@jkyberneees

@jkyberneees jkyberneees commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

This PR introduces the opt-in Extended Memory module for odek.

What is included

  • New internal/memory/extended/ package with atom-based semantic memory.
  • Dedicated or fallback memory LLM support.
  • Per-turn atom extraction from user messages (trusted user_said source).
  • Tainted-atom quarantine for indirect content with TTL eviction.
  • Configurable semantic search with top-K, overfetch, min-score, and optional LLM rerank.
  • 100 MB default disk cap with retention_decay eviction.
  • Vector index with background compaction.
  • Memory tool actions: add_atom, search_atoms, forget_atom, pin_atom, list_quarantine.
  • CLI commands: odek memory extended promote|pin|forget|quarantine|compact.
  • Updated documentation in docs/EXTENDED_MEMORY.md, docs/MEMORY.md, and docs/CONFIG.md.

Trust model

  • User inputs are trusted and extracted automatically.
  • Indirect inputs (tool output, file reads outside workspace, web, MCP, subagents) are tainted and quarantined; they are never recalled without explicit user promotion.
  • SourceInferred is treated as tainted until promotion exists.

Tests

  • go test ./... -count=1 passes.
  • go test -race ./internal/memory/extended/... -count=1 passes.
  • internal/memory/extended coverage: 82.3%.

Design docs

  • docs/EXTENDED_MEMORY.md

This is P0-P2 of the Extended Memory roadmap; P3-P5 (user-state model, predictive recall, proactive behaviors) are reserved for follow-up work.

…nd 80.5% coverage

- Add opt-in Extended Memory atom store, vector recall, and retention-decay eviction
- Wire dedicated/fallback memory LLM and forward MemoryConfig in run/continue
- Fix loop dedup for extended context injection
- Implement quarantine store with TTL and scan tainted atoms before persistence
- Add promote/pin/list_quarantine tool actions and CLI commands
- Harden size cap accounting, vector file permissions, and anti-injection framing
- Update MEMORY.md, EXTENDED_MEMORY.md, and CONFIG.md docs

Delete docs/EXTENDED_MEMORY_IMPLEMENTATION_PLAN.md
- Evictor.SelectForEviction now returns (ids, freed, ok) so callers know
  whether the requested bytes could be covered by non-pinned atoms.
- enforceCap uses per-atom effective size (chunk + atoms.json share +
  amortized vector cost) instead of raw store+index disk size, avoiding
  double-counting and making fail-closed checks reliable.
- After eviction, if the evictor reports it cannot free enough, AddAtom
  returns an error and the atom is not written.
- Add ExtendedMemory.Close() and atomVectorIndex.Wait() so tests can
  drain background compaction goroutines before TempDir cleanup.
The odek memory extended * subcommands (forget, promote, pin, quarantine,
compact) construct ExtendedMemory using DefaultConfig, which has Enabled
= false, so all operations silently failed. Force Enabled = true for the
CLI surface only; the agent tool surface still does not expose promote.
- Cap tests now use mock embedder, small per-test caps via testCapBytes,
  smaller atoms, and defer em.Close() so race tests run quickly and
  TempDir cleanup succeeds reliably.
- TestEvictionAllPinned now expects AddAtom to fail when pinned atoms
  fill the cap, matching the fail-closed behavior.
- Add TestCapFailClosedWhenAllPinned.
- Add coverage tests for SetEmbedder, SetEmbedderFactory, MarkDirty,
  Compact, UserModel, Associations, and AddAtoms batch failure path.
- Move eviction after the ok check so atoms are only removed when the cap can be satisfied.
- gofmt changed files in internal/memory/extended.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 10, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
odek 93ba487 Commit Preview URL

Branch Preview URL
Jul 10 2026, 08:39 PM

- P3: user-state model with pending-review queue and background inference
- P4: predictive intent generation, atom associations, predictive recall augmentation
- P5: return-after-break summaries, anaphora resolution, follow-up anticipation, style mirroring
- Add memory tool actions and CLI commands for pending review, promotion, and pinning
- Update docs in MEMORY.md, EXTENDED_MEMORY.md, and CONFIG.md
… predictor wiring, and context propagation

- Replace only the first pronoun when the top atom's semantic score is
  above SemanticSearchMinScore; return (resolved, bool).
- Guard triggerBackgroundInference with a closed flag and lock to prevent
  starting goroutines after Close.
- Add a 10-message recent-user buffer and pass it plus UserState to the
  Predictor in Recall.Query.
- Thread context.Context through AnaphoraResolve, FormatExtendedContext,
  FormatUserStateContext, and FormatReturnAfterBreak; update loop engine
  callbacks and MemoryManager callers to use caller-provided contexts with
  timeouts.
- Add TestExtendedMemoryCloseRace.
…e promote lock

- Scan every string value in UserState on Load; drop fields/entries that
  fail ScanContent so a tampered user_model.json cannot poison prompts.
- Scan the formatted Summary() output before returning it.
- Use Lock instead of RLock in Quarantine.Promote because loadLocked may
  call saveLocked during eviction.
…turnAfterBreak

- Remove SourceInferred from TrustBoost (treated as tainted/quarantined
  until promotion), matching IsTaintedSourceClass.
- Wrap the return-after-break summary with wrapUntrusted before injecting
  it as a system message in odek continue; attach the audit recorder early
  so the ingestion is recorded.
- Update docs to mark inferred as tainted/quarantined and document the
  first-pronoun, score-gated anaphora behavior.
…d branch

- Update EXTENDED_MEMORY.md to reflect implemented P3-P5 features
- Add inferred to tainted source classes in MEMORY.md
- Remove dead no-op branch in UserModel.Update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant