Overseer agent: one cross-repo conversational AI (smart auto voice mode) with memory + self-awareness

[serge-ivo]

Vision (user's framing)

A hands-off smart auto mode: a non-stop voice conversation with ONE agent that sits across ALL your repos — knows what's happening in each, tells you when/what finished, explains things, and chats to the remote Claudes on your behalf. Explicitly one agent across everything, with memory + self-awareness — NOT a voice agent per repo.

Why one agent, not per-repo (agreed)

Per-repo co-pilots can't answer "what's the status of everything / what just finished / which needs me?" and can't move work between repos. A single Overseer holds the global picture and routes to the right repo's Claude. It's the natural home for memory + continuous conversation.

Architecture

• Scope: one Overseer per user (or per coding instance). Not tied to a session.
• Global awareness: subscribes to ALL the user's coding sessions — live run-state (working/idle/needs-you — already built), each session's recent coding_timeline, deploy status. This is its always-on context.
• Memory: persistent, cross-session (a new overseer_memory or reuse per-instance memory) — ongoing goals, your preferences, "what we're doing in each repo". Survives reloads.
• Tools (tool-calling loop, BYOK Claude):
  • repos_status() — every repo's state at a glance.
  • repo_detail(repoId) / search_history(repoId?, query) — read a repo's activity (vector + timeline).
  • drive_claude(repoId, instruction) — delegate work to a specific repo's Claude (reuse /coding/act + the finish watcher).
  • focus(repoId) / navigation for voice.
• Proactive narration: when any repo's finish-watcher fires, the Overseer announces it ("platform finished adding auth — want the diff?"). Reuses the notification/watch machinery.
• Continuous voice: continuous SpeechRecognition + TTS; barge-in (recording cancels playback — already built). A real back-and-forth.
• Backend: POST /instances/:id/overseer — a tool-calling loop fed {user message + global context + memory}; returns a reply and/or delegates. Generalises issue Conversational agent routing: one tool-calling co-pilot (answer-from-context vs delegate-to-Claude) #3's per-session routing to global scope.

Relationship to #3

This subsumes #3 (conversational routing): #3 is the same answer-from-context vs drive_claude decision, but per-session. The Overseer is that decision at GLOBAL scope. Recommend: build #3's tool-loop as the reusable core, then lift it to the Overseer with global context + memory + multi-repo drive_claude.

Build order (proposed)

1. The tool-loop core (Conversational agent routing: one tool-calling co-pilot (answer-from-context vs delegate-to-Claude) #3) — one session, answer/delegate. (Reusable engine.)
2. Global context + repos_status/drive_claude(repoId) — the Overseer reads all repos and routes.
3. Persistent Overseer memory + proactive finish narration.
4. Continuous-voice UI (ties in Mode 1's command layer).

Acceptance

A single voice conversation where I can ask "what's everyone doing / what finished?", tell it "have the platform repo run the tests", and it routes to that repo's Claude, narrates completions, and remembers context across the session.

[edihasaj]

Nice design, this Overseer idea needs a durable, project-aware memory layer exactly like you describe. We built Recall to persist vetted, high-signal memories locally per project and surface only what an agent should see, which fits the Overseer memory role and keeps data off the cloud. If helpful, our GitHub has the code and integration points so you can plug it into your tool-loop and have per-repo contexts aggregated into a single Overseer memory: https://github.com/edihasaj/recall

[serge-ivo]

Closing as superseded by the current supervision/coordination design. The old hardcoded Overseer route exists (/coding/overseer), but the repo has moved the durable design to configurable supervision/coordination primitives: agent_supervision, list_subordinates/delegate_goal/check_delegation, Teamwork UI, and the coordination/supervision roadmap. Keeping this old 'one Overseer' architecture open now conflicts with #154/#160/#182 and the supervision docs.

[serge-ivo]

Supersession note: closed because the old hardcoded Overseer framing was replaced by the current coordination/supervision primitives (#154/#160/#182 and the supervision connector/tools). No single implementing PR exists for this old design.