diff --git a/README.md b/README.md index 6ef314c..bea2882 100644 --- a/README.md +++ b/README.md @@ -78,6 +78,9 @@ ak status read-only dashboard: what's true, what's drifted [--json] [--d ak sync converge to good: upgrade + heal + verify [--dry-run] [--no-upgrade] ak dashboard open the local web dashboard (auto-opens your browser) [--port N] [--no-open] [--live-source 'surface=path'] +ak about what each installed component is and why it's there +ak system machine footprint: install size, runtime, storage, catalog, projects + [--deep] [--json] ak usage inspect/refresh offline provider account analytics status | refresh openrouter ak host manage execution hosts, routing, and provider bindings diff --git a/bin/agentic-kit.mjs b/bin/agentic-kit.mjs index a38a3cd..0df99c9 100755 --- a/bin/agentic-kit.mjs +++ b/bin/agentic-kit.mjs @@ -22,6 +22,8 @@ const PORCELAIN = Object.assign(Object.create(null), { dashboard: () => import('../src/commands/x/dashboard.mjs'), admin: () => import('../src/commands/x/admin.mjs'), usage: () => import('../src/commands/usage.mjs'), + system: () => import('../src/commands/system.mjs'), + about: () => import('../src/commands/about.mjs'), run: () => import('../src/commands/run.mjs'), host: () => import('../src/commands/x/host.mjs'), uninstall: () => import('../src/commands/uninstall.mjs'), @@ -49,6 +51,8 @@ Usage (ak = alias of agentic-kit): ak dashboard open the local web dashboard (localhost; auto-opens browser) [--port N] [--no-open] ak admin maintainer-only telemetry admin (localhost; GitHub/npm egress) [--port N] [--no-open] ak usage inspect/refresh offline provider analytics [status|refresh openrouter] + ak system what this stack occupies on your machine [--deep] [--json] + ak about what agentic-kit installs and configures, and why [--category N] ak run execute a host-neutral activity pipeline [template "task"] [--dry-run] ak host manage agent hosts, routing, and provider bindings [status|pick|refresh|off] ak uninstall leave cleanly [--this-project] [--purge] diff --git a/docs/DASHBOARD.md b/docs/DASHBOARD.md index f941b0a..c9bff3b 100644 --- a/docs/DASHBOARD.md +++ b/docs/DASHBOARD.md @@ -11,19 +11,32 @@ ak dashboard ak dashboard --no-open ``` -The dashboard has exactly three stable primary areas: **Overview**, **Usage**, and -**Observability**. The row directly below them is one fixed, left-aligned secondary navigation rail. -Its position does not move when the primary area changes; only the choices inside it change. +The dashboard has five primary areas, in this order: **About**, **Overview**, **Usage**, +**Observability**, and **System**. The row directly below them is one fixed, left-aligned secondary +navigation rail. Its position does not move when the primary area changes; only the choices inside +it change — About is the one area with no views to choose between, so its rail carries jump links +to the sections of its single page instead. + +About is leftmost because it is the reading-order entry point for someone who does not yet know +what the other tabs mean. **Overview remains the landing view on every open, including the first.** +A first visit shows a dismissible "new here?" nudge on Overview pointing at About; dismissing it is +permanent. ## Navigation and deep links | Primary area | Secondary view | Canonical hash | Heading | What it answers | |--------------|----------------|----------------|---------|-----------------| +| About | (whole page) | `#about` | Meet your toolkit | What agentic-kit installs and configures, what each piece is for, and where to read more | +| About | Hosts | `#about/hosts` | Hosts | The coding agents themselves: Claude Code, Codex, OpenCode | +| About | Engine & memory | `#about/engine` | Engine & memory | ruflo and agentdb — orchestration, cross-session memory, learning | +| About | Quality · Safety · Knowledge | `#about/quality` | Quality · Safety · Knowledge | agentic-qe, the prompt-injection defense, and the offline knowledge base | +| About | The kit | `#about/kit` | The kit | agentic-kit itself — what installs, heals, and explains the rest | +| About | Configured for you | `#about/configured` | Configured for you | Non-package surfaces ak set up on your behalf, each with the command that manages it | | Overview | Summary | `#overview/summary` | System overview | Overall readiness, configuration health, and items needing attention | | Overview | Hosts & Routing | `#overview/hosts` | Hosts & routing | Enabled execution hosts, activity assignments, primary-host policy, and escalation paths | | Overview | Providers | `#overview/providers` | Inference providers | Provider bindings, availability, provenance, and configuration health | | Overview | Runtime | `#overview/runtime` | Runtime health | Local services, MCP connections, processes, and operational readiness | -| Overview | Intelligence | `#overview/intelligence` | Intelligence & learning | Machine-wide learning rollup across every ruflo-initialized project, plus near-live detail for one explicitly selected project | +| Overview | Intelligence | `#overview/intelligence` | Intelligence & learning | Machine-wide learning rollup across every project with memory or intelligence state, plus near-live detail for one explicitly selected project | | Usage | Scorecard | `#usage/score` | Usage scorecard | Token consumption, API-equivalent cost, efficiency, and trends | | Usage | Limits | `#usage/limits` | Provider limits | Current provider windows, reset timing, and available capacity | | Usage | Findings | `#usage/findings` | Usage findings | Actionable anomalies, efficiency opportunities, and evidence-backed recommendations | @@ -31,6 +44,16 @@ Its position does not move when the primary area changes; only the choices insid | Usage | Transcript | `#usage/transcript` | Transcript detail | The selected session's locally retained, server-masked evidence | | Observability | Live | `#observability/live` | Observability · Live | Projects and roots with current presence or fresh meaningful activity | | Observability | History | `#observability/history` | Observability · History | Retained roots that are not currently Live | +| System | Summary | `#system/summary` | Summary | Install size, retained data, live resource use, deployed inventory, and the machine's largest storage consumers in one glance | +| System | Advisory | `#system/advisory` | Advisory | What could be reclaimed, in two safety tiers reported separately and never added — the only System area that suggests an action, and it still has no delete control | +| System | Sessions | `#system/sessions` | Sessions | The largest individual session files, the project each belongs to, and its share of that host's retained bytes | +| System | Storage | `#system/storage` | Storage | Where the retained bytes are, by category and host — learning stores counted separately because they dwarf everything else — plus per-series growth | +| System | Runtime | `#system/runtime` | Runtime | Live host processes, their CPU and memory, background daemons, and machine denominators — refreshed on the header's poll clock while open | +| System | Catalog | `#system/catalog` | Catalog | Every deduplicated skill, agent, command, plugin and MCP server across user scope and all projects on disk, with a per-host presence matrix and kind/host filters | +| System | Projects | `#system/projects` | Projects | Every repository with a remote that a host has recorded a session in — its approximate lines of code, language mix, total disk size and last activity. Worktrees, sub-folders and remote-less repositories are counted below the table, not listed | + +About is one scrolling page, so its hashes scroll to a section rather than swapping panels; `#about` +alone opens the page at the top. Opening a transcript replaces `transcript` with the URL-encoded session ID: `#usage/`. These hashes select state inside the one page; they do not create separate @@ -51,6 +74,34 @@ Every view supplies its own heading and short description beneath the shared nav Overview child views remain scoped to those views, while Summary aggregates the items that need attention. +## About + +About introduces every component agentic-kit installs or configures, for a reader who has not met +any of them. It opens with one orientation sentence and a short map of how the pieces relate, then +one card per component grouped into sections: hosts, engine and memory, quality/safety/knowledge, +the kit itself, and the surfaces ak configured for you. + +Each card carries an icon, the component name with a state chip, a plain-language tagline, one +short paragraph explaining what the thing does for you, and a row of link pills — source (GitHub), +package (npm), and public docs. Configured surfaces use the same card shape but swap the link pills +for the command that manages them, because "where do I change this" is their equivalent of "where +do I read more". + +Two rules make the page trustworthy: + +- **The prose is authored; the chip is measured.** Every paragraph reads the same on a machine + where the component is absent. Whether something is installed, at what version, or configured + comes only from the same detection Overview reports — never from the copy. +- **Absent evidence reads as unknown, not as good news.** If the status fetch fails, every chip + reads `state unknown` and every card still renders. A surface whose verdict `ak status` does not + publish — currently the permission allowlist — reads `state unknown` permanently rather than + claiming to be configured. + +Links are ordinary anchors you click in your own browser. The dashboard never fetches them. + +`ak about` prints the same directory in a terminal; `ak about --category configured` narrows it, +and `ak about --json` emits the entries with their detected state. + ## Overview Overview keeps status and routing in one health-first area: @@ -62,7 +113,9 @@ Overview keeps status and routing in one health-first area: - **Providers** presents inference-provider bindings and their configuration provenance. - **Runtime** presents operational services, processes, and MCP readiness. - **Intelligence** presents memory, learning, and quality-improvement signals machine-wide: an - always-visible rollup folded across every ruflo-initialized project on this machine, plus detail + always-visible rollup folded across every project on this machine where memory or intelligence has + been activated — a `.claude-flow`, `.agentic-qe` or `.swarm` directory, whichever host created it + — plus detail for one explicitly selected, explicitly labeled project — the neural pattern store's current size, its separate lifetime patterns-learned counter, and reasoning-graph growth. Project selection defaults to whichever discovered project was most recently active; there is no implicit @@ -71,8 +124,32 @@ Overview keeps status and routing in one health-first area: ruflo/agentic-qe already write under `.claude-flow/` and updates near-live over a per-project SSE stream while the view is open, falling back to the general status poll otherwise. See [Project intelligence](ddd/project-intelligence.md) and - [ADR-0024](adr/0024-project-intelligence-telemetry.md) for the full model, the project-discovery - mechanism, and the two learning metrics' load-bearing distinction, now also at machine scope. + [ADR-0024](adr/0024-project-intelligence-telemetry.md) for the full model and the two learning + metrics' load-bearing distinction, and [ADR-0027](adr/0027-shared-project-census.md) for project + discovery. + +### Why project counts differ between tabs + +Every area derives its project list from one census, so a project means the same thing everywhere — +a session run in `myrepo/backend` belongs to `myrepo`, not to a project called `backend`, and an +agent worktree is not a peer of the repository it was cut from. + +The totals still differ, because the tabs ask different questions: + +| Tab | Counts | Over | +|---|---|---| +| Overview → Intelligence | projects with learning state | all time | +| Observability → History | projects with retained sessions | the selected history window | +| Usage → Scorecard | projects with recorded usage | the selected day window | +| System → Projects | **directories** ever seen, and the subset still on disk | all time | + +Two of those differences are structural rather than temporal. System counts **directories**, because +only a directory has bytes and lines to measure; the other tabs count **projects**. And a windowed +count is smaller than a lifetime one by exactly the projects you have not touched lately — a shorter +window, not a missing project. + +Each count carries the sentence explaining what it counted; on Intelligence it is behind +**how these projects were counted**, next to the rollup. ## Usage @@ -126,13 +203,137 @@ updates. See [Observability](OBSERVABILITY.md) for the map legend, workspace facts, host capability coverage, History/Review semantics, privacy limits, and troubleshooting. +## System + +System answers what this toolchain costs the machine itself — a different question from health +(Overview), spend (Usage), or activity (Observability). Its five views are Summary, Storage, +Runtime, Catalog, and Projects. Projects stays separate from Storage on purpose: lines of code and +a git remote answer "what have I built here", not "where are my bytes". + +### Two tiers, and why nothing scans on open + +Opening System costs almost nothing. The cheap tier — the live process census, individually known +file sizes, and the figures carried forward from the last deep scan — is served on every read and +cached briefly. + +Everything else comes from a **deep scan**, which walks the install trees, the retained-data roots, +every host's catalog surfaces, and every known project. That is real I/O and takes tens of seconds +on a large machine, so it runs **only when you press Rescan** (or run `ak system --deep`). Opening +the tab never triggers it, and System does not poll: it fetches once, then again only while a scan +you started is still running. + +The trade is stated rather than hidden. Deep-tier figures always render with when they were +measured, and once a snapshot passes seven days the freshness label turns amber and reads +`stale, rescan`. A scan writes one file — its own snapshot — and nothing else; the reclaimable-space +rows are advisory, with their rationale and their path, and there is no delete button anywhere in +this area. + +### Largest consumers, and the project-trees toggle + +The Summary strip ranks the biggest storage roots on the machine — not just the kit's own. On a +working machine the top of that list is usually local model weights, package caches and toolchain +installs, so the strip covers around fifty known cache roots (Ollama, LM Studio, Hugging Face, +npm/pnpm/yarn/bun, rustup and Cargo, Go, uv and pip, Maven and Gradle, Playwright and Puppeteer, +mise, Homebrew, Docker) alongside the kit's own. **Ranked** and **By ecosystem** re-shape the +same measurement; the ecosystem view is usually the more actionable one, because four Node caches +at 5 GB each is a Node answer, not four unrelated rows. + +Nested roots are counted **once**, at the outermost row. `~/.npm/_cacache` sits inside `~/.npm`, +`~/.cache/huggingface` inside `~/.cache`, the npm global root inside mise's Node install — so a +row inside another row is shown as a *breakdown* of its parent and is left out of the ranking and +the totals. Every parent with breakdowns also gets an "everything else" row, so a breakdown always +adds up to its parent. Roots that do not exist on this machine are listed as absent rather than +ranked at 0 B, and roots that could not be read say so with their reason. + +**Project trees** are excluded by default, and the chip that includes them is a *scan* control, +not a filter. One large repository can outweigh every shared cache combined, and a chart +containing it is a chart of one repository — so the ranking says, in the panel, that they were +left out. Turning the chip on starts a new deep scan that walks them (and turning it off starts +one that does not); it is disabled while a scan is running. `ak system --deep` scans without +project trees. + +### Two reclaimable tiers, never one total + +Reclaimable rows come in two tiers, rendered as two separate blocks because they are two +different promises: + +- **regenerable** — the owning tool refetches it on demand (package caches, superseded knowledge- + base copies, stale npx envs). This block states a total. +- **review** — plausible, but not safe to call removable: aged transcripts are the only copy of + the sessions they record, an extra runtime version may be the one a live toolchain resolves + through, a browser build may still be pinned. This block deliberately has **no total**. Its + bytes appear per row as context, and some rows show what is installed at that path rather than + a measured removable subset, labeled as such. + +The two are never added together. A combined "you could free N" would be the one number you would +act on and the one number the measurement cannot stand behind. Where a tier's own rows overlap on +disk, its total reads unknown-with-reason instead of counting the same bytes twice. Nothing here +removes anything; where a CLI already owns the cleanup, the row names it. + +### Reading the numbers honestly + +- **A section that has never been scanned says so.** It reads "not measured yet — press Rescan", + never `0`. A zero here means a real, measured zero. +- **A total whose inputs were incomplete renders as `≥ N`.** If one subtree could not be read or a + walk hit its cap, the sum is a floor, not a total, and is labeled that way. +- **A failed measurement names its reason.** An unreadable directory degrades that node alone; its + siblings and the rest of the scan are unaffected. +- **Lines of code are approximate and say so**, counted by extension with `node_modules`, vendored + trees, and binary files excluded. Only *languages* carry lines. Frameworks, SDKs and tools are + shown as present or not — React does not own lines, the `.tsx` files do — and what the registry + could not name is listed by name rather than swept into an "Other" slice. +- **Projects are counted twice, and the two numbers differ.** The KPI reads + `N ever · M on disk`: *ever* is every project any host has ever recorded a session in, including + ones you have since deleted or moved; *on disk* is the subset that still exists, and only those + become rows in the Projects table — a deleted project has no bytes and no lines to measure. + A large gap is a fact about your history, not an error. +- **A project whose path cannot be recovered is counted, not invented.** Claude stores transcripts + in a directory name that encodes the project path lossily (`/`, `.` and `-` all become `-`), so + it cannot simply be decoded back. The path is read from the session record instead; where no + session recorded one and the encoded name cannot be confirmed against your filesystem, the + project is reported as unresolved and the *ever seen* count is shown as a floor. You will never + see a guessed path here. +- **Growth per day is approximate too** — a file counts its whole size on the day it was last + written, which is exact for append-only transcripts and over-counts rewritten databases. +- **Some things cannot be attributed, and say that instead of guessing.** Codex transcripts are + stored by date rather than by project, so those bytes render as unattributable. + +### Platforms + +All five views work on macOS, Linux, and Windows. On Windows the process census (host, pid, CPU, +memory, uptime) is always available; the bound project is a best-effort read that can be blocked by +antivirus, execution policy, or permissions, in which case that one column reads +"not attributable on Windows" with the reason and every other figure in the row still renders. + +One field is honestly missing everywhere: the ruflo daemon budget has no local source this +collector can read, so it reports unknown rather than a number inferred from nothing. + +`ak system` prints the same collector output in a terminal, `ak system --deep` runs the scan, and +`ak system --json` emits the payload verbatim. See +[Machine footprint](ddd/machine-footprint.md) and +[ADR-0025](adr/0025-machine-footprint-metrics.md) for the full model and its invariants. + ## Local state and security -Theme, polling preference, the selected primary/Overview view, and the Session Stream collapse choice -are stored in browser-local storage. Canonical hashes make views linkable without putting the -dashboard token in the path or query string. +Theme, polling preference, the selected primary/Overview/System view, whether the About nudge has +been dismissed, and the Session Stream collapse choice are stored in browser-local storage. +Canonical hashes make views linkable without putting the dashboard token in the path or query +string. The launch token initially arrives in the URL fragment and is then stored locally for authenticated -API requests. The dashboard remains localhost-only and offline-first. Usage and Observability may -show sensitive local project or transcript information; use them only where that local information -may be viewed. +API requests. The dashboard remains localhost-only and offline-first. Usage, Observability, and +System may show sensitive local project, transcript, or filesystem-path information; use them only +where that local information may be viewed. System deliberately shows absolute paths — a storage +breakdown that hides where the bytes live answers nothing — behind the same token-gated loopback +delivery as every other route. + +What System reads is a short, fixed list: directory entries and file `stat` results; your +`.git/config` origin remote (so a project can link to its repository page — the kit never fetches +it, you click it); a linked worktree's `gitdir` pointer; the `cwd` **field** recorded at the top +of a session transcript, and OpenCode's per-session `directory` column, so a session can be +attributed to the right project; your projects' manifest **dependency names**, which are neither +evaluated nor resolved; and your source files' bytes, streamed through a fixed buffer purely to +count newlines. Each of those yields a path, a name, or a number. **No message, prompt, tool call, +tool result, or model output is ever read** — those stay in Usage and Observability, which have +their own contracts for them. The full enumeration is +[Machine footprint § The read surface](ddd/machine-footprint.md#the-read-surface). diff --git a/docs/PROVIDERS.md b/docs/PROVIDERS.md index 0217f2c..568657b 100644 --- a/docs/PROVIDERS.md +++ b/docs/PROVIDERS.md @@ -231,12 +231,24 @@ Defaults (all overridable; your edits are marked `custom` and never re-seeded): |---|---|---| | specification, review, release | claude | `claude-sonnet-5` | | architecture, design, debugging, security-analysis | claude | `claude-opus-5` | -| implementation, testing, security-scan | codex | `gpt-5.4` | -| documentation, packaging | codex | `gpt-5.3-codex` | +| implementation, testing, security-scan | codex | `gpt-5.6-terra` | +| documentation, packaging | codex | `gpt-5.6-luna` | *(packaging & release are `ak`-added — ruflo ships templates for feature/security/refactor only.)* -**Known-good model choices** (verified 2026-07; any model your host CLI accepts also works): +**Retired codex models.** `gpt-5.4` and `gpt-5.4-mini` retire from Codex on **2026-08-31**, and +`gpt-5.3-codex` is already withdrawn for ChatGPT sign-in +([Codex models](https://developers.openai.com/codex/models)) — which is why the execution defaults +above moved to the 5.6 line. `ak` substitutes a retired model at read time, so no run dispatches to +one even if your `kit.json` still names it, and `ak sync` rewrites `seeded` routes that do. A `user` +pin is reported but never rewritten on disk (see +[ADR-0003](adr/0003-auto-seed-dual-host-provenance.md)). + +`claude-opus-4-8` is **not** retired — it carries no deprecation notice and stays pinnable. It is +merely no longer the default, which `ak status` reports as routing *divergence*: a trade for you to +weigh, cleared with `ak x host refresh` if you want the newer default. + +**Known-good model choices** (verified 2026-08; any model your host CLI accepts also works): > **Per-token price ≠ per-task cost.** A model that needs more agentic turns costs more > per task at the same per-token price. On subscription (`claude-code` oauth) billing the @@ -245,15 +257,14 @@ Defaults (all overridable; your edits are marked `custom` and never re-seeded): | Host | Model | When to use | |---|---|---| -| claude | `claude-opus-5` | new top Opus — same per-token price as 4.8, but ~2–3× the agentic turns on routine work; earns it at the hard end | +| claude | `claude-opus-5` | top Opus — the deepest reasoning, at ~2–3× the agentic turns of a balanced model on routine work; earns it at the hard end | | claude | `claude-sonnet-5` | near-Opus capability at a lower per-token price — review, spec, release | | claude | `claude-fable-5` | top capability (Mythos-class, above Opus 5) — hardest problems | | claude | `claude-haiku-4-5-20251001` | cheap/fast — high-volume mechanical work | -| claude | `claude-opus-4-8` | prior Opus generation — same per-token price, roughly half the turns on routine work | -| codex | `gpt-5.4` | coding + reasoning + agentic — recommended execution default | -| codex | `gpt-5.6-sol` | newest line; first-class max reasoning effort | -| codex | `gpt-5.3-codex` | pure coding-tuned — mechanical implementation & docs | -| codex | `gpt-5-codex-mini` | smallest/cheapest — escalation floor, high volume | +| claude | `claude-opus-4-8` | prior Opus generation — same per-token price as Opus 5, roughly half the agentic turns on routine work | +| codex | `gpt-5.6-sol` | flagship 5.6 — strongest on complex coding, computer use and security work; first-class max reasoning effort | +| codex | `gpt-5.6-terra` | balanced 5.6 — everyday implementation and testing at a materially lower per-token price than sol; the gpt-5.4 replacement | +| codex | `gpt-5.6-luna` | fastest/cheapest 5.6 — mechanical implementation, docs and packaging; the gpt-5.4-mini replacement | > **Where Opus 5 sits** ([announcement](https://www.anthropic.com/news/claude-opus-5), July 2026): > same $5/$25 per-Mtok pricing as Opus 4.8 with roughly double the Frontier-Bench diff --git a/docs/TRANSCRIPTS.md b/docs/TRANSCRIPTS.md index ce27c9c..c94b5e0 100644 --- a/docs/TRANSCRIPTS.md +++ b/docs/TRANSCRIPTS.md @@ -36,12 +36,12 @@ rewritten; rule 3 of the module header, `usage-index.mjs:22-29`): | Host | Store | Discovered by | |---|---|---| -| Claude Code | `~/.claude/projects//.jsonl` | `listClaude` (`usage-index.mjs:684`) — exactly one level of project directories | -| Codex CLI | `~/.codex/sessions///
/rollout--.jsonl` | `listCodex` (`usage-index.mjs:705`) — the `yyyy/mm/dd` tree walk | +| Claude Code | `~/.claude/projects//.jsonl` | `listClaude` (`usage-index.mjs:786`) — exactly one level of project directories | +| Codex CLI | `~/.codex/sessions///
/rollout--.jsonl` | `listCodex` (`usage-index.mjs:758`) — the `yyyy/mm/dd` tree walk | -Roots come from `defaultRoots()` (`usage-index.mjs:697-701`) and are injectable +Roots come from `defaultRoots()` (`usage-index.mjs:750-754`) and are injectable for tests. A malformed line is skipped, never fatal (`jsonLines`, -`usage-index.mjs:328-334` — one corrupt line must not cost a whole file). +`usage-index.mjs:381-387` — one corrupt line must not cost a whole file). Host evidence is not inference-provider proof. A Claude transcript may describe Anthropic-, OpenRouter-, or Ollama-served inference. ADR-0016 defines separate @@ -52,34 +52,34 @@ transcript host/parser identity unless other evidence grounds the inference prov ### 1.1 Claude entry vocabulary Each line has a top-level `type`. The parser (`parseClaude`, -`usage-index.mjs:427-510`) reads: +`usage-index.mjs:480-563`) reads: | `type` | What the parser takes from it | |---|---| -| `ai-title` | The model-written session title (`usage-index.mjs:435`) — preferred over the first-prompt fallback | +| `ai-title` | The model-written session title (`usage-index.mjs:488`) — preferred over the first-prompt fallback | | `user` | A user-**role** turn — which is *not* the same as "the human"; see §3 | -| `assistant` | A model turn: `model` id, per-turn `usage` token counts, `tool_use` blocks (`usage-index.mjs:455-498`) | -| any | Side-band fields read regardless of type: `attributionSkill`/`attributionPlugin` (`usage-index.mjs:436-437`), `isSidechain` (`usage-index.mjs:438`), `cwd` for project derivation | +| `assistant` | A model turn: `model` id, per-turn `usage` token counts, `tool_use` blocks (`usage-index.mjs:508-551`) | +| any | Side-band fields read regardless of type: `attributionSkill`/`attributionPlugin` (`usage-index.mjs:489-490`), `isSidechain` (`usage-index.mjs:491`), `cwd` for project derivation | An assistant entry with `isApiErrorMessage: true` is a **local placeholder** Claude Code writes when a request dies before a real completion (connection drop, rate limit, auth failure — `model: ""`, all-zero usage). It is real engaged time but not a model attempt: counted as an *exception*, never -pushed into `models` or priced (`usage-index.mjs:498-517`; the full story is +pushed into `models` or priced (`usage-index.mjs:551-570`; the full story is [`USAGE-SCORECARD-METRICS.md`](USAGE-SCORECARD-METRICS.md) §10). ### 1.2 Codex entry vocabulary Codex rollout lines carry `type` + `payload`. The parser (`parseCodex`, -`usage-index.mjs:527-628`) reads: +`usage-index.mjs:580-681`) reads: | `type` / `payload.type` | What the parser takes from it | |---|---| -| `session_meta` | Authoritative session id, `cwd`, and `thread_source` (`usage-index.mjs:539-544`) — `"subagent"` marks a thread_spawn replay whose tokens are excluded from aggregation (`usage-index.mjs:609`; `USAGE-SCORECARD-METRICS.md` Appendix A, Bug B) | -| `turn_context` | The model id in effect from this point on (`usage-index.mjs:545`) | -| `event_msg` → `token_count` | A **cumulative** usage snapshot; only the last one is kept (`usage-index.mjs:609-611`) | -| `event_msg` → `user_message` | A real human prompt — Codex does not route tool output through this event (`usage-index.mjs:584-592`) | -| `event_msg` → `agent_message` | A model response (`usage-index.mjs:594-605`) | +| `session_meta` | Authoritative session id, `cwd`, and `thread_source` (`usage-index.mjs:592-597`) — `"subagent"` marks a thread_spawn replay whose tokens are excluded from aggregation (`usage-index.mjs:662`; `USAGE-SCORECARD-METRICS.md` Appendix A, Bug B) | +| `turn_context` | The model id in effect from this point on (`usage-index.mjs:598`) | +| `event_msg` → `token_count` | A **cumulative** usage snapshot; only the last one is kept (`usage-index.mjs:662-664`) | +| `event_msg` → `user_message` | A real human prompt — Codex does not route tool output through this event (`usage-index.mjs:637-645`) | +| `event_msg` → `agent_message` | A model response (`usage-index.mjs:647-658`) | Codex tool calls and tool outputs travel in event types the parser does not surface as turns at all — so a Codex transcript renders as a prompt/response @@ -95,8 +95,8 @@ The same parsers serve two very different callers, switched by `withTurns`: | Path | Entry point | `withTurns` | Message bodies | Cached? | |---|---|---|---|---| -| **Scan** — the aggregate index behind the Scorecard/Findings/Sessions views | `buildIndex` → `parseFile` (`usage-index.mjs:693`) | `false` | never held — holding them would balloon memory across 3,000+ files (`usage-index.mjs:437-440`) | yes: per-file derived records in `~/.config/agentic-kit/usage-index.json`, keyed `(path, mtime, size)`, invalidated wholesale by `SCHEMA_VERSION` (`usage-index.mjs:51`) | -| **Reader** — one transcript for the Transcript view | `readSession` (`usage-index.mjs:1310`) | `true` | full turn list built | **never** — every call re-reads and re-parses the one file | +| **Scan** — the aggregate index behind the Scorecard/Findings/Sessions views | `buildIndex` → `parseFile` (`usage-index.mjs:746`) | `false` | never held — holding them would balloon memory across 3,000+ files (`usage-index.mjs:490-493`) | yes: per-file derived records in `~/.config/agentic-kit/usage-index.json`, keyed `(path, mtime, size)`, invalidated wholesale by `SCHEMA_VERSION` (`usage-index.mjs:51`) | +| **Reader** — one transcript for the Transcript view | `readSession` (`usage-index.mjs:1363`) | `true` | full turn list built | **never** — every call re-reads and re-parses the one file | ![Figure: one parser, two read paths — the scan path (withTurns false) caches per-file records keyed by path, mtime and size; the reader path (withTurns true) builds full turns and is never cached](assets/transcript-read-paths.svg) @@ -117,11 +117,11 @@ recorded in `USAGE-SCORECARD-METRICS.md` Appendix A). |---|---|---| | `role` | all | `"user"` or `"assistant"` — the **Messages-API role**, not the author (see below) | | `at` | all | ISO timestamp | -| `text` | all | Flattened display text (`claudeText`, `usage-index.mjs:384` — binary payloads dropped: a pasted screenshot renders as `[image]`, a tool result is prefixed `[tool result]`) | -| `model` | assistant | The model id; the literal string `exception` for an API-error placeholder turn (`usage-index.mjs:473`) | +| `text` | all | Flattened display text (`claudeText`, `usage-index.mjs:437` — binary payloads dropped: a pasted screenshot renders as `[image]`, a tool result is prefixed `[tool result]`) | +| `model` | assistant | The model id; the literal string `exception` for an API-error placeholder turn (`usage-index.mjs:526`) | | `tools` | assistant | Tool names invoked in the turn | -| `prompt` | user | `isHumanPrompt`'s verdict (`usage-index.mjs:388-397`) — drives the **prompt counts** | -| `kind` | user | `'prompt'` \| `'tool-result'` \| `'context'` — drives the **attribution label** (`userTurnKind`, `usage-index.mjs:415-426`) | +| `prompt` | user | `isHumanPrompt`'s verdict (`usage-index.mjs:441-450`) — drives the **prompt counts** | +| `kind` | user | `'prompt'` \| `'tool-result'` \| `'context'` — drives the **attribution label** (`userTurnKind`, `usage-index.mjs:468-479`) | | `exception` | assistant | `true` on API-error placeholder turns | | `truncated`, `originalChars` | any | Present **only** when the turn was abridged (§4.3) | @@ -141,7 +141,7 @@ story is [Appendix A](#appendix-a--fix-history).) ### 3.2 `kind` — the attribution field -`userTurnKind` (`usage-index.mjs:451-455`) classifies every user-role turn: +`userTurnKind` (`usage-index.mjs:523-528`) classifies every user-role turn: | `kind` | Test | Meaning | |---|---|---| @@ -164,7 +164,7 @@ Two deliberate subtleties: - **`tool-result` outranks `context`**: a `tool_result` block on an `isMeta` entry is still tool feedback. -Codex user turns are `kind: 'prompt'` by construction (`usage-index.mjs:584-592`) +Codex user turns are `kind: 'prompt'` by construction (`usage-index.mjs:710`) — rollouts only record real prompts as `user_message` events (§1.2). Coverage: `tests/kit/usage-index.test.mjs` — "user-role turns carry a kind" @@ -175,18 +175,18 @@ and image-only pastes get the right kind" (the two edges). ## 4. The `readSession` pipeline — how one session becomes a payload -`readSession(id, opts)` (`usage-index.mjs:1297-1385`) is the only way +`readSession(id, opts)` (`usage-index.mjs:1350-1438`) is the only way transcript content leaves the module, and every step is a gate: ### 4.1 Locate, contain, bound 1. **Id grammar before any filesystem access** — `VALID_ID` (`/^[A-Za-z0-9._-]{1,128}$/`, `usage-index.mjs:83`) rejects traversal - shapes with `ERR_INVALID_SESSION_ID` (`usage-index.mjs:1329-1333`). -2. **Locate by id** across both roots (`locate`, `usage-index.mjs:1339`), + shapes with `ERR_INVALID_SESSION_ID` (`usage-index.mjs:1382-1386`). +2. **Locate by id** across both roots (`locate`, `usage-index.mjs:1392`), consulting the scan cache when present but never requiring it — `readSession` works with no prior `buildIndex`. -3. **Realpath containment** (`usage-index.mjs:1335-1349`) — the resolved file +3. **Realpath containment** (`usage-index.mjs:1388-1402`) — the resolved file must live under a transcript root *after* `realpathSync` collapses symlinks; a symlink planted inside a root pointing at `/etc/anything` passes a lexical `startsWith` but fails this. Roots are realpath'd too so @@ -198,8 +198,8 @@ transcript content leaves the module, and every step is a gate: ### 4.2 Parse and price The file is parsed with `withTurns: true` by the provider's parser -(`usage-index.mjs:1430-1437`), and `meta` is assembled -(`usage-index.mjs:1414-1442`) with the same fields the Sessions view rows +(`usage-index.mjs:1483-1490`), and `meta` is assembled +(`usage-index.mjs:1467-1495`) with the same fields the Sessions view rows carry — `prompts`, `responses`, `exceptions`, `sidechain`, `threadSource`, `models`, `tools`, `skill`/`plugin`, worktree — plus a `cost` priced from the same per-model usage rows `aggregate()` uses (the header used to render a @@ -211,7 +211,7 @@ Every turn body is passed through `maskSecrets` (`usage-index.mjs:196` — the 23 secret shapes) **server-side, before serialization**, then length-capped at `MAX_TURN_CHARS` (40,000, `usage-index.mjs:77`) with the marker appended -(`usage-index.mjs:1477-1487`). Two invariants: +(`usage-index.mjs:1530-1540`). Two invariants: - **Presence is the signal.** `truncated`/`originalChars` are emitted only when the slice fired, so a complete turn cannot be misread as abridged. diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index f747b84..2f70aab 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -132,6 +132,21 @@ sources establish facts with observed, configured, inferred, or unknown provenan not silently create, adopt, or rewrite these bindings, and credentials remain environment-only. See [ADR-0016](adr/0016-capability-driven-integration-adapters.md). +## `ak system --json` fields removed in 4.0.0-alpha.41 + +Two fields left the runtime census. If you parse `ak system --json` (or `GET /api/system`), read +them defensively or drop them: + +| Removed | Where | Why | +|---|---|---| +| `runtime.daemons.budget` | daemon census | No local source exists for ruflo's launch budget — not circumstantially, structurally — so the field could only ever read `unknown`. A permanently unknowable quantity is removed rather than reported as degraded ([ADR-0023](adr/0023-fail-closed-operations-and-explicit-degradation.md) §9). `ruflo daemon budget` remains the way to ask. | +| `runtime.childProcessCount` | runtime census | Still counted by the process survey — it is what makes the per-host rows correct — but no longer republished. As a rendered figure it was a bare number with no denominator, no history and no action attached. | + +Nothing else was removed. `storage.topSessions` rows **gained** `projectLabel` and +`projectResolved`; the raw `project` key is unchanged. `catalog.items` now also covers +project-scoped `.claude/skills|agents|commands` across every project on disk, so the list is +longer — the shape is identical and deduplication by `(kind, name)` is unchanged. + ## QE-Court configs created before agentic-qe 3.13.3 `agentic-qe` 3.13.3 fixed its shipped QE-Court default and made configuration validation diff --git a/docs/USAGE-SCORECARD-METRICS.md b/docs/USAGE-SCORECARD-METRICS.md index fb24edf..d27e0ec 100644 --- a/docs/USAGE-SCORECARD-METRICS.md +++ b/docs/USAGE-SCORECARD-METRICS.md @@ -64,8 +64,8 @@ schema change — `src/lib/usage-index.mjs:65`): | Claude Code | `~/.claude/projects//.jsonl` | one JSON object per line: `user`/`assistant` turns, each assistant turn carrying its own `usage` object | | Codex CLI | `~/.codex/sessions///
/rollout--.jsonl` | one JSON object per line: `session_meta`, `turn_context`, and `event_msg` records, the latter carrying **cumulative** `token_count` snapshots, not per-turn deltas | -The parsers are `parseClaude` (`usage-index.mjs:427-510`) and `parseCodex` -(`usage-index.mjs:527-628`). Both are pure functions over the raw file bytes — +The parsers are `parseClaude` (`usage-index.mjs:480-563`) and `parseCodex` +(`usage-index.mjs:580-681`). Both are pure functions over the raw file bytes — no network, no clock dependency beyond the transcript's own timestamps — so every downstream number traces back to bytes already on the user's disk. Nothing in this transcript pipeline calls a provider API or a billing endpoint; **no transcript @@ -128,14 +128,14 @@ responses = Σ over included sessions of session.responses **Source:** - Filter: a parsed record with zero assistant turns is dropped entirely — "no - assistant turn → not a session" (`usage-index.mjs:923`) — and a record whose + assistant turn → not a session" (`usage-index.mjs:988`) — and a record whose last activity falls outside the requested window is dropped too - (`usage-index.mjs:924`). + (`usage-index.mjs:989`). - `responses` accumulation: Claude increments per assistant message - (`usage-index.mjs:504-509`); Codex increments per `agent_message` event - (`usage-index.mjs:651-655`). + (`usage-index.mjs:557-562`); Codex increments per `agent_message` event + (`usage-index.mjs:704-708`). - Totals: `totals.responses += s.responses` per included session - (`usage-index.mjs:884`). + (`usage-index.mjs:937`). - Render: `kpi("sessions", fmtNum(t.sessions), fmtNum(t.responses)+" assistant turns", "")` (`dashboard/client.mjs`). @@ -174,7 +174,7 @@ cost = (inputUnits × rate_in + output × rate_out) / 1,000,000 summed across every row in the window. -**Source:** `costOf()`, `src/lib/pricing.mjs:247-253`, reproduced verbatim: +**Source:** `costOf()`, `src/lib/pricing.mjs:253-259`, reproduced verbatim: ```js export function costOf(usage) { @@ -188,17 +188,17 @@ export function costOf(usage) { ``` `CACHE_READ_MULTIPLIER = 0.1`, `CACHE_WRITE_MULTIPLIER = 1.25` -(`pricing.mjs:154-155`) — see §13 for why these two numbers are correct for +(`pricing.mjs:160-161`) — see §13 for why these two numbers are correct for **both** Anthropic and OpenAI, which is why `costOf` needs no per-provider branch on the multiplier (only on the base `rate_in`/`rate_out`, resolved by -`priceFor`, `pricing.mjs:214-227`). +`priceFor`, `pricing.mjs:220-233`). Rate resolution is **longest-prefix match** on a normalized model id -(`pricing.mjs:163-172`), so a dated release (`claude-haiku-4-5-20251001`) +(`pricing.mjs:169-178`), so a dated release (`claude-haiku-4-5-20251001`) resolves to the same entry as its bare alias, and a more specific entry (`gpt-5.6-sol`) is never shadowed by a shorter one (`gpt-5.6`). An id matching nothing gets `FALLBACK_PRICE` — Sonnet-class rate, `$3`/`$15` -(`pricing.mjs:151`) — rather than `$0`, so an unrecognized model can never be +(`pricing.mjs:157`) — rather than `$0`, so an unrecognized model can never be silently free; `matched: false` travels with the result so a maintainer can find fallback-priced rows if the table needs a new entry. @@ -208,12 +208,12 @@ Each table entry is a **schedule** — an ordered list of periods, each with the day it takes effect. Nearly every entry has exactly one period that has always applied (`anthropic(5, 25)` builds that shape); an entry whose rate the vendor has *published* a change to carries more than one (`schedule`, -`pricing.mjs:41-55`). `periodOn` (`pricing.mjs:187`) picks the last period +`pricing.mjs:41-55`). `periodOn` (`pricing.mjs:193`) picks the last period already in effect on the given day, comparing ISO date strings lexicographically so no `Date` parsing is involved and the module stays clock-free. -`aggregate()` passes each usage row's own `day` (`usage-index.mjs:911`), which +`aggregate()` passes each usage row's own `day` (`usage-index.mjs:979`), which it already has because rows are keyed by `(day, model)`. **This is the whole point:** tokens metered in August must still read as August's rate when the panel is opened in December. Pricing by *today's* date instead would restate a @@ -235,7 +235,7 @@ Two rules bound the mechanism: Codex promo would be a one-line edit rather than new machinery. Rates that vary by *how* a request was served — regional uplift, large-prompt surcharge, service tiers — are a different axis, are deliberately **not** expressible - here, and remain in `UNMODELLED_PRICING_FACTORS` (`pricing.mjs:142-144`) + here, and remain in `UNMODELLED_PRICING_FACTORS` (`pricing.mjs:148-150`) because a transcript does not record the endpoint or tier. A `priceFor` call with no day prices as of `PRICES_AS_OF`, the table's @@ -295,9 +295,9 @@ tokens = input + output + cacheRead + cacheWrite (summed across all rows in wi ``` **Source:** `t.tokens` from `totals`, accumulated per row at -`usage-index.mjs:935` (`rowTokens = row.input + row.output + row.cacheRead + +`usage-index.mjs:988` (`rowTokens = row.input + row.output + row.cacheRead + row.cacheWrite`) and rolled into `totals.tokens` via `addTo` -(`usage-index.mjs:843-852`). Rendered with `fmtTok()` +(`usage-index.mjs:1080-1086`). Rendered with `fmtTok()` (`dashboard/client.mjs`): `≥1e9` → `"X.XB"`, `≥1e6` → `"X.XM"`, `≥1e3` → `"X.XK"`, else the rounded integer. @@ -309,9 +309,9 @@ numbers as percentages of `t.tokens` (`dashboard/client.mjs`, **What "input" excludes.** For both providers, the `input` counter recorded per row is **gross input minus cached input** — Claude's parser reads `cache_read_input_tokens` and `cache_creation_input_tokens` as separate fields -the provider already reports separately (`usage-index.mjs:523-524`); Codex's +the provider already reports separately (`usage-index.mjs:598-599`); Codex's parser subtracts `cached_input_tokens` from `input_tokens` explicitly -(`usage-index.mjs:666-675`, `input: Math.max(0, gross - cacheRead)`) because +(`usage-index.mjs:719-728`, `input: Math.max(0, gross - cacheRead)`) because Codex's own `input_tokens` field **includes** cached tokens and would double-count them against the separately-reported `cacheRead` figure if left as-is. This is asserted by test: @@ -396,7 +396,7 @@ session data, and each needs its own fix: human, or genuinely idle) donates its *entire* idle stretch to the span, even though no work happened during it. Fix: split each session into active sub-intervals wherever the gap between two consecutive timestamps - exceeds `IDLE_GAP_MS` (15 minutes, `usage-index.mjs:62`), then union + exceeds `IDLE_GAP_MS` (15 minutes, `usage-index.mjs:80`), then union *those* sub-intervals — this is `engagedSeconds`. **Source:** @@ -405,15 +405,15 @@ session data, and each needs its own fix: sorts intervals and merges any two that overlap **or exactly touch** (`s <= curEnd`, `usage-index.mjs:105`), returning total covered seconds rounded to the nearest second. -- `activeIntervals()` (`usage-index.mjs:352-363`) — splits one session's +- `activeIntervals()` (`usage-index.mjs:405-416`) — splits one session's sorted timestamp list into sub-intervals wherever a gap exceeds `IDLE_GAP_MS`; "a run of one timestamp yields a zero-length interval and so - contributes nothing" (comment, `usage-index.mjs:346-350`). + contributes nothing" (comment, `usage-index.mjs:399-403`). - Aggregation: `totals.engagedSeconds = mergeIntervals(sessions.flatMap(s => - s._active))` (`usage-index.mjs:927`); `totals.spanUnionSeconds = - mergeIntervals(sessions.map(s => s._span))` (`usage-index.mjs:926`); + s._active))` (`usage-index.mjs:980`); `totals.spanUnionSeconds = + mergeIntervals(sessions.map(s => s._span))` (`usage-index.mjs:979`); `totals.spanMinutes` is a running sum of `s._span[1] - s._span[0]` across - the loop (`usage-index.mjs:888`, finalized `usage-index.mjs:925`). + the loop (`usage-index.mjs:941`, finalized `usage-index.mjs:978`). - Render: `fmtHours()` (`dashboard/client.mjs`, `≥10h` rounds to the nearest hour, else one decimal place) and `fmtMins()` (`dashboard/client.mjs`, `≥60min` rounds to hours, else whole @@ -462,12 +462,12 @@ byDay[day].cost = Σ costOf(row) for every usage row whose day == that key **Source:** the day key is the row's own `row.day`, computed once at parse time as **local calendar day**, not UTC -(`usage-index.mjs:530`/`usage-index.mjs:667` call `localDay(at)`) — so a +(`usage-index.mjs:595`/`usage-index.mjs:732` call `localDay(at)`) — so a session that runs from 23:58 local to 00:05 local is billed to the day its *first* row landed on (test: `tests/kit/usage-index.test.mjs:634`, "a session that opens before midnight is counted on its first billed day"). Accumulation: -`byDay[row.day].cost += rowCost` (`usage-index.mjs:846`). Bar height: +`byDay[row.day].cost += rowCost` (`usage-index.mjs:899`). Bar height: `h = maxDay ? max(2, cost/maxDay*100) : 2` (`dashboard/client.mjs`) — every non-empty day gets a visually nonzero bar (floor of 2%), so a very cheap day is never rendered as invisible. @@ -488,11 +488,11 @@ renders "no sessions in window" instead of zeroed figures (`dashboard/client.mjs`). **Formula:** identical aggregation to every other bucket -(`byProvider[s.provider]`, populated via `addTo()`, `usage-index.mjs:889-898`, - called once per session at `usage-index.mjs:1000`), keyed by the literal string +(`byProvider[s.provider]`, populated via `addTo()`, `usage-index.mjs:942-951`, + called once per session at `usage-index.mjs:1084`), keyed by the literal string `"claude"` or `"codex"` assigned at parse time (`blankSession(id, 'claude')` / `blankSession(id, 'codex')`, -`usage-index.mjs:291-301`, `parseClaude`/`parseCodex` entry points). +`usage-index.mjs:398-408`, `parseClaude`/`parseCodex` entry points). **Why this pairing is the one under the most scrutiny.** Both providers' tokens are summed into the *same* `tokens`/`cost` fields using the *same* @@ -528,9 +528,9 @@ punchcard[dow + "-" + hour] += 1 per assistant/agent_message response, at its ``` **Source:** incremented once per Claude assistant turn -(`usage-index.mjs:504-509`, keyed by `punchKey(at)`) and once per Codex -`agent_message` (`usage-index.mjs:651-655`), merged into the window-level -`punchcard` object per session (`usage-index.mjs:1000`). Cell intensity is +(`usage-index.mjs:557-562`, keyed by `punchKey(at)`) and once per Codex +`agent_message` (`usage-index.mjs:704-708`), merged into the window-level +`punchcard` object per session (`usage-index.mjs:1053`). Cell intensity is linear against the single busiest cell in the window: `v = pcMax ? n/pcMax : 0` (`dashboard/client.mjs`) — this is a **relative**, not absolute, scale, so the heatmap's brightest cell is always @@ -563,9 +563,9 @@ byModel[model].sessions = count of DISTINCT sessions whose s.models includes th ``` **Source:** cost/tokens/responses accumulate inside the usage-row loop -(`usage-index.mjs:896-922`); the `sessions` count is deliberately computed +(`usage-index.mjs:949-975`); the `sessions` count is deliberately computed **separately**, once per session over its `s.models` array -(`usage-index.mjs:898-903`) rather than inside the cost loop, precisely +(`usage-index.mjs:951-956`) rather than inside the cost loop, precisely **so that a model can appear in `byModel` — with a nonzero session count — even in a session that contributed zero cost/tokens/responses for that model.** This is not an edge case invented for this document: it is the @@ -575,11 +575,11 @@ excluded subagent-replay session still shows up as "used," at zero cost, rather than vanishing. `byModel[...].responses` is populated from `row.responses` -(`usage-index.mjs:921`), which in turn comes from the `responses` field +(`usage-index.mjs:1017`), which in turn comes from the `responses` field passed into `addUsage()` at the call site — `1` per Claude assistant turn -(`usage-index.mjs:484-490`), or `rec.responses` (the session's whole response +(`usage-index.mjs:537-543`), or `rec.responses` (the session's whole response count) once per Codex session, passed at the single point Codex calls -`addUsage` (`usage-index.mjs:613-624`). +`addUsage` (`usage-index.mjs:732-738`). **Render:** `bar(name, fmtUsd(cost), fmtTok(tokens)+" · "+fmtNum(responses)+" resp", pct(cost, topModelCost), false)` (`dashboard/client.mjs`), @@ -595,14 +595,14 @@ split `server_error` 27, `authentication_failed` 3, `rate_limit` 3 — three distinct underlying causes, one placeholder shape). The parser branches on `isApiErrorMessage === true` -(`usage-index.mjs:469-478`): the turn still increments `rec.responses` -and the punchcard (`usage-index.mjs:457-460`) — it *is* real engaged +(`usage-index.mjs:522-531`): the turn still increments `rec.responses` +and the punchcard (`usage-index.mjs:510-513`) — it *is* real engaged time, someone was genuinely waiting on it — but it is never pushed into `rec.models` and `addUsage()` is never called for it, so it can no longer create a `byModel` row of any kind. It increments a separate -`rec.exceptions` counter instead (`usage-index.mjs:470`), rolled up into -`totals.exceptions` (`usage-index.mjs:995-999`) and surfaced per-session -(`usage-index.mjs:920-934`, alongside the existing `sidechain`/`threadSource` +`rec.exceptions` counter instead (`usage-index.mjs:523`), rolled up into +`totals.exceptions` (`usage-index.mjs:1048-1052`) and surfaced per-session +(`usage-index.mjs:973-987`, alongside the existing `sidechain`/`threadSource` flags — inspectable in the Sessions tab, never hidden). When `totals.exceptions > 0`, the panel header shows a small `"· N dropped/errored turns excluded"` note (`dashboard/client.mjs`); @@ -756,7 +756,7 @@ own published table, not a transcription from a secondary source: | Claude Sonnet 4.6 / 4.5 | $3/MTok | $3.75/MTok | $6/MTok | $0.30/MTok | $15/MTok | | Claude Haiku 4.5 | $1/MTok | $1.25/MTok | $2/MTok | $0.10/MTok | $5/MTok | -Every value in `pricing.mjs`'s Anthropic entries (`pricing.mjs:74-108`) matches +Every value in `pricing.mjs`'s Anthropic entries (`pricing.mjs:74-114`) matches this table's "Base input" and "Output" columns exactly. Note that the two Sonnet 5 rows above are not a documentation convenience — they are exactly what the code encodes, as the two periods of that entry's schedule (§3a), so the table @@ -777,7 +777,7 @@ Anthropic's own prompt-caching documentation **[C2]** states the multipliers in prose, independent of the pricing table: *"5-minute cache write tokens are 1.25 times the base input tokens price... Cache read tokens are 0.1 times the base input tokens price."* This is the second, independent confirmation of -`CACHE_READ_MULTIPLIER`/`CACHE_WRITE_MULTIPLIER` (`pricing.mjs:154-155`). +`CACHE_READ_MULTIPLIER`/`CACHE_WRITE_MULTIPLIER` (`pricing.mjs:160-161`). ### 13.2 OpenAI (Codex) — hand-maintained, no canonical machine-readable source @@ -826,8 +826,8 @@ hand-maintained and drift-prone, not vendor-confirmed via automated fetch. ### 13.3 What the pricing table deliberately does not model -Recorded verbatim from `pricing.mjs:125-141` (`UNMODELLED_PRICING_FACTORS`, -`pricing.mjs:142-144`) because listing known gaps is what makes the +Recorded verbatim from `pricing.mjs:131-147` (`UNMODELLED_PRICING_FACTORS`, +`pricing.mjs:148-150`) because listing known gaps is what makes the *modelled* factors credible: - **Regional-processing uplift.** OpenAI charges +10% on data-residency @@ -886,7 +886,7 @@ both credential-free for ak: `windowDurationMins: 10080` (the weekly). Windows are therefore keyed and labelled by duration (`windowLabel`, `quota.mjs:44`), never by slot name. The same rule applies to the historical snapshots parsed out of rollouts: the -normalizer at `usage-index.mjs:591-615` keeps a flat `windows` list keyed by +normalizer at `usage-index.mjs:644-668` keeps a flat `windows` list keyed by `window_minutes`. **Freshness is part of the number.** Both sides carry `fetchedAt`; the view @@ -911,13 +911,13 @@ Codex ≥0.140 maintains its own SQLite thread ledger (`~/.codex/state_N.sqlite` — the `N` is a migration generation, so `codexStateDb` (`codex-state.mjs:30`) globs and takes the newest). `readCodexState` (`:49`) reads per-thread `thread_source` (`user` vs `subagent`) plus `thread_spawn_edges`, and -`applyCodexLedger` (`usage-index.mjs:1264-1275`) overlays that onto parsed +`applyCodexLedger` (`usage-index.mjs:1317-1328`) overlays that onto parsed sessions: a ledger-identified subagent has its token usage stripped — its rollout replays the parent's entire token history (ccusage/ccusage#950 measured up to 91× inflation) — while the session record stays visible. The rollout's own `session_meta.thread_source` sniff remains as the fallback when the ledger is absent or migrated beyond recognition. Codex sessions also carry -`reasoningOutput` (`usage-index.mjs:677-678`) — reasoning tokens are a **subset** +`reasoningOutput` (`usage-index.mjs:739-740`) — reasoning tokens are a **subset** of output tokens and are annotation only, never added to any sum. ## 14. Known limitations, restated as a single checklist @@ -962,14 +962,14 @@ commit `540be18` on this branch. `parseCodex`'s single `addUsage()` call never included a `responses` field — Claude's parser passes `responses: 1` per assistant turn -(`usage-index.mjs:503`, the current equivalent), but Codex's call +(`usage-index.mjs:556`, the current equivalent), but Codex's call passed no such field at all. Because `byModel[model].responses` is summed -directly from each usage row's `responses` field (`usage-index.mjs:921`, +directly from each usage row's `responses` field (`usage-index.mjs:974`, `m.responses += row.responses`), **every** Codex model in §10's "Models in Play" list displayed `0 resp` regardless of real token/cost volume or actual `agent_message` count. **Fix:** `parseCodex` now passes `responses: rec.responses` (the session's own tallied response count, -`usage-index.mjs:654`) on its `addUsage()` call. +`usage-index.mjs:707`) on its `addUsage()` call. #### Bug B — subagent thread-replay could double-bill tokens @@ -989,13 +989,13 @@ at face value (correctly avoiding the separate naive-summing bug **[C5]** documents, since it already used last-event-only logic — see §4's worked example) but performed **no de-duplication** against a parent session a subagent file might be replaying. **Fix:** the parser now reads -`session_meta.thread_source` (`usage-index.mjs:575-579`, confirmed as a real +`session_meta.thread_source` (`usage-index.mjs:628-632`, confirmed as a real Codex rollout field by **[C7]**) and skips the `addUsage()` call entirely -when its value is `'subagent'` (`usage-index.mjs:609`, guard condition +when its value is `'subagent'` (`usage-index.mjs:662`, guard condition `rec.threadSource !== 'subagent'`). The session record itself is **not** dropped — it remains visible in the Sessions tab with `threadSource` surfaced (mirroring the existing `sidechain` flag Claude sessions already -carry, `usage-index.mjs:294`), so a maintainer auditing the raw data can +carry, `usage-index.mjs:347`), so a maintainer auditing the raw data can still see it; it simply contributes zero tokens/cost, exactly as intended by the "models still shows up in §10's list, with zero cost" mechanism §10 describes. diff --git a/docs/adr/0002-activity-vocabulary-defaults-from-ruv-templates.md b/docs/adr/0002-activity-vocabulary-defaults-from-ruv-templates.md index 959e09c..eac846f 100644 --- a/docs/adr/0002-activity-vocabulary-defaults-from-ruv-templates.md +++ b/docs/adr/0002-activity-vocabulary-defaults-from-ruv-templates.md @@ -38,6 +38,13 @@ as such** wherever surfaced (an `ak` tag in the UI, a comment in the defaults ta Default *models* map to the host's appropriate tier (Opus for deep reasoning, Sonnet for review, a Codex model for execution) and are treated as **soft defaults** — see the "open question" on pinning live model IDs. +Tier is the pairing key, not the model id: `MODEL_CATALOG` spells `flagship`/`balanced`/`fast` +identically on both hosts so primary-host mirroring can map a route to its counterpart's equivalent. +As of 2026-08-07 execution routes to `gpt-5.6-terra` (balanced) and mechanical work to `gpt-5.6-luna` +(fast), following OpenAI's own migration off `gpt-5.4`/`gpt-5.4-mini` before their 2026-08-31 Codex +retirement; deep reasoning routes to `claude-opus-5`. Withdrawn ids are handled by the retirement +mechanism in [ADR-0003](0003-auto-seed-dual-host-provenance.md), not by editing this table alone. + ## Consequences - Defaults are **provably grounded** and explainable ("architect→claude because `featureDevelopment` does"). diff --git a/docs/adr/0003-auto-seed-dual-host-provenance.md b/docs/adr/0003-auto-seed-dual-host-provenance.md index 868888d..53bc3ec 100644 --- a/docs/adr/0003-auto-seed-dual-host-provenance.md +++ b/docs/adr/0003-auto-seed-dual-host-provenance.md @@ -2,9 +2,9 @@ - **Status:** Amended by [ADR-0020](0020-ga-stable-surfaces.md) - **Date:** 2026-07-23 -- **Updated:** 2026-07-30 +- **Updated:** 2026-08-07 - **Update note:** Preserved subscription-safe seeding and provenance while moving intent to the - canonical routing envelope. + canonical routing envelope; separated model **retirement** from route **divergence** (2026-08-07). - **Deciders:** agentic-kit maintainers > **GA amendment:** subscription-safe seeding and user-intent preservation remain. The persisted @@ -40,6 +40,34 @@ Every `ActivityRoute` carries **provenance**: `source: 'default' | 'seeded' | 'u **never clobbered**. `ak sync` reasserts the policy but never re-seeds or overwrites `user` routes. `ak x provider off` clears the whole policy and reverts projections. +### Amendment (2026-08-07) — retired models are not divergence + +Provenance answers "may ak change this value?". It does **not** answer "does this value still work?", +and one case needs both: a model the host has **withdrawn**. A route naming one is not a stale +preference, it is a scheduled hard failure — `gpt-5.4` and `gpt-5.4-mini` stop answering in Codex on +2026-08-31, and `gpt-5.3-codex` already has. + +So retirement is separated from divergence, and the two are handled differently: + +| | divergence (`divergedRoutes`) | retirement (`RETIRED_MODELS`) | +|---|---|---| +| what happened | the defaults moved | the model stops answering | +| is there a trade? | yes — a newer default can cost 2–3× the agentic turns | no | +| reported as | a choice, with both models' cost-per-task notes | a fact, with the retirement date | +| cleared by | an explicit `ak x host refresh` | `ak sync`, automatically | +| touches a `user` pin? | never | **substituted at read time, never rewritten on disk** | + +That last row is the only place ak overrides deliberate user intent, and it is deliberate: honoring a +pin into a model that no longer exists fails the run rather than respecting the pin. The override is +confined to `resolveRoutes()` — the read boundary every dispatch path already goes through — so the +file on disk still says what the user wrote, and every surface that shows the substituted model also +names the id it replaced (`retiredFrom`). + +Because this power is easy to misuse, `RETIRED_MODELS` admits **only** models with a published +withdrawal notice from the host, cited in the map. "We would rather they used the newer one" is +divergence, not retirement — `claude-opus-4-8` is the worked example: superseded as a default, no +deprecation notice, therefore deliberately absent from the map and still pinnable. + ## Consequences - **Zero-step** onboarding for dual-host users; **no cost surprise** (subscription/local only). @@ -47,9 +75,16 @@ Every `ActivityRoute` carries **provenance**: `source: 'default' | 'seeded' | 'u - Single-host users are unaffected — nothing is seeded, projections stay empty, behavior is unchanged. - Requires tracking `source` on each route and honoring it in every write path (seed, pick, sync, refresh). - If a route later points at a now-disabled host, `ak status` warns rather than silently mutating it. +- A retired model can never be dispatched, whatever the policy on disk says — but the substitution is + invisible unless surfaces render `retiredFrom`, so every route-rendering surface must carry it. +- `RETIRED_MODELS` needs maintaining against host deprecation notices. A missed entry degrades to a + failed run with a clear upstream error; a *wrong* entry silently ignores a user's pin, which is far + worse — hence the citation requirement. ## References -- `src/lib/providers.mjs` `bothHostsEnabled`, `_managedBy` / `.bak` discipline (`applyAqeRouter`, - `undoAqeRouter`, `writeJsonWithBackup`) +- `src/lib/routing.mjs` `RETIRED_MODELS`, `retirementOf`, `migrateRetiredRoutes`, `resolveRoutes` +- `src/lib/providers.mjs` `bothHostsEnabled`, `migrateRetiredRoutesInConfig`, `_managedBy` / `.bak` + discipline (`applyAqeRouter`, `undoAqeRouter`, `writeJsonWithBackup`) +- `tests/kit/routing-retirement.test.mjs` - ADR-0001, ADR-0002 diff --git a/docs/adr/0023-fail-closed-operations-and-explicit-degradation.md b/docs/adr/0023-fail-closed-operations-and-explicit-degradation.md index 5a9fedf..d81d674 100644 --- a/docs/adr/0023-fail-closed-operations-and-explicit-degradation.md +++ b/docs/adr/0023-fail-closed-operations-and-explicit-degradation.md @@ -1,6 +1,7 @@ # ADR-0023 — Fail-closed mutations and explicit degraded operation evidence - **Status:** Implemented +- **Updated:** 2026-08-07 — §9 permanently unmeasurable quantities, §10 stated exclusions - **Date:** 2026-08-04 - **Updated:** 2026-08-06 - **Update note:** Generalized setup preflight into a required host-adapter trust contract, added @@ -173,9 +174,34 @@ PATH before loading or launching the CLI. The ordinary matrix runs these tests o Windows. The scheduled/manual nightly additionally packs the current artifact and performs real setup on `macos-latest` with all global packages and user/project files under `runner.temp`. +### 9. A permanently unmeasurable quantity is deleted, not degraded (2026-08-07) + +The rules above cover a quantity that *could* be measured and was not: it renders as unknown with a +reason. They did not cover a quantity that no code path can *ever* produce. The System area shipped +one — an "AI-worker budget" tile hardcoded to unknown with the reason "ruflo exposes no local budget +state this collector can read". It could never render anything else. + +An always-unknown tile is not honest degradation, it is a promise the product cannot keep, occupying +space and teaching the reader to ignore unknowns. So: a field whose reason is *structural* rather +than circumstantial is **removed from the collector, the payload and the panel**, not rendered as +degraded. The distinction is whether a plausible future state of the machine would populate it. If +one would, it stays and degrades; if none would, it was never a measurement. + +### 10. An excluded figure is still stated (2026-08-07) + +Storage lifts learning stores out of its shared charts because at ~99% of retained bytes they flatten +every other series. Exclusion for legibility is allowed; *silent* exclusion is not. The excluded +category is reported on its own card, the charts say what they leave out, and the per-host split +names the non-host bytes it drops with their figure — so the bars still account for the whole of the +donut beside them. A reader must never have to reconcile two panels and find the difference +unexplained. + ## Consequences - A fallback can keep work available without being mislabeled healthy. +- An unknown carries information, because nothing that is permanently unknowable is rendered as one. +- A chart may exclude a category for legibility, but the panel says so and the excluded figure is + still reachable. - Temporary SQLite locks and corrupt stores no longer become measured zero usage. - Backup-path problems stop setup/sync instead of destroying the only recoverable pre-write state. - Status-line failures stay silent for ordinary users and become diagnosable without logging content. diff --git a/docs/adr/0024-project-intelligence-telemetry.md b/docs/adr/0024-project-intelligence-telemetry.md index 3f2e41b..45120a8 100644 --- a/docs/adr/0024-project-intelligence-telemetry.md +++ b/docs/adr/0024-project-intelligence-telemetry.md @@ -1,8 +1,8 @@ # ADR-0024 — Project intelligence: live learning telemetry from ruflo/agentic-qe's own state -- **Status:** Implemented +- **Status:** Implemented; discovery amended by [ADR-0027](0027-shared-project-census.md) - **Date:** 2026-08-05 -- **Updated:** 2026-08-05 +- **Updated:** 2026-08-07 - **Update note:** Extended Intelligence from one project's telemetry, implicitly tied to the dashboard server's own launching cwd, to a machine-wide catalog of every ruflo-initialized project plus an explicitly selected, explicitly labeled detail project (defaulting to @@ -13,9 +13,22 @@ - **Deciders:** agentic-kit maintainers - **Related:** [ADR-0005](0005-dashboard-in-page-routing-reveal.md), [ADR-0009](0009-usage-scorecard-local-transcript-analytics.md), - [ADR-0012](0012-observability.md) + [ADR-0012](0012-observability.md), + [ADR-0027](0027-shared-project-census.md) -**2026-08-05 machine-wide discovery amendment:** `src/lib/dashboard/project-discovery.mjs` adds +> **Superseded 2026-08-07 by [ADR-0027](0027-shared-project-census.md).** The machine-wide +> discovery amendment below is retained as the record of how discovery worked from 2026-08-05; its +> mechanism is gone. `discoverRuvfloProjects()` and `project-discovery.mjs` are retired, and +> `registryWorkspaces()` is module-private again. Two things it got wrong, both visible only at +> scale: requiring `.claude-flow/neural/` answered "has ruflo *trained* here" rather than "is +> memory or intelligence active here", excluding agentic-qe and swarm state and every non-Claude +> host; and the 150-transcript bound made discovery a function of recency. On the machine this +> amendment was verified against, it found 4 projects where 17 had learning state. Project +> discovery now comes from the shared census, which lists directories and folds them onto project +> identity. Everything else in this ADR — the intel payload, the SSE pool, the panel itself — +> stands unchanged. + +**2026-08-05 machine-wide discovery amendment (superseded):** `src/lib/dashboard/project-discovery.mjs` adds `discoverRuvfloProjects()`, unioning three sources into one deduplicated, most-recently-active-first catalog of every project on this machine ruflo has genuinely initialized — a `.claude-flow/neural/` subdirectory present, not merely a bare `.claude-flow/`. Source 1 reuses `registryWorkspaces()` from @@ -237,7 +250,7 @@ path. - `src/lib/dashboard/intel-history.mjs` (`readIntelHistory`, `readMachineWideIntel`), `tests/kit/intel-history.test.mjs` -- `src/lib/dashboard/project-discovery.mjs` (`discoverRuvfloProjects`), +- `src/lib/project-census.mjs` (`projectCensus`, `projectsInScope`) — replaced `src/lib/dashboard/project-discovery.mjs` (`discoverRuvfloProjects`) per ADR-0027, `tests/kit/project-discovery.test.mjs` - `src/lib/live/intelligence-watch.mjs`, `tests/kit/intelligence-watch.test.mjs` - `src/lib/dashboard-server.mjs` (`collectData`, `buildProjectSnapshotCache`, diff --git a/docs/adr/0025-machine-footprint-metrics.md b/docs/adr/0025-machine-footprint-metrics.md new file mode 100644 index 0000000..a8cd668 --- /dev/null +++ b/docs/adr/0025-machine-footprint-metrics.md @@ -0,0 +1,464 @@ +# ADR-0025 — Machine footprint: infrastructure metrics for install, runtime, storage, and catalog + +- **Status:** Implemented +- **Date:** 2026-08-06 +- **Updated:** 2026-08-06 — accepted and implemented; the open points below are resolved decisions +- **Updated:** 2026-08-07 — §7 replaced by an enumerated read surface (the collectors now read a + transcript head's `cwd` field and project manifests' dependency keys); §6 gains reclaimable + safety tiers; §8 and §9 added for the widened scan surface, the corrected brain/Playwright + figures, and project accounting +- **Deciders:** agentic-kit maintainers +- **Related:** [ADR-0005](0005-dashboard-in-page-routing-reveal.md), + [ADR-0007](0007-maintainer-admin-local-telemetry.md), + [ADR-0009](0009-usage-scorecard-local-transcript-analytics.md), + [ADR-0012](0012-observability.md), + [ADR-0014](0014-dashboard-auth-and-remediation.md), + [ADR-0023](0023-fail-closed-operations-and-explicit-degradation.md), + [ADR-0024](0024-project-intelligence-telemetry.md) + +> **2026-08-07 amendment.** Four changes, each recorded where it belongs: +> project discovery moved to the shared census ([ADR-0027](0027-shared-project-census.md)); +> Storage reports learning stores on their own card and its per-host split covers real hosts only, +> naming the excluded figures rather than dropping them ([ADR-0023](0023-fail-closed-operations-and-explicit-degradation.md) §10); +> the Runtime census dropped its launch-budget and child-process fields, the first because no code +> path could ever populate it ([ADR-0023](0023-fail-closed-operations-and-explicit-degradation.md) §9); +> and the Catalog now covers project scope across every project on disk, not just the launching repo. +> The Projects table dropped its forge sub-line, its presence-only stack chips and its +> tree/.git/node_modules breakdown from the RENDERING — all three are still measured and still ship +> on `ak system --json` — and now lists only repositories with a remote that a host has recorded a +> session in, counting the excluded directories beneath it. +> +> The area also grew from five sub-views to seven: **Advisory** and **Sessions** split out of +> Storage. Advisory earns its own tab because it is the only part of System that suggests an +> action while every other part reports what is; that distinction was invisible while it sat as a +> card under a byte chart. Neither split changes a measurement, and Advisory still has no delete +> verb (§6 stands). + +## Context + +The dashboard currently answers three families of questions, each owned by its own context: + +- **Overview** — is the kit healthy and configured (subsystem status, hosts, routing, providers, + Intelligence's learning trends per [ADR-0024](0024-project-intelligence-telemetry.md))? +- **Usage** — what did sessions cost in tokens and API-equivalent dollars + ([ADR-0009](0009-usage-scorecard-local-transcript-analytics.md))? +- **Observability** — what are agents doing right now, and what did completed sessions do + ([ADR-0012](0012-observability.md))? + +Nobody answers the fourth family: **what does this toolchain cost the machine itself.** Real +questions a user of this stack has today, none answerable without hand-rolled `du`/`ps`/`find` +archaeology: + +- How big is the whole install — ruflo + agentic-qe + Claude Code + Codex + OpenCode + their + native addons and caches — and which tool owns how much of it? +- How much disk do six months of transcripts hold, broken down by host, by project, by session? + Which single sessions are the giants? +- How much of the disk is host ledgers and logs (Codex's `state_N.sqlite` thread ledgers, the + statusline tee files, OpenCode's store) as opposed to transcripts proper? +- How much RAM and CPU are three concurrent host processes plus background daemons consuming + right now? +- How many distinct skills, agents, and slash commands are actually deployed across hosts — and + which hosts have which? +- How many projects has this machine touched, and per project: how many lines of code, how much + disk (working tree, `.git`, `node_modules`)? + +Everything needed is already locally readable at trust boundaries the kit already crosses: + +- the install trees it manages and their install methods + ([MANAGED-TOOLS.md](../MANAGED-TOOLS.md); npm/mise/brew awareness already exists for drift + reporting); +- the current-user, argv-minimized process survey Observability already runs + (`src/lib/live/process-sessions.mjs`, hardened under + [ADR-0023](0023-fail-closed-operations-and-explicit-degradation.md)); +- the transcript roots Historical usage and Observability already walk + (`~/.claude/projects`, `~/.codex/sessions`, OpenCode's store); +- the catalog surfaces Integration management already projects into (`.claude/agents`, + `.claude/commands`, skills listings, OpenCode's converted agents, MCP registrations); +- the project catalog machine-wide discovery already assembles + ([ADR-0024](0024-project-intelligence-telemetry.md)'s transcript-cwd source); +- the third-party cache roots any developer machine accumulates — model weights, package caches, + toolchain installs — which are plain directories under `$HOME` and need naming, not access + (§8). + +The gap is a **domain and a UX**, not access. + +This record was written as a proposal and is retained in that voice; the decision has since been +accepted and shipped. Where the draft left a choice open, the [resolved +decisions](#resolved-decisions) section states what was decided and why. + +## Decision + +### 1. A new bounded context: Machine footprint + +Machine footprint is its own bounded context — see +[Machine footprint](../ddd/machine-footprint.md) for the model, boundaries, and invariants. It is +deliberately **not** part of: + +- **Historical usage** — no tokens, no cost, no model identity here; a byte of transcript on disk + is a storage fact, not a spend fact; +- **Observability** — no session lifecycle, no evidence confidence, no transcript content; this + domain reads `stat` metadata, paths, and declared names, never message bodies (§7 enumerates + every read); +- **Project intelligence** — no learning counters; the shared piece is project *discovery* as a + candidate-path source, on the same boundary ADR-0024 draws when it reuses Observability's + workspace store, though this domain runs its own discovery (§9). + +### 2. A fourth primary area: System + +The dashboard gains a fourth primary area — **System** — alongside Overview, Usage, and +Observability. This deliberately amends the "three stable primary areas" layout decision +([ADR-0005](0005-dashboard-in-page-routing-reveal.md) and `page.mjs`'s own header comment). +Machine-scoped resource facts are a peer question family, not a subsection of configuration +health; folding them under Overview would bury a whole domain under a tab whose contract is +"readiness and attention," and Overview's secondary rail is already five views deep. + +Information architecture ([ADR-0026](0026-about-component-directory.md) later added **About** +left of Overview; the System area itself is unaffected): + +```text +[ Overview | Usage | Observability | System ] + +System secondary rail: + [ Summary | Storage | Runtime | Catalog | Projects ] [ as of 2h ago · ⟳ rescan ] + +Summary KPI band: install size · data size · live processes (combined RSS) · projects + ("N ever · M on disk") · skills/agents/commands counts · machine free-space + denominator. "Largest consumers" strip (top-20 roots, ranked or grouped by + ecosystem, project trees opt-in). Freshness label. +Storage Breakdown tree: category → host → project → session. Transcripts vs ledgers/logs vs + learning stores vs kit caches. Trailing-30d growth sparkline per host. Top-N largest + sessions/files. Advisory reclaimable candidates in two safety tiers, never one total. +Runtime Live process table (host, pid, CPU%, RSS, uptime, bound project) + combined totals. + Daemon census (count, age vs TTL). +Catalog Deduplicated skills / agents / commands / plugins / MCP servers, each with a per-host + presence matrix (which hosts carry it). +Projects Table: project (name links to its git remote's web page when one exists — derived + from .git/config, "local only" otherwise), lines of code (by language), detected + frameworks/SDKs/tools (presence only), working-tree bytes, .git bytes, + node_modules bytes, last activity. Rows are the on-disk subset; ever-seen is a count. +``` + +The window/refresh control sits in the secondary-actions slot, the same pattern as Usage's +day-window chips and Observability's history-window chips. + +### 3. Tiered, honest collection + +Two tiers, because the metrics differ by orders of magnitude in cost: + +- **Cheap tier** (served on every `GET /api/system`, TTL-cached ~60s like the project-snapshot + cache): the process census, sizes of individually known files (ledgers, caches, index files), + install-tree sizes carried forward from the last deep scan, and the persisted deep snapshot's + contents with their `asOf`. +- **Deep tier** (explicit, user-triggered): the full storage walk, per-project LOC counting, and + cross-host catalog deduplication. Single-flight — concurrent requests attach to the in-flight + scan. The result is persisted to `~/.config/agentic-kit/footprint-snapshot.json` so the panel + paints instantly on every later open, labeled with when it was measured. + +The honest-measurement contract of +[ADR-0023](0023-fail-closed-operations-and-explicit-degradation.md) applies verbatim: a section +that has never been deep-scanned renders "not measured yet," never `0`; a measured zero is a real +zero; a subtree that fails to stat degrades that node to unknown without discarding its siblings. + +### 4. Initial metric taxonomy (non-exhaustive by design) + +Metrics marked ✚ are additions beyond the requesting examples; the taxonomy is expected to grow. + +| Section | Metric | Source | +|---------|--------|--------| +| Install | Per managed tool (ruflo, agentic-qe, claude, codex, opencode, ak, brain KB): version, install method, root path, tree bytes | managed-tools detection + walk | +| Install | Native-addon inventory and duplicate builds across trees (better-sqlite3, hnswlib, onnxruntime) ✚ | walk | +| Install | Shared caches: npx cache envs, brain KB, browser binaries ✚ | known roots | +| Install | Total install bytes + machine free-space denominator ✚ | walk + `statfs` | +| Runtime | Per live host process: pid, host, CPU%, RSS, uptime, bound project | existing runtime survey + `ps -o pcpu,rss` | +| Runtime | Daemon census: count, age vs 12h TTL ✚ | existing daemon registry | +| Runtime | Child / MCP-server process count ✚ | survey process tree | +| Storage | Transcript bytes + file counts: host → project → session | transcript-root walk | +| Storage | Host ledgers/logs: Codex `state_N.sqlite`, statusline tee, runtime-debug log, OpenCode store | known paths | +| Storage | Learning/memory stores: per-project `.claude-flow`, `.agentic-qe`, agentdb/HNSW/RVF files ✚ | project catalog + walk | +| Storage | ak's own caches: usage-index.json, observability-workspaces.json, footprint snapshot itself ✚ | known paths | +| Storage | Top-N largest sessions / files ✚ | walk | +| Storage | Trailing-30d growth per host (from mtime + size) ✚ | walk metadata | +| Storage | Advisory reclaimable candidates (stale npx envs, aged transcripts, superseded cache snapshots, regenerable package caches, redundant browser revisions, extra runtime versions, orphaned worktrees), each with a `safety` tier ✚ | walk + heuristics | +| Storage | Ranked largest consumers across ~50 curated third-party cache roots, grouped by ecosystem, with containment/residual accounting ✚ | consumer registry + walk | +| Catalog | Unique skills / agents / commands across hosts, per-host presence matrix | host catalog surfaces | +| Catalog | Plugins and registered MCP servers ✚ | settings surfaces | +| Catalog | Config surface: managed CLAUDE.md/AGENTS.md block count, settings file sizes ✚ | managed-blocks registry | +| Projects | Projects **ever seen** across hosts and the **on-disk** subset ✚; per on-disk project: LOC by language, detected frameworks/SDKs/tools by presence ✚, the unrecognized extension/dependency tail ✚, working-tree bytes, `.git` bytes ✚, `node_modules` bytes ✚, last activity | own cross-host discovery + walk | +| Projects | Git remote web link per project (origin URL → GitHub/GitLab/etc. page; "local only" when absent) ✚ | `.git/config` remote parse — the admin collector's `parseRepoSlug` shapes, reused | + +LOC counting is a zero-dependency extension-bucketed line count by the kit's own bounded walker +(no `cloc`/`tokei` dependency), excluding `node_modules`, vendored trees, and binary extensions — +stated as approximate, which is what the question needs. **Lines belong to languages only.** +Frameworks, SDKs and tools are detected by presence and carry no line count in the payload at +all: React does not own lines, the `.tsx` files do, and putting both on one proportional bar +would count the same bytes twice. What the registry cannot name is published as an +**unrecognized tail** — extensions by name, dependency keys by name — which turns the usual +silent "Other" slice into a to-do list a release can close. + +### 5. Delivery + +- `GET /api/system` — cheap tier + last persisted deep snapshot; same loopback bind, token auth + ([ADR-0014](0014-dashboard-auth-and-remediation.md)), and zero egress + ([ADR-0007](0007-maintainer-admin-local-telemetry.md)'s offline side of the line) as every + other dashboard route. +- `GET /api/system?refresh=deep` — starts or attaches to the single-flight deep scan. The + dashboard server is deliberately GET-only; a refresh is a re-*measurement* of local state, not + a mutation of user data, so it stays within that contract. `&trees=1|0` sets whether that scan + walks project working trees; it is a **measurement** parameter, not a view filter, because + trees that were never walked cannot be un-hidden client-side. +- `ak system [--deep] [--json]` — CLI parity sharing the same collector, following the + usage-scorecard precedent of one collector behind both surfaces. + +### 6. Read-only; reclaimables are advisory, in two safety tiers + +v1 computes reclaimable-space *candidates* and renders them with their rationale; it deletes +nothing. Cleanup remains CLI-owned where it already lives (`ak x daemon-gc`, npx cache tooling). +A future `ak system clean` would be its own decision with its own safety contract. + +Advisory is not enough on its own, because "advisory" is a tone of voice and users act on +numbers. Safety is therefore a **field**. Every candidate carries `safety`: + +- **`regenerable`** — the owning tool refetches it on demand: the npm content cache, the Homebrew + download cache, the brain's superseded `kb.bak-*` copies, stale npx envs. +- **`review`** — plausible but not safe to *state* as removable: aged transcripts (a transcript is + the only copy of the session it records, and Historical usage is denominated in them), extra + runtime versions (mise holds eight Node entries on this machine and some are aliases a live + toolchain resolves through), a browser revision that may still be pinned by an installed + package. + +**The two tiers are totalled separately and never added.** `combined` is `null` by design: a +single "you could free N" that mixed a regenerable cache with a possibly-live runtime tree would +be the one number a reader would act on and the one number this domain cannot stand behind. Two +supporting rules keep the per-tier totals honest — only rows whose `bytesMeaning` is `candidate` +are summable (an `installed` figure is context on a review row, not a claim about removable +space), and a tier whose rows describe overlapping paths reports its total as +unknown-with-reason rather than counting the same bytes twice. The `review` block is rendered +without a leading total at all, so it cannot read as available space. + +### 7. A deliberate path-visibility exception, and an enumerated read surface + +Observability's `publicLivePayload` reduces absolute paths to leaf names because its payloads +describe *sessions* and a path is incidental provenance. In this domain the paths **are the +subject matter** — a storage breakdown that hides where the bytes live answers nothing. The +System payload therefore carries absolute paths, protected by the same token-gated loopback +delivery as everything else. + +The content boundary is stated as an **enumeration**, not as "file contents are never read" — +that phrasing was true of the first collectors and became false the moment project discovery +needed a real path and the Projects table needed a language breakdown. What is read, and what is +taken from it: + +| Read | What is taken | What is never taken | +|------|---------------|---------------------| +| Directory entries, `lstat` | name, kind, size, mtime, blocks | anything inside a file | +| `.git/config` | the origin remote URL | every other key | +| `.git/worktrees//gitdir` | one path, bounded to 4 KB | — | +| A transcript's head (≤256 KB, ≤40 parsed lines) | the session's `cwd` **field** | every message, prompt, tool call, tool result and model output | +| OpenCode's session store (read-only) | the `directory` column | every other column and every message row | +| A project's own manifests (≤3 deep, ≤64 files, ≤512 KB each) | dependency **keys** | values, scripts, anything executable — nothing is evaluated or resolved | +| A project's own source files | the count of `\n` bytes | the text: each 64 KB chunk is counted and overwritten | + +Each of those yields a path, a name, or an integer. **No message body, prompt, tool call, tool +result, model output, or manifest value enters this domain, in any tier, on any path.** + +Three of the rows are justifications rather than mere disclosures. The transcript `cwd` read is +the *only* honest way to know which project a session belonged to — the alternative is decoding +Claude's transcript-directory name, which is a lossy encoding that a naive decode gets wrong for +85% of directories on a real corpus (§9). It is also the same read +`native-transcript-discovery.mjs` already performs for Observability at the same trust boundary, +and it is *discovery*, which supplies a candidate path and never a measurement. The manifest read +is the same class of datum as `.git/config`'s remote: a bounded read of a declaration file, for +names. And the source-file read is what "count lines of code" always meant — §4 sanctioned an +extension-bucketed line count from the first draft, and a line count is a byte scan; what is new +here is saying so. + +Rendering the git-remote link stays inside the zero-egress contract: the kit never fetches it; +navigation is the user clicking a link in their own browser. + +[Machine footprint § The read surface](../ddd/machine-footprint.md#the-read-surface) holds the +normative list; a read not on it is a defect, and adding one amends both documents. + +### 8. The scan surface is wider than the kit's own trees, and three figures were wrong + +The first implementation answered "where are *the kit's* bytes" and presented it as "what is +eating this disk". Those are different questions, and the gap was not a rounding error. Measured +on the authoring machine, the top consumers were **entirely invisible** to the panel: +`~/.ollama` 141.31 GB, `~/.lmstudio` 48.78 GB, `~/.cache/huggingface` 35.92 GB, `~/.npm/_cacache` +22.26 GB, mise installs 16.30 GB, Docker 15.47 GB, rustup 14.62 GB, the pnpm store 13.40 GB, the +brain 13.18 GB. The panel reported the npx cache at 6.64 GB as the machine's number one; it is +number eleven. + +The Summary strip's ranking is therefore its own collector over a curated registry of ~50 +third-party cache roots grouped by ecosystem, not a client-side sort over whatever other sections +happened to walk. A ranking whose breadth is an accident of what other sections needed is a +ranking that reads as an answer and is not one. Because these roots nest — `~/.npm` ⊃ `_cacache` +and `_npx`; `~/.cache` ⊃ huggingface, ruvnet-brain, puppeteer, pnpm, uv; `~/.local/share` ⊃ mise, +pnpm, claude, opencode; mise installs ⊃ node ⊃ the npm global root the Install section already +walks — the accounting rules are structural rather than editorial: **containment** (bytes counted +once, at the outermost row; enclosed rows are breakdowns, excluded from the ranking and the group +totals), **residuals** (a synthesized "everything else" row so a breakdown always sums to its +parent), and **project trees opt-in** (one repository here is 175 GB and would flatten the chart, +so `includeProjectTrees` defaults off and the exclusion is stated in the payload, never silent). +Absent roots are reported as absent, not ranked at zero; Docker's sparse VM image is measured in +allocated blocks because its apparent size (~4 TB) is larger than the disk. + +Two individual figures were wrong at the source, and both changed a headline: + +- **The brain was measured at its KB, not at its install root.** `kbDir()` alone under-reported + it by 85%: 1.9 GB of active KB inside a 13.18 GB cache root. The remainder is chiefly five + dated `kb.bak-*` copies the installer leaves behind on every update — ~11 GB that nothing + reads, and the single largest reclaimable on this machine — plus ~234 MB of embedding models. + The root is now measured whole *and* broken into components with a remainder row, because a + user watching a figure move from 1.9 GB to 13.18 GB is owed the reason. The upward resolution + is conservative: both `…/ruvnet-brain/kb` segments must match, so a relocated + `RUVNET_BRAIN_KB` cannot bill a shared volume to the brain. +- **Playwright's macOS location was not probed.** `playwright install` writes to + `~/Library/Caches/ms-playwright` on macOS; only the XDG and Windows paths were checked, so a + mac holding 1.86 GB of browser builds reported a *measured zero* — honest about the XDG path + that genuinely does not exist, wrong about the question the row asks. All three locations are + now probed, realpath-collapsed (two can be real at once), and summed into one row. + +The consumer walk carries raised caps of its own (depth 24, 2,000,000 entries) because the +walker's defaults were sized for install trees: rustup exhausted the default entry cap at 9.4 of +its 14.62 GB, and the pnpm store, LM Studio and `~/.claude` all bottomed out at depth 16. A +ranking whose deepest trees are systematically floors does not merely under-report them, it +**mis-orders** the answer. The caps are raised, not removed — a root that exhausts even these +still reports `≥`. + +### 9. Projects are counted twice, on purpose, and never given a fabricated path + +The Projects section publishes **`everSeen`** (every distinct project any host ever recorded a +session in, deletions included — the deletions are the point) and **`onDisk`** (the subset that +still resolves to a directory, the only projects a byte or line measurement can be taken of). +Here: 50 ever seen, 25 on disk, 21 of those git repositories. Only on-disk projects become table +rows; the vanished ones survive in `everSeen` rather than as unmeasurable rows. + +Discovery is this domain's own (`project-sources.mjs`), not `discoverRuvfloProjects()`. That +function answers "which projects carry ruflo learning state" by requiring `.claude-flow/neural/` +and reading only the 150 newest transcripts per host; both narrowings are right there and wrong +here — on this machine they collapse ~50 projects to 5. Sightings are de-duplicated by +**resolved real path**, so one project touched by Claude, Codex and OpenCode is one project and a +symlinked route is not a second one; a deleted path that cannot be resolved falls back to +`path.resolve` so it is still counted. + +**Claude's transcript-directory encoding is lossy and is not safely decodable.** +`~/.claude/projects//` maps `/`, `.` and a literal `-` all to `-`, so +`-Users-me-ai-agentic-kit` reads equally as `/Users/me/ai/agentic/kit` and +`/Users/me/ai/agentic-kit`. Measured on this corpus, naive `-`→`/` substitution resolves **8 of +52** directories — wrong for 85% of them. The path therefore comes from the transcript's declared +`cwd` (2,538 of 2,585 Claude transcripts here, all 175 Codex rollouts, and OpenCode's per-session +`directory` column). The encoded name is consulted only as a fallback for a directory where no +transcript declares a `cwd`, and even then only through a decoder that walks candidate segments +against the real filesystem and returns a path only when the filesystem confirms it — which can +name just 18 of the 52 on its own. When neither route resolves, the group is reported as +**`unresolved`**, contributes no row, and makes `everSeen` an explicit lower bound +(`complete: false`). It is never given a guessed path. Every row carries `origins` naming which +route produced it, so a fallback-derived path can never be read as a declared one. + +## Consequences + +### Positive + +- The "what is this costing my machine" question family gets a real answer, from data the kit + already has trust-boundary access to — no new privileges, no egress. +- Sprawl becomes visible and actionable: duplicate native builds, giant sessions, stale caches, + and per-project bloat all surface with their locations. +- The consumer ranking answers the question a user actually asks — "what is eating this disk" — + rather than the narrower one the kit could answer without leaving its own trees. On the + authoring machine that moved nine roots totalling >320 GB from invisible to ranked, and + demoted the previously-reported number one to eleventh (§8). +- The four-family dashboard (health / spend / activity / footprint) completes a coherent mental + model, each family in its own context with clean boundaries. + +### Negative + +- A fourth primary area is a permanent navigation-surface expansion and an explicit amendment of + ADR-0005's three-area layout. +- Deep scans on large corpora (multi-GB transcript trees, many projects) take real time and I/O; + the tiered model contains but does not eliminate that cost, and widening the scan surface to + ~50 third-party cache roots (§8) widened that cost with it — tens of seconds on this machine. +- A curated registry of third-party cache locations is a maintenance surface: a tool that moves + its cache goes unmeasured until the registry learns the new path. It fails visibly (the row + reads absent with the path it looked at) rather than silently, which is the trade taken. +- A persisted snapshot is a new on-disk artifact with its own staleness to manage honestly. + +### Risks and mitigations + +| Risk | Mitigation | +|------|------------| +| Deep scan runs long on huge corpora and reads as a hang | Single-flight with progress state surfaced in the panel; bounded walkers with entry caps; persisted snapshot means the panel is never blank while a scan runs | +| Stale snapshot presented as current | Every deep-tier figure carries `asOf`; the freshness label and rescan control are part of the Summary view's contract, not an afterthought | +| LOC figures treated as precise | Labeled approximate, extension-bucketed; the DDD doc forbids presenting them as authoritative | +| Walker follows a symlink cycle or escapes a root | Symlinks never followed; depth and entry caps; one bad subtree degrades to unknown for that node only | +| Scope creep into cleanup/mutation | v1 invariant: this context mutates nothing; reclaimables are advisory rows with rationale and a safety tier, and the tiers are never summed into one actionable number | +| Boundary erosion into Usage/Observability | DDD invariants forbid tokens/cost and session evidence; the read surface is enumerated (§7) and admits paths, names and counts only, never message content; cross-links replace duplication | +| A ranking that reads as an answer and is not one | Breadth is a curated registry, not an accident of what other sections walked; containment/residual accounting so nested roots cannot double-count; raised walk caps because a floored deep tree mis-*orders* the list; excluded categories stated in the payload | +| A size figure measured at the wrong root | Both known cases (brain KB vs cache root, Playwright's macOS location) are recorded in §8 with what they cost; new roots carry a component breakdown with a remainder row so a corrected figure arrives with its explanation | + +## Resolved decisions + +The draft left four points open. All four are decided; this section is the record. + +1. **Naming — the tab and the CLI are both "System" (`ak system`).** One word for one area is + what a user can guess and what documentation can state without a translation table; a name the + UI and the terminal disagree about is a support burden with no upside. **Footprint** survives + only as the *domain* name inside the DDD documents ([Machine footprint](../ddd/machine-footprint.md)) + and the module directory (`src/lib/footprint/`), where it names a bounded context rather than a + user-facing surface. The command is `src/commands/system.mjs`; no user-facing string says + "footprint". +2. **Projects stays its own sub-view — five sub-views, not four.** LOC, git-remote identity, and + `node_modules` bloat answer "what have I built here", not "where are my bytes"; merging them + into Storage would put two different questions behind one heading and force the Storage tree to + grow a column vocabulary it does not otherwise need. +3. **Deep scans re-run on an explicit click only, with a visible staleness nudge past ~7 days.** + Auto-scanning on open would make simply *looking* at the tab cost tens of seconds of I/O on a + large corpus — the surprise cost is worse than a stale figure that says how stale it is. The + snapshot's `asOf` is always rendered, and beyond `SNAPSHOT_STALE_AFTER_MS` (7 days) the + freshness label turns amber and reads "stale, rescan". Opening the System tab issues a plain + `GET /api/system`; only the Rescan control adds `?refresh=deep`. +4. **Windows ships a guaranteed census plus a best-effort true `cwd`, degrading honestly, with no + dependency added.** The draft's "unsupported on win32" answer would have blanked the whole + Runtime view on a supported platform. Instead `src/lib/live/win-process-survey.ps1` — a plain text + script invoked the same way the POSIX path already invokes `ps` and `lsof`, no npm package and + no compiled artifact — provides two layers: + - a **guaranteed** census (host, pid, ppid, start time, CPU, working set) from + `Get-CimInstance Win32_Process`, plus current-user-only command lines proven via `GetOwner`; + - a **best-effort** true per-process working directory via inline `Add-Type` P/Invoke + (`NtQueryInformationProcess` → PEB → `RTL_USER_PROCESS_PARAMETERS` → `CurrentDirectory`). + + If the P/Invoke path fails for any reason — antivirus block, execution policy, insufficient + rights, WOW64 bitness mismatch — every other field still returns and the Project column + degrades to an explicit "not attributable on Windows" carrying the failure reason. An empty + census is treated as a broken survey, not an idle machine. This is genuinely verified rather + than asserted: `windows-latest` is already in the CI matrix (`.github/workflows/ci.yml`), so + the Windows path runs on every push alongside Linux and macOS. + +## Follow-ups on acceptance + +All complete: + +- The context is on the [context map](../ddd/context-map.md) (context description, ASCII map, and + relationships table) and its terms are merged into + [ubiquitous language](../ddd/ubiquitous-language.md). +- This ADR is in the [ADR index](README.md) and the theme narrative. + +## References + +- [Machine footprint domain](../ddd/machine-footprint.md) — the domain model and invariants +- [Design mock-up](../assets/system-tab-mock.html) — a self-contained, both-theme HTML mock of + the System area with illustrative data: per-metric chart forms (odometer KPIs, radial disk + gauge, donut, stacked bars, small-multiple growth areas, radar, presence matrix, composed + project bars), each card annotated with its form choice and rationale +- [Dashboard guide](../DASHBOARD.md) +- [Managed tools](../MANAGED-TOOLS.md) +- `src/lib/footprint/` — the collectors: `walk.mjs` (the bounded walker and the `Measurement` + vocabulary), `install.mjs`, `storage.mjs`, `runtime.mjs`, `catalog.mjs`, `projects.mjs`, + `consumers.mjs` (the ranked largest-consumers view and its containment/residual accounting), + `project-sources.mjs` (cross-host project discovery: ever-seen vs on-disk), + `stack-registry.mjs` + `stack-detect.mjs` (languages carry lines; frameworks/SDKs/tools are + presence-only; the unrecognized tail), `snapshot.mjs` (the persisted deep-tier snapshot), + `index.mjs` (the two-tier collector) +- `src/commands/system.mjs` (the CLI twin) +- `src/lib/live/win-process-survey.ps1` (the Windows stand-in for `ps` + `lsof`) +- `src/lib/live/process-sessions.mjs` (the runtime survey this reuses) +- `src/lib/dashboard/project-discovery.mjs` (the project catalog this reuses) diff --git a/docs/adr/0026-about-component-directory.md b/docs/adr/0026-about-component-directory.md new file mode 100644 index 0000000..6a97bd6 --- /dev/null +++ b/docs/adr/0026-about-component-directory.md @@ -0,0 +1,189 @@ +# ADR-0026 — About: a component directory that explains everything ak installs + +- **Status:** Implemented +- **Date:** 2026-08-06 +- **Updated:** 2026-08-06 — accepted and implemented; the open points below are resolved decisions +- **Deciders:** agentic-kit maintainers +- **Related:** [ADR-0005](0005-dashboard-in-page-routing-reveal.md), + [ADR-0007](0007-maintainer-admin-local-telemetry.md), + [ADR-0016](0016-capability-driven-integration-adapters.md), + [ADR-0023](0023-fail-closed-operations-and-explicit-degradation.md), + [ADR-0025](0025-machine-footprint-metrics.md) + +## Context + +agentic-kit installs and configures a lot on a user's behalf: three frontier-agent CLIs, an +orchestration engine, a memory layer, a quality fleet, security scanners, an offline knowledge +base, MCP registrations, guidance blocks, statuslines, a routing policy, background daemons. +Every existing dashboard area assumes the user already knows what these things *are*: + +- **Overview** grades their health, **Usage** their spend, **Observability** their activity, and + **System** ([ADR-0025](0025-machine-footprint-metrics.md)) their machine cost — all operational + views of components the user is presumed to recognize. +- The only place that *introduces* the components is prose documentation + ([MANAGED-TOOLS.md](../MANAGED-TOOLS.md), [SETUP.md](../SETUP.md)) — maintainer-register + reference material, not a new user's first five minutes. + +A new user's first honest question is prior to all of that: **"what did this thing just put on +my machine, and why should I be glad it's there?"** Nothing answers it today. The result is a +trust gap exactly where trust matters most — at first contact, right after `ak setup` printed a +long list of installs. + +This record was written as a proposal and is retained in that voice; the decision has since been +accepted and shipped. Where the draft left a choice open, the [resolved +decisions](#resolved-decisions) section states what was decided and why. + +## Decision + +### 1. A new leftmost primary area: About + +The dashboard gains a fifth primary area — **About** — placed **left of Overview**, first in +the tab order. This amends [ADR-0005](0005-dashboard-in-page-routing-reveal.md)'s layout a +second time (after ADR-0025's fourth area). Leftmost placement is deliberate: About is the +reading-order entry point for someone who doesn't yet know what the other tabs are about. +**Overview remains the default landing view** — About is discoverable first position, not a +gate returning users must click past. + +```text +[ About | Overview | Usage | Observability | System ] + +About (no secondary rail — one scrolling page with category anchors): + Hero One friendly sentence of orientation + component count + a five-word map of how + the pieces relate (hosts ⟷ engine ⟷ memory/quality/security, kit around all). + Hosts The agent CLIs the user already recognizes — Claude Code, Codex, OpenCode. + Engine & ruflo (orchestration, cross-session memory, learning), agentdb (the memory + memory layer, version-pinned to ruflo). + Quality agentic-qe — test generation, coverage, quality gates. + Safety @claude-flow/aidefence + @claude-flow/security — prompt-injection defense and + scanning. + Knowledge RuvNet Brain — the offline KB that grounds answers about this stack. + The kit agentic-kit itself — the caretaker that installs, heals, and explains the rest. + Configured for you Non-package surfaces: MCP registrations, guidance blocks, + statuslines, dual-host routing & bridge, background daemon, permission + allowlists — each with "configured by setup/sync · yours to change". +``` + +Quality, Safety, and Knowledge are three separate categories in the data and one section on the +page — three one-card categories read as one cluster rather than three sparse sections. The page +therefore carries five anchors (`hosts`, `engine`, `quality`, `kit`, `configured`) over seven +categories. + +### 2. A new bounded context: Component directory + +Curated editorial content joined with live detection facts — see +[Component directory](../ddd/component-directory.md). The split is the decision: + +- **Editorial** (this context owns): per-component tagline, one friendly paragraph of value + proposition, outbound links (GitHub / npm / docs), icon spec, category, and curated order. + Checked into the repo as a directory module, versioned with the release — never generated at + runtime, never fetched. +- **Detection** (borrowed, read-only): installed/not-installed, version, install method — + the facts `/api/status` and the managed-tools machinery already produce. About performs + **no probing of its own and adds no endpoint**; the page renders the directory module joined + client-side with the existing status payload. + +### 3. Completeness is a gate, not a hope + +Every tool in the managed-tools registry must have a directory entry — enforced by a parity +test, so a newly managed tool cannot ship without its About card. The reverse also holds: a +directory entry for something ak does not install or configure is a lie and fails the same +gate. + +### 4. Card anatomy and the new-user register + +Each card: an icon tile; the component name with an honest state chip (`installed v3.34.0` · +`not installed — ak setup adds it` · `configured`); a bold plain-language tagline; **one** +paragraph (~50 words) of value proposition written to a reader who has never heard of the +tool; and a row of link pills (GitHub / npm / Docs). Editorial register is a contract, not a +style hope: friendly, concrete, jargon-free — every term of art either avoided or explained in +the sentence that uses it ("agent swarms" → "teams of specialist agents"). No marketing +superlatives; the value prop states what the thing *does for the user*. + +### 5. Iconography: official marks where they exist, honest monograms elsewhere + +The three hosts reuse the exact official SVG marks the dashboard already ships for +Observability's session rows. Components without an official mark get a **monogram tile** +(letterform on a category-hued rounded tile) — deliberately not a fabricated logo. All icons +are inline SVG/CSS (the dashboard is self-contained; no remote images), and the same icon for +a component everywhere it appears. + +### 6. Links are outbound and user-initiated; the kit stays offline + +Every link is a plain `https` anchor the user clicks in their own browser — GitHub repo, npm +package page, public docs. The kit fetches nothing ([ADR-0007](0007-maintainer-admin-local-telemetry.md)'s +offline side), link URLs live in the versioned directory module, and the nightly external link +check covers them like every other doc link. + +## Consequences + +### Positive + +- The first-five-minutes trust gap closes: every installed thing explains itself, in one + place, in plain language, with the receipts (source, package, docs) one click away. +- The parity gate turns "docs drift from reality" into a test failure instead of a review hope. +- Zero new collection surface: no endpoint, no probe, no cache — editorial content plus a join + with facts the dashboard already has. + +### Negative + +- A fifth primary tab; the tab bar is reaching its comfortable ceiling, and any sixth area + should trigger a navigation rethink rather than another amendment. +- Curated copy is a maintenance duty: component descriptions, links, and value props must be + kept truthful as upstreams evolve (the parity gate catches presence, not prose accuracy). +- Editorial tone is subjective; the register contract reduces but cannot eliminate review churn. + +### Risks and mitigations + +| Risk | Mitigation | +|------|------------| +| Copy claims something is installed when it isn't | Editorial/detection split is structural: state chips render only detection facts; copy is forbidden (by DDD invariant and review checklist) from asserting runtime state | +| A new managed tool ships without an About card | Registry↔directory parity test fails the build | +| Links rot | Nightly external link check already covers `docs/**`; the directory module's URLs join that sweep | +| Brand misrepresentation via invented logos | Official marks only where already shipped as official; everything else is an explicit monogram tile | +| The page drifts into a second status/metrics view | DDD invariant: no numbers beyond version strings — health belongs to Overview, cost to Usage, footprint to System | + +## Resolved decisions + +The draft left three points open. All three are decided; this section is the record. + +1. **First run shows a dismissible nudge on Overview; it never hijacks the landing view.** + Overview stays the default on every open, including the first. A one-time "new here? start + with About" nudge sits on Overview and is dismissed for good (`ak-dash-about-nudge` in + browser-local storage). Redirecting the first open would make the landing view depend on + invisible browser state — two users, or one user on two machines, would see different first + screens with no way to tell why — and it would put a page between an operator and the health + verdict they opened the dashboard to read. +2. **`ak about` ships in v1.** The directory is data with no I/O of its own, and the state chips + join the same `ak status` collection the dashboard uses, so the terminal twin costs one + renderer (`src/commands/about.mjs`) and keeps the one-collector-two-surfaces symmetry the + usage scorecard and `ak system` already follow. It also answers the question in the place the + question is usually asked — right after `ak setup` finishes printing. +3. **Six separate "Configured for you" cards, not one card with a list.** Each configured + surface — MCP registrations, guidance blocks, statuslines, dual-host routing and bridge, the + background daemon, permission allowlists — has its own state chip and its own managing + command. Collapsing them into one card would force a single chip to summarize six independent + health facts, which is exactly the kind of averaging the honest-degradation contract forbids, + and would hide the `manage:` line that makes "yours to change" actionable. + +## Follow-ups on acceptance + +All complete: + +- The context is on the [context map](../ddd/context-map.md) and its terms are merged into + [ubiquitous language](../ddd/ubiquitous-language.md); this ADR is in the [index](README.md) + and the theme narrative. +- The registry↔directory parity test ships in `tests/kit/about-directory.test.mjs`, checked in + both directions against the managed-tools registry, the heal/detection paths, and + [MANAGED-TOOLS.md](../MANAGED-TOOLS.md). + +## References + +- [Component directory domain](../ddd/component-directory.md) — the domain model and invariants +- [Design mock-up](../assets/about-tab-mock.html) — self-contained both-theme HTML mock with + the full card grid, category sections, per-section design rationale, and an annotated card + anatomy +- [Managed tools](../MANAGED-TOOLS.md) — the registry this directory must stay in parity with +- [Dashboard guide](../DASHBOARD.md) +- `src/lib/dashboard/about-directory.mjs` — the directory module (pure data plus accessors) +- `src/commands/about.mjs` — the CLI twin +- `tests/kit/about-directory.test.mjs` — the parity gate and the register-contract checks diff --git a/docs/adr/0027-shared-project-census.md b/docs/adr/0027-shared-project-census.md new file mode 100644 index 0000000..a62d775 --- /dev/null +++ b/docs/adr/0027-shared-project-census.md @@ -0,0 +1,105 @@ +# ADR-0027 — One project census, four scopes, every count explains itself + +- **Status:** Implemented +- **Date:** 2026-08-07 +- **Deciders:** agentic-kit maintainers +- **Related:** [ADR-0012](0012-observability.md), + [ADR-0023](0023-fail-closed-operations-and-explicit-degradation.md), + [ADR-0024](0024-project-intelligence-telemetry.md), + [ADR-0025](0025-machine-footprint-metrics.md) +- **Supersedes:** the machine-wide discovery amendment in ADR-0024 (`discoverRuvfloProjects`) + +## Context + +Four dashboard areas answered "what projects does this machine have" four different ways, from +four different sources, with four different naming rules — and reported four different numbers for +the same machine: + +| Area | Source of truth | Naming | Reported | +|---|---|---|---| +| Overview → Intelligence | directories with `.claude-flow/neural/`, from the 150 most recent transcripts | basename | **4** | +| Observability → History | git-root identity over a 14-day transcript window | identity hash | **14** | +| Usage → Scorecard | `projectLabel()` basename heuristic, over its own day window | cwd basename | **14** (a *different* 14) | +| System → Projects | every `cwd` any host ever recorded | resolved real path | **~48** | + +No single number was wrong. What was wrong is that nothing told the user which question each was +answering, so four honest answers read as one broken feature. Worse, the Intelligence number was +answering a question nobody asked: `.claude-flow/neural/` means "ruflo has *trained* here", which +silently excluded every project whose memory came from agentic-qe or swarm storage, and every +project driven by Codex or OpenCode rather than Claude. On the deciding machine it reported 4 +projects where 17 had memory or intelligence active. + +## Decision + +**One census. Four named scopes. Every count is rendered with the scope that produced it.** + +`discoverProjectSources()` (`src/lib/footprint/project-sources.mjs`) becomes the single census, +reused verbatim rather than reimplemented. It is already the widest and most carefully bounded of +the four sources: it reads exactly one field — the session `cwd` — out of the head of every Claude +and Codex transcript plus the OpenCode session store, dedupes by resolved real path, and reports +three deliberately distinct figures instead of one lossy total. + +`src/lib/project-census.mjs` wraps it with the scope vocabulary: + +| Scope | Means | Level | +|---|---|---| +| `everSeen` | every project any host ever recorded a session in, deletions included | directory | +| `onDisk` | the subset that still resolves — the only projects that can be measured | directory | +| `gitRepos` | the on-disk subset under version control | directory | +| `learning` | on-disk projects with `.claude-flow`, `.agentic-qe` or `.swarm` state, whichever host created it | **project** | + +Three consequences of that table are the actual decision: + +**1. The learning scope is about activation, not training.** Its markers are exactly the ones +`defaultStorageRoots()` already treats as a project's learning stores, so the Intelligence panel and +the Storage panel cannot disagree about what a learning store *is*. Host-independence is the point: +memory activated by Codex counts. + +**2. Directory scopes and project scopes are different levels, deliberately.** The System area +measures **directories** because that is what has bytes and lines in it ([ADR-0025](0025-machine-footprint-metrics.md)); +folding them would destroy the figures it exists to report. The Intelligence panel aggregates +**projects**, because a project is what a user picks. So the learning scope folds a repository's +sub-directories and its ephemeral `.claude/worktrees/agent-*` checkouts onto the repository root, +keyed by `resolveProjectIdentity()` — the same git-root identity Observability already uses. + +This is not cosmetic. The project picker keys on that identity. Listing 24 directories against 17 +identities made 7 rows unreachable and let the wrong project be selected. + +**3. Counts still differ, and that is correct.** A lifetime census, a 14-day session window and a +learning-state subset are three questions. Shared identity removes the *spurious* differences — +`backend` and `emailibrium` are no longer two projects — and leaves the real ones. For the real ones +the answer is not to force agreement but to explain: `describeScope()` returns one sentence naming +what a count counted and how it narrowed, and **no surface may render a project count without one.** +Where a count is windowed, the window is named, because that is then the only remaining reason two +counts legitimately differ. + +## Consequences + +- The Intelligence panel reports every project with memory or intelligence state, whichever host + created it — 17 rather than 4 on the deciding machine. +- The redundant "N projects tracked on this machine" and "N projects available" captions are + **deleted**, not reworded: with a scope explainer and a KPI card, they restated a number twice and + explained it zero times. +- `discoverRuvfloProjects()` and `project-discovery.mjs` are retired, and `registryWorkspaces()` + reverts to module-private. ruflo's machine-level registries do not exist on every machine — which + is exactly why they made a poor discovery source and a fine daemon-accounting one. +- A census walk is not free (~650 ms over ~3,200 transcripts on the deciding machine). It stays + behind the existing 60-second snapshot cache, walked once per snapshot and exposed two ways. +- `learningState` rides onto each project row so a project reporting zero patterns can say *why* — + a project with `.agentic-qe` but no ruflo counters is genuinely active and genuinely has no + patterns, and without that field it is indistinguishable from a failed read ([ADR-0023](0023-fail-closed-operations-and-explicit-degradation.md)). +- The census reports `complete: false` when a transcript could not be read, and every figure derived + from it is then rendered as a lower bound rather than a fact. +- Adding a fifth scope means adding a row to `SCOPE_NOTE`; a scope with no explanation returns the + empty string rather than rendering an unexplained number. + +## References + +- `src/lib/project-census.mjs` — `projectCensus`, `projectsInScope`, `describeScope`, + `hasLearningState`, `LEARNING_MARKERS` +- `src/lib/footprint/project-sources.mjs` — `discoverProjectSources` (the census itself) +- `src/lib/live/project-label.mjs` — `resolveProjectIdentity` (the shared identity) +- `src/lib/dashboard-server.mjs` — `censusBackedDiscovery`, `buildProjectSnapshotCache` +- `tests/kit/project-census.test.mjs` +- [docs/ddd/project-intelligence.md](../ddd/project-intelligence.md), + [docs/ddd/context-map.md](../ddd/context-map.md) diff --git a/docs/adr/README.md b/docs/adr/README.md index 86012e0..4e39037 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -33,6 +33,9 @@ Consequences**, and cites the grounded source it rests on where relevant. | [0022](0022-metaharness-as-optional-assurance-companion.md) | MetaHarness as an optional assurance companion | Proposed | | [0023](0023-fail-closed-operations-and-explicit-degradation.md) | Fail-closed mutations and explicit degraded operation evidence | Implemented | | [0024](0024-project-intelligence-telemetry.md) | Project intelligence: live learning telemetry from ruflo/agentic-qe's own state | Implemented | +| [0025](0025-machine-footprint-metrics.md) | Machine footprint: infrastructure metrics for install, runtime, storage, and catalog | Implemented | +| [0026](0026-about-component-directory.md) | About: a component directory that explains everything ak installs | Implemented | +| [0027](0027-shared-project-census.md) | One project census, four scopes, every count explains itself | Implemented | Theme: ADRs **0001–0006** define **dual-host LLM routing and leadership** — how `ak` lets ruflo route each development activity (architecture, implementation, testing, review, …) to the right host (Claude @@ -135,3 +138,52 @@ counter, and reasoning-graph size samples that ruflo/agentic-qe already write un intelligence as its own bounded context rather than an Observability extension — this telemetry carries no session, actor, host, provider, or lifecycle identity and needs no per-field evidence confidence, and the panel it feeds lives under Overview, not Observability's Live/History scope. + +**0025** answers the fourth question family the dashboard had no owner for — not health, spend, or +activity, but **what this toolchain costs the machine itself**: install bytes per managed tool with +their install methods and duplicate native builds, live CPU/RSS per host process and daemon, +retained data broken down category → host → project → session with growth and advisory reclaimable +candidates, the deduplicated cross-host inventory of skills/agents/commands/plugins/MCP servers, +and per-project approximate LOC alongside working-tree, `.git`, and `node_modules` bytes. It +establishes Machine footprint as its own bounded context — metadata-only by construction, so +transcript content cannot enter it — and adds a fourth primary area, **System**, deliberately +amending 0005's three-area layout. Collection is tiered because the costs differ by orders of +magnitude: a cheap TTL-cached tier on every read, and an explicit single-flight deep scan persisted +to a snapshot so the panel paints instantly and always states how old its figures are. 0023's +honesty contract is load-bearing throughout — a never-scanned section reads "not measured yet", +never `0`; a partial total renders as a lower bound; an unreadable subtree degrades that node +alone. The tab and the CLI share one name (`ak system`); "footprint" survives only as the domain +name. Windows gets a guaranteed `Get-CimInstance` process census plus a best-effort P/Invoke +working-directory probe that degrades to an explicit "not attributable" rather than blanking the +view — no dependency added, and `windows-latest` is already in the CI matrix. + +**0026** closes the trust gap the other four areas assume away: every existing view is an +*operational* view of components the user is presumed to already recognize, and nothing answered +"what did this thing just put on my machine, and why should I be glad it's there?" It adds +**About** as a fifth primary area, leftmost in reading order but never the landing view — Overview +stays the default and a dismissible first-run nudge points newcomers left. Its bounded context, +Component directory, owns the one kind of fact no neighboring context can supply: editorial +identity — a tagline, one ~50-word plain-language paragraph, outbound source/package/docs links, an +icon spec, and a curated order — authored and versioned with the release. The editorial/detection +split is the decision: prose never claims runtime state, chips render only detection facts already +produced by `/api/status`, and About adds no endpoint, no probe, and no cache. Completeness is a +test rather than a review hope — every managed tool must have exactly one entry, and no entry may +exist for something ak neither installs nor configures. Official marks appear only where the +dashboard already ships them as official; everything else gets an explicit monogram tile rather +than a fabricated logo. `ak about` prints the same directory in a terminal. + +**0027** ends a four-way disagreement about what a project *is*. Overview, Usage, Observability and +System each discovered projects their own way, from their own source, with their own naming rule, +and reported four different numbers for the same machine — 4, 14, another 14, and ~48. None was +wrong; nothing said which question each answered, so four honest answers read as one broken +feature. It makes `discoverProjectSources()` the single census and names four **scopes** over it +(`everSeen`, `onDisk`, `gitRepos`, `learning`), with the rule that no surface may render a project +count without the sentence explaining what that count counted. Two decisions carry the weight. The +*learning* scope asks whether memory or intelligence has been **activated** — `.claude-flow`, +`.agentic-qe` or `.swarm`, whichever host created it — rather than whether ruflo has *trained*, +which is why Intelligence went from 4 projects to 17 on the deciding machine. And directory scopes +stay separate from project scopes: System measures directories because that is what has bytes in +it, while Intelligence folds a repo's sub-directories and throwaway agent worktrees onto one +identity because that is what a user picks — a distinction that was also a live bug, since keying +the picker off identity while listing directories made 7 of 24 rows unreachable. Counts that remain +different stay different, and say why. diff --git a/docs/assets/about-tab-mock.html b/docs/assets/about-tab-mock.html new file mode 100644 index 0000000..b1007ee --- /dev/null +++ b/docs/assets/about-tab-mock.html @@ -0,0 +1,469 @@ + + + + + +ak · About — design mock (ADR-0026) + + + + +
+ Design mock · illustrative copy, versions, and links — final content lives in the versioned directory module + accompanies ADR-0026 + About primary area +
+ +
+ +

agentic-kit

v4.0.0-alpha.40 · local diagnostic panel
+
+ + + + + +
+ +
+ About +

Meet your toolkit

+

agentic-kit set up 9 components and 6 configurations on this machine. + This page says what each one is, in plain words — and where to read more. Health lives in + Overview, spend in Usage, activity in Observability; here, everything just introduces itself.

+
+ How it fits +
+
Coding agentsClaude Code · Codex · OpenCode
+ ⇆ +
Engine + memoryruflo · agentdb
+ ⇆ +
Quality · Safety · Knowledgeagentic-qe · aidefence · Brain
+ ⤺ +
agentic-kitinstalls & heals it all
+
+
+
+ + +
+ About +

Hosts — the agents you talk to

+

The coding agents themselves. Everything further down exists to make these + smarter, safer, and easier to watch.

+
+
+
+ + Claude Codeinstalled · v2.1.223 +
+ Anthropic's coding agent, in your terminal. +

Point it at a repository and talk — it reads your code, writes changes, runs + the tests, and explains what it did as it goes. Most of what agentic-kit sets up exists to + make this tool and its peers below work better together.

+
GitHub ↗ + npm ↗ + Docs ↗
+
+
+
+ + Codexinstalled · v0.29.0 +
+ OpenAI's coding agent — a second pair of eyes. +

The same idea from OpenAI: an agent that codes with you in the terminal. With + both installed, agentic-kit runs them as peers — each can lead, review the other's work, or + pick up when one gets stuck.

+
GitHub ↗ + npm ↗ + Docs ↗
+
+
+
+ + OpenCodenot installed — ak setup adds it +
+ An open-source coding agent, supervised. +

A community-built agent CLI that agentic-kit can manage as an opt-in worker: + explicitly routed for specific jobs, observable like the others, and never handed the lead + by accident.

+
GitHub ↗ + npm ↗ + Docs ↗
+
+
+
Design note: hosts lead because they're what a new user already + recognizes — familiar things first, infrastructure after. Official marks are reused + byte-identically from Observability's session rows. The OpenCode card deliberately shows the + honest not-installed state: absence is information, not an empty slot.
+
+ + +
+ About +

Engine & memory — what makes sessions smarter

+
+
+
+ rf + rufloinstalled · v3.34.0 +
+ The orchestration engine underneath. +

Gives your agents what a single session can't: memory that survives restarts, + teams of specialists working in parallel, hooks that learn your patterns, and security + scanning along the way. agentic-kit keeps it current and healthy.

+
GitHub ↗ + npm ↗
+
+
+
+ db + agentdbinstalled · 3.0.0-alpha.20 · pinned to ruflo +
+ Where agent memory actually lives. +

A fast, local database for what your agents learn and remember — no server, no + cloud. It's version-matched to ruflo on purpose, so the engine and its memory can never + drift apart.

+
GitHub ↗ + npm ↗
+
+
+
Design note: monogram tiles (letters on a category hue) are the honest + icon for tools without an official mark — consistent, self-contained under the CSP, and never a + fabricated logo. The pinned-version chip explains itself inline rather than hiding a deliberate + choice behind "outdated".
+
+ + +
+ About +

Quality · Safety · Knowledge — evidence, defense, grounding

+

Three specialists with one theme: making agent work trustworthy — tested, + protected from smuggled instructions, and answered from real source.

+
+
+
+ qe + agentic-qeinstalled · v3.13.9 +
+ A quality-engineering fleet on call. +

Specialist agents for the unglamorous parts: generating tests, finding + coverage gaps, and gating changes on real quality signals — so "the agent says it works" + turns into evidence you can check.

+
GitHub ↗ + npm ↗
+
+
+
+ + aidefence + securityinstalled · bundled with ruflo +
+ Checks text before your agents trust it. +

A pasted issue, a web page, a stranger's README — any of them can carry + instructions smuggled into content ("prompt injection"). These scanners check untrusted + text before it reaches your agents. ak reinstalls them if an upgrade ever drops them.

+
GitHub ↗ + npm ↗
+
+
+
+ B + RuvNet Braininstalled · v0.9.4 +
+ Answers about these tools, from their real source. +

The tools on this page evolve faster than any AI model's training data. The + Brain is a local, offline knowledge base built from their actual source code — so when you + ask your agent about this stack, it answers from evidence, not stale memory.

+
GitHub ↗
+
+
+
Design note: three one-tool categories share a row instead of three + sparse sections — the hero map already groups them as one cluster. The shield is the page's one + hand-drawn glyph ("safety" is where a monogram would undersell the job), and the aidefence + paragraph defines "prompt injection" in the same breath it uses it: the register contract in + action.
+
+ + +
+ About +

The kit — who takes care of all this

+
+
+
+ + agentic-kitinstalled · v4.0.0-alpha.40 +
+ The caretaker for everything above. +

One command installs the set; one command heals it after upgrades; this + dashboard shows what it's all doing. If a tool on this page breaks, drifts, or goes + missing, ak notices and says so — that is its whole job.

+
GitHub ↗ + npm ↗
+
+
+
+ + +
+ About +

Configured for you

+

Not packages — settings and wiring ak set up on your behalf. Each names the + command that manages it: yours to change, never a black box.

+
+
+
⎈ + MCP registrationsconfigured
+

Agents reach tools through MCP — the plug-in protocol for AI tooling. ak + registers this stack's tools once, at user scope, so every project gets them without + per-repo setup.

+
manage: ak x mcp pick
+
+
+
+ Guidance blocksconfigured
+

Short managed sections in CLAUDE.md and AGENTS.md that teach each agent the + house rules for this stack. ak keeps them current — and never touches text you wrote + yourself.

+
manage: ak sync
+
+
+
⏯ + Statuslinesconfigured
+

A live footer inside Claude Code and Codex showing the model in use, session + usage, and remaining limits at a glance — so you never wonder what a session is costing.

+
manage: ak setup
+
+
+
⇆ + Dual-host routing & bridgeconfigured
+

With both Claude and Codex enabled, each kind of work is routed to the host + that's best at it — and each host can hand a subtask to the other mid-flight.

+
manage: ak host pick
+
+
+
⚙ + Background daemonconfigured
+

Local-only workers that keep learning and maintenance ticking between + sessions. Free by default — anything that would spend AI tokens stays strictly opt-in.

+
manage: ruflo daemon
+
+
+
✓ + Permission allowlistconfigured
+

Pre-approved command patterns for the stack's own tools, so they don't + interrupt you with permission prompts. Every rule is disclosed at setup, none added + silently.

+
manage: ak setup
+
+
+
Design note: configured surfaces get the same card shape as packages — + one grammar to learn — but a distinct configured chip, no version, and a + manage: line instead of source links, because "where do I change + this" is their equivalent of "where do I read more".
+
+ + +
+ Design +

Anatomy of a card

+
+
+ 1 + 2 + 3 + 4 + 5 +
+
+ rf + rufloinstalled · v3.34.0 +
+ The orchestration engine underneath. +

Gives your agents what a single session can't: memory that survives + restarts, teams of specialists working in parallel…

+
GitHub ↗ + npm ↗
+
+
+
+
1Icon tile. Official mark where one + genuinely exists (the three hosts); otherwise a monogram on the category hue. Never an + invented logo.
+
2Name + state chip. The only place + runtime facts appear — fed by detection the dashboard already does, never by the prose. + Absent reads "not installed — ak setup adds it"; a failed status fetch reads "state + unknown" while the card still renders.
+
3Tagline. Ten plain words or fewer — + the card's thesis, scannable on its own.
+
4One paragraph, ~50 words. What it does + for you, active voice, every term of art explained where it's used. Depth lives + behind the Docs link, not here.
+
5Link pills. GitHub / npm / Docs — the + receipts. Plain outbound anchors the user clicks; the kit itself never fetches them.
+
+
+
+ +
ak · About — design mock for ADR-0026 · illustrative content · both themes via the dashboard's own tokens
+
+ + diff --git a/docs/assets/system-tab-mock.html b/docs/assets/system-tab-mock.html new file mode 100644 index 0000000..0aad1a3 --- /dev/null +++ b/docs/assets/system-tab-mock.html @@ -0,0 +1,643 @@ + + + + + +ak · System — design mock (ADR-0025) + + + + +
+ Design mock · illustrative data throughout — nothing here is measured + accompanies ADR-0025 + System primary area +
+ +
+ +

agentic-kit

v4.0.0-alpha.40 · local diagnostic panel
+
+ + + +
+
+ + + + + +
+
+ deep scan · 2h ago + +
+
+ +
+ + +
+
+ System +

Summary

+

The four families in one glance: install size, retained data, live resource use, and deployed + inventory — every deep-tier figure stamped with when it was measured.

+
+ +
+
+
Install footprint +
GB
+
7 managed tools · 318 native addons
+
Data retained +
GB
+
transcripts, ledgers, stores, caches
+
Live processes +
+
1.9 GB RSS · 11% CPU combined
+
Projects +
+
412k LOC across catalog
+
Catalog +
+
skills · 23 agents · 41 commands
+
+ +
+

Disk denominator

radial gauge
+ + + + + + 12.7 GB + toolchain · of 994 GB disk + 213 GB free + + +
Why a gauge: one ratio against a limit. The denominator keeps "3.8 GB + install" honest — big enough to see, small enough to not panic about. Accent = toolchain; + everything else stays de-emphasis gray so the accent has one meaning.
+
+ +
+

Largest consumers — all categories

ranked bars
+
+
codex transcripts
3.9 GB
+
claude transcripts
2.5 GB
+
ruflo install tree
1.4 GB
+
codex state ledgers
1.1 GB
+
learning stores (all projects)
0.9 GB
+
+
Why ranked bars: "what's eating the disk" is a magnitude comparison — + the top-5 across every category beats five per-category charts for a summary. Host hue + carries identity where a row belongs to one host; install/shared rows stay neutral.
+
+
+
+ + +
+
+ System +

Storage

+

Retained data by category, then by host, down to the session leaf — plus growth, giants, and + advisory reclaimables. Paths are the answer here, so rows carry them.

+
+ +
+
+

Retained data by category

donut
+
+ + + + + + + + 8.9 + GB retained + +
+ transcripts 5.2 GB · 58% + ledgers & logs 2.0 GB · 22% + learning stores 1.0 GB · 12% + kit caches 0.7 GB · 8% +
+
+
Why a donut: part-to-whole with exactly four slices and a center total — + within honest-pie territory. Values ride the legend (light-mode aqua/yellow sit under 3:1, so + the relief rule makes labels mandatory, never color-alone). More than ~5 categories → switch + to a stacked bar.
+
+ +
+

Per-host split by category

stacked bars
+
+
codex +
+ + + +
5.2 GB
+
claude +
+ + + +
3.2 GB
+
opencode +
+ + +
0.5 GB
+
+
transcripts + ledgers & logs + learning stores + kit caches
+
Why horizontal stacks: the same four category hues as the donut — color + follows the entity across the whole view, so the reader learns the mapping once. Rows share + one scale; 2px gaps separate segments. Clicking a row would drill to project → session leaves.
+
+ +
+

Growth — bytes added per day, 30d

small-multiple areas
+
+
claude
+ + + + + + +
+
codex
+ + + + + + +
+
opencode
+ + + + + + +
+
+
Why small multiples, not one chart: three hosts on one axis would make + opencode invisible against codex. Each panel keeps its own emphasized endpoint; host hue + matches the host's color everywhere else. Derived from mtime + size only — no content reads.
+
+ +
+

Reclaimable — advisory only

annotated list
+
+
⚠ 1.9 GB +
Transcripts older than 180d (312 files)
~/.codex/sessions/2026-0[1-2]/**
+
⚠ 640 MB +
Stale npx cache envs (9, unused > 90d)
~/.npm/_npx/*
+
⚠ 210 MB +
Orphaned worktrees (2, branch merged)
~/dev/keel/.autopilot/wt-*
+
+
Why a list, not a chart: each row is a decision, not a magnitude — the + rationale and the path are the content. Warning color ships with icon + label, never alone. + No delete button exists by invariant; the row points at the CLI that owns cleanup.
+
+ +
+

Largest sessions

table + inline bars
+
+ + + + + + + +
SessionHostProjectFilesSizeShare of host
rollout-2026-08-01T09-14…codexagentic-kit61412 MB
94d88189-5c24-420a…claudeagentic-kit18287 MB
rollout-2026-07-28T14-02…codexemailibrium44231 MB
a5e3220a-2e76-4e94…claudekeel9164 MB
+
Why a table: a session id, host, project, and path are lookup facts — the + bar is a garnish for scanning, the row is the unit. Doubles as the accessibility table view for + this section's charts.
+
+
+
+ + +
+
+ System +

Runtime

+

A point-in-time census — computed on request, never persisted. CPU and memory of live host + processes, daemons against their TTL, and combined totals.

+
+ +
+
+

Live host processes

table + inline bars
+
+ + + + + + + + + + +
HostpidProjectUptimeCPURSS
claude48112agentic-kit3h 12m6.2%
1.31 GB
codex50871emailibrium1h 40m3.9%
0.48 GB
opencode51230keel22m0.8%
0.14 GB
+
Why a table with bars, not gauges per process: the census is a scan-and- + compare surface — bars share one RSS scale so "claude is 3× codex" is visible without reading + numbers. Reuses the argv-minimized survey; a row is a resource consumer, not a session actor.
+
+ +
+

Combined memory

meter
+
+
1.93GB RSS combined
+
+
of 32 GB physical · 11.9% CPU combined (8 cores)
+
+

Daemons

stat tiles
+
+
+
2
+
running · oldest 4.2h of 12h TTL
+
+
$0
+
AI workers off · budget idle
+
+
Why a meter, not a radial: one ratio against a hard limit (physical RAM) + reads fastest as a straight track. The daemon tiles are single current values — stat tiles, not + a one-bar chart.
+
+
+
+ + +
+
+ System +

Catalog

+

What is actually deployed, deduplicated across hosts — and which host carries what.

+
+ +
+
+

Host inventory profile

radar · axis-normalized
+ + + + + + + + + + + + + + + skills + agents + commands + plugins + MCP + + +
claude + codex + opencode
+
Why a radar (and its caveat): the question is shape — "is codex a + commands-heavy host?" — not precise magnitude. Three series is the all-pairs-safe cap for + overlapping fills. Each axis is normalized to its own max, and the tooltip carries the real + counts, because radar area is not a quantity.
+
+ +
+

Unique across hosts

stat row + presence matrix
+
+
+
57
unique skills
+
+
23
unique agents
+
+
41
unique commands
+
+
12
plugins
+
+
9
MCP servers
+
+
+ claudecodexopencode + security-testing + qe-court + spring-m11n:migrate + autopilot:orchestrate + frontend-design +
+
…52 more · search + full table in the real view
+
Why a matrix, not more charts: "which hosts carry it" is boolean identity — + a dot grid answers per-item questions the radar's aggregate cannot. Counts get stat tiles + because each is a single current value. Observed inventory only; never desired state.
+
+
+
+ + +
+
+ System +

Projects

+

Every project the machine knows, sized two ways — code and disk — with the overhead + (.git, node_modules) kept separate so it can't + masquerade as "your project got big."

+
+ +
+
+

Project footprints

table + composed bars
+
+ LOC: JS/MJS + TS + Rust + other + Disk: tree + .git + node_modules +
+
+ + + + + + + + + + + + + + + + + + + +
ProjectLOC ≈By languageDisktree · .git · node_modulesLast active
agentic-kit ↗
github · pacphi/agentic-kit
96k
1.9 GB
2h ago
emailibrium ↗
github · pacphi/emailibrium
201k
3.4 GB
1d ago
keel
local only — no git remote
64k
1.1 GB
3d ago
+
…11 more projects · sortable by any column in the real view
+
Two bars, two color jobs: LOC is identity (languages → categorical + slots, "other" folds to gray past three). Disk composition is one entity's ranked parts — + shades of a single hue, darkest = the part you wrote, faintest = reinstallable overhead. LOC is + labeled approximate by invariant; extension-bucketed, stated exclusions. + Remote links: the project name is the link (one obvious click target, ↗ marks it as + leaving the panel), the host · slug subline carries provenance, and + the tooltip holds the raw origin URL — parsed from .git/config via + the same shapes parseRepoSlug already handles. No remote renders an + explicit "local only", never a dead link; the kit itself never fetches the URL.
+
+
+
+ +
ak · System — design mock for ADR-0025 · illustrative data · both themes via the dashboard's own tokens
+
+ + + + + + diff --git a/docs/ddd/component-directory.md b/docs/ddd/component-directory.md new file mode 100644 index 0000000..458904f --- /dev/null +++ b/docs/ddd/component-directory.md @@ -0,0 +1,208 @@ +# Component Directory Domain + +This document specifies the domain decided by +[ADR-0026](../adr/0026-about-component-directory.md) and implemented in +`src/lib/dashboard/about-directory.mjs`. Its terms are merged into +[Ubiquitous language](ubiquitous-language.md) and the context is on the +[context map](context-map.md). + +## Purpose + +Component directory answers a new user's first question — **"what did agentic-kit put on my +machine, and why is each thing worth having?"** — inside the dashboard's **About** primary area, +placed first in the tab order. It owns the curated editorial identity of every +component the kit installs or configures: a plain-language tagline, one friendly paragraph of +value proposition, outbound links to the source (GitHub), package (npm), and public docs, an +icon, a category, and a deliberate reading order. It joins that editorial content, at render +time, with detection facts the dashboard already has — installed or not, version, install +method — and never collects anything of its own. + +The shared terms in [Ubiquitous language](ubiquitous-language.md) are normative; the +[terms table](#ubiquitous-language-additions) below restates this context's contributions to it. + +## Why this is a separate context + +The neighboring contexts each own a different kind of fact about the same components; this one +owns the only kind none of them can: **editorial identity**. + +- **[Integration management](integration-management.md)** owns capability registries, bindings, + and ownership — what a component *can do* and what ak *manages about it*. It has no voice: no + value proposition, no links for a human, no reading order. The directory borrows its registry + as a parity gate (every managed tool must have an entry) and borrows nothing else. +- **Overview / status** owns health verdicts. The directory renders a version chip from the + same detection facts, but a card is an introduction, not a verdict — About never says + "degraded," it says what the thing is for and where to read more. +- **[Machine footprint](machine-footprint.md)** ([ADR-0025](../adr/0025-machine-footprint-metrics.md)) + owns measured cost. The directory renders no numbers beyond version strings — bytes, counts, + and processes belong there. +- **Documentation** (`docs/*.md`) explains ak to maintainers in reference register. The + directory is the new-user register, structured as data (per-component entries) so + completeness is testable, which prose cannot be. + +The editorial/detection split is the load-bearing boundary: editorial content is **authored, +versioned, and reviewed** with the release; detection facts are **observed at render**. A card +can therefore never claim runtime state in prose — the chip says `installed v3.34.0` because +detection said so, while the paragraph would read identically on a machine where the tool is +absent (with the chip honestly reading `not installed — ak setup adds it`). + +## Model + +```text +src/lib/dashboard/about-directory.mjs (pure data + tiny accessors; frozen, no I/O) + +DirectoryEntry { + id, // stable key; the managed-tools registry key where one exists + category, // 'hosts' | 'engine-memory' | 'quality' | 'safety' | 'knowledge' + // | 'kit' | 'configured' + name, tagline, // tagline: plain-language, ≤ 10 words + paragraph, // one paragraph, ~50 words, new-user register (see contract below) + links: [ { kind: 'github'|'npm'|'docs', label, url } ], // https only; [] for configured + icon: { kind: 'official', ref } // ref = 'claude' | 'codex' | 'opencode' + | { kind: 'monogram', ref, hue }, // ref = letterform, hue = a CSS token name + + // packaged entries only: + detectionKey, // join key into existing status/managed-tools facts + npmPackage, // package identity; the version chip's join key into the drift report + + // configured entries only: + subsystem, // the `ak status` subsystem row whose verdict feeds the chip + manage // the command that manages this surface ("yours to change") +} + +CATEGORY_ORDER // frozen curated order, hosts first, configured last +directoryEntries() // -> frozen DirectoryEntry[] in curated order +entryById(id) / entriesByCategory(category) + + + joined at render with existing facts (no new collection): +/api/status rows + managed-tools detection -> { state, version } + + | + v +About primary area (leftmost tab): hero orientation strip + category sections of cards + Card = icon tile · name + state chip · tagline · paragraph · link pills + (configured surfaces swap the link pills for a `manage:` line) +``` + +Two fields exist because the render would otherwise have to guess. `npmPackage` names the package +a version chip reports on, so the client joins the drift report by package identity instead of +hardcoding an id→package map or parsing an npm URL. `manage` makes the "yours to change" line +data rather than prose the client parses back out. + +### The editorial register contract + +The paragraph and tagline are covered by an explicit writing contract, enforced in review (and +lintable where mechanical): + +1. **One paragraph, ~50 words, no more.** A card is an introduction, not documentation — the + Docs link is where depth lives. +2. **Plain language.** Every term of art is either avoided or explained in the sentence using + it. "Agent swarms" renders as "teams of specialist agents"; "MCP" as "the plug-in protocol + agents use to reach tools." +3. **Value stated as what it does for the user**, in active voice — never marketing + superlatives, never claims about quality ("best," "blazing") that detection can't back. +4. **No runtime claims in prose.** "Installed," "running," "healthy" are chip words, fed by + detection; prose describes purpose, which is true whether or not the tool is present. +5. **Friendly ≠ cute.** Warmth comes from clarity and usefulness; no exclamation marks doing + the work of substance. + +### Iconography + +Official marks are used only where the dashboard already ships them as official — the three +host SVGs Observability renders on session rows — and are reused byte-identically so a +component looks the same everywhere. Every other component gets a **monogram tile**: its +initial(s) on a rounded tile in its category hue. A monogram is an honest "no official mark" +statement, not a stand-in logo; if an upstream later publishes a usable mark, swapping it in is +a one-entry change. All icons are inline (SVG or styled text) — the page stays self-contained +under the dashboard's CSP. + +### Configured surfaces + +Non-package things ak sets up — MCP registrations, managed guidance blocks +(CLAUDE.md/AGENTS.md), statuslines, the dual-host routing policy and Claude↔Codex bridge, the +background daemon, permission allowlists — are directory entries too, in the `configured` +category. Their chip reads `configured` (joined from the relevant status subsystem row), their +paragraph explains what was configured and why it helps, and each names the command that +manages it (`ak host`, `ak sync`, `ak x mcp pick`) so "yours to change" is actionable, not a +platitude. + +## Delivery + +No new endpoint. The directory module is imported by the page/client the same way `groups.mjs` +already is, and the state chips join client-side against the `/api/status` payload the +dashboard polls anyway. A status fetch that fails degrades every chip to `state unknown` while +the editorial content — which needs no network and no facts — renders in full: the page's +purpose survives its join's absence ([ADR-0023](../adr/0023-fail-closed-operations-and-explicit-degradation.md) +honesty, applied to a page whose primary content is static). + +Outbound links open the user's browser; the kit performs no egress. The directory's URLs are +swept by the same nightly external link check that covers `docs/**` — a rotted link is CI red, +not a permanent dead end. + +About is leftmost in the tab order but is **not** the landing view: Overview stays the default on +every open, including the first, and a one-time dismissible nudge on Overview points newcomers +left. Making the landing view depend on invisible browser state would give two machines two +different first screens with no way to tell why. + +`ak about` renders the same directory in a terminal from the same module, joined against one +`ak status` collection — the rows `/api/status` serves. Each detection source is guarded +independently, so one dead collector degrades one chip rather than the page. + +Two join limits are stated rather than papered over. A surface whose verdict `ak status` does not +emit — today, the permission allowlist — renders `state unknown` with that reason, because +claiming `configured` from the absence of a row would invent the one fact nobody supplied. And a +surface backed by more than one status row — statuslines, which span Claude's and Codex's — +takes the worst of the pair, so the quieter one's health cannot hide behind the other's. + +## Invariants + +1. **Editorial content is authored and versioned with the release** — checked in, reviewed, + never generated or fetched at runtime. +2. **Detection facts come only from existing collectors.** The directory probes nothing, adds + no endpoint, and a failed status join degrades chips to `unknown` without hiding cards. +3. **Prose never claims runtime state.** Installed/version/configured render exclusively as + chips fed by detection; the paragraph reads true on any machine. +4. **Registry↔directory parity is a test.** Every managed tool has exactly one entry; no entry + exists for something ak neither installs nor configures. +5. **Links are `https`, named-host, user-initiated**; the kit fetches none of them; all are + covered by the nightly external link sweep. +6. **Official marks only where genuinely official and already shipped**; everything else is an + explicit monogram tile. No fabricated brand assets, ever. +7. **No numbers beyond version strings.** Health, cost, counts, and activity belong to their + own contexts; About introduces, it does not measure. +8. **The register contract governs every entry** (length, plain language, active voice, no + superlatives, no runtime claims). +9. **Order is curated and stable** — hosts first, the kit's own card near the end, configured + surfaces last — never derived from popularity, size, or health. +10. **One icon per component, everywhere.** The directory's icon spec is the single source for + that component's mark across the dashboard. +11. **A chip states only what detection said.** An informational row carries a fact without a + verdict, and a missing row carries nothing at all; neither is upgraded to "installed". Absent + evidence renders `unknown` with its reason. + +## Ubiquitous language additions + +These terms are merged into [Ubiquitous language](ubiquitous-language.md); that glossary is +normative and this table restates it for readers of this document. + +| Term | Meaning | +|------|---------| +| Component directory | The curated catalog of everything ak installs or configures, with editorial identity per entry | +| DirectoryEntry | One component's editorial identity: category, tagline, paragraph, links, icon, and a detection join key | +| Editorial content | Authored, versioned prose and links — the part of a card that is true regardless of machine state | +| Detection fact | An observed install/version/configured fact borrowed read-only from existing collectors, rendered only as chips | +| State chip | The card element that renders detection facts (`installed v…` / `not installed — ak setup adds it` / `configured` / `unknown`) | +| Monogram tile | The honest icon for a component with no official mark: initials on a category-hued tile | +| Register contract | The editorial writing rules (one ~50-word paragraph, plain language, no runtime claims, no superlatives) | +| Parity gate | The test asserting managed-tools registry ↔ directory completeness in both directions | +| Configured surface | A non-package thing ak sets up, carrying a managing command instead of package links | + +## References + +- [ADR-0026](../adr/0026-about-component-directory.md) — the decision record this domain implements +- [Integration management](integration-management.md) — the registry this stays in parity with +- [Machine footprint](machine-footprint.md) — the measurement context this deliberately isn't +- [Context map](context-map.md) — where this context sits +- [Dashboard guide](../DASHBOARD.md) +- `src/lib/dashboard/about-directory.mjs` — the directory module; + `src/commands/about.mjs` — the CLI twin; + `tests/kit/about-directory.test.mjs` — the parity gate and register-contract checks diff --git a/docs/ddd/context-map.md b/docs/ddd/context-map.md index 3e0beee..a57816a 100644 --- a/docs/ddd/context-map.md +++ b/docs/ddd/context-map.md @@ -21,7 +21,9 @@ Native Evidence ----> Evidence Acquisition ----> Canonical Evidence | | | +-----------------------+----> Dashboard Delivery <----+ ^ -Project State (.claude-flow/*) ----> Project Intelligence ┘ +Project State (.claude-flow/*) ----> Project Intelligence-+ +Local filesystem + process table ---> Machine Footprint --+ +Curated editorial content ----------> Component Directory-+ Maintainer Administration is a separate, deliberately-egressing context. ``` @@ -87,11 +89,40 @@ provider, or lifecycle identity and grades no per-field evidence confidence. See [Project intelligence](project-intelligence.md). +### Machine footprint + +Owns read-only measurement of what the toolchain costs this machine: install bytes and install +method per managed tool, the ephemeral runtime census of host processes and daemons, the retained +data breakdown (category → host → project → session) with growth and advisory reclaimable +candidates, the deduplicated cross-host catalog inventory, and per-project approximate lines of +code and disk. Its sources are the local filesystem and the current-user process table — the same +trust boundary `ak status` and the runtime survey already cross — so no anti-corruption adapter is +required. It is structurally metadata-only: collectors read directory entries, `stat` results, +manifest names, and `.git/config`'s remote URL, so transcript, prompt, and tool-payload content +cannot enter the model. It owns no spend, activity, or learning facts, and it mutates nothing +beyond its own snapshot file. + +See [Machine footprint](machine-footprint.md). + +### Component directory + +Owns the curated editorial identity of every component agentic-kit installs or configures: a +tagline, one plain-language paragraph, outbound source/package/docs links, an icon spec, a +category, and a stable curated order. That content is authored, checked in, and versioned with the +release — never generated, fetched, or derived at runtime. It performs no detection of its own: the +installed/version/configured chips are joined at render time from facts existing collectors already +produce, and a failed join degrades chips to unknown while the editorial content still renders in +full. It renders no numbers beyond version strings; health, cost, activity, and measurement belong +to their own contexts. + +See [Component directory](component-directory.md). + ### Dashboard delivery Owns protected HTTP/SSE delivery, browser DTOs, filters, presentation, and interaction state. It -may combine read models from Observability, Historical Usage, Project Intelligence, routing, and -integration facts. It cannot manufacture or strengthen domain facts. +may combine read models from Observability, Historical Usage, Project Intelligence, Machine +Footprint, Component Directory, routing, and integration facts. It cannot manufacture or strengthen +domain facts. ### Maintainer administration @@ -114,6 +145,29 @@ and credential policy is distinct from the offline-first dashboard and integrati | Historical usage | Dashboard delivery | Historical aggregates and findings | | Project state (`.claude-flow/*`) | Project intelligence | Direct local file reads; no anti-corruption adapter needed | | Project intelligence | Dashboard delivery | Read-model projection, delivered by poll (`/api/status`) and SSE push (`/api/live/intelligence`) | +| Local filesystem and process table | Machine footprint | Direct metadata-only reads; no anti-corruption adapter needed | +| Project census | Machine footprint | Candidate paths only, at directory granularity; every rendered figure is measured by this context's own collectors | +| Project census | Project intelligence | The `learning` scope, folded onto project identity — the project list and the selectable key | +| Project census | Historical usage | Repository roots, so a session in a sub-directory labels as its repository rather than as a peer project | +| Machine footprint | Dashboard delivery | Two-tier measurement read model over `GET /api/system`, and the same collector behind `ak system` | +| Integration management | Component directory | Registry consumed only as a parity gate; no editorial content flows either way | +| Detection facts (`/api/status`, managed-tools) | Component directory | Read-only join at render; a failed join degrades chips to unknown, never hides cards | +| Component directory | Dashboard delivery | Versioned editorial entries imported by the page; no endpoint, no probe, no cache | + +### Project census (shared kernel) + +`src/lib/project-census.mjs` is the one enumeration of this machine's projects +([ADR-0027](../adr/0027-shared-project-census.md)). It is a **shared kernel**, not a context: it +owns no domain logic, produces no rendered figure, and every consumer applies its own named scope +and takes its own measurements. Four contexts derive their project list from it, and the identity +it keys on (`resolveProjectIdentity`) is Observability's, reused rather than reinvented — which is +what makes a project mean the same thing in all four. + +The kernel deliberately serves **two granularities**. Machine footprint consumes directories, +because directories are what have bytes and lines in them. Project intelligence consumes projects, +folded onto identity, because a project is what a user selects. Collapsing those would either +destroy the System area's per-directory figures or break the Intelligence picker; both were +observed before the split was made explicit. ## Boundary rules @@ -123,6 +177,14 @@ and credential policy is distinct from the offline-first dashboard and integrati - Dashboard presentation cannot upgrade provenance. - Historical usage and live topology share identifiers, not aggregate ownership. - Network egress occurs only in commands and contexts whose contract explicitly permits it. +- Every project count is rendered with the scope that produced it; two contexts may report + different totals, but neither may report an unexplained one. - Project intelligence reads local project state directly; it never enters Evidence Acquisition's anti-corruption layer or Observability's canonical event model, and it establishes no session, actor, host, provider, or lifecycle identity. +- Machine footprint measures metadata only. It never opens transcript, prompt, or tool-payload + content, never publishes spend, session, or learning facts, and never renders an unmeasured + value as zero. +- Component directory authors identity, it does not observe it. Editorial prose never asserts + runtime state; installed, version, and configured render exclusively as chips fed by detection + facts borrowed from existing collectors. diff --git a/docs/ddd/machine-footprint.md b/docs/ddd/machine-footprint.md new file mode 100644 index 0000000..7de1bbb --- /dev/null +++ b/docs/ddd/machine-footprint.md @@ -0,0 +1,633 @@ +# Machine Footprint Domain + +This document specifies the domain decided by +[ADR-0025](../adr/0025-machine-footprint-metrics.md) and implemented in `src/lib/footprint/`. +Its terms are merged into [Ubiquitous language](ubiquitous-language.md) and the context is on the +[context map](context-map.md). + +## Purpose + +Machine footprint answers **what this toolchain costs the machine itself**: how many bytes the +managed install occupies and where; how much CPU and RAM live host processes and daemons are +consuming right now; how retained data (transcripts, ledgers, logs, learning stores, caches) +breaks down by category, host, project, and session; and what is actually deployed — the +deduplicated inventory of skills, agents, commands, plugins, and MCP servers across hosts, plus +every known project's size in lines of code and disk. + +It is a read-only measurement domain over local state the kit already has trust-boundary access +to. It renders in the dashboard's **System** primary area and through a CLI twin (`ak system`), +and it mutates nothing — including the reclaimable-space candidates it computes, which are +advisory rows with rationale, never delete actions. + +**Footprint** is the name of this context, not of a user-facing surface: every surface a user +touches — the tab, the command, the route — is called *System*. + +The shared terms in [Ubiquitous language](ubiquitous-language.md) are normative; the +[terms table](#ubiquitous-language-additions) below restates this context's contributions to it. + +## Why this is a separate context + +Each neighboring context owns a different kind of fact about overlapping raw material, and the +boundaries are what keep all four honest: + +- **[Historical usage](context-map.md)** owns *spend* facts — tokens, API-equivalent cost, model + identity — parsed from transcript **content**. Machine footprint reads transcript `stat` + metadata, plus one declared field — the session's `cwd` — and never a message body + ([the read surface](#the-read-surface)). A 400 MB session is a storage fact here and a token + fact there; the two figures answer different questions and neither substitutes for the other. +- **[Observability](observability.md)** owns *activity* evidence — session lifecycle, actors, + per-field confidence, a protected transcript plane. Machine footprint's runtime census reuses + the same process survey as a *source*, but publishes resource rows (CPU%, RSS, uptime), not + `ObservedSession` evidence; a process here is a consumer of memory, not an actor in a session + graph. +- **[Project intelligence](project-intelligence.md)** owns *learning* trends from + `.claude-flow/` state. Machine footprint measures those same directories only as bytes on + disk. The shared piece is the *idea* of project discovery, not the implementation: this domain + runs its own (`project-sources.mjs`) because Intelligence's catalog answers "which projects + carry ruflo learning state" and collapses ~50 projects to 5 here. Either way a path is not + evidence, and discovery supplies nothing this domain renders as a measurement — exactly the + boundary ADR-0024 draws when it reuses Observability's workspace store. +- **[Integration management](integration-management.md)** owns what *should* be deployed + (bindings, projections, ownership). Machine footprint reports what *is* on disk and how big it + is; catalog counts here are observed inventory, never desired state. + +Because every source is the local filesystem and the current-user process table — the same trust +boundary `ak status` and the runtime survey already cross — no anti-corruption adapter guards +this domain's reads. What *is* guarded is content, and the guarantee is narrower and more precise +than "the collectors never open a file": see [The read surface](#the-read-surface) for the +enumerated list of what is opened, what is taken out of it, and why every one of those reads is +still a metadata read. + +## The read surface + +Invariant 1 is an enumeration, not a slogan. Every read this domain performs is listed here; a +read not on this list is a defect, and adding one is an amendment to this document and to +[ADR-0025 §7](../adr/0025-machine-footprint-metrics.md#7-a-deliberate-path-visibility-exception-and-an-enumerated-read-surface). + +| Read | Collector | What is taken | What is never taken | +|------|-----------|---------------|---------------------| +| Directory entries and `lstat` | `walk.mjs`, every collector | name, kind, size, mtime, block count | anything inside a file | +| `.git/config` | `projects.mjs` | the origin remote URL | every other config key | +| `.git/worktrees//gitdir` | `storage.mjs` | one filesystem path, bounded to 4 KB | — | +| A transcript's **head** | `project-sources.mjs` | the session's `cwd` **field** | every message, prompt, tool call, tool result and model output in the file | +| OpenCode's session store | `project-sources.mjs` | the `directory` column, read-only | every other column, and every message row | +| A project's own manifests | `stack-detect.mjs` | dependency **keys** (and, for `path:`/`workspace:` entries, enough of the value to reject them) | manifest values, scripts, and anything executable | +| A project's own source files | `stack-detect.mjs` | the count of `\n` bytes, and whether byte 0 of the first chunk region is NUL | the text — each 64 KB chunk is counted and immediately overwritten | + +Three of those rows are new since the first draft of this document, and they are the reason this +section exists rather than a one-line invariant. + +**A transcript's `cwd` is a path, not content.** Project discovery opens each Claude and Codex +transcript, reads at most its first 256 KB, JSON-parses at most its leading 40 non-blank lines, +and takes exactly one field: `record.cwd` (Claude) or `record.payload.cwd` on the `session_meta` / +`turn_context` records that open a rollout (Codex). The parsed records are discarded at the end of +the loop; nothing but the path string survives the function. This is the same read +`native-transcript-discovery.mjs` already performs for Observability at the same trust boundary, +and it is what makes discovery honest: the alternative is guessing the path from the directory +name, which [Project accounting](#project-accounting) shows is wrong four times out of five. + +The head bound is a correctness statement as much as a cost one. A session's `cwd` is declared in +its opening records or nowhere, so reading further would cost the whole corpus (~2,700 files here) +to learn nothing — and it would put the collector's read window over message bodies for no gain. + +**A manifest's dependency keys are names, not code.** Stack detection reads `package.json`, +`Cargo.toml`, `go.mod`, `pyproject.toml`, `pom.xml`, `build.gradle`, `mix.exs`, `Gemfile`, +`pubspec.yaml` and their siblings to at most 3 directories deep, at most 64 of them, at most +512 KB each, and extracts dependency **keys**. Nothing is evaluated, executed, resolved or +fetched: `mix.exs` and `build.gradle` are Elixir and Groovy *source*, and they are scanned with +regexes, never run. The depth bound is not cosmetic — a scan of this machine found 1,884 +`Cargo.toml` files, nearly all of them inside Cargo's registry cache, and a deeper search would +report a dependency's dependencies as the project's own. + +**Counting lines requires reading bytes, and the ADR always required counting lines.** LOC was +sanctioned from the first draft (`§4`: "a zero-dependency extension-bucketed line count by the +kit's own bounded walker"). What was left implicit is that a line count is a byte scan: the +counter streams the file through one fixed 64 KB buffer, increments on `0x0a`, bails out on a NUL +byte in the first chunk (binary), and never retains, concatenates, decodes or emits the text. The +figure that leaves the function is an integer. + +The boundary that still holds, stated positively: **no message body, prompt, tool call, tool +result, model output, or manifest value enters this domain, in any tier, on any path.** Every +figure the System area renders is either a `stat` result, a count, or a path. + +Two neighbouring boundaries are untouched by the above. The `cwd` read is *discovery* +(invariant 9) and supplies a candidate path, never a measurement. And nothing here reads a +transcript for what Historical usage reads it for — tokens, cost and model identity stay in that +context, parsed from the message bodies this one does not open. + +## Model + +```text +Sources (all local; see "The read surface" for exactly what is read) + install roots (managed-tools detection) process table (runtime survey + pcpu/rss) + transcript roots (~/.claude, ~/.codex, opencode store) + consumer registry (~50 curated third-party cache roots — consumers.mjs) + known files (ledgers, tee logs, caches, indexes) + project sources (own cross-host discovery) host catalog surfaces (agents/skills/commands/MCP) + | + v +Collectors (bounded walkers; two tiers — src/lib/footprint/) + walk.mjs the one bounded walker + the Measurement vocabulary every collector shares + project-sources.mjs cross-host project discovery (everSeen / onDisk) + stack-registry.mjs + stack-detect.mjs — languages (lines) vs frameworks/SDKs/tools (presence) + cheap tier: runtime.mjs census + known-file stats + carry-forward of last deep scan (TTL 60s) + deep tier: install.mjs + storage.mjs + catalog.mjs + projects.mjs + consumers.mjs + (explicit, single-flight) + snapshot.mjs persists the deep sections; index.mjs is the two-tier collector façade + | + v +FootprintSnapshot { asOf, completeness, install, runtime, storage, catalog, projects, consumers } + install: HostInstallation[] { tool, version, installMethod, root, bytes, nativeAddons[] } + runtime: RuntimeCensus { processes[], daemons[], totals } (ephemeral, never + persisted) + storage: StorageBreakdown { nodes: category → host → project → session, growth, topN, + reclaimables[], reclaimSummary: { tiers[], combined: null } } + catalog: CatalogInventory { skills[], agents[], commands[], plugins[], mcpServers[], + each with per-host presence } + projects: ProjectFootprint[] { path, label, remote?: {host, slug, webUrl}, stack: {languages, + stack, unrecognized}, treeBytes, gitBytes, nodeModulesBytes, + lastActivity } + + counts { everSeen, onDisk, gitRepos, unresolved } + consumers: ConsumerRanking { rows[] (root | breakdown | residual), top[], groups[], + totals, absent[], unmeasured[], includeProjectTrees } + | + v +Delivery + GET /api/system → cheap tier + persisted snapshot (token auth, loopback, no egress) + GET /api/system?refresh=deep → start-or-attach the single-flight deep scan + ak system [--deep] [--json] → the same collector, CLI-rendered + | + v + System primary area: Summary | Storage | Runtime | Catalog | Projects +``` + +### Measurement semantics + +Every figure in this domain is a `Measurement` — a value plus how it was obtained: + +- **measured** — a real walk/stat/count produced it, stamped with the snapshot's `asOf`; +- **carried forward** — from the persisted snapshot, presented with that snapshot's `asOf`, + never as current; +- **unknown** — never measured, or the measurement failed for that node; rendered as + "not measured yet" / "unavailable," **never as `0`**. + +A measured zero is a real zero and renders as one. This is the same zero-vs-unknown discipline +the dashboard's `metric()` helper and [ADR-0023](../adr/0023-fail-closed-operations-and-explicit-degradation.md) +already enforce elsewhere. + +A measurement additionally carries `partial`. A total whose inputs included an unreadable or +capped subtree is a **lower bound**, not a total: unknown inputs never contribute a zero, they set +`partial` on the sum. Every surface renders a partial figure as "≥ N" — a partial total presented +as a total would be the honest-degradation contract broken at the last mile. + +### Install footprint + +One `HostInstallation` per managed tool (ruflo, agentic-qe, Claude Code, Codex, OpenCode, +agentic-kit itself, the brain KB), carrying the version and install method the kit's existing +detection already knows (npm/mise/brew/self-managed), the resolved root path, tree bytes from +the deep walk, and a native-addon inventory — including the *duplicate-builds* view (the same +native module compiled into multiple trees), which is sprawl the user cannot see today. Shared +caches (npx envs, browser binaries) are install-adjacent nodes with their own rows, not smeared +into a tool's tree. The machine's free-space figure is the section's denominator, so "the +install is X GB" always has a "of Y free" next to it. + +Two of those roots were measured at the wrong place, and the corrections are recorded here +because both changed a headline figure rather than a detail. + +**The brain is measured whole, not at its KB.** `install.mjs`'s `brainRoot()` resolves the +installer's own `…/ruvnet-brain/kb` layout upward to the cache root and measures that. Measuring +`kbDir()` alone under-reported this machine's brain by 85%: 1.9 GB of active KB inside a 13.2 GB +cache root. The remainder is not one thing, so it is not one number — the row carries components +(`BRAIN_COMPONENTS`) that break the root into the active KB, the superseded `kb.bak-*` copies the +installer leaves behind on every update (5 of them, ~11 GB together on this machine), the +embedding models (~234 MB), and a remainder row so the parts add up to the whole. A user watching +a figure move from 1.9 GB to 13.2 GB is owed that breakdown, and the `kb.bak-*` copies are the +answer: nothing reads them, and they are the largest single reclaimable on this machine. + +The upward resolution is deliberately conservative. Both path segments must match, because +`RUVNET_BRAIN_KB` can relocate the KB anywhere: a KB at `/mnt/data/kb` would make the parent a +directory the brain does not own, and billing a shared volume to the brain is a worse error than +under-reporting it. An unrecognized layout stays measured at the KB dir. + +**Playwright has three platform locations and they are one cache.** Only the XDG and Windows +paths were probed, so on macOS — where `playwright install` writes to +`~/Library/Caches/ms-playwright` — a machine holding 1.86 GB of browser builds reported a +*measured zero*: honest about the XDG path that genuinely does not exist, wrong about the +question the row asks. All three are now probed and collapse into a single row. Two can be real +at once (a cache migrated between layouts leaves both; on macOS `~/.cache` is sometimes a symlink +into `~/Library/Caches`), so candidates are realpath-collapsed before summing and an aliased +target is measured once. When none exists, the platform-canonical path is the one named, so the +measured zero still says where it looked. + +### Runtime census + +A point-in-time table of live host processes — reusing the existing current-user, argv-minimized +survey and extending its `ps` read with `pcpu`/`rss` — plus the daemon census (count, and the +oldest daemon's age against the 12h TTL). The census is +**ephemeral**: computed per request, never persisted into the snapshot file, because a process +table is a moment, not a fact worth retaining, and persisting it would create a stale-liveness +trap. `snapshot.mjs` enforces this structurally — it serializes only the deep-tier keys (`install`, +`storage`, `catalog`, `projects`, `consumers`), so a census handed to it is dropped rather than +written. + +Two figures this census once carried are deliberately gone. A **launch-budget** field could never +be populated: ruflo exposes no local state to read it from, so it was a permanent "unavailable" +rather than an honest degradation, and a permanently unknowable quantity is removed rather than +rendered (see [ADR-0023](../adr/0023-fail-closed-operations-and-explicit-degradation.md) §9). A +**child/MCP-server process count** is still computed by the survey — it is what makes the per-host +rows correct — but is no longer republished: as a rendered figure it was a bare number with no +denominator, no history, and no action attached to it. + +The census is also the one deep-tier neighbour that refreshes on the dashboard's ordinary poll +clock while its view is open. It measures liveness, so a figure loaded once when the tab was first +opened is the one kind of staleness this area cannot tolerate — but only the cheap tier polls; the +filesystem walk stays behind an explicit rescan. + +On **Windows** the census is real, not unsupported. `src/lib/live/win-process-survey.ps1` — a plain +text script invoked the way the POSIX path already invokes `ps` and `lsof`, with no npm dependency +and no compiled artifact — returns a guaranteed census (host, pid, ppid, start time, CPU, working +set) from `Get-CimInstance Win32_Process`, and command lines only for processes `GetOwner` proves +belong to the current user. The bound project comes from a **best-effort** P/Invoke read of the +process's own `CurrentDirectory` (`NtQueryInformationProcess` → PEB → +`RTL_USER_PROCESS_PARAMETERS`). When that probe fails — antivirus block, execution policy, +insufficient rights, WOW64 bitness mismatch — every other field still returns and the project +column degrades to an explicit "not attributable on Windows" carrying the reason. A row is never +dropped for being unattributable: a process we can measure but not attribute still consumes RAM, +and hiding it would understate the totals this whole area is denominated in. An empty census on +Windows is treated as a broken survey, never as an idle machine. + +One field is honestly absent everywhere: the daemon **budget** state. `ruflo daemon budget` is a +CLI with no local file this collector can read, so budget reports `unknown` with that reason +rather than a figure inferred from absence of evidence. + +### Storage breakdown + +A tree of `StorageNode`s: category (transcripts / ledgers-and-logs / learning stores / kit +caches) → host → project → session leaf, each with bytes and file count. Derived views over the +same walk: trailing-30d growth per host (from mtime + size — no content reads), top-N largest +sessions and files, and advisory `ReclaimableCandidate` rows (stale npx envs, transcripts beyond +a stated age, superseded cache snapshots, regenerable package caches, redundant browser +revisions, extra runtime versions, orphaned worktrees), each carrying its rationale and its path. +Candidates are information, not actions — this context has no delete verb. + +**Learning stores are reported on their own, not mixed into the shared charts.** On a real +machine they are ~99% of retained bytes, so a donut that includes them renders as a solid ring +and a per-host bar chart flattens every other series to a sliver — the chart stops being a +measurement and becomes a picture of one category. They get a single-figure card; the donut and +the per-host split cover the remaining three categories and **state the exclusion on the panel**. +This is presentation only: the collector measures all four categories and `ak system --json` +emits all four, unchanged. The per-host split likewise shows only real hosts (claude, codex, +opencode) and names the remaining bytes — ak's own state — in a footnote with its figure, so the +bars still account for the whole of the donut beside them rather than silently disagreeing with it. + +Two honest limits belong with the numbers. **Growth is approximate and says so**: a file +contributes its whole size on its mtime day, which is exact for append-only transcripts and +over-counts rewritten SQLite ledgers, so the figure carries its own `basis` string. And **Codex +transcripts carry no project attribution**: rollout paths are dated, not project-scoped, and the +project name lives inside the file, which this domain may not open. Those nodes are marked +`attribution: 'none'` and render as "unattributable" — never blank, never zero. + +### Largest consumers + +The storage tree answers "where are *the kit's* bytes"; it does not answer "what is eating this +disk", and for a while the panel pretended it did. The Summary strip's ranking was assembled in +the browser from whatever the install and storage sections happened to carry, which made it wrong +in the one way a size ranking must never be wrong: it named the npx cache (6.64 GB) this +machine's largest consumer while `~/.npm/_cacache` — three and a half times larger — was not +scanned at all. The real order on this machine begins `~/.ollama` 141.31 GB, `~/.lmstudio` +48.78 GB, `~/.cache/huggingface` 35.92 GB, `~/.npm/_cacache` 22.26 GB, mise installs 16.30 GB, +Docker 15.47 GB, rustup 14.62 GB, the pnpm store 13.40 GB, the brain 13.18 GB. The npx cache is +eleventh. + +`consumers.mjs` therefore owns the ranking as a first-class view over a **curated registry** of +~50 third-party cache roots — Ollama, LM Studio, Hugging Face, the npm/pnpm/yarn/bun caches, +rustup and Cargo, Go's module cache, uv and pip, Maven and Gradle, Playwright and Puppeteer, +mise, Homebrew, Docker — grouped by *ecosystem*, because the actionable question is which +toolchain is costing the disk: four Node package caches at 5 GB each is a Node answer, not four +unrelated rows. Third-party cache conventions live in that module rather than in `paths.mjs`, +which owns the kit's and its hosts' own locations and should stay auditable as such. + +A size ranking is trivially made dishonest, so three accounting rules are structural: + +- **Containment.** Roots nest — `~/.npm` contains `_cacache` and `_npx`; `~/.cache` contains + `huggingface`, `ruvnet-brain`, `puppeteer`, `pnpm` and `uv`; `~/.local/share` contains `mise`, + `pnpm`, `claude` and `opencode`; mise's installs tree contains the Node install that contains + the npm global root the install section already walks. Bytes are counted **once**, at the + outermost row (`kind: 'root'`); every enclosed row is a `kind: 'breakdown'` that explains its + parent instead of competing with it. Containment is *derived* from resolved paths, not + hand-declared, so a registry edit cannot silently start double-counting. Only roots are ranked + and only roots are summed. Without this rule the list is self-similar and the total is fiction. +- **Residuals.** A parent with breakdowns also gets a synthesized `:other` row — parent + minus its direct children — so a breakdown always adds up. This is what makes the brain row + legible: 13.18 GB, of which 1.9 GB is the active KB and ~11 GB is superseded `kb.bak-*` copies. + Merging the two figures and reporting only the KB are both ways of being wrong about the same + 11 GB. An unknown input makes the residual unknown, never a difference taken against a + fabricated zero, and a negative residual reports itself rather than rendering as a plausible + small number. +- **Project trees are opt-in.** One repository on a working machine can outweigh every shared + cache combined (175 GB here), and a bar chart containing it is a bar chart of one repository. + `includeProjectTrees` defaults to false and the exclusion is **stated in the payload** + (`consumers.projectTrees.reason`), never silent — an omitted category the reader cannot see is + the same failure as an unknown rendered as zero. + +Absent roots are not consumers: a cache root that does not exist is reported in `absent` with its +path — "we looked, it is not here" — and kept out of both the ranking and the group totals rather +than ranked as a zero-byte consumer. Unreadable roots are reported in `unmeasured` with their +errno and make the affected group total `partial`; they are never a zero. + +Two roots need a basis other than apparent size, and say so. Docker Desktop's VM image is a +sparse file whose apparent size here is ~4 TB against ~15 GB actually written, so it is measured +in **allocated blocks** and carries both figures plus a `basis` string; an apparent-size reading +would put a number larger than the disk at the top of the ranking. And figures the install or +projects scan already measured for the same path are **adopted** rather than re-walked, which is +why a row can name its `measuredBy`. + +This is deep-tier work by construction — a 22 GB content-addressable cache is ~10⁵ files — and it +carries its own raised walk caps (`CONSUMER_WALK_LIMITS`: depth 24, 2,000,000 entries) because +the walker's defaults were sized for install trees and transcript roots. Measured: rustup +exhausted the default entry cap at 9.4 of its 14.62 GB, and pnpm's content store, LM Studio and +`~/.claude` all bottomed out on depth 16. A ranking whose deepest trees are systematically floors +does not merely under-report them, it **mis-orders** the answer. Raised, not removed: a root that +exhausts even these still reports `≥`. + +### Reclaimable safety tiers + +Safety is a field, not a tone of voice. Every `ReclaimableCandidate` carries `safety`: + +- **`regenerable`** — the owning tool refetches it on demand. The npm content cache, the Homebrew + download cache, the brain's superseded `kb.bak-*` copies, stale npx envs. +- **`review`** — plausible but **not** safe to state as removable. Aged transcripts (a transcript + is the only copy of the session it records, and Historical usage is denominated in them); extra + runtime versions (mise has eight Node entries on this machine and some are the aliases a live + toolchain resolves through); a browser revision that may still be pinned by an installed + package. A review row is a pointer at something to look at. + +Each row also carries `bytesMeaning`: `candidate` (the bytes the row is actually about) or +`installed` (what is on disk at that path, offered as context on a review row that has no +defensible candidate subset). `keeps` names what was excluded from the figure because it is in +use, and `cleanupHint` names the CLI that already owns removal — documentation, not a command +this module runs. + +**The tiers never sum into one headline number.** `summarizeReclaimables` totals each tier +separately and sets `combined: null`, permanently. A combined "you could free N" that mixed a +regenerable cache with a runtime tree that may be live would produce the one number a reader +would act on and the one number this domain cannot stand behind. Two further rules keep even the +per-tier totals honest: only `bytesMeaning: 'candidate'` rows are summable, and a tier whose own +rows describe overlapping paths (an aged transcript can also sit under a project that no longer +exists) reports its total as unknown-with-reason rather than counting the same bytes twice — the +row count still stands, and every row still carries its own measured figure. + +Reclaimables are their own area rather than a card under a measurement. Everything else in this +context reports what **is**; this is the only surface that suggests what a user might **do**, and +that difference is worth a tab of its own — buried under a byte chart it read as another statistic. +It still has no delete verb and may never gain one. + +The rows render as one scrollable table ordered regenerable-first, not as two stacked blocks of +paragraph-bearing cards. The tier pill survives that change because it is what distinguishes the +two promises at a glance, and only the regenerable pill carries bytes: on a row that may be in +use, a figure in the pill would read as "this much is yours to take back", which is a claim the +measurement does not support. A review row's bytes stay in the size column, marked as context. + +The surfaces honour the split structurally: the `review` tier is rendered without a leading total +at all, and its bytes ride each row as context, so the block cannot be read as "N GB available +here". A row from a snapshot predating the `safety` field lands in `review` — the tier that +promises less — never in the one that reads as free space. + +### Catalog inventory + +Deduplicated `CatalogItem`s across hosts — skills, agents, commands, plugins, MCP servers — +keyed by normalized name, each with a per-host presence matrix (which hosts carry it, from which +surface it was observed). Counting is by manifest/directory-entry **names** on the host catalog +surfaces Integration management already projects into; item file contents are not parsed beyond +what naming requires. Scope is **user plus every project on disk**, not user plus the launching +repo: a skill defined in a repository is as deployed as one in `~/.claude`, and the question this +inventory answers is what the machine carries. Deduplication by `(kind, name)` means a name +defined in five projects is still one row, so the inventory grows with distinct names rather than +with project count. + +The presence matrix carries two independent multi-select filters — by kind and by host — with every +option selected at first paint, so the default remains the whole inventory. The host filter matches +**any** selected host rather than all of them: "carried by codex" is the question a reader is +asking, and intersecting would answer a different one. A filtered view states how much it is +hiding, and each row carries its own kind, because a filtered list must never leave a name +unexplained. The config-surface row (managed CLAUDE.md/AGENTS.md block count, settings +file sizes) lives here because it answers the same "what is deployed" question. + +### Project accounting + +**The Projects table lists repositories, not directories.** A row is rendered only when it has a +remote and a host has recorded a session in it. Without that rule the table listed ephemeral +`.claude/worktrees/agent-*` checkouts, sub-folders a session happened to run in, and home +directories, each beside its own parent repository as though it were a peer with its own +multi-gigabyte figure. The session test excludes an empty host list, never a missing one — a +snapshot predating that field cannot answer the question, and reading absent as zero would blank +the table. Excluded directories are counted and characterised beneath the table. + +**Two numbers, not one.** `discoverProjectSources()` publishes `everSeen` and `onDisk`, and they +are different questions: + +- **`everSeen`** — every distinct project any host has ever recorded a session in, *including* + the ones since deleted or moved. The deletions are the point, so they are never dropped. +- **`onDisk`** — the subset that still resolves to a directory, i.e. the only projects a byte or + line measurement can be taken of at all. Only these become table rows; the vanished ones + survive in `everSeen`, not as unmeasurable rows. + +On this machine: **50 ever seen, 25 still on disk, 21 of those git repositories.** The gap is the +figure, not an error. + +This domain deliberately does **not** reuse `discoverRuvfloProjects()`. That function answers a +different question — "which projects carry ruflo learning state" — by requiring a +`.claude-flow/neural/` directory and by reading only the 150 most-recently-modified transcripts +per host. Both narrowings are correct there and wrong here: on this machine they collapse ~50 +projects to 5. The Intelligence panel keeps that meaning; System has its own source with its own +stated method. + +**De-duplication is by resolved real path.** One project touched by Claude, Codex and OpenCode is +one project; a project reached through a symlink is the same project as the one reached directly. +Each sighting's `cwd` is `realpath`-resolved before it keys the map, falling back to +`path.resolve` when the target cannot be resolved — a deleted project has no real path and must +still be counted. A row therefore carries the set of hosts that saw it, its session count across +all of them, and its most recent sighting. + +**Claude's transcript-directory encoding is lossy, and this domain refuses to fake a decode.** +`~/.claude/projects//` encodes the project path by replacing `/`, `.` **and a literal +`-`** all with `-`, so `-Users-me-ai-agentic-kit` reads equally as `/Users/me/ai/agentic/kit` and +`/Users/me/ai/agentic-kit`. There is no safe pure-string decode. Measured on this machine's +corpus: naive `-`→`/` substitution resolves **8 of 52** directories — wrong for 85% of them. +`decodeClaudeProjectDir` therefore walks the candidate segments against the real filesystem and +returns a path only when the filesystem confirms it, under its own `lstat` budget; even that +verified walk can name only 18 of the 52 from the directory name alone. + +Which is why the encoded name is a **fallback, not the source**. The path comes from the +transcript's declared `cwd` (2,538 of 2,585 Claude transcripts here carry one; all 175 Codex +rollouts do; OpenCode's store carries an absolute `directory` per session). The decoder is +consulted only for a project directory where *no* transcript declares a `cwd` — one such group on +this machine. When neither route resolves, the group is counted as **`unresolved`** and +contributes no row: it is never given a fabricated path, and its existence makes `everSeen` a +**lower bound** rather than a total (`complete: false`). Every project row carries `origins` +saying which route named it (`cwd` or `encoded-dir`), so a fallback-derived path can never be +mistaken for a declared one. + +Codex is left out of the fallback entirely: its rollout directories are dated, not +project-scoped, so there is nothing to decode. + +### Project footprint + +One `ProjectFootprint` per **on-disk** project: working-tree bytes, `.git` bytes, `node_modules` +bytes (kept separate precisely because it dominates and distorts), last activity, and a detected +stack. LOC figures are labeled approximate wherever they render; this domain forbids presenting +them as authoritative. + +**Lines belong to languages; frameworks are presence only.** `stack-detect.mjs` returns +`languages` with a line count, because a file extension is what a line belongs to — and `stack` +entries (frameworks, SDKs, tools) with **no `lines` field at all**. React does not own lines, the +`.tsx` files do, and stacking both on one proportional bar would count the same bytes twice. +Nothing downstream can make that mistake by accident because the number simply is not in the +payload. `stack-registry.mjs` is the versioned data behind both, and every detection carries its +`registryVersion` and its `via` (which manifest or signature named it). + +**The unrecognized tail is the point.** An extension the registry does not map is never counted +as lines — most unmapped extensions on a real machine are binaries and data — so it is tallied +*by name* instead, as is every declared dependency that matched no registry entry. That converts +the usual silent "Other" slice into a to-do list a release can close. The tail admits only what +belongs in it: an extension the registry has already ruled out as non-source (`.png`, `.sqlite`) +is a stated exclusion counted separately, and a key that is not shaped like an extension at all +(`.2026-08-06`, from a rotated log) collapses into a named bucket rather than minting a thousand +single-file "extensions". `STACK_EXCLUSIONS` ships attached to the figure, so no surface can +render a line count without being able to state what it left out. + +A project additionally carries an optional `remote` — host, slug, and derived web URL — parsed +from `.git/config`'s origin remote through the same URL-shape handling the admin collector's +`parseRepoSlug` already proves (git+https / ssh / scp / bare). In the Projects table the project +name renders as a link to that page, with the raw remote in the tooltip; a recognized host +(GitHub, GitLab, Bitbucket, or a self-hosted URL that is already web-shaped) yields a link, an +unrecognized remote shape renders the remote name unlinked, and a project with no remote renders +an explicit "local only" — absence stated, never guessed. The link is user-initiated browser +navigation; the kit itself never fetches the remote. + +## Delivery + +`GET /api/system` follows Dashboard delivery's existing contract: loopback bind, per-session +token auth ([ADR-0014](../adr/0014-dashboard-auth-and-remediation.md)), `no-store`, zero egress. +The response is the cheap tier computed fresh (TTL ~60s, shared-cache pattern like the +project-snapshot cache) merged with the persisted deep snapshot and its `asOf`. +`?refresh=deep` starts the deep scan or attaches to the one in flight (single-flight, like the +usage index's coalesced builds); progress is surfaced so a long scan reads as working, not hung. +The server stays GET-only: a rescan re-measures local state and writes only this domain's own +snapshot file — it mutates no user data. + +**A deep scan never runs on its own.** Opening the System area issues a plain `GET /api/system`; +only the Rescan control adds `?refresh=deep`. A deep scan costs tens of seconds of I/O on a large +corpus, and making the act of *looking* cost that is a worse trade than a stale figure that states +how stale it is. Staleness is therefore surfaced rather than pre-empted: every deep-tier figure +renders with its snapshot's `asOf`, and past `SNAPSHOT_STALE_AFTER_MS` (7 days) the freshness +label turns amber and reads "stale, rescan". The client polls only while a user-started scan is +running, and stops when it finishes. + +One deliberate divergence from Observability's delivery: absolute paths are **part of this +payload**. `publicLivePayload`'s leaf-only rule exists to keep incidental provenance out of +session payloads; here the path is the answer ("where are the bytes"), and the same token-gated +loopback delivery protects it. Message content remains structurally absent from the payload: the +collectors never take it out of a file ([the read surface](#the-read-surface)), so delivery has +nothing to leak. + +The CLI twin (`ak system`) renders the same collector output, `--json` emitting the collector's +payload verbatim, following the one-collector-two-surfaces precedent of the usage scorecard. +`ak system --deep` is the terminal spelling of the Rescan control and writes the same snapshot. + +## Invariants + +1. **Metadata only, ever — and the reads are enumerated, not asserted.** The complete list is + [The read surface](#the-read-surface); it is normative, and a read not on it is a defect. + Collectors read directory entries and `stat` results; `.git/config`'s remote URL; + `.git/worktrees//gitdir` (bounded to 4 KB, validated as an absolute path) for the + orphaned-worktree candidate, which no `stat` can identify; a transcript head's `cwd` **field** + and OpenCode's session `directory` column, for project discovery; a project manifest's + dependency **keys**; and a source file's bytes streamed through a fixed buffer to count + newlines. Every one of those yields a path, a name, or an integer. **No message body, prompt, + tool call, tool result, model output, or manifest value enters this domain, in any tier, on + any path**, and nothing read is retained past the function that read it. +2. **Unknown is never zero.** An unmeasured or failed measurement renders as unknown with a + reason; a measured zero renders as zero. A total built over an unknown or capped input is + `partial` and renders as a lower bound. No fabricated figures. +3. **Freshness is part of the value.** Every deep-tier figure carries the snapshot `asOf` it came + from; carried-forward data is never presented as current. A deep scan runs only when a user + asks for one — opening the area never triggers it — and staleness past the stated threshold is + surfaced as a nudge, not silently repaired. +4. **This context mutates nothing.** No delete, prune, or cleanup verb exists here; reclaimable + candidates are advisory rows with rationale, each carrying a `safety` tier and a + `bytesMeaning`. (The snapshot file it owns is the sole write.) +5. **The runtime census is ephemeral.** It is computed per request and never persisted; a stale + process table is never replayed as liveness. +6. **Bounded walkers.** Symlinks are never followed; depth and entry caps apply; one unreadable + subtree degrades that node to unknown without discarding siblings or aborting the scan. +7. **Single-flight deep scans.** Concurrent refresh requests attach to the in-flight scan; two + scans never race each other or double-write the snapshot. +8. **No spend, no activity, no learning facts.** Tokens/cost stay in Historical usage; session + lifecycle/evidence stays in Observability; learning counters stay in Project intelligence. + Cross-links, not duplication. +9. **Discovery supplies paths, not measurements.** The project catalog contributes candidate + locations only; everything rendered is measured by this domain's own collectors. A path that + cannot be resolved is reported as `unresolved` and never fabricated, which makes `everSeen` a + lower bound rather than a guess. +10. **Catalog counts are observed inventory.** They state what is on disk per host surface, + never desired state, and never upgrade Integration management's ownership facts. +11. **LOC is approximate and says so.** Extension-bucketed line counts with stated exclusions; + no rendering presents them as authoritative. Lines belong to **languages** only: frameworks, + SDKs and tools are detected by presence and carry no line count in the payload at all, so no + surface can double-count the same bytes under a framework's name. +12. **Same delivery protections as the rest of the dashboard.** Loopback, token auth, GET-only, + zero egress; the absolute-path exception is deliberate, documented, and content-free. +13. **Every platform reports what it can, and names what it cannot.** No section is switched off + for a platform. Where a per-platform probe fails, that field alone degrades with its reason + and the row keeps every other measurement; a row is never dropped for being unattributable. +14. **Sizes are counted once, and every exclusion is stated.** In the largest-consumers ranking + nested roots are counted at the outermost row only; enclosed rows are breakdowns that explain + their parent rather than competing with it; a residual row makes every breakdown add up to + its parent; absent roots are listed as absent rather than ranked as zero-byte consumers; and + a category excluded by default — project working trees — states its exclusion in the payload. +15. **Reclaimable tiers are never summed together.** `regenerable` and `review` are separate + promises with separate totals; `combined` is `null` by design. Only `bytesMeaning: + 'candidate'` rows are summable, and a tier whose rows describe overlapping paths reports + unknown-with-reason rather than counting the same bytes twice. + +## Ubiquitous language additions + +These terms are merged into [Ubiquitous language](ubiquitous-language.md); that glossary is +normative and this table restates it for readers of this document. + +| Term | Meaning | +|------|---------| +| Footprint | The machine-resource cost of the toolchain: install bytes, runtime CPU/RSS, retained-data bytes, deployed inventory. The context's name; the surface is **System** | +| FootprintSnapshot | The persisted result of a deep scan: `asOf`, completeness, and the deep-tier section models (install, storage, catalog, projects, consumers) | +| Measurement | A value plus provenance: measured (with `asOf`), carried forward, or unknown-with-reason — unknown is never zero | +| Partial measurement | A measured value known to be a lower bound because a contributing subtree was unreadable or capped; rendered as "≥ N" | +| HostInstallation | One managed tool's install facts: version, install method, root, tree bytes, native addons | +| RuntimeCensus | The ephemeral point-in-time table of live host processes, daemons, and machine denominators | +| StorageNode | One node in the category → host → project → session breakdown: bytes + file count | +| ReclaimableCandidate | An advisory row naming reclaimable space, its path, and its rationale — never an action | +| Safety tier | A candidate's `regenerable` (the owning tool refetches it) or `review` (plausible, not safe to call removable). The two are totalled separately and never combined | +| Bytes meaning | Whether a candidate's bytes are the `candidate` subset it is about, or the `installed` size at that path offered as context on a review row | +| Consumer root | A ranked top-level storage root. Nested rows are `breakdown`s of it, plus a synthesized residual, so bytes are counted once | +| Ever seen / on disk | `everSeen` is every project any host ever recorded a session in, deletions included; `onDisk` is the measurable subset. Different questions, never one number | +| Unresolved project | A transcript directory whose project path neither a declared `cwd` nor a filesystem-verified decode can name. Reported as such, never given a fabricated path; it makes `everSeen` a lower bound | +| Stack detection | Per-project `languages` (which carry lines) and `stack` — frameworks, SDKs, tools — which carry presence only, plus the unrecognized tail of extensions and dependency names the registry could not name | +| CatalogItem | A deduplicated deployed artifact (skill, agent, command, plugin, MCP server) with a per-host presence matrix | +| ProjectFootprint | One project's size facts: approximate LOC by language, tree/`.git`/`node_modules` bytes, last activity, and an optional git-remote web link ("local only" when absent) | +| Deep scan | The explicit, user-triggered, single-flight full measurement pass that produces a FootprintSnapshot | +| Cheap tier | The per-request census + known-file stats + snapshot carry-forward served on every read | + +## References + +- [ADR-0025](../adr/0025-machine-footprint-metrics.md) — the decision record this domain implements +- [Context map](context-map.md) — where this context sits +- [Historical usage / Observability / Project intelligence](context-map.md) — the neighboring + contexts this domain is deliberately distinct from +- [Dashboard guide](../DASHBOARD.md) +- `src/lib/footprint/` — the collectors: `walk.mjs` (the bounded walker and the `Measurement` + vocabulary), `install.mjs`, `storage.mjs`, `runtime.mjs`, `catalog.mjs`, `projects.mjs`, + `consumers.mjs` (the ranked largest-consumers view), `project-sources.mjs` (cross-host project + discovery), `stack-registry.mjs` + `stack-detect.mjs` (languages, frameworks and the + unrecognized tail), `snapshot.mjs`, `index.mjs` +- `src/commands/system.mjs` — the CLI twin; `src/lib/live/win-process-survey.ps1` — the Windows + process survey diff --git a/docs/ddd/project-intelligence.md b/docs/ddd/project-intelligence.md index d0208f3..a630bc8 100644 --- a/docs/ddd/project-intelligence.md +++ b/docs/ddd/project-intelligence.md @@ -4,9 +4,14 @@ This document describes the domain implemented by [ADR-0024](../adr/0024-project `src/lib/dashboard/intel-history.mjs`, `src/lib/dashboard/project-discovery.mjs`, and `src/lib/live/intelligence-watch.mjs`. +> **2026-08-07 amendment:** project discovery moved to the shared census +> ([ADR-0027](../adr/0027-shared-project-census.md)). "Every ruflo-initialized project" below now +> reads "every project with memory or intelligence state, whichever host created it" — see +> [Project discovery](#project-discovery). +> > **2026-08-05 amendment:** extended from one project's telemetry, implicit and bound to the > dashboard server's own launching working directory, to a machine-wide catalog of every -> ruflo-initialized project on this machine, a machine-wide aggregate that is always shown, and an +> project on this machine, a machine-wide aggregate that is always shown, and an > explicitly selected, explicitly labeled detail project (defaulting to most-recently-active). See > [ADR-0024](../adr/0024-project-intelligence-telemetry.md)'s update note for the full amendment > record, including why the `/api/status` payload shape is a clean break rather than a preserved @@ -66,50 +71,42 @@ never a remote or unregistered one. ## Project discovery -`discoverRuvfloProjects()` (`src/lib/dashboard/project-discovery.mjs`) returns every project on -this machine ruflo has genuinely initialized — a `.claude-flow/neural/` subdirectory present, not -merely a bare `.claude-flow/` (a project that only ever ran, say, `ruflo daemon start` without ever -training or learning anything is correctly excluded) — deduplicated by resolved absolute path and -sorted most-recently-active first by `.claude-flow/neural/stats.json`'s `lastAdaptation`. - -Three sources are unioned: - -1. **Registry.** `registryWorkspaces()`, reused verbatim from `daemons.mjs` (imported, not - reimplemented), walks `~/.claude-flow/{ai-jobs.json,workspace-leases.json,repo-supervisors.json}` - for every workspace path recorded there that carries a `.claude-flow` directory. `daemons.mjs`'s - own header comment assumes ruflo 3.28+ reliably writes these; verified false on a real ruflo - 3.34.0 machine with real, populated ruflo projects — none of the three files existed. Kept as a - source (cheap, and correct wherever those files do exist), but no longer described as the - guaranteed-correct primary. -2. **Observability cross-reference.** `WorkspaceSnapshotStore` (`src/lib/live/workspace-store.mjs`, - [Observability](observability.md)) is checked for any record whose workspace carries a - genuinely resolvable absolute path. That store's own privacy sanitizers — `repositoryLabel` - rejects any path separator, `directoryLabel` rejects anything absolute-looking — mean a real - record never carries one, so this source is structurally empty by that store's own design, not - a gap. The check remains a real, defensive one rather than being skipped outright, so it starts - contributing automatically if that schema ever grows a genuine path field. -3. **Transcript content (the source that matters in practice).** Real absolute `cwd` values read - directly out of Claude and Codex transcript content under `~/.claude/projects/**/*.jsonl` and - `~/.codex/sessions/**/*.jsonl`, via the same `discoverJsonl()`/`bootstrapRecords()` functions - `live-sessions-service.mjs` already trusts for Observability's own live session tracking — flat - `record.cwd` for Claude, `record.payload.cwd` for Codex's `session_meta`/`turn_context` records. - Unlike source 2's sanitized, persisted registry, raw transcripts are not sanitized and do carry - a resolvable path — legitimately readable at this trust boundary since it's the same user, same - machine, same files Observability already parses. Bounded to the 150 most-recently-modified - transcripts per host so cost stays flat regardless of session count. On the real machine where - source 1 returned nothing, this source alone found all 4 real ruflo-initialized projects. - -Each discovered row is `{ path, label, source }`, where `label` reuses Observability's own -`resolveProjectLabel` for the same path (falling back to the bare directory name) so a project -reads identically wherever it is named, and `source` is `'registry'`, `'observability'`, -`'transcript'`, or `'both'` when a project was found by two or more sources (not necessarily -exactly two). +Intelligence does not discover projects. It consumes the **learning scope** of the shared project +census ([ADR-0027](../adr/0027-shared-project-census.md), `src/lib/project-census.mjs`), which is +the same census the System area measures directories from and the same identity Observability keys +sessions by. A project therefore means the same thing here as it does there. + +The learning scope is every project still on disk that carries **learning state** — a +`.claude-flow`, `.agentic-qe` or `.swarm` directory — regardless of which host created it. + +Two properties of that definition are deliberate and both were once wrong: + +- **Activation, not training.** The predicate this replaced required `.claude-flow/neural/`, which + answers "has ruflo trained here". That excluded every project whose memory came from agentic-qe + or swarm storage, and every project driven by Codex or OpenCode. On a real machine it reported 4 + projects where 17 had learning state. The markers above are exactly the ones the Storage context + already treats as a project's learning stores, so the two contexts cannot disagree about what a + learning store is. +- **One row per project identity, not per directory.** Sessions get recorded in a repository's + sub-directories and in ephemeral `.claude/worktrees/agent-*` checkouts. The census lists those as + the distinct directories they are — the System context measures bytes per directory — and the + learning scope folds them onto the repository root via `resolveProjectIdentity()`. Without the + fold, a picker keyed by identity silently loses every folded row. + +A merged row is anchored on the path that actually carries the learning state (the repository root +in the ordinary case), because that is the path `readIntelHistory()` reads. It keeps `paths`, every +directory that contributed, and `learningState`, which markers were found — so a project reporting +zero patterns can say *why* it reports zero rather than being indistinguishable from a failed read. + +`label` reuses Observability's own `resolveProjectLabel` so a project reads identically wherever it +is named, and the key a client echoes back as `?project=` is `resolveProjectIdentity(path).key`. ## Model ```text -discoverRuvfloProjects() -> ProjectRow[] { path, label, source } (every ruflo-initialized - project on this machine) +projectsInScope(census, 'learning') (every project on this machine + -> ProjectRow[] { path, label, paths, learningState, hosts } with memory or intelligence + state, any host — ADR-0027) .claude-flow/neural/patterns.json -> PatternStoreEntry[] { createdAt, type } .claude-flow/neural/stats.json -> GlobalLearningStats { patternsLearned, trajectoriesRecorded, diff --git a/docs/ddd/routing-and-orchestration.md b/docs/ddd/routing-and-orchestration.md index 3044c28..33b65b5 100644 --- a/docs/ddd/routing-and-orchestration.md +++ b/docs/ddd/routing-and-orchestration.md @@ -43,6 +43,24 @@ Route provenance is: Refresh may update diverged seeded routes. It does not overwrite user routes. +### Divergence and retirement + +Two different things can be wrong with a route's model, and they are separate concepts. + +A route is **diverged** when the defaults have moved past its seeded model. Which side is better is +activity-dependent, so divergence is a trade to weigh, never a lag to clear: it is reported with both +models' cost-per-task characteristics and is only ever resolved by an explicit refresh. + +A model is **retired** when its host has published a withdrawal notice for it. There is no trade — +the model stops answering — so a retired model is substituted for its replacement at the point every +dispatch path reads the policy, and seeded routes naming one are rewritten on the next sync. A route +that was substituted reports the id it replaced, so a surface never silently disagrees with the file +on disk. + +Retirement is the single case in which a `user` route is overridden, and only for the run: honoring +a pin into a withdrawn model fails the work rather than respecting the intent. The persisted value is +left exactly as the user wrote it. A model that is merely superseded is diverged, not retired. + ### Primary host The primary host determines which peer leads the default table and how missing-host status is diff --git a/docs/ddd/ubiquitous-language.md b/docs/ddd/ubiquitous-language.md index 6c4e7eb..57004d2 100644 --- a/docs/ddd/ubiquitous-language.md +++ b/docs/ddd/ubiquitous-language.md @@ -81,6 +81,63 @@ scope and at machine-wide-rollup scope alike. There is no unlabeled "this projec Intelligence: the panel always shows an explicitly selected, explicitly labeled project alongside the always-visible machine-wide rollup. See [Project intelligence](project-intelligence.md). +**Project census** — the one enumeration of this machine's projects, read from the session `cwd` +recorded in every Claude and Codex transcript plus the OpenCode session store. Every area derives +its project list from it; none discovers projects independently ([ADR-0027](../adr/0027-shared-project-census.md)). + +**Scope** — the named filter an area applies to the census, and the reason two areas can report +different totals without either being wrong. `everSeen` (all, deletions included), `onDisk` (still +resolvable), `gitRepos` (under version control) and `learning` (carries learning state). A count is +never rendered without the sentence naming its scope. + +**Learning state** — a `.claude-flow`, `.agentic-qe` or `.swarm` directory in a project: memory or +intelligence has been *activated* there, by any host. Distinct from having been *trained*, which is +what ruflo pattern counters measure and what the retired `.claude-flow/neural/` predicate required. + +**Directory scope vs project scope** — `everSeen`/`onDisk`/`gitRepos` count directories, because +directories are what have bytes and lines in them. `learning` counts projects, folding a +repository's sub-directories and its ephemeral agent worktrees onto one identity, because a project +is what a user selects. + +## Machine footprint language + +| Term | Meaning | +|------|---------| +| Footprint | The machine-resource cost of the toolchain: install bytes, runtime CPU/RSS, retained-data bytes, deployed inventory. The bounded context's name; the user-facing surface is called **System** | +| FootprintSnapshot | The persisted result of a deep scan: `asOf`, completeness, and the four deep-tier section models (install, storage, catalog, projects) | +| Measurement | A value plus provenance: measured (with `asOf`), carried forward, or unknown-with-reason — unknown is never zero | +| Partial measurement | A measured value known to be a lower bound because a contributing subtree was unreadable or capped; rendered as "≥ N", never as a total | +| HostInstallation | One managed tool's install facts: version, install method, root, tree bytes, native addons | +| RuntimeCensus | The ephemeral point-in-time table of live host processes, daemons, and machine denominators | +| StorageNode | One node in the category → host → project → session breakdown: bytes + file count | +| ReclaimableCandidate | An advisory row naming reclaimable space, its path, and its rationale — never an action | +| CatalogItem | A deduplicated deployed artifact (skill, agent, command, plugin, MCP server) with a per-host presence matrix | +| ProjectFootprint | One project's size facts: approximate LOC by language, tree/`.git`/`node_modules` bytes, last activity, and an optional git-remote web link ("local only" when absent) | +| Deep scan | The explicit, user-triggered, single-flight full measurement pass that produces a FootprintSnapshot | +| Cheap tier | The per-request census + known-file stats + snapshot carry-forward served on every read | + +A measured zero is a real zero and renders as one; an unmeasured or failed figure renders as +unknown with its reason. Storage bytes and Usage tokens are different facts about the same +transcript and never substitute for each other. See [Machine footprint](machine-footprint.md). + +## Component directory language + +| Term | Meaning | +|------|---------| +| Component directory | The curated catalog of everything ak installs or configures, with editorial identity per entry | +| DirectoryEntry | One component's editorial identity: category, tagline, paragraph, links, icon, and a detection join key | +| Editorial content | Authored, versioned prose and links — the part of a card that is true regardless of machine state | +| Detection fact | An observed install/version/configured fact borrowed read-only from existing collectors, rendered only as chips | +| State chip | The card element that renders detection facts (`installed v…` / `not installed — ak setup adds it` / `configured` / `unknown`) | +| Monogram tile | The honest icon for a component with no official mark: initials on a category-hued tile | +| Register contract | The editorial writing rules (one ~50-word paragraph, plain language, active voice, no runtime claims, no superlatives) | +| Parity gate | The test asserting managed-tools registry ↔ directory completeness in both directions | +| Configured surface | A non-package thing ak sets up — MCP registrations, guidance blocks, statuslines, routing and bridge, the daemon, permission allowlists — carrying a managing command instead of package links | + +Editorial content states purpose and reads true on a machine where the component is absent; +runtime state is a chip word, never a prose word. See +[Component directory](component-directory.md). + ## Usage rules - Say **host** when referring to Claude Code, Codex, OpenCode, session drivers, leadership, or @@ -92,11 +149,17 @@ the always-visible machine-wide rollup. See [Project intelligence](project-intel - Qualify **adapter** as integration adapter or source adapter when ambiguity is possible. - Do not infer an inference provider from a transcript host alone. - Do not replace an unknown fact with a convenient default. +- Say **System** for the dashboard area and the command; say **Machine footprint** only for the + bounded context and its module directory. No user-facing string says "footprint". +- Say **About** for the dashboard area and the command; say **Component directory** only for the + bounded context. ## Persisted names - `ak host` and `ak x host` manage execution hosts and routing; they do not redefine inference providers. +- `ak system [--deep] [--json]` renders the Machine footprint collector; `ak about [--category] + [--json]` renders the Component directory. Both are read-only twins of a dashboard area. - `kit.json.integrations.hosts` records enabled hosts. Top-level `routing` records `version`, `primaryHost`, and per-activity `routes`; route entries use `provenance` and `escalation`. - Derived exports in `hosts.mjs`, `providers.mjs`, and `routing.mjs` are views, not independent diff --git a/eslint.config.mjs b/eslint.config.mjs index 2805cb3..6224dda 100644 --- a/eslint.config.mjs +++ b/eslint.config.mjs @@ -13,6 +13,11 @@ export default [ '.agents/**', 'docs/archive/**', 'coverage/**', + // Gitignored tool output, same category as coverage/: tests/ui writes + // screenshots here and debugging sessions leave browser-context scratch + // scripts behind. Linting them fails `check` for anyone who has run + // `pnpm test:ui`, over globals (document, location) that are correct there. + '.ui-artifacts/**', ], }, js.configs.recommended, diff --git a/src/commands/about.mjs b/src/commands/about.mjs new file mode 100644 index 0000000..9d1fb8b --- /dev/null +++ b/src/commands/about.mjs @@ -0,0 +1,296 @@ +// ak about — the component directory in the terminal (ADR-0026). +// +// The CLI twin of the dashboard's About area, reading the SAME frozen editorial +// data (src/lib/dashboard/about-directory.mjs). There is deliberately no second +// copy of the copy: a paragraph reviewed for the release must read identically +// in both surfaces, or one of them shipped text nobody signed off on. +// +// The editorial/detection split survives the port. Prose comes from the +// directory and never claims runtime state; the state chip is the only place a +// runtime fact appears, and it is fed by collectors that ALREADY EXIST — the +// version primitives `ak status` itself calls, and `ak status`'s own rows for +// the configured surfaces. This command adds no probe of its own. When a +// detection source fails the chip degrades to `state unknown — ` and the +// entry still renders: an unmeasured component is never drawn as absent +// (ADR-0023 / component-directory invariants 2 and 3). +import { heading, dim, bold, glyph, green, yellow } from '../lib/output.mjs'; +import { CATEGORY_ORDER, directoryEntries } from '../lib/dashboard/about-directory.mjs'; + +export const options = { + json: { type: 'boolean', default: false }, + category: { type: 'string' }, + 'no-detect': { type: 'boolean', default: false }, +}; + +export const help = `ak about — what agentic-kit installs and configures, and why + +One entry per component: what it is, what it does for you, where to read more, +and an honest state chip. Prose is authored with the release; the chip is the +only runtime fact, read from the same detection \`ak status\` uses. The dashboard's +About tab renders this identical directory. + +Usage: + ak about [entry-id] + +Options: + --category only one category: ${CATEGORY_ORDER.join(', ')} + --no-detect editorial only — resolve no state chips at all (instant) + --json emit the directory plus resolved state chips + +Examples: + ak about the whole directory, grouped by category + ak about ruflo one entry + ak about --category hosts just the agent CLIs + ak about --no-detect the authored copy alone, no detection at all + ak about --json machine-readable directory + state`; + +// The directory exports no category headings on purpose: section copy belongs to +// the page that renders it, not to the shared data. These are the terminal's. +const CATEGORY_LABELS = Object.freeze({ + hosts: 'Hosts — the agents you talk to', + 'engine-memory': 'Engine and memory — what runs underneath', + quality: 'Quality — proving the work is done', + safety: 'Safety — checking untrusted text', + knowledge: 'Knowledge — grounded answers about this stack', + kit: 'The kit itself', + configured: 'Configured for you — surfaces ak sets up', +}); + +const installed = (version, note = null) => ({ state: 'installed', version: version ?? null, note }); +const absent = () => ({ state: 'absent', version: null, note: null }); +const configured = () => ({ state: 'configured', version: null, note: null }); +const attention = (note) => ({ state: 'attention', version: null, note: String(note) }); +const unknown = (why) => ({ state: 'unknown', version: null, note: String(why) }); +// Distinct from `unknown`: nothing failed here, the user asked for no detection. +// Sharing the chip would flag all fifteen entries as if something were wrong. +const skipped = () => ({ state: 'skipped', version: null, note: 'detection skipped (--no-detect)' }); + +const reasonOf = (error) => String(error?.message ?? error); + +/** + * State chips for the packaged entries, from the primitives `ak status` already + * calls. Each source is guarded independently so one unavailable collector + * degrades one chip, never the page. Network-free by construction: every call + * here reads the filesystem or resolves a binary — the drift/latest-version + * lookups that DO reach npm are deliberately not consulted, because "is it + * installed" is a local question and About asks nothing else. + * + * @param {{ pkgRoot?: string }} input + * @returns {Promise>} + */ +async function detectPackaged({ pkgRoot }) { + const states = new Map(); + + try { + const { HOSTS, hostInstallState } = await import('../lib/providers.mjs'); + for (const host of HOSTS) { + try { + const state = await hostInstallState(host); + states.set(`hosts.${host.id}`, state.method === 'absent' + ? absent() + // An externally-installed host (mise, brew, a native installer) is + // present and ak says so — while naming the owner, because ak does + // not manage its updates (MANAGED-TOOLS "honest disowning"). + : installed(state.version, + state.method === 'external' ? 'external install — self-managed' : null)); + } catch (error) { + states.set(`hosts.${host.id}`, unknown(reasonOf(error))); + } + } + } catch (error) { + for (const id of ['claude', 'codex', 'opencode']) states.set(`hosts.${id}`, unknown(reasonOf(error))); + } + + const { installedVersion, KIT_PKG } = await import('../lib/versions.mjs'); + for (const [key, pkg] of [['ruflo', 'ruflo'], ['agentic-qe', 'agentic-qe']]) { + try { + const version = installedVersion(pkg); + states.set(key, version ? installed(version) : absent()); + } catch (error) { + states.set(key, unknown(reasonOf(error))); + } + } + + try { + const { coherence } = await import('../lib/agentdb.mjs'); + const c = coherence(); + states.set('agentdb', c.present ? installed(c.global) : absent()); + } catch (error) { + states.set('agentdb', unknown(reasonOf(error))); + } + + // aidefence and @claude-flow/security are nested under global ruflo, not + // global packages, so there is no version to read — presence is the whole + // fact ak has. Reporting the pair separately matters: security-without- + // aidefence is the state in which `security defend` silently does nothing. + try { + const { aidefencePresent, securityPresent } = await import('../lib/natives.mjs'); + const security = securityPresent(); + if (security && aidefencePresent()) states.set('security', installed(null)); + else if (security) states.set('security', attention('@claude-flow/security present, aidefence missing')); + else states.set('security', absent()); + } catch (error) { + states.set('security', unknown(reasonOf(error))); + } + + try { + const brain = await import('../lib/ruvnet-brain.mjs'); + states.set('ruvnet-brain', brain.present() ? installed(brain.installedVersion()) : absent()); + } catch (error) { + states.set('ruvnet-brain', unknown(reasonOf(error))); + } + + // The kit's own version: the globally installed copy when there is one, else + // the running checkout's manifest — the same order collectInstall resolves + // `self` in, so a linked dev install reports a version rather than "absent". + try { + let version = installedVersion(KIT_PKG); + if (!version && pkgRoot) { + const [{ readFileSync }, { default: path }] = await Promise.all([ + import('node:fs'), import('node:path'), + ]); + version = JSON.parse(readFileSync(path.join(pkgRoot, 'package.json'), 'utf8')).version ?? null; + } + states.set('self', version ? installed(version) : absent()); + } catch (error) { + states.set('self', unknown(reasonOf(error))); + } + + return states; +} + +/** + * State chips for the configured surfaces, joined from `ak status` rows exactly + * as the dashboard joins them from /api/status. A subsystem that emits NO row is + * unknown, never `configured`: an unjoined key is an unmeasured fact, not a + * satisfied one (the permission allowlist is in that position today). + */ +function detectConfigured(rows) { + const bySubsystem = new Map(); + for (const row of rows) { + if (!bySubsystem.has(row.subsystem)) bySubsystem.set(row.subsystem, []); + bySubsystem.get(row.subsystem).push(row); + } + return (subsystem) => { + const matched = bySubsystem.get(subsystem); + if (!matched?.length) return unknown(`ak status emits no '${subsystem}' row`); + const bad = matched.find((row) => row.level === 'fail') ?? matched.find((row) => row.level === 'warn'); + return bad ? attention(bad.message) : configured(); + }; +} + +/** The chip's rendered text. `installed` without a version is a real state — + * some components are nested packages with no manifest ak may read. */ +function chipText(state) { + if (state.state === 'installed') return state.version ? `installed · v${state.version}` : 'installed'; + if (state.state === 'absent') return 'not installed — ak setup adds it'; + if (state.state === 'configured') return 'configured'; + if (state.state === 'attention') return 'needs attention'; + if (state.state === 'skipped') return 'state not resolved — detection skipped'; + return `state unknown — ${state.note}`; +} + +const chipLevel = (state) => (state.state === 'installed' || state.state === 'configured' ? 'ok' + : state.state === 'attention' || state.state === 'unknown' ? 'warn' : 'none'); + +function paint(state, text) { + const level = chipLevel(state); + if (level === 'ok') return green(text); + if (level === 'warn') return yellow(text); + return dim(text); +} + +/** Greedy wrap. Long tokens (URLs) are never broken — a split URL is unusable. */ +function wrap(text, width) { + const lines = []; + let line = ''; + for (const word of String(text).split(/\s+/).filter(Boolean)) { + if (line && line.length + 1 + word.length > width) { lines.push(line); line = word; } + else line = line ? `${line} ${word}` : word; + } + if (line) lines.push(line); + return lines; +} + +function renderEntry(entry, state, width) { + const indent = ' '; + const body = Math.max(40, width - indent.length); + console.log(` ${glyph(chipLevel(state))} ${bold(entry.name)} ${paint(state, chipText(state))}`); + console.log(`${indent}${dim(entry.tagline)}`); + for (const line of wrap(entry.paragraph, body)) console.log(`${indent}${line}`); + for (const link of entry.links) console.log(`${indent}${dim(link.label.padEnd(6))} ${link.url}`); + if (entry.manage) console.log(`${indent}${dim('manage'.padEnd(6))} ${entry.manage}`); + console.log(''); +} + +/** + * @param {{ flags: Record, positionals: string[], pkgRoot?: string, + * deps?: { collectStatus?: Function, cwd?: string } }} input + */ +export async function run({ flags, positionals, pkgRoot, deps = {} }) { + const wanted = positionals[0]; + // The directory is a frozen literal, so its inferred type is a union of 15 + // distinct shapes and `entry.subsystem` is only on some of them. Widening here + // keeps the field access honest at runtime (a missing key is `undefined`, + // which is exactly the packaged/configured discriminator) without weakening + // any type the directory itself publishes. + /** @type {Array>} */ + const entries = directoryEntries().filter((entry) => ( + (!wanted || entry.id === wanted) && (!flags.category || entry.category === flags.category) + )); + + if (!entries.length) { + const what = [wanted && `id '${wanted}'`, flags.category && `category '${flags.category}'`] + .filter(Boolean).join(' and '); + console.log(`${glyph('warn')} no directory entry matches ${what}`); + console.log(dim(`ids: ${directoryEntries().map((entry) => entry.id).join(', ')}`)); + return 2; + } + + const states = new Map(); + if (flags['no-detect']) { + for (const entry of entries) states.set(entry.id, skipped()); + } else { + const packaged = await detectPackaged({ pkgRoot }); + // One `ak status` collect for every configured chip. Skipped entirely when + // no configured entry survived the filter, so `ak about ruflo` stays cheap. + /** @type {(subsystem: string) => { state: string, version: string|null, note: string|null }} */ + let configuredState = () => unknown('status collection not run'); + if (entries.some((entry) => entry.subsystem)) { + try { + const collectStatus = deps.collectStatus + ?? (await import('./status.mjs')).collect; + configuredState = detectConfigured(await collectStatus({ pkgRoot, cwd: deps.cwd ?? process.cwd() })); + } catch (error) { + const why = reasonOf(error); + configuredState = () => unknown(why); + } + } + for (const entry of entries) { + states.set(entry.id, entry.subsystem + ? configuredState(entry.subsystem) + : packaged.get(entry.detectionKey) ?? unknown(`no detection source for '${entry.detectionKey}'`)); + } + } + + if (flags.json) { + console.log(JSON.stringify({ + detection: !flags['no-detect'], + entries: entries.map((entry) => ({ ...entry, state: states.get(entry.id) })), + }, null, 2)); + return 0; + } + + const width = Math.max(60, Math.min(100, process.stdout.columns || 80)); + heading('ak about — what agentic-kit installs and configures'); + console.log(dim(' purpose is authored; presence is measured — only the chip claims runtime state')); + + for (const category of CATEGORY_ORDER) { + const inCategory = entries.filter((entry) => entry.category === category); + if (!inCategory.length) continue; + heading(CATEGORY_LABELS[category] ?? category); + console.log(''); + for (const entry of inCategory) renderEntry(entry, states.get(entry.id), width); + } + return 0; +} diff --git a/src/commands/sync.mjs b/src/commands/sync.mjs index 4983563..cbd3ec2 100644 --- a/src/commands/sync.mjs +++ b/src/commands/sync.mjs @@ -12,7 +12,7 @@ import { OPENCODE_LIFECYCLE_ADAPTER } from '../lib/opencode.mjs'; import { runLifecycle } from '../lib/adapters/lifecycle.mjs'; import { listDaemons, staleDaemons, reap } from '../lib/daemons.mjs'; import { loadKitConfig, saveKitConfig } from '../lib/config.mjs'; -import { commandHosts, applyHosts, applyProviders, hostInstallState, installHost, applyAqeRouter, seedActivityRoutesIfMultiHost, ensureCodexMcp, ensureRufloMcpInCodex, bothHostsEnabled } from '../lib/providers.mjs'; +import { commandHosts, applyHosts, applyProviders, hostInstallState, installHost, applyAqeRouter, seedActivityRoutesIfMultiHost, migrateRetiredRoutesInConfig, ensureCodexMcp, ensureRufloMcpInCodex, bothHostsEnabled } from '../lib/providers.mjs'; import { driftReport, selfDrift } from '../lib/versions.mjs'; import { RUVECTOR_PKG, managed as ruvectorManaged } from '../lib/ruvector.mjs'; import { pruneNpxStale } from '../lib/npx.mjs'; @@ -209,6 +209,20 @@ export async function run({ flags, pkgRoot }) { // eligible (e.g. aqe upgraded ≥3.13.1 since enablement), before materializing. const seed = seedActivityRoutesIfMultiHost(cfg); if (seed.seeded) { saveKitConfig(cfg); report('routing', { ok: true, changed: true, detail: `seeded ${seed.count} activities` }); } + // Retire withdrawn models from the persisted policy. Distinct from divergence + // (which stays an explicit `ak x host refresh` decision): a retired model + // stops answering, so leaving it named on disk is a scheduled failure. Only + // seeded entries are rewritten; a user pin is reported and left alone. + const retired = migrateRetiredRoutesInConfig(cfg); + if (retired.changes.length > 0) { + if (retired.changed) saveKitConfig(cfg); + for (const c of retired.changes) { + const when = c.retiresOn ? `retires ${c.retiresOn}` : 'already withdrawn'; + report('routing', c.rewritten + ? { ok: true, changed: true, detail: `${c.activity} ${c.field}: ${c.from} → ${c.to} (${when})` } + : { ok: true, changed: false, detail: `${c.activity} ${c.field} pins ${c.from} (${when}) — user pin kept; ak runs ${c.to}` }); + } + } const router = applyAqeRouter(cfg, cwd); if (router.changed || !router.ok) report('aqe router', router); const mcp = await ensureCodexMcp(cfg, cwd); diff --git a/src/commands/system.mjs b/src/commands/system.mjs new file mode 100644 index 0000000..bfdd474 --- /dev/null +++ b/src/commands/system.mjs @@ -0,0 +1,346 @@ +// ak system — the machine footprint in the terminal (ADR-0025). +// +// The CLI twin of the dashboard's System area, driving the SAME composed +// collector (src/lib/footprint/index.mjs) over the same two tiers. Reading is +// cheap and always safe: the live process census, the individually-known files, +// and whatever the last deep scan persisted, carried forward with THAT scan's +// timestamp. The expensive walk runs only under --deep, only when a human asked +// for it — never on open, never on a nudge (the nudge just says the figures are +// getting old). +// +// Every number on this page is a Measurement, and this file's whole job is to +// render one honestly. A measured zero prints as 0 because it IS zero. An +// unmeasured quantity prints the reason it is missing and NEVER a 0 (ADR-0023, +// machine-footprint invariant 2). A capped or partially-degraded walk prints +// with a `>=` because what it measured is a floor, not a total. +import { heading, info, ok, warn, dim, bold, withProgress } from '../lib/output.mjs'; +import { createSystemCollector } from '../lib/footprint/index.mjs'; +import { UNKNOWN } from '../lib/footprint/walk.mjs'; + +export const options = { + json: { type: 'boolean', default: false }, + deep: { type: 'boolean', default: false }, +}; + +export const help = `ak system — what this stack occupies on your machine + +Reads the cheap tier by default: the live agent-process census, the files that +grow fastest between scans, and the last deep scan's figures carried forward with +the date they were taken. --deep re-walks install trees, storage, the cross-host +catalog, and every discovered project, then persists the result. + +Usage: + ak system [options] + +Options: + --deep re-run the full scan now (minutes on a large machine), then persist it + --json emit the snapshot payload verbatim — the same shape /api/system serves + +Examples: + ak system install totals, runtime census, storage, catalog, projects + ak system --deep re-measure everything, then print it + ak system --json machine-readable snapshot (no scan) + ak system --deep --json re-measure, then emit the fresh snapshot`; + +const UNITS = ['B', 'KB', 'MB', 'GB', 'TB', 'PB']; + +/** Decimal units, matching the System design mock. */ +function fmtBytes(n) { + if (!Number.isFinite(n)) return String(n); + let value = Math.abs(n); + let unit = 0; + while (value >= 1000 && unit < UNITS.length - 1) { value /= 1000; unit += 1; } + const digits = unit === 0 ? 0 : value >= 100 ? 0 : value >= 10 ? 1 : 2; + return `${(n < 0 ? -value : value).toFixed(digits)} ${UNITS[unit]}`; +} + +const fmtCount = (n) => (Number.isFinite(n) ? n.toLocaleString('en-US') : String(n)); +const fmtPercent = (n) => (Number.isFinite(n) ? `${n.toFixed(1)}%` : String(n)); + +function fmtDuration(ms) { + if (!Number.isFinite(ms) || ms < 0) return String(ms); + const s = Math.floor(ms / 1000); + if (s < 60) return `${s}s`; + if (s < 3600) return `${Math.floor(s / 60)}m`; + if (s < 86_400) return `${Math.floor(s / 3600)}h ${Math.floor((s % 3600) / 60)}m`; + return `${Math.floor(s / 86_400)}d ${Math.floor((s % 86_400) / 3600)}h`; +} + +const fmtAgo = (at, now) => (Number.isFinite(at) ? `${fmtDuration(now - at)} ago` : String(at)); +const fmtStamp = (at) => (Number.isFinite(at) + ? new Date(at).toISOString().replace('T', ' ').slice(0, 16) + : 'unknown'); + +/** + * Render a Measurement as a line value. An unknown carries its reason with it — + * that reason is the whole point of the type, and dropping it here would leave + * a bare "unknown" indistinguishable from a rendering bug. + */ +function meas(value, fmt = fmtCount) { + if (!value || typeof value !== 'object') return dim('unknown — no measurement reported'); + if (value.status === UNKNOWN) return dim(`unknown — ${value.reason}`); + return `${value.partial ? '>= ' : ''}${fmt(value.value)}`; +} + +/** + * Table cells cannot carry a reason: column widths are computed on the raw text, + * so a long reason (or an ANSI escape) would wreck the alignment. The sink keeps + * every distinct reason it swallowed and the caller prints them under the table, + * so "unknown" in a cell is still traceable to why. + */ +function reasonSink() { + const seen = new Set(); + return { + cell(value, fmt = fmtCount) { + if (!value || typeof value !== 'object') { seen.add('no measurement reported'); return 'unknown'; } + if (value.status === UNKNOWN) { seen.add(value.reason); return 'unknown'; } + return `${value.partial ? '>= ' : ''}${fmt(value.value)}`; + }, + report(indent = ' ') { + for (const reason of seen) console.log(`${indent}${dim(`unknown: ${reason}`)}`); + }, + }; +} + +/** Left-aligned columns; the last column is never padded so lines do not carry + * trailing whitespace. Cells must be plain text (see reasonSink). */ +function table(headers, rows, indent = ' ') { + if (!rows.length) return; + const widths = headers.map((header, i) => Math.max( + header.length, ...rows.map((row) => String(row[i] ?? '').length), + )); + const line = (cells) => cells + .map((cell, i) => (i === cells.length - 1 ? String(cell ?? '') : String(cell ?? '').padEnd(widths[i]))) + .join(' ') + .trimEnd(); + console.log(`${indent}${dim(line(headers))}`); + for (const row of rows) console.log(`${indent}${line(row)}`); +} + +const field = (label, value) => console.log(` ${dim(label.padEnd(16))}${value}`); + +/** A deep section's provenance. Every figure inside it was taken at one instant + * and every one of them reads back as `carried-forward` (the section is always + * served from the persisted snapshot, even microseconds after --deep wrote it), + * so the scan date is stated ONCE here instead of on all several hundred lines + * — invariant 3 satisfied without burying the figures. */ +function deepHeading(name, section) { + const asOf = section?.asOf; + return heading(`${name}${Number.isFinite(asOf) ? dim(` — measured ${fmtStamp(asOf)}`) : ''}`); +} + +function renderSummary(snapshot) { + const { snapshot: snap, runtime, knownFiles } = snapshot; + heading('ak system — machine footprint'); + field('platform', snapshot.platform); + field('census', `${runtime?.ephemeral ? 'live' : 'reported'} · ${snapshot.generatedAt}`); + const present = knownFiles?.nodes?.filter((node) => node.presence === 'present').length ?? 0; + field('known files', `${present}/${knownFiles?.nodes?.length ?? 0} present`); + if (!snap?.present) { + field('deep scan', dim(`never run — ${snap?.reason ?? 'no snapshot'}`)); + } else { + const missing = snap.completeness?.missing ?? []; + field('deep scan', `${fmtStamp(snap.asOf)} (${fmtDuration(snap.ageMs)} ago)` + + (missing.length ? dim(` · ${missing.join(', ')} not measured`) : '')); + } +} + +function renderInstall(install) { + deepHeading('Install', install); + if (!install) { + info(dim('not measured yet — run: ak system --deep')); + return; + } + field('tools present', meas(install.totals?.toolsPresent)); + field('install size', meas(install.totals?.installBytes, fmtBytes)); + field('shared caches', meas(install.totals?.cacheBytes, fmtBytes)); + field('native addons', meas(install.totals?.nativeAddons)); + field('disk', `${meas(install.disk?.freeBytes, fmtBytes)} free of ` + + `${meas(install.disk?.totalBytes, fmtBytes)}`); + if (install.globalRootReason) field('npm root', dim(install.globalRootReason)); + + const sink = reasonSink(); + const rows = (install.tools ?? []).map((tool) => [ + tool.label, + tool.present ? (tool.version ? `v${tool.version}` : 'present') : 'absent', + tool.installMethod, + sink.cell(tool.bytes, fmtBytes), + tool.rootReason ?? '', + ]); + console.log(''); + table(['TOOL', 'VERSION', 'METHOD', 'SIZE', 'NOTE'], rows); + sink.report(); + const dupes = install.duplicateNatives ?? []; + if (dupes.length) info(`${dupes.length} native module(s) compiled into more than one tree`); +} + +function renderRuntime(runtime) { + heading(`Runtime${dim(' — live, never persisted')}`); + if (!runtime) { + info(dim('no census reported')); + return; + } + field('processes', meas(runtime.totals?.processCount)); + field('memory (RSS)', meas(runtime.totals?.rssBytes, fmtBytes)); + field('cpu', meas(runtime.totals?.cpuPercent, fmtPercent)); + field('daemons', `${meas(runtime.daemons?.count)} running · ${meas(runtime.daemons?.staleCount)} stale`); + field('machine', `${meas(runtime.machine?.physicalMemoryBytes, fmtBytes)} memory · ` + + `${meas(runtime.machine?.freeMemoryBytes, fmtBytes)} free · ${meas(runtime.machine?.cpuCount)} cores`); + + const census = runtime.processes; + if (census?.status === UNKNOWN) { + console.log(` ${dim(`process table unavailable — ${census.reason}`)}`); + return; + } + const rows = census?.value ?? []; + if (!rows.length) { + console.log(` ${dim('no agent processes are running')}`); + return; + } + const sink = reasonSink(); + console.log(''); + table(['HOST', 'PID', 'CPU', 'RSS', 'UPTIME', 'PROJECT'], rows.map((row) => [ + row.host, + String(row.pid), + sink.cell(row.cpuPercent, fmtPercent), + sink.cell(row.rssBytes, fmtBytes), + sink.cell(row.uptimeMs, fmtDuration), + row.project?.status === UNKNOWN ? 'unattributed' : (row.project?.value?.label ?? 'unattributed'), + ])); + sink.report(); + // `project` degrades per process (a cwd the platform will not disclose); its + // reason lives on the row, not in the numeric sink above. + for (const reason of new Set(rows.filter((row) => row.project?.status === UNKNOWN) + .map((row) => row.project.reason))) { + console.log(` ${dim(`unattributed: ${reason}`)}`); + } +} + +function renderStorage(storage) { + deepHeading('Storage', storage); + if (!storage) { + info(dim('not measured yet — run: ak system --deep')); + return; + } + field('total', `${meas(storage.totals?.bytes, fmtBytes)} · ${meas(storage.totals?.files)} files`); + + const sink = reasonSink(); + console.log(''); + table(['CATEGORY', 'SIZE', 'FILES'], (storage.categories ?? []).map((category) => [ + category.label, + sink.cell(category.bytes, fmtBytes), + sink.cell(category.files), + ])); + sink.report(); + + const reclaimables = storage.reclaimables ?? []; + if (reclaimables.length) { + const advisory = reasonSink(); + console.log(''); + console.log(` ${dim('reclaimable (advisory only — ak system removes nothing)')}`); + table(['CANDIDATE', 'SIZE', 'CLEANUP'], reclaimables.map((row) => [ + row.label, advisory.cell(row.bytes, fmtBytes), row.cleanupHint ?? '—', + ])); + advisory.report(); + } +} + +function renderCatalog(catalog) { + deepHeading('Catalog', catalog); + if (!catalog) { + info(dim('not measured yet — run: ak system --deep')); + return; + } + const kinds = catalog.kinds ?? []; + field('deduplicated', kinds.map((kind) => `${meas(catalog.counts?.[kind])} ${kind}`).join(' · ')); + + const sink = reasonSink(); + console.log(''); + table(['HOST', ...kinds.map((kind) => kind.toUpperCase())], (catalog.hosts ?? []).map((host) => [ + host, ...kinds.map((kind) => sink.cell(catalog.perHost?.[host]?.[kind])), + ])); + sink.report(); + if (catalog.degraded?.length) info(dim(`unreadable surfaces: ${catalog.degraded.join(', ')}`)); + if (catalog.truncated?.length) { + info(dim(`capped surfaces (counts are floors): ${catalog.truncated.join(', ')}`)); + } +} + +function renderProjects(projects, now) { + deepHeading('Projects', projects); + if (!projects) { + info(dim('not measured yet — run: ak system --deep')); + return; + } + field('discovered', `${meas(projects.count)}${projects.truncated ? dim(' · list truncated') : ''}`); + if (!projects.locMeasured) field('lines of code', dim('not measured in this scan')); + + const sink = reasonSink(); + console.log(''); + table(['PROJECT', 'SIZE', 'LOC', 'LAST ACTIVE'], (projects.projects ?? []).map((project) => [ + project.label, + sink.cell(project.totalBytes, fmtBytes), + sink.cell(project.loc?.total), + sink.cell(project.lastActivity, (at) => fmtAgo(at, now)), + ])); + sink.report(); +} + +/** The staleness nudge — the ONLY thing that ever suggests a rescan. It never + * triggers one: a deep walk costs minutes, so it stays a human's decision. */ +function renderNudge(snap) { + if (!snap?.present) { + info(`no deep scan on this machine yet — run: ${bold('ak system --deep')}`); + } else if (snap.stale) { + warn(`deep figures are ${fmtDuration(snap.ageMs)} old — refresh with: ${bold('ak system --deep')}`); + } +} + +/** + * @param {{ flags: Record, + * deps?: { collector?: ReturnType, + * cwd?: string, now?: () => number } }} input + */ +export async function run({ flags, deps = {} }) { + const collector = deps.collector ?? createSystemCollector({ cwd: deps.cwd ?? process.cwd() }); + const now = deps.now ?? Date.now; + + let scan = null; + if (flags.deep) { + // refreshDeep never rejects by contract; it reports failure in its result so + // a partly-completed scan still yields the sections that DID finish. + scan = await withProgress('deep scan', () => collector.refreshDeep(), { + // The ticker owns a stdout line via \r-rewrites — it must never interleave + // with a --json payload. + tty: process.stdout.isTTY && !flags.json, + }); + } + + // The collector assembles the deep sections by spread, so the inferred return + // type cannot name install/storage/catalog/projects. The wire shape is the + // contract (ADR-0025); widening here reads it without restating it. + /** @type {Record} */ + const snapshot = await collector.read(); + + if (flags.json) { + console.log(JSON.stringify(snapshot, null, 2)); + return scan && !scan.ok ? 1 : 0; + } + + renderSummary(snapshot); + if (scan) { + if (scan.ok) ok(`deep scan complete in ${fmtDuration(snapshot.scan?.durationMs ?? 0)}`); + else warn(`deep scan failed: ${scan.error}`); + // A scan that measured everything but could not write the file is still a + // successful measurement — say which of the two happened. + if (scan.ok && scan.error) warn(scan.error); + } + renderInstall(snapshot.install); + renderRuntime(snapshot.runtime); + renderStorage(snapshot.storage); + renderCatalog(snapshot.catalog); + renderProjects(snapshot.projects, now()); + console.log(''); + renderNudge(snapshot.snapshot); + return scan && !scan.ok ? 1 : 0; +} diff --git a/src/lib/daemons.mjs b/src/lib/daemons.mjs index f82fd39..447d2ea 100644 --- a/src/lib/daemons.mjs +++ b/src/lib/daemons.mjs @@ -14,11 +14,15 @@ const alive = (pid) => { /** Known-workspace discovery from ruflo's machine-level registries * (~/.claude-flow/*.json record workspaces; each workspace has - * .claude-flow/daemon.pid + daemon-state.json with startedAt). Exported - * (visibility-only change, behavior unchanged) so project-discovery.mjs can - * reuse it verbatim as its guaranteed-correct primary source instead of - * reimplementing the same registry walk. */ -export function registryWorkspaces() { + * .claude-flow/daemon.pid + daemon-state.json with startedAt). + * + * Module-private again: it was exported so the (now retired) + * project-discovery.mjs could reuse it verbatim as a project-discovery + * source. Project discovery moved to the shared census (ADR-0027), which + * reads session transcripts rather than these registries — the registries do + * not exist on every machine, which is precisely why they made a poor + * discovery source. Only daemon accounting reads them now. */ +function registryWorkspaces() { const out = new Set(); const reg = path.join(home, '.claude-flow'); for (const f of ['ai-jobs.json', 'workspace-leases.json', 'repo-supervisors.json']) { diff --git a/src/lib/dashboard-server.mjs b/src/lib/dashboard-server.mjs index d4f20fe..d29f9ba 100644 --- a/src/lib/dashboard-server.mjs +++ b/src/lib/dashboard-server.mjs @@ -31,6 +31,13 @@ // GET /api/usage → the usage Aggregate MINUS sessions[] (ADR-0009) // GET /api/sessions → the session list, filtered + paginated // GET /api/session/:id → one transcript, secrets masked SERVER-side +// GET /api/system → the machine-footprint payload (ADR-0025): the cheap +// tier (runtime census + known-file stats, TTL-cached +// ~60s) merged with the last persisted deep snapshot, +// carried forward with ITS asOf. `?refresh=deep` starts +// or attaches to the single-flight deep scan and returns +// immediately with progress state; `&trees=1|0` sets +// whether that scan walks project working trees. // // The status rows are gathered by SHELLING OUT to the installed CLI // (`node bin/agentic-kit.mjs status --json`) so we never duplicate status.mjs's @@ -45,11 +52,13 @@ import fs from 'node:fs'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { execFile } from 'node:child_process'; -import { driftReport, selfDrift } from './versions.mjs'; +import { driftReport, selfDrift, installedVersion } from './versions.mjs'; +import { HOSTS, collectIntegrationFacts } from './providers.mjs'; +import { globalRoot } from './paths.mjs'; import { drift as ruvnetBrainDrift } from './ruvnet-brain.mjs'; import { drift as ruvectorDrift, managed as ruvectorManaged } from './ruvector.mjs'; import { loadKitConfig } from './config.mjs'; -import { resolveRoutes, routingSummary, ACTIVITIES } from './routing.mjs'; +import { resolveRoutes, routingSummary, divergedRoutes, retirementOf, ACTIVITIES } from './routing.mjs'; import { renderPage } from './dashboard/page.mjs'; import { requestRejection } from './dashboard/request-security.mjs'; import { tokenMatches } from './admin-server.mjs'; @@ -59,7 +68,7 @@ import { sseChannel, reserveClientSlot, clientGone } from './dashboard/sse.mjs'; // here because readIntelHistory() already composes it (as `.healthRing`, // forwarded verbatim below); a bare `readHealthRing` import would be unused. import { readIntelHistory, readMachineWideIntel } from './dashboard/intel-history.mjs'; -import { discoverRuvfloProjects } from './dashboard/project-discovery.mjs'; +import { projectCensus, projectsInScope, describeScope } from './project-census.mjs'; import { resolveProjectIdentity, safeProjectKey } from './live/project-label.mjs'; import { TRANSCRIPT_ROOTS, @@ -199,7 +208,7 @@ function resolveSelectedProject(projects, rawParam) { * discovery reads real machine-global state (~/.claude-flow/*.json, * ~/.config/agentic-kit/observability-workspaces.json) that a test must * never depend on. */ -function buildProjectSnapshotCache(discoverProjectsFn, machineWideIntelFn) { +function buildProjectSnapshotCache(discoverProjectsFn, machineWideIntelFn, readCensusFn) { let snapshot = null; let fetchedAt = 0; return () => { @@ -208,13 +217,39 @@ function buildProjectSnapshotCache(discoverProjectsFn, machineWideIntelFn) { const projects = discoverProjectsFn().map((project) => ( { ...project, key: keyForProject(project) } )); - snapshot = { projects, machineWide: machineWideIntelFn(projects) }; + // The scope counts that let the panel say what it counted (ADR-0027). + // Null when a caller injected its own discoverProjects: that seam yields + // a project list with no census behind it, and reporting a machine-wide + // total derived from a fixture would be a fabricated number. Absent is + // the honest reading, and the renderer omits the line rather than + // printing a zero (ADR-0023). + const census = typeof readCensusFn === 'function' ? readCensusFn() : null; + snapshot = { projects, machineWide: machineWideIntelFn(projects), census }; fetchedAt = now; } return snapshot; }; } +/** The default discovery seam: one census walk, exposed two ways — the + * learning-scoped project rows the Intelligence panel aggregates, and the + * scope counts that explain how that number relates to what the other tabs + * show. Paired so the corpus is walked ONCE per snapshot rather than twice. */ +function censusBackedDiscovery() { + let last = null; + return { + discover: () => { + last = projectCensus(); + return projectsInScope(last, 'learning'); + }, + readCensus: () => (last ? { + everSeen: last.everSeen, onDisk: last.onDisk, + gitRepos: last.gitRepos, learning: last.learning, + complete: last.complete, + } : null), + }; +} + /** Assemble the full /api/status payload. */ async function collectData({ cwd, fetchStatus, projectParam, getProjectSnapshot }) { let status; @@ -244,6 +279,7 @@ async function collectData({ cwd, fetchStatus, projectParam, getProjectSnapshot try { if (ruvectorManaged()) drift = foldRuvectorDrift(drift, await ruvectorDrift()); } catch { /* banner is best-effort — the subsystem card still carries the ruvector row */ } + try { drift = await foldKnownVersions(drift); } catch { /* version chips degrade, nothing else */ } } // Machine-wide Intelligence (clean break from the old cwd-hardcoded, @@ -251,7 +287,7 @@ async function collectData({ cwd, fetchStatus, projectParam, getProjectSnapshot // `getProjectSnapshot()` is the shared, TTL-cached {projects, machineWide} // read; `projectParam` is the raw ?project= query value, resolved the exact // same way /api/live/intelligence resolves it (resolveSelectedProject). - const { projects, machineWide } = getProjectSnapshot(); + const { projects, machineWide, census } = getProjectSnapshot(); const selected = resolveSelectedProject(projects, projectParam); const selectedHistory = selected ? readIntelHistory(selected.path) : EMPTY_SELECTED_HISTORY; @@ -306,6 +342,18 @@ async function collectData({ cwd, fetchStatus, projectParam, getProjectSnapshot patternStore: selectedHistory.patternStore, graph: selectedHistory.graph, machineWide, + // census — how this panel's project count relates to the counts the + // other tabs show (ADR-0027). `scope` names the filter Intelligence + // applies; `counts` are the same machine census under every scope, so + // a user can see that 14 and 48 and this number are three questions + // rather than three answers. Null when discovery was injected — there + // is no census behind a fixture, and inventing one would be worse + // than saying nothing. + census: census ? { + scope: 'learning', + note: describeScope('learning'), + counts: census, + } : null, }, routing: routingPayload(), }; @@ -318,15 +366,28 @@ export function routingPayload(cfg = loadKitConfig()) { const policy = cfg.routing?.routes ?? {}; if (!Object.keys(policy).length) return null; const routes = resolveRoutes(policy); + // Two DIFFERENT signals, deliberately kept apart on the wire so the panel can + // say different things about them (see RETIRED_MODELS in routing.mjs): + // retiredFrom — the host withdrew this model; ak already substituted it, so + // the row shows what will actually run. Actionable but not a + // choice. + // diverged — a seeded route the defaults moved past. A trade to weigh, + // cleared only by an explicit `ak x host refresh`. + const diverged = new Map(divergedRoutes(policy).map((d) => [d.activity, d])); return { primaryHost: cfg.routing?.primaryHost ?? 'claude', summary: routingSummary(policy), routes: ACTIVITIES.map((activity) => { const r = routes[activity]; + const d = diverged.get(activity); return { activity, host: r.host, model: r.model ?? '', provenance: r.provenance, akOriginated: !!r.akOriginated, escalation: (r.escalation ?? []).map((e) => e.host), + ...(r.retiredFrom + ? { retiredFrom: r.retiredFrom, retiresOn: retirementOf(r.retiredFrom)?.retiresOn ?? null } + : {}), + ...(d ? { diverged: { defaultModel: d.defaultModel, defaultNote: d.defaultNote, currentNote: d.currentNote } } : {}), }; }), }; @@ -368,6 +429,76 @@ export function foldRuvectorDrift(drift, r) { }]; } +// `collectIntegrationFacts` probes each host's binary for its version, which +// costs ~300ms and must never ride the dashboard's ~30s poll. Host versions +// change only when someone installs or upgrades a CLI, so an in-process window +// this long is generous and still catches an upgrade within one coffee break. +const HOST_FACTS_TTL_MS = 5 * 60_000; +let hostFactsCache = null; + +async function cachedHostFacts(now) { + if (hostFactsCache && now - hostFactsCache.at < HOST_FACTS_TTL_MS) return hostFactsCache.hosts; + const facts = await collectIntegrationFacts({ cwd: process.cwd(), cfg: loadKitConfig() }); + hostFactsCache = { at: now, hosts: facts?.hosts ?? {} }; + return hostFactsCache.hosts; +} + +/** + * Fold in the installed versions `driftReport()` structurally cannot report, so + * every About card states a version rather than some silently omitting one. + * + * driftReport only walks npm-managed globals. Two managed components are + * knowable but sit outside it: a host installed by mise/brew/the native + * installer (present and versioned, just not an npm global), and agentdb (a + * real npm global, but deliberately pinned to ruflo's bundled version rather + * than latest, so MANAGED-TOOLS.md excludes it from the update banner). + * + * Both fold in with `outdated: false` and no `latest`, which is inert for the + * banner — `noticeHtml` renders only entries where `outdated` is true — while + * giving the version chip the structured fact it needs. Nothing here parses a + * version out of a status row's prose; every value is a structured probe. + * Existing entries always win, so this can only ever add. + * @param {Array<{pkg:string, installed?:string|null, latest?:string|null, + * outdated?:boolean}>|null} drift the drift array, in the shape driftReport() + * and the selfDrift/brain/ruvector folds all already emit + * @param {{ now?: number, hostFacts?: Record }} [deps] test seam + * @returns {Promise>} the input array plus the folded entries; incoming + * entries are passed through untouched, so their fields stay as optional as + * whichever fold produced them + */ +export async function foldKnownVersions(drift, { now = Date.now(), hostFacts } = {}) { + const out = [...(drift ?? [])]; + const seen = new Set(out.map((d) => d?.pkg).filter(Boolean)); + const add = (pkg, installed) => { + if (!pkg || !installed || seen.has(pkg)) return; + seen.add(pkg); + out.push({ pkg, installed, latest: null, outdated: false }); + }; + const hosts = hostFacts ?? await cachedHostFacts(now); + for (const host of HOSTS) add(host.pkg, hosts?.[host.id]?.version); + add('agentdb', installedVersion('agentdb')); + add('@claude-flow/aidefence', bundledVersion('ruflo', '@claude-flow/aidefence')); + return out; +} + +/** The version of a package that ships INSIDE another rather than as its own + * global — aidefence lives in ruflo's dependency tree, so `installedVersion` + * cannot see it and its card was the last one silently missing a version. + * Both npm layouts are checked directly (hoisted beside the host, then nested + * under it) because Node's resolver refuses `/package.json` whenever the + * package publishes an `exports` map, which aidefence does. */ +function bundledVersion(hostPkg, pkg) { + const root = globalRoot(); + for (const file of [ + path.join(root, pkg, 'package.json'), + path.join(root, hostPkg, 'node_modules', pkg, 'package.json'), + ]) { + try { return JSON.parse(fs.readFileSync(file, 'utf8')).version ?? null; } catch { /* try next */ } + } + return null; +} + // ───────────────────────────────────────────────────────────────────────────── // Usage tab plumbing (ADR-0009). Everything here is either a pure guard or a // thin adapter over usage-index.mjs — no parsing, no aggregation, no pricing. @@ -412,6 +543,20 @@ function clampInt(raw, fallback, min, max) { return Math.min(max, Math.max(min, n)); } +// Observability History's window control — day-count approximations (a +// calendar "month" is treated as 30 days, matching clampDays' own plain-day +// semantics rather than adding calendar-aware month math for a browse filter). +const HISTORY_WINDOW_DAYS = { + '1d': 1, '7d': 7, '14d': 14, '1mo': 30, '3mo': 90, '6mo': 180, '1y': 365, +}; +/** ?window= → an epoch-ms cutoff, or null for "all time". Unknown/ + * missing tokens fall back to 14d, the same default clampDays uses. */ +function windowToSinceMs(raw, now = Date.now()) { + if (raw === 'all') return null; + const days = Object.hasOwn(HISTORY_WINDOW_DAYS, raw) ? HISTORY_WINDOW_DAYS[raw] : HISTORY_WINDOW_DAYS['14d']; + return now - days * 86_400_000; +} + function sendJson(res, status, payload) { res.writeHead(status, { 'content-type': 'application/json; charset=utf-8', 'cache-control': 'no-store', 'x-content-type-options': 'nosniff' }); res.end(JSON.stringify(payload)); @@ -487,6 +632,21 @@ function sendTranscriptJson(res, status, payload) { res.end(JSON.stringify(payload)); } +/** Lazily bind the machine-footprint collector (ADR-0025), for the same reason + * lazyUsage is lazy: the System area pulls in five walkers plus the runtime + * survey, and a panel that never opens that tab must not pay for them. One + * instance per dashboard server — it owns the cheap tier's TTL cache and the + * deep scan's single-flight slot, and a second instance would defeat both. */ +function lazySystem(systemOptions = {}) { + let instancePromise; + return async () => { + instancePromise ||= import('./footprint/index.mjs').then(({ createSystemCollector }) => ( + createSystemCollector(systemOptions) + )); + return instancePromise; + }; +} + /** Load the collector only when Live is requested. This keeps dashboard startup * cheap and permits tests/embedders to inject a source without touching real * transcript stores. */ @@ -511,7 +671,8 @@ function lazyLive(liveOptions = {}) { * any|Promise)|any, * intelClientBuffer?: number, intelMaxClients?: number, * discoverProjects?: () => Array<{ path: string, label: string, source?: string }>, - * machineWideIntel?: (projects: Array) => any }} [opts] + * machineWideIntel?: (projects: Array) => any, + * system?: any, systemOptions?: any }} [opts] * @returns {Promise<{ url: string, urlWithToken: string, port: number, token: string, close: () => Promise }>} */ export function startDashboard({ @@ -520,7 +681,7 @@ export function startDashboard({ liveOptions = {}, liveIdleMs = 30_000, transcripts, transcriptOptions = {}, transcriptClientBuffer = 64, transcriptMaxClients = 16, intelWatch, intelClientBuffer = 256, intelMaxClients = 32, - discoverProjects, machineWideIntel, + discoverProjects, machineWideIntel, system, systemOptions = {}, } = {}) { const provide = fetchStatus || shellOutStatus(cwd); const usageApi = usage || lazyUsage(); @@ -528,6 +689,22 @@ export function startDashboard({ // real ~/.config through this route. Lazy for the same reason lazyUsage is. const provideLimits = limits || (async () => (await import('./quota.mjs')).readLimits()); const provideLive = typeof live === 'function' ? live : live ? async () => live : lazyLive(liveOptions); + // Same injection contract as `live`: a function is called to produce the + // collector, a value is reused verbatim (tests hand over a fake so the route + // never walks the real machine), and the default is the lazy real one. The + // collector must expose read/refreshDeep — checked at use, not here, so an + // unopened System tab still costs nothing. + const provideSystem = typeof system === 'function' + ? system : system ? async () => system : lazySystem({ cwd, ...systemOptions }); + let systemPromise; + const getSystem = async () => { + const collector = await (systemPromise ||= Promise.resolve().then(provideSystem)); + if (!collector || typeof collector.read !== 'function' + || typeof collector.refreshDeep !== 'function') { + throw new TypeError('system collector must implement read and refreshDeep'); + } + return collector; + }; let transcriptServicePromise; const provideTranscripts = typeof transcripts === 'function' ? transcripts : transcripts ? async () => transcripts : async () => { @@ -605,9 +782,12 @@ export function startDashboard({ // server, used by BOTH /api/status and /api/live/intelligence so they can // never scan the machine independently or disagree on results within the // same TTL window. + const censusBacked = censusBackedDiscovery(); + const injectedDiscovery = typeof discoverProjects === 'function'; const getProjectSnapshot = buildProjectSnapshotCache( - typeof discoverProjects === 'function' ? discoverProjects : discoverRuvfloProjects, + injectedDiscovery ? discoverProjects : censusBacked.discover, typeof machineWideIntel === 'function' ? machineWideIntel : readMachineWideIntel, + injectedDiscovery ? null : censusBacked.readCensus, ); // ── /api/live/intelligence pool plumbing ────────────────────────────────── @@ -765,6 +945,25 @@ export function startDashboard({ return; } + if (url === '/api/live/history') { + // On-demand, not part of the SSE stream: a fresh scan per request, same + // privacy scrubbing (publicLivePayload) as every other live surface. + try { + const service = await getLive(); + if (typeof service.historySnapshot !== 'function') { + sendJson(res, 501, { error: 'history browsing not supported by this live service' }); + return; + } + const sinceMs = windowToSinceMs(query.get('window')); + sendJson(res, 200, publicLivePayload(await service.historySnapshot({ sinceMs }))); + } catch { + sendJson(res, 503, { error: 'live telemetry unavailable' }); + } finally { + scheduleLiveIdle(); + } + return; + } + if (url === '/api/live/events') { // Reserve the client-cap slot BEFORE the first await (getLive() may do a // dynamic import() + service.start()). Concurrent requests arriving @@ -1203,6 +1402,55 @@ export function startDashboard({ return; } + // ── System / machine footprint (ADR-0025). Lazy, like Usage above. ────── + // + // DELIBERATE DIVERGENCE from publicLivePayload, documented in ADR-0025 §7 + // and the machine-footprint DDD's delivery section: this payload carries + // ABSOLUTE PATHS and is NOT run through publicLivePayload's leaf-only + // reduction. That reduction exists because a path is incidental provenance + // on a *session* payload; here the path IS the answer — a storage + // breakdown that hides where the bytes live answers nothing. The exposure + // is bounded by what the collectors structurally cannot do: file CONTENTS + // are never read (stat metadata, directory entries, and manifest names + // only), so no transcript, prompt, or tool payload can reach this route to + // leak. Delivery protections are otherwise identical to every route above + // — loopback bind, per-session token auth, no-store, nosniff, zero egress. + if (url === '/api/system') { + try { + const collector = await getSystem(); + // ORDER IS LOAD-BEARING: assemble the payload BEFORE starting a scan. + // The deep collectors are synchronous, so the first phase occupies the + // event loop the moment it gets a turn — and `read()` awaits, which + // hands it that turn. Starting first therefore made the *initiating* + // request wait out the phase it had just kicked off (measured: 9s), + // which is precisely the hang the progress state exists to avoid. + const payload = await collector.read(); + if (query.get('refresh') === 'deep') { + // Start-or-attach and answer NOW. The collector's single flight means + // a second refresh joins the running scan rather than racing it, and + // it never rejects — the catch guards an injected collector that does + // not honour that contract, so a bad one cannot take the process down + // with an unhandled rejection. + // `trees` is a MEASUREMENT parameter, not a view filter: project + // working trees are only walked when it is set, and one large + // repository outweighs every shared cache combined — so the ranking + // has to be re-measured, not re-sorted. Absent means "keep whatever + // the collector already defaults to". + const trees = query.get('trees'); + Promise.resolve(collector.refreshDeep( + trees == null ? undefined : { includeProjectTrees: trees === '1' }, + )).catch(() => {}); + // The payload predates the start by microseconds; re-stamp the live + // scan block so this response reads "running", not "idle". + if (typeof collector.scanState === 'function') payload.scan = collector.scanState(); + } + sendJson(res, 200, payload); + } catch (e) { + sendJson(res, 503, { error: 'system footprint unavailable', reason: String(e && e.message || e) }); + } + return; + } + if (url === '/api/sessions') { try { const agg = await usageApi.readIndex({ days: clampDays(query.get('days')) }); diff --git a/src/lib/dashboard/about-directory.mjs b/src/lib/dashboard/about-directory.mjs new file mode 100644 index 0000000..fd7c321 --- /dev/null +++ b/src/lib/dashboard/about-directory.mjs @@ -0,0 +1,343 @@ +// The About area's component directory: the authored editorial identity of every +// component agentic-kit installs or configures (ADR-0026 / docs/ddd/component-directory.md). +// +// THE EDITORIAL/DETECTION SPLIT is why this module exists and why it is pure data. +// Editorial content — tagline, paragraph, links, icon, category, order — is authored, +// reviewed, and versioned WITH THE RELEASE. Detection facts — installed, version, +// install method, configured — are observed at render by collectors that already exist +// (/api/status rows, the drift array, the managed-tools machinery). The directory owns +// only the first kind and joins the second client-side; it probes nothing, fetches +// nothing, and adds no endpoint. Importing this module must therefore have zero side +// effects so a unit test can assert the whole catalog without a machine to measure. +// +// PROSE NEVER CLAIMS RUNTIME STATE. "Installed", "running", "healthy" are chip words fed +// by detection. Every paragraph below reads true on a machine where the component is +// absent — that is the structural guarantee behind the state chip, not a style +// preference: if copy could assert presence, a stale sentence would contradict an honest +// chip on the same card. Purpose is timeless; presence is measured. +// +// The register contract governing this copy (one paragraph, ~50 words, plain language, +// active voice, no superlatives, no runtime claims) is specified in the DDD doc. + +/** Curated, stable reading order. Never derived from popularity, size, or health: + * the things a new user already recognizes come first, infrastructure after, and the + * kit's own card sits near the end because it is the caretaker, not the point. */ +export const CATEGORY_ORDER = Object.freeze([ + 'hosts', 'engine-memory', 'quality', 'safety', 'knowledge', 'kit', 'configured', +]); + +// Official marks are reused BYTE-IDENTICALLY from the marks the dashboard already ships +// for Observability's session rows (`hostIcon()` in live/client.mjs, `sourceHostIcon()` +// in client.mjs), which dispatch on these exact host ids. Only the three hosts have a +// genuinely official, already-shipped mark; everything else is an explicit monogram +// tile — initials on a category hue — because a fabricated logo would misrepresent a +// project we do not speak for. `hue` names a CSS custom property the About stylesheet +// defines; the directory names the token, it does not pick the colour. +const OFFICIAL = (ref) => Object.freeze({ kind: 'official', ref }); +const MONOGRAM = (ref, hue) => Object.freeze({ kind: 'monogram', ref, hue }); + +const link = (kind, label, url) => Object.freeze({ kind, label, url }); + +// detectionKey joins a packaged entry to the managed-tools detection facts. The keys are +// heterogeneous because the facts are: hosts resolve through the host registry +// (`hosts.`), agentdb / agentic-qe / ruvnet-brain have their own `ak status` +// subsystem row, aidefence is reported by the `security` row, and the kit reports as +// `self`. `npmPackage` is carried alongside because the version chip's other source — +// the drift array — is keyed by package name, and deriving that from a link URL would be +// a parsing trick rather than a stated fact. Configured surfaces have no detectionKey: +// they are not packages, so they carry the status `subsystem` row their chip joins on +// plus the `manage` command that changes them. `subsystem` names the row a chip SHOULD +// join; where no such row is emitted yet the chip must degrade to `unknown` rather than +// assume `configured` — an unjoined key is an unmeasured fact, not a satisfied one. +const ENTRIES = Object.freeze([ + Object.freeze({ + id: 'claude-code', + category: 'hosts', + name: 'Claude Code', + tagline: "Anthropic's coding agent, in your terminal.", + paragraph: + 'Point it at a repository and talk to it: it reads your code, proposes and makes ' + + 'changes, runs your tests, and explains each step as it goes. It is one of the ' + + 'agent CLIs — coding assistants that live in your terminal — the rest of this ' + + 'toolkit exists to strengthen.', + links: Object.freeze([ + link('github', 'GitHub', 'https://github.com/anthropics/claude-code'), + link('npm', 'npm', 'https://www.npmjs.com/package/@anthropic-ai/claude-code'), + link('docs', 'Docs', 'https://docs.claude.com/en/docs/claude-code/overview'), + ]), + icon: OFFICIAL('claude'), + detectionKey: 'hosts.claude', + npmPackage: '@anthropic-ai/claude-code', + }), + Object.freeze({ + id: 'codex', + category: 'hosts', + name: 'Codex', + tagline: "OpenAI's coding agent — a second pair of eyes.", + paragraph: + "OpenAI's take on the same idea: an agent that works alongside you in the terminal " + + 'on real repositories. When both it and Claude Code are enabled, agentic-kit runs ' + + "them as peers — either can lead a job, review the other's output, or pick up a " + + 'step the other stalled on.', + links: Object.freeze([ + link('github', 'GitHub', 'https://github.com/openai/codex'), + link('npm', 'npm', 'https://www.npmjs.com/package/@openai/codex'), + link('docs', 'Docs', 'https://developers.openai.com/codex/cli'), + ]), + icon: OFFICIAL('codex'), + detectionKey: 'hosts.codex', + npmPackage: '@openai/codex', + }), + Object.freeze({ + id: 'opencode', + category: 'hosts', + name: 'OpenCode', + tagline: 'An open-source coding agent, run as a worker.', + paragraph: + 'A community-built agent CLI from the OpenCode project. agentic-kit treats it as an ' + + 'opt-in worker: you route particular jobs to it by name, its sessions appear ' + + 'alongside the others in Observability, and it never takes the lead on a job ' + + 'unless you say so.', + links: Object.freeze([ + // The upstream repository moved from sst/opencode; github.com/sst/opencode still + // resolves only by redirect, so the canonical owner is named here directly. + link('github', 'GitHub', 'https://github.com/anomalyco/opencode'), + link('npm', 'npm', 'https://www.npmjs.com/package/opencode-ai'), + link('docs', 'Docs', 'https://opencode.ai/docs/'), + ]), + icon: OFFICIAL('opencode'), + detectionKey: 'hosts.opencode', + npmPackage: 'opencode-ai', + }), + Object.freeze({ + id: 'ruflo', + category: 'engine-memory', + name: 'ruflo', + tagline: 'The orchestration engine under your agents.', + paragraph: + 'Gives your agents what a single session cannot: decisions remembered across ' + + "restarts, teams of specialist agents working one task in parallel (a 'swarm'), " + + 'hooks that send each job to the agent suited to it, and security checks along ' + + 'the way. agentic-kit installs it and keeps it healthy.', + links: Object.freeze([ + // The upstream repository was renamed from ruvnet/claude-flow; npm still records + // the old URL, so the canonical name is stated here rather than derived. + link('github', 'GitHub', 'https://github.com/ruvnet/ruflo'), + link('npm', 'npm', 'https://www.npmjs.com/package/ruflo'), + link('docs', 'Docs', 'https://github.com/ruvnet/ruflo/tree/main/docs'), + ]), + icon: MONOGRAM('rf', '--hue-engine'), + detectionKey: 'ruflo', + npmPackage: 'ruflo', + }), + Object.freeze({ + id: 'agentdb', + category: 'engine-memory', + name: 'agentdb', + tagline: 'Where what your agents learn is stored.', + paragraph: + 'The store behind that memory: a single local file holding what agents recorded — ' + + 'decisions, state, and the reasons behind them — searchable by meaning as well as ' + + 'by keyword, with the links between entries kept too. agentic-kit pins its version ' + + 'to the one ruflo ships, so the two cannot drift apart.', + links: Object.freeze([ + link('github', 'GitHub', 'https://github.com/ruvnet/agentdb'), + link('npm', 'npm', 'https://www.npmjs.com/package/agentdb'), + link('docs', 'Docs', 'https://github.com/ruvnet/agentdb/tree/main/docs'), + ]), + icon: MONOGRAM('db', '--hue-engine'), + detectionKey: 'agentdb', + npmPackage: 'agentdb', + }), + Object.freeze({ + id: 'agentic-qe', + category: 'quality', + name: 'agentic-qe', + tagline: 'A quality-engineering fleet you can call on.', + paragraph: + "Specialist agents for the unglamorous half of shipping: writing tests in your " + + "project's own framework, finding which untested code carries the most risk, " + + 'spotting tests that pass and fail at random, and gating a change on those ' + + "results. 'The agent says it works' becomes something you can check.", + links: Object.freeze([ + link('github', 'GitHub', 'https://github.com/proffesor-for-testing/agentic-qe'), + link('npm', 'npm', 'https://www.npmjs.com/package/agentic-qe'), + link('docs', 'Docs', 'https://github.com/proffesor-for-testing/agentic-qe#readme'), + ]), + icon: MONOGRAM('qe', '--hue-quality'), + detectionKey: 'agentic-qe', + npmPackage: 'agentic-qe', + }), + Object.freeze({ + id: 'aidefence', + category: 'safety', + name: 'aidefence + security', + tagline: 'Checks untrusted text before your agents act on it.', + paragraph: + "A pasted issue, a fetched web page, a stranger's README — any of them can carry " + + "instructions aimed at your agent rather than at you ('prompt injection'). " + + 'aidefence scans text for those patterns, for jailbreak attempts, and for exposed ' + + 'personal data; @claude-flow/security supplies the primitives ruflo\'s security ' + + 'commands are built on.', + links: Object.freeze([ + link('github', 'GitHub', + 'https://github.com/ruvnet/ruflo/tree/main/v3/@claude-flow/aidefence'), + link('npm', 'npm', 'https://www.npmjs.com/package/@claude-flow/aidefence'), + // No upstream doc site exists for these two packages; ak's own troubleshooting + // page is the honest "where to read more", not a stand-in for one. + link('docs', 'Docs', + 'https://github.com/pacphi/agentic-kit/blob/main/docs/TROUBLESHOOTING.md'), + ]), + icon: MONOGRAM('ad', '--hue-safety'), + detectionKey: 'security', + npmPackage: '@claude-flow/aidefence', + }), + Object.freeze({ + id: 'ruvnet-brain', + category: 'knowledge', + name: 'RuvNet Brain', + tagline: 'Answers about this stack, from its real source.', + paragraph: + "These tools change faster than any model's training data, so an agent asked about " + + 'them tends to guess. The Brain is a local knowledge base built from those ' + + "projects' real source, and it answers with the file path each answer came from — " + + 'so you can check the claim yourself.', + links: Object.freeze([ + link('github', 'GitHub', 'https://github.com/stuinfla/ruvnet-brain'), + link('npm', 'npm', 'https://www.npmjs.com/package/ruvnet-brain'), + link('docs', 'Docs', 'https://isovision.ai/ruvnet-brain/'), + ]), + icon: MONOGRAM('B', '--hue-knowledge'), + detectionKey: 'ruvnet-brain', + npmPackage: 'ruvnet-brain', + }), + Object.freeze({ + id: 'agentic-kit', + category: 'kit', + name: 'agentic-kit', + tagline: 'The caretaker for everything above.', + paragraph: + 'One command sets this collection up, one command repairs it after an upgrade moves ' + + 'something, and this dashboard shows what all of it is doing. When a piece drifts ' + + 'from the version it should be on, breaks, or disappears, ak names the problem and ' + + 'the command that fixes it.', + links: Object.freeze([ + link('github', 'GitHub', 'https://github.com/pacphi/agentic-kit'), + link('npm', 'npm', 'https://www.npmjs.com/package/@pacphi/agentic-kit'), + link('docs', 'Docs', 'https://github.com/pacphi/agentic-kit/tree/main/docs'), + ]), + icon: MONOGRAM('ak', '--hue-kit'), + detectionKey: 'self', + npmPackage: '@pacphi/agentic-kit', + }), + Object.freeze({ + id: 'mcp-registrations', + category: 'configured', + name: 'MCP registrations', + tagline: 'Your tools, plugged in once for every project.', + paragraph: + 'Agents reach outside tools through MCP, the plug-in protocol for AI tooling. ak ' + + "registers this stack's MCP servers once at your user level, so every repository " + + 'you open gets them without per-project setup, and lets you switch off tool groups ' + + 'you would rather not have. Run `ak x mcp pick` to change it.', + links: Object.freeze([]), + icon: MONOGRAM('M', '--info'), + subsystem: 'mcp', + manage: 'ak x mcp pick', + }), + Object.freeze({ + id: 'guidance-blocks', + category: 'configured', + name: 'Guidance blocks', + tagline: 'House rules your agents read at startup.', + paragraph: + 'Short managed sections inside CLAUDE.md and AGENTS.md that tell each agent how this ' + + 'stack fits together and which tool to reach for. ak writes only between its own ' + + 'markers, so anything you wrote stays untouched, and refreshes the text as the ' + + 'tools change. Run `ak sync` to reapply it.', + links: Object.freeze([]), + icon: MONOGRAM('G', '--info'), + subsystem: 'blocks', + manage: 'ak sync', + }), + Object.freeze({ + id: 'statuslines', + category: 'configured', + name: 'Statuslines', + tagline: 'A live footer showing what a session costs.', + paragraph: + 'A footer line inside Claude Code and Codex showing the model in use, what the ' + + "session has spent so far, and how much of your plan's limit is left — so cost " + + 'stays visible while you work instead of arriving later. Run `ak sync` when an ' + + 'upgrade wipes it.', + links: Object.freeze([]), + icon: MONOGRAM('S', '--info'), + subsystem: 'statusline', + manage: 'ak sync', + }), + Object.freeze({ + id: 'dual-host-routing', + category: 'configured', + name: 'Dual-host routing & bridge', + tagline: 'Each kind of work goes to the better host.', + paragraph: + 'With Claude Code and Codex both enabled, ak records which host handles which kind ' + + 'of work — coding, testing, review, security — and wires each one to reach the ' + + 'other as a tool, so a job can be handed across mid-flight. You pick the table and ' + + 'which host leads: `ak host pick`.', + links: Object.freeze([]), + icon: MONOGRAM('R', '--info'), + subsystem: 'routing', + manage: 'ak host pick', + }), + Object.freeze({ + id: 'background-daemon', + category: 'configured', + name: 'Background daemon', + tagline: 'Local workers that keep things tidy between sessions.', + paragraph: + 'A background process ruflo runs between your sessions for upkeep — learning from ' + + 'finished work, tidying up — staffed by local workers that cost nothing. Anything ' + + 'that would spend money on a model stays opt-in, and each daemon expires on its ' + + 'own. List or stop them with `ak x daemon-gc`.', + links: Object.freeze([]), + icon: MONOGRAM('D', '--info'), + subsystem: 'daemons', + manage: 'ak x daemon-gc', + }), + Object.freeze({ + id: 'permission-allowlist', + category: 'configured', + name: 'Permission allowlist', + tagline: 'Fewer prompts for the tools you already trust.', + paragraph: + "Pre-approved command patterns for this stack's own tools, so routine calls stop " + + 'asking your permission every time. Every rule is shown to you during setup and ' + + "written into your project's settings file, and ak strips out any rule it finds " + + 'there that it never disclosed. Run `ak setup` to review them.', + links: Object.freeze([]), + icon: MONOGRAM('P', '--info'), + // `ak status` emits no `permissions` row today; until one exists this chip reads + // unknown. Naming the row it would join is the honest placeholder — inventing a + // green `configured` for an unmeasured surface is the failure this avoids. + subsystem: 'permissions', + manage: 'ak setup', + }), +]); + +/** The whole directory in curated order. Frozen: a card renderer reads it, a parity test + * asserts over it, and neither may edit the release's authored copy in place. */ +export function directoryEntries() { + return ENTRIES; +} + +/** One entry by its stable id, or null — never a partial object, so a caller that + * mistypes an id gets an obvious absence rather than a card with empty prose. */ +export function entryById(id) { + return ENTRIES.find((e) => e.id === id) || null; +} + +/** Entries in one category, in curated order; an unknown category yields []. */ +export function entriesByCategory(category) { + return ENTRIES.filter((e) => e.category === category); +} diff --git a/src/lib/dashboard/client.mjs b/src/lib/dashboard/client.mjs index d8a9814..bb09e53 100644 --- a/src/lib/dashboard/client.mjs +++ b/src/lib/dashboard/client.mjs @@ -1,4 +1,5 @@ import { CAT, RANK, PREF, esc, catOf, groupRows, rowLine, groupCard, gridHtml, noticeHtml } from './groups.mjs'; +import { directoryEntries } from './about-directory.mjs'; // The classification/grouping/card/notice logic lives in ./groups.mjs (pure — // unit-testable in node without a DOM). Here those exact function sources and @@ -11,12 +12,19 @@ const objLiteral = (o) => `{${Object.entries(o).map(([k, v]) => `${ident(k) ? k const CAT_JS = objLiteral(CAT); const RANK_JS = objLiteral(RANK); const PREF_JS = JSON.stringify(PREF); +// The About area's authored directory, serialized into the bundle rather than +// fetched: it is release-versioned editorial content, not machine state, so it +// ships with the page and needs no endpoint of its own (ADR-0026). Runtime +// facts arrive from the /api/status payload the dashboard already polls and are +// joined in the browser. +const ABOUT_JS = JSON.stringify(directoryEntries()); export const JS = ` (function(){ "use strict"; var root=document.documentElement; var LS="ak-dash-theme", LS_TAB="ak-dash-tab", LS_OVERVIEW="ak-dash-overview-view"; + var LS_SYSTEM="ak-dash-system-view", LS_ABOUT_NUDGE="ak-dash-about-nudge"; // Dashboard-wide session token (ADR-0014). Bootstrap is idempotent and // duplicated from live-view.mjs's copy (separate diff --git a/src/lib/dashboard/project-discovery.mjs b/src/lib/dashboard/project-discovery.mjs deleted file mode 100644 index b6f0daf..0000000 --- a/src/lib/dashboard/project-discovery.mjs +++ /dev/null @@ -1,254 +0,0 @@ -// project-discovery.mjs — machine-wide discovery of every ruflo-initialized -// project on this machine, for a later Intelligence-panel feature that will -// aggregate across all of them and let a user pick one to view. This module's -// ONLY concern is discovery: it returns plain { path, label, source } rows -// and does no aggregation, no reading of neural PATTERNS, and no dependency -// on intel-history.mjs (sorting reads .claude-flow/neural/stats.json -// directly, deliberately duplicating intel-history.mjs's tiny `?? 0` default -// rather than importing it, per this module's brief). -// -// "Ruflo has genuinely initialized" a directory means it has a -// .claude-flow/neural/ subdirectory (fs.existsSync check below) — a bare -// .claude-flow/ with no neural/ subdir is NOT enough; that's how a project -// that only ever ran e.g. `ruflo daemon start` without ever training/learning -// anything gets correctly excluded. -// -// ── Two sources, unioned ──────────────────────────────────────────────────── -// -// SOURCE 1 (primary, guaranteed-correct): registryWorkspaces() from -// ../daemons.mjs, reused VERBATIM (imported, not reimplemented) — it walks -// ~/.claude-flow/{ai-jobs.json,workspace-leases.json,repo-supervisors.json} -// and returns every workspace path recorded there that has a .claude-flow -// directory. That function was previously module-private in daemons.mjs; it -// has been given a bare `export` (visibility only, zero behavior change) so -// it could be imported here instead of duplicated — the only edit made -// outside this file, and made because the frozen contract for this module -// requires verbatim reuse, which is otherwise impossible to satisfy. -// -// SOURCE 2 (cross-reference, structurally empty today): Observability's own -// persisted notion of known projects — WorkspaceSnapshotStore -// (../live/workspace-store.mjs) reading -// ~/.config/agentic-kit/observability-workspaces.json. Investigated against -// the REAL file on this machine (229 session records, schemaVersion 1) and -// against workspace-store.mjs's `safeRecord`/`safeWorkspace` sanitizers: -// every record's workspace carries only `repositoryLabel` (a single -// path-separator-free leaf name, e.g. "emailibrium") and `directoryLabel` (a -// relative fragment like "backend" or "repo root", with absolute-looking or -// ".."-traversal strings explicitly rejected by workspace-store.mjs's own -// `workspaceText()` sanitizer). That sanitizer exists specifically so this -// store never retains a real filesystem path — it is a deliberately -// privacy-safe label store, not a path index. A scan of the real file found -// ZERO absolute-path-shaped strings in any record, and structurally cannot -// find one as long as that schema holds. The code below still does a real, -// defensive per-record check — via resolvableAbsolutePath() — rather than -// skipping source 2 outright, so it starts pulling genuine extra projects -// automatically if that schema ever grows a real cwd/path field. -// -// SOURCE 3 (primary in practice): raw transcript content under -// ~/.claude/projects/**/*.jsonl and ~/.codex/sessions/**/*.jsonl, read via -// native-transcript-discovery.mjs's discoverJsonl()/bootstrapRecords() — the -// exact same functions live-sessions-service.mjs already trusts for -// Observability's live session tracking. Unlike source 2's *persisted, -// privacy-sanitized* registry, these are the raw session transcripts -// themselves: legitimately readable at this trust boundary (same user, same -// machine, same files Observability already parses), and unlike the -// sanitized store they DO carry a real absolute cwd — flat `record.cwd` for -// Claude records, `record.payload.cwd` for Codex's session_meta/turn_context -// records. Bounded to the MAX_TRANSCRIPT_FILES most-recently-modified -// transcripts (discoverJsonl's own recency sort) so this stays cheap -// regardless of how many total sessions a project has accumulated; dedup -// against sources 1/2 happens for free through record()'s existing -// resolved-path Map. -// -// ── Real-machine result the above implies ─────────────────────────────────── -// ~/.claude-flow/{ai-jobs.json,workspace-leases.json,repo-supervisors.json} -// do not exist on this machine, so source 1 (registryWorkspaces()) returns -// an empty Set here — despite daemons.mjs's own header comment assuming -// ruflo 3.28+ reliably writes those files (this machine runs ruflo 3.34.0 -// and does not have them; a per-project .claude-flow/daemon.pid still -// exists, so daemons DO run here, just without that specific central -// registry). Source 2 is structurally empty per its own privacy design -// (above). Source 3 is what actually makes discoverRuvfloProjects() return -// real projects on this machine today — verified against this repo's own 4 -// real ruflo-initialized projects. -// -// ── Testability ────────────────────────────────────────────────────────── -// registryWorkspaces() takes no arguments and reads a `home` binding that -// paths.mjs captures ONCE from os.homedir() at module-load time — there is -// no runtime-overridable home-dir seam on the real function to inject -// against without fragile pre-import env-var tricks. WorkspaceSnapshotStore, -// by contrast, already takes its file path as a plain constructor argument -// (see live-sessions-service.mjs's own `options.workspaceFile ?? ...` -// pattern). So rather than inventing a parallel home-dir-resolver override, -// discoverRuvfloProjects() accepts optional { registryWorkspaces, -// resolveProjectLabel, observabilityFile, readObservabilityRecords } -// overrides — each defaulting to the real import — which lets tests -// substitute a fake registry sweep and a fixture observability file without -// touching HOME or module caching at all. readObservabilityRecords is a -// second, narrower seam on source 2 specifically: WorkspaceSnapshotStore's -// own sanitizer makes it IMPOSSIBLE to get an absolute path into a record -// even via a crafted fixture file (repositoryLabel rejects any path -// separator, directoryLabel rejects anything absolute-looking), so the -// merge/"both"-tagging logic is exercised by injecting already-sanitized- -// looking records directly; the realistic "skip" behavior is exercised via -// observabilityFile against the real WorkspaceSnapshotStore, matching what -// actually happens on a real machine. Source 3 accepts { claudeProjectsRoot, -// codexSessionsRoot, discoverJsonl, bootstrapRecords, maxTranscriptFiles } -// overrides on the same principle — each defaulting to the real -// claudeDir()/codexDir() paths and the real native-transcript-discovery.mjs -// functions, so tests can point at fixture transcript trees without -// touching HOME. -import fs from 'node:fs'; -import path from 'node:path'; -import { registryWorkspaces } from '../daemons.mjs'; -import { resolveProjectLabel } from '../live/index.mjs'; -import { WorkspaceSnapshotStore } from '../live/workspace-store.mjs'; -import { bootstrapRecords, discoverJsonl } from '../live/native-transcript-discovery.mjs'; -import { claudeDir, codexDir, observabilityWorkspacePath } from '../paths.mjs'; -import { readJson } from '../settings.mjs'; - -const MAX_TRANSCRIPT_FILES = 150; - -/** True only when `candidate` has real ruflo learning state, not merely a - * .claude-flow directory. */ -function hasNeuralState(candidate) { - return fs.existsSync(path.join(candidate, '.claude-flow', 'neural')); -} - -/** Canonicalize for dedup so a symlinked/relative variant of the same - * project never appears twice. Falls back to path.resolve when the target - * can't be realpath'd (e.g. permissions), matching project-label.mjs's own - * fallback convention. */ -function resolvePath(candidate) { - try { return fs.realpathSync.native(candidate); } - catch { return path.resolve(candidate); } -} - -/** Reuse Observability's own label for the same path so a project reads - * identically here and there; falls back to the bare directory name if - * resolveProjectLabel can't be used for any reason. */ -function labelFor(resolveProjectLabelImpl, resolved) { - try { - const label = resolveProjectLabelImpl(resolved); - if (typeof label === 'string' && label && label !== 'unknown') return label; - } catch { /* fall through to basename */ } - return path.basename(resolved) || resolved; -} - -/** Look for a genuinely resolvable absolute filesystem path inside a - * WorkspaceSnapshotStore record's workspace. Per the header notes above, - * the real schema never carries one (repositoryLabel/directoryLabel are - * sanitized labels, not paths) — this returns null in that case rather than - * reconstructing anything from a label. */ -function resolvableAbsolutePath(workspace) { - if (!workspace || typeof workspace !== 'object') return null; - for (const candidate of [workspace.directoryLabel, workspace.repositoryLabel]) { - if (typeof candidate === 'string' && path.isAbsolute(candidate) && fs.existsSync(candidate)) { - return candidate; - } - } - return null; -} - -/** Default source-2 reader: the real WorkspaceSnapshotStore against the real - * (or injected) file. Kept as its own named function so tests can override - * just this seam (readObservabilityRecords) when they need to exercise the - * merge logic with record shapes the store's own sanitizer would otherwise - * make unreachable. */ -function readObservabilityRecords(file) { - return new WorkspaceSnapshotStore(file).records(); -} - -/** Source-3 candidates: real absolute cwd values read directly out of Claude - * and Codex transcript content under `root`, via the exact discoverJsonl/ - * bootstrapRecords functions live-sessions-service.mjs already trusts for - * Observability. Bounded to the maxFiles most-recently-modified transcripts - * (discoverJsonl's own recency sort) so a project with hundreds of sessions - * costs no more to scan than one with a handful — dedup happens for free - * downstream in record()'s resolved-path Map, so returning the same cwd - * from many sessions in one project is harmless, not wasted work avoided - * here on purpose. Returns [] on any read/parse failure for a given file - * (bootstrapRecords already degrades that way) rather than aborting the - * whole scan over one bad transcript. */ -function transcriptCwdCandidates(root, adapter, { maxFiles, discoverJsonl: discoverJsonlImpl, bootstrapRecords: bootstrapRecordsImpl }) { - const files = discoverJsonlImpl(root, { maxDepth: 6, maxFiles, accept: () => true }); - const candidates = []; - for (const file of files) { - const records = bootstrapRecordsImpl(file, adapter); - for (const rec of records) { - const cwd = adapter === 'codex' ? rec?.payload?.cwd : rec?.cwd; - if (typeof cwd === 'string' && cwd) candidates.push(cwd); - } - } - return candidates; -} - -/** .claude-flow/neural/stats.json's lastAdaptation, read directly (no - * intel-history.mjs dependency, per this module's discovery-only brief). - * Missing/unreadable/non-numeric defaults to 0, oldest-first among ties. */ -function lastAdaptationOf(projectPath) { - const stats = readJson(path.join(projectPath, '.claude-flow', 'neural', 'stats.json')); - const value = stats?.lastAdaptation; - return typeof value === 'number' && Number.isFinite(value) ? value : 0; -} - -/** - * Every project on this machine ruflo has genuinely initialized, deduplicated - * by resolved absolute path and sorted most-recently-active first. - * @returns {Array<{ path: string, label: string, source: 'registry'|'observability'|'transcript'|'both' }>} - * 'both' means 2+ sources agree on the same resolved path, not literally - * exactly two. - */ -export function discoverRuvfloProjects({ - registryWorkspaces: registryWorkspacesImpl = registryWorkspaces, - resolveProjectLabel: resolveProjectLabelImpl = resolveProjectLabel, - observabilityFile = observabilityWorkspacePath(), - readObservabilityRecords: readObservabilityRecordsImpl = readObservabilityRecords, - claudeProjectsRoot = path.join(claudeDir(), 'projects'), - codexSessionsRoot = path.join(codexDir(), 'sessions'), - discoverJsonl: discoverJsonlImpl = discoverJsonl, - bootstrapRecords: bootstrapRecordsImpl = bootstrapRecords, - maxTranscriptFiles = MAX_TRANSCRIPT_FILES, -} = {}) { - const projects = new Map(); // resolved absolute path -> row - - const record = (candidate, tag) => { - if (typeof candidate !== 'string' || !candidate || !path.isAbsolute(candidate)) return; - if (!hasNeuralState(candidate)) return; - const resolved = resolvePath(candidate); - const existing = projects.get(resolved); - if (existing) { - if (existing.source !== tag) existing.source = 'both'; - return; - } - projects.set(resolved, { path: resolved, label: labelFor(resolveProjectLabelImpl, resolved), source: tag }); - }; - - // Source 1 — the guaranteed-correct registry sweep. - for (const workspace of registryWorkspacesImpl()) record(workspace, 'registry'); - - // Source 2 — cross-reference Observability's persisted workspace snapshots. - let snapshotRecords; - try { snapshotRecords = readObservabilityRecordsImpl(observabilityFile) ?? []; } - catch { snapshotRecords = []; } - for (const entry of snapshotRecords) { - const candidate = resolvableAbsolutePath(entry?.workspace); - if (candidate) record(candidate, 'observability'); - } - - // Source 3 — real cwd values read out of raw Claude/Codex transcript - // content. In practice, the primary source of real results on a machine - // where source 1's central registry files don't exist (see header notes). - const transcriptOpts = { maxFiles: maxTranscriptFiles, discoverJsonl: discoverJsonlImpl, bootstrapRecords: bootstrapRecordsImpl }; - for (const candidate of transcriptCwdCandidates(claudeProjectsRoot, 'claude', transcriptOpts)) { - record(candidate, 'transcript'); - } - for (const candidate of transcriptCwdCandidates(codexSessionsRoot, 'codex', transcriptOpts)) { - record(candidate, 'transcript'); - } - - const rows = [...projects.values()]; - const lastAdaptation = new Map(rows.map((row) => [row.path, lastAdaptationOf(row.path)])); - rows.sort((a, b) => lastAdaptation.get(b.path) - lastAdaptation.get(a.path)); - return rows; -} diff --git a/src/lib/dashboard/styles.mjs b/src/lib/dashboard/styles.mjs index 3c571cf..cefe084 100644 --- a/src/lib/dashboard/styles.mjs +++ b/src/lib/dashboard/styles.mjs @@ -17,8 +17,21 @@ export const CSS = ` --line:rgba(255,255,255,.09); --line-2:rgba(255,255,255,.17); --accent:#0a84ff; --accent-soft:rgba(10,132,255,.16); --ok:#30d158; --warn:#ff9f0a; --fail:#ff453a; --info:#98989d; --purple:#bf5af2; + --ok-soft:rgba(48,209,88,.16); --material:rgba(16,16,18,.72); --shadow:0 1px 2px rgba(0,0,0,.4),0 12px 32px -20px rgba(0,0,0,.9); + /* About: category hues for monogram tiles. IDENTITY, not a data series — + they say "this card belongs to that family", never "this value is larger". + Named by the directory module (about-directory.mjs's icon.hue), defined + here, so the data never picks a colour. */ + --hue-engine:#7d7aff; --hue-quality:#30d158; --hue-safety:#ff9f0a; + --hue-knowledge:#bf5af2; --hue-kit:#0a84ff; + /* System: the four-step categorical series. Four is the cap — past that the + steps stop being separable at 3:1 against this panel, which is why the + Storage donut is capped at four slices and every chart carries labels + rather than relying on colour alone. --dim is de-emphasis (free space, + "other", reinstallable overhead), never a category. */ + --s1:#3987e5; --s2:#d95926; --s3:#199e70; --s4:#c98500; --dim:#48484a; } :root[data-theme="light"]{ --bg:#f5f5f7; --panel:#ffffff; --panel-2:#f2f2f7; --raised:#ffffff; --thumb:#ffffff; @@ -26,8 +39,14 @@ export const CSS = ` --line:rgba(60,60,67,.12); --line-2:rgba(60,60,67,.22); --accent:#007aff; --accent-soft:rgba(0,122,255,.12); --ok:#34c759; --warn:#ff9500; --fail:#ff3b30; --info:#8e8e93; --purple:#af52de; + --ok-soft:rgba(52,199,89,.14); --material:rgba(249,249,251,.78); --shadow:0 1px 2px rgba(0,0,0,.05),0 12px 30px -22px rgba(0,0,0,.22); + /* Light-theme steps of the same two families — darkened where the dark-theme + value would not clear 3:1 against a #ffffff panel. */ + --hue-engine:#5e5ce6; --hue-quality:#1baf7a; --hue-safety:#eb6834; + --hue-knowledge:#af52de; --hue-kit:#007aff; + --s1:#2a78d6; --s2:#eb6834; --s3:#1baf7a; --s4:#eda100; --dim:#c7c7cc; } @media (prefers-color-scheme:light){ :root:not([data-theme]){ color-scheme:light; } @@ -330,6 +349,27 @@ body.gated .band,body.gated .tabbar,body.gated main{display:none} } .strip-head{display:flex; align-items:baseline; justify-content:space-between; gap:12px; margin-bottom:14px} .strip-title{font-size:16px; font-weight:600; letter-spacing:-.014em; margin:0} + +/* Panel-level collapse. A real