Skip to content

llmdoc V3: CLI-as-runtime rewrite (@tokenroll/llmdoc), plain-directory topics, git-native validity - #33

Merged
Disdjj merged 24 commits into
mainfrom
v3
Aug 24, 2026
Merged

llmdoc V3: CLI-as-runtime rewrite (@tokenroll/llmdoc), plain-directory topics, git-native validity#33
Disdjj merged 24 commits into
mainfrom
v3

Conversation

@Disdjj

@Disdjj Disdjj commented Aug 24, 2026

Copy link
Copy Markdown
Member

Implements the V3 milestone from #32. Design docs with the full decision log live in docs/v3-design/; the landing report with diffs vs the original spec drafts is in this comment.

What's in here (21 commits)

CLI runtime — npm @tokenroll/llmdoc (published 3.1.0, 3.1.1 pending)

  • Progressive retrieval: tree / index / show / search / context with token budgets + cursors
  • Validity: status / delta / validate / fingerprint / init-state — git-revision anchored, no custom hashes
  • One-step finalize: commit (validate gate → pathspec-scoped docs commit → fingerprint → meta follow-up commit)
  • Hooks: hook session-start/stop/compact (fail-open, silent outside llmdoc projects)
  • Maintenance: new / mv (reference rewrite + rollback) / prune --report / upgrade
  • Web viewer: serve — topics topology graph, document graph, per-doc freshness (127.0.0.1 only)
  • 36 tests incl. regressions for non-ASCII paths, ancestor-dir-named-llmdoc moves, hostile YAML, supply-chain-safe npx

Knowledge model

  • .mdx = Markdown + YAML front matter (description/kind required, code.paths for impact mapping) + single <CodeRef /> whitelist
  • Root singletons + one-level plain-directory topics; no index/entry nodes anywheretree aggregates summaries
  • meta.json = validity ledger only; kinds reduced to architecture/guide/reference; reflections replaced by folding lessons into topic docs on update

Plugin surface

  • Repo root is the canonical Claude Code plugin; commands/ merged into skills/ (five SKILL.md incl. operating skill; /llmdoc:init etc. unchanged; upgrade keeps disable-model-invocation)
  • Two agents only (investigator/recorder); recorder never hand-edits meta.json
  • Codex surface regenerated via acplugin (conversion gaps filed as fix: PreCompact hook degradation, model:inherit passthrough, AGENTS.md hook section formatting acplugin#9); Codex install: codex plugin marketplace add TokenRollAI/llmdoc
  • All invocations use the scoped npx @tokenroll/llmdoc — zero wrong-package risk vs the bare npm name

Dogfooded twice

  • This repo: V2 startup pack → 7 docs / cold-start map ≈ 124 tokens
  • tool-bridge (9-package monorepo, 24 V2 docs): full V2→V3 migration; caught 8 stale-doc claims; 17 docs passed validate first try; the 5 tooling frictions it exposed are all fixed in this PR

Closes the implementation phases 1–10 and 12 of #32 (phase 11 V2→V3 upgrade is CLI-inventory + recorder-driven, validated in the tool-bridge migration).

Disdjj added 24 commits August 24, 2026 05:19
TypeScript npm package (bin: llmdoc) that owns all mechanical work:
- progressive retrieval: tree / index / show / search / context
- validity state: status / delta / validate / fingerprint (revision-based)
- hooks: session-start / stop / compact (fail-open, silent without llmdoc)
- maintenance: new / mv (git mv + reference rewrite + rollback) / prune --report / upgrade inventory
- web viewer: serve (topics topology + document graph + freshness, 127.0.0.1 only)

Topics are plain directories (no index.mdx entry nodes); topic summaries
are aggregated from document front matter. 32 tests, output schemas
validated at runtime, prompt-budget checks in scripts/.
- commands/ merged into skills/ (Claude unified commands into skills):
  five SKILL.md files — llmdoc operating skill + init/update/prune/upgrade
  explicit workflows; upgrade keeps disable-model-invocation
- two agent roles only (investigator / recorder); worker & reflector removed
- recorder never hand-edits meta.json: all ledger changes go through the CLI
- write protocol: clean check -> write -> validate gate -> commit write-set
  -> fingerprint -> amend meta; rollback is git checkout, spelled out per command
- hooks.json delegates SessionStart/Stop/PreCompact to npx llmdoc hook *
- V2 startup pack, references/, templates/, watermark state all deleted
Snapshot of the Codex-compatible surface (.agents/skills, .codex/agents,
converted from the Claude canonical surface). NOTE: canonical prompts have
changed since this conversion (commands->skills merge, index-node removal,
recorder meta.json rule); re-run ACPlugin conversion before release and
verify against tests/parity-checklist.md.
- docs/v3-design/: six-part design (knowledge model, meta/validity, CLI,
  workflows, packaging) with decision log and diffs vs the issue #32 spec
- llmdoc/: this repo's own V3 dogfood surface — root architecture.mdx plus
  three plain-directory topics (cli-runtime, workflows, plugin-packaging);
  no index entry nodes, summaries aggregate via llmdoc tree
- bare 'llmdoc' on npm is taken by an unrelated third-party package;
  publish under the org scope (@tokenroll, alongside acplugin) with bin
  name 'llmdoc' so local invocation stays 'npx llmdoc'
- all agent-facing prompts (skills, agents) now use 'npx --no-install
  llmdoc' so an uninstalled environment fails cleanly instead of npx
  downloading and executing the unrelated 'llmdoc' package (supply-chain
  guard); hooks.json already did this
- READMEs: install instructions + explicit warning about the bare-name
  npx pitfall
AGENTS/CLAUDE example recipes, the Codex-converted skills, and the README
hook table are all executed or copied verbatim by agents/hosts — switch
their invocations to 'npx --no-install llmdoc' so an environment without
@tokenroll/llmdoc fails cleanly instead of pulling the unrelated bare
'llmdoc' package. Human-facing README prose keeps plain 'npx llmdoc'
behind the install-first instruction and warning.
For plugin users (Claude Code / Codex) the hooks + operating skill already
cover everything the examples said, so they only invite drift. The one
audience that still needs a recipe — platforms without a plugin system —
now gets a copy-pasteable AGENTS.md block directly in both READMEs.
…, wikilink warning

From the tool-bridge V2->V3 migration dogfood:
- 'llmdoc commit': one-step finalize (validate gate -> pathspec-scoped docs
  commit -> fingerprint -> meta follow-up commit); kills the manual
  three-step ritual and the --amend hash-chasing trap; --no-verify for
  husky-heavy repos
- 'llmdoc init-state': meta.json skeleton for init/upgrade (null revisions
  + measured convergence); was previously hand-written
- .llmdocignore: minimatch patterns excluded from unmapped/dirty signals
  (local runtime files like data/*.sqlite3)
- validate now warns on [[wikilink]] syntax (writers produce it; llmdoc
  never parses it)
- upgrade inventory no longer suggests the removed <topic>/index.mdx
…kilink warning)

Tests now read the expected version from package.json instead of
hardcoding it.
….1.0 canonical

skills convert 1:1 (no more cmd-* prefixes), disable-model-invocation maps
to both frontmatter and openai.yaml allow_implicit_invocation, argument-hint
passes through. Manually patched .codex-plugin version to 3.1.0 (acplugin
emits 3.0.0 regardless of source manifest — reported upstream). Known gaps
kept as-is: hooks degrade to prose (PreCompact skipped entirely), agent
'model: inherit' hardcoded to a model name.
…stall docs

- All prompts, agents, docs, dogfood, and Codex-converted surfaces now use
  the full scoped name. Unlike the bare 'npx llmdoc' (which resolves to an
  unrelated third-party npm package when not installed), the scoped name
  has zero wrong-package risk and even works uninstalled. Hooks keep
  --no-install to stay offline-safe and fail open.
- CLI hint strings (tree, init-state) updated to the full invocation.
- README: install sections now include the official Codex flow
  ('codex plugin marketplace add TokenRollAI/llmdoc' per
  developers.openai.com/plugins/build/plugins); warning block rewritten.
- .agents/plugins/marketplace.json fixed to match the official catalog
  schema: source.path './' (was pointing at a non-existent ./plugins/
  subdir), added description and category.
- Manifests bumped to 3.1.1 incl. .claude-plugin/marketplace.json (missed
  in the 3.1.0 bump — this is what acplugin reads for the plugin version).
…n scanner action

- The viewer integration test (server start + fixture git init + 6
  requests) exceeds vitest's 5s default on CI runners; give it 20s.
- hashgraph-online/codex-plugin-scanner's pinned SHA vanished after an
  upstream history rewrite; re-pin to the current head.
…in-repo codex surface check

- validate:dogfood ran 'npx --no-install @tokenroll/llmdoc' but npm skips
  bin links for file: deps whose bin target doesn't exist at install time
  (cli/dist is built later in CI); invoke cli/dist/bin/llmdoc.js directly.
- hashgraph-online/codex-plugin-scanner moved repos and currently ships no
  action.yml at either location; replaced with scripts/check-codex-surface.mjs
  covering manifest schema, marketplace policy fields, skill frontmatter,
  and the scoped --no-install hook invocation convention.
… CI full-depth checkout

CI checkouts default to fetch-depth: 1, where every historical commit in
meta.json is unreachable — validate falsely reported the whole ledger as
stale. In shallow repositories these become warnings with an unshallow
hint; this repo's CI also checks out full history for exact validation.
@Disdjj
Disdjj merged commit dd3f57d into main Aug 24, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant