feat(docs): add agent playbook and plain-text Markdown mirrors - #57
Conversation
Make the documentation ingestible by AI agents. Every page now has a plain-text Markdown mirror at the same path with a `.md` suffix — previously a 404. `src/lib/to-markdown.ts` converts MDX to clean Markdown rather than serving raw source: imports are dropped, `<Aside>` becomes a blockquote, `<TabItem>`/`<Card>` become bold labels with dedented bodies, `<LinkCard>` becomes a list item, and `<Tabs>`/`<Steps>`/`<FileTree>` are unwrapped. The transform is code-fence aware so pages documenting JSX pass through untouched. Adds three entry points for agents: - `/agents.md` — a single-page integration playbook: decision tree, install matrix, recipes for SPA migration, Docker (both modes), Kubernetes and local dev, the client API with its behavioural contracts, gateway reference, and common failure modes. - `/llms.txt` — indexed by sidebar section, linking to `.md` URLs. - `/llms-full.txt` — the full documentation set as one file. Cloudflare Pages serves `.md` as `text/markdown`, which browsers download instead of render. Pages splat patterns match greedily and a `/*.md` suffix pattern is not documented as supported, so `src/integrations/md-headers.mjs` enumerates the rules explicitly from the build output (49 of a 100-rule limit, with a warning if that is approached). `public/_headers` is now generated rather than maintained by hand. The sidebar moves to `src/sidebar.mjs` so Starlight and the `/llms.txt` route share one ordering, and a Starlight `Head` override advertises each page's mirror via `<link rel="alternate" type="text/markdown">`. Also syncs the stale published copy of the manifest JSON schema.
Greptile SummarySummary
Confidence Score: 5/5The documented update command should be corrected, but this is a non-blocking documentation defect. The remaining issue is limited to the Kubernetes documentation: the example patch creates a second ConfigMap entry while the configured gateway continues reading the unchanged Files Needing Attention: docs/src/content/docs/deployment/kubernetes.mdx
What T-Rex did
|
`Reload()` re-reads configuration via `config.ReadAndClassify(s.cfg.EnvFile)` (gateway/internal/server/server.go:398). With `EnvFile` empty — it is set only by `--env-file`/`REP_GATEWAY_ENV_FILE` (config.go:155) — classify.go builds the variable set from `os.Environ()` alone (classify.go:101-122). Both Kubernetes recipes supplied configuration through `envFrom` while watching a mounted ConfigMap. `envFrom` is captured once at container start, so a volume refresh fired the watcher and the reload succeeded, but the injected payload kept its original values. The documented "no pod restart" workflow did not work. Both recipes now mount a single env-format ConfigMap key and pass `--env-file=/config/.env`, and the ConfigMap shape is corrected — a `data:` map of one key per variable mounts as one file per key, not as an env file. Also documents the precedence trap: ReadAndClassify merges the env file first and lets the process environment overwrite it, so a key set through both `envFrom` and `--env-file` silently keeps its `envFrom` value. Verified against a locally built gateway using a kubelet-style volume (timestamped directory + atomic `..data` symlink rename): the corrected recipe moves the injected value across the swap and emits `rep.config.changed` for each key, and the same key set in both sources reproduces the stale-value behaviour.
Makes the documentation site ingestible by AI agents, and fixes the 404 every docs path returned when suffixed with
.md.Plain-text Markdown mirrors
Every page now has a mirror at the same path with a
.mdsuffix —/reference/sdk.md,/concepts/wire-format.md, and so on (46 pages, plus/index.mdfor the landing page).docs/src/lib/to-markdown.tsconverts MDX to clean Markdown rather than serving raw source, so agents don't have to parse JSX:import … from '@astrojs/starlight/components'<Aside type="caution">> **Caution**blockquote<TabItem label="Docker">/<Card title="…"><LinkCard title href description />- [title](href) — description<Tabs>/<Steps>/<FileTree>The transform is code-fence aware, so the pages that document JSX (
<RepScript />, Next.js layouts, component examples) pass through untouched. Verified across all 46 files: no leftover component tags, no stray imports, even code-fence parity everywhere.Entry points for agents
/agents.md— a single-page integration playbook. Decision tree for choosing gateway mode, dev tooling and client API; tier classification rules; install matrix across all npm packages, the Docker image and the raw binary; five copy-paste recipes (SPA migration, Docker embedded, Docker proxy, Kubernetes sidecar, local dev); the client API with the behavioural contracts that change how code is written against it (get()is synchronous, the firstgetSecure()decrypts everything, SSE connects lazily); gateway flag/env/endpoint tables; ten failure modes; and a verification checklist. It also renders as a normal docs page and appears in the sidebar./llms.txt— indexed by sidebar section with descriptions, linking to the.mdURLs./llms-full.txt— the complete documentation set as one file (~187 KB).Content-type handling
Cloudflare Pages serves
.mdastext/markdown, which browsers download rather than render. A_headersoverride fixes that, but the wildcard form wasn't safe to assume: Cloudflare documents a splat as greedily matching all characters and doesn't document a/*.mdsuffix pattern as supported.docs/src/integrations/md-headers.mjstherefore enumerates the rules explicitly from the real build output — 49 rules against a 100-rule limit, with a build warning if that is ever approached.public/_headersis generated rather than hand-maintained, so it can't drift from the routes.Supporting changes
astro.config.mjsintodocs/src/sidebar.mjs, shared by Starlight and the/llms.txtroute so the index can't drift from the nav.Headoverride adds<link rel="alternate" type="text/markdown">to each page, making mirrors discoverable from the HTML without guessing the convention.SITE_URLconsolidated intodocs/src/site.mjs.rep-manifest.schema.json(copy-schemas.mjsoutput that had drifted fromschema/).Verification
pnpm buildsucceeds; all 48 mirrors plus bothllmsfiles generated.pnpm check— 0 errors, 0 warnings, 0 hints./reference/sdk.md,/llms.txtand/agents.mdwithcontent-type: text/plain; charset=utf-8.agents.mdcross-checked against source: gateway env var names againstgateway/internal/config/config.go, the Kubernetes sidecar againstdeployment/kubernetes.mdx, CLI flags and therepbin name againstcli/package.json.