Skip to content

feat: make blog content consumable by AI agents - #15

Open
bobbyg603 wants to merge 2 commits into
mainfrom
feat/agent-readable-content
Open

feat: make blog content consumable by AI agents#15
bobbyg603 wants to merge 2 commits into
mainfrom
feat/agent-readable-content

Conversation

@bobbyg603

Copy link
Copy Markdown
Collaborator

Summary

Makes every blog article (current and future) directly consumable by AI agents, with zero per-post maintenance — everything derives from the content collection at build time.

  • Markdown twin of every post at /blog/<slug>.md — title, metadata, canonical URL, and the raw article body served as text/markdown (~5-10x fewer tokens than the HTML page)
  • /llms.txt (llmstxt.org) — site overview plus a linked index of every post's .md twin, newest first
  • /llms-full.txt — full text of all articles in one fetch (~80 KB today)
  • JSON-LD BlogPosting structured data on every post — author, dates, publisher, keywords, for answer-engine attribution
  • RSS feed at /rss.xml via @astrojs/rss, with an autodiscovery <link> on every page
  • rel=alternate type=text/markdown link on each post pointing at its .md twin
  • Drafts are excluded everywhere (matches the blog index convention)

Implementation notes

  • The generators live as pure functions in src/utils/agentContent.ts; the Astro endpoints (llms.txt.ts, llms-full.txt.ts, blog/[...slug].md.ts) are thin wrappers, so future posts are picked up automatically on every build
  • BaseLayout gained a head slot so pages can inject head tags (used by BlogPost for JSON-LD + alternate links)
  • Site stays fully static — no adapter, no worker code; Cloudflare serves the new files as plain assets
  • Added .wrangler/ (local wrangler dev cache) to .gitignore

Testing

  • npm test — 10 new vitest unit tests covering the markdown/llms.txt/llms-full.txt generators (new test script)
  • npm run build — verified dist/ contains llms.txt, llms-full.txt, rss.xml, and a .md file per post; JSON-LD and alternate links present in built post HTML; sitemap not polluted by the new endpoints
  • Served dist/ through npx wrangler dev (the production serving layer) and confirmed 200s with correct content types: text/markdown for .md, text/plain for the llms files, application/xml for RSS

🤖 Generated with Claude Code

- Markdown twin of every post at /blog/<slug>.md, generated at build
  time from the content collection
- /llms.txt index and /llms-full.txt full-content file (llmstxt.org)
- RSS feed at /rss.xml with autodiscovery link
- JSON-LD BlogPosting structured data and rel=alternate markdown link
  on every post
- Unit tests for the markdown/llms.txt generators (vitest)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 17, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
web 201281e Commit Preview URL

Branch Preview URL
Jul 17 2026, 01:06 AM

- Escape < in JSON-LD so set:html cannot break out of <script>
- Format agent markdown dates in America/New_York (not UTC day)
- Restore valid package name workingdevshero-web
- Add unit test for late-evening offset pubDate calendar day

Co-Authored-By: Grok <noreply@x.ai>
@bobbyg603

Copy link
Copy Markdown
Collaborator Author

Fixed review body nit on package name in 201281e.

  • Restored valid unscoped name workingdevshero-web in package.json and the lockfile root (npm names cannot contain / unless scoped as @scope/name).
  • Left license: MIT as-is to match package.json (the lockfile had been out of sync with ISC).

Also addressed the two inline suggestions in the same commit:

  1. JSON-LD < escape before set:html
  2. America/New_York calendar dates for agent markdown + regression test

Verification: npm test (11 passed) and npm run build (success).

@bobbyg603 bobbyg603 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Solid, well-scoped PR that makes the blog consumable by AI agents via /llms.txt, /llms-full.txt, per-post .md twins, RSS, and BlogPosting JSON-LD plus discovery links. Core logic is cleanly centralized in agentContent.ts with good unit coverage; static build output confirms routes, draft filtering, sorting, and content shape all work. Residual risks are small hardening gaps (JSON-LD injection edge case, UTC calendar-date formatting) and an incidental invalid package name change, not structural flaws.

Issue counts by severity

  • bugs: 0
  • suggestions: 2
  • nits: 1

Issues outside the diff

These findings reference lines that are not present in the diff and could not be posted as inline comments:

  • [nit] package.json:2 -- Package name changed from workingdevshero-web to workingdevshero/web. An unscoped npm name cannot contain /; scoped packages must be @scope/name. This is unrelated to the agent-content feature and can confuse tooling that validates package names. The same edit also flips license from ISC to MIT in the lockfile root metadata.
    • Suggestion: Keep workingdevshero-web, or use @workingdevshero/web if scoping is intentional. Revert the name/license drive-by if it was accidental.

Comment thread src/layouts/BlogPost.astro Outdated
<BaseLayout title={title} description={description} image={heroImage} type="article">
<Fragment slot="head">
<link rel="alternate" type="text/markdown" href={markdownURL} title="Markdown version of this article" />
<script is:inline type="application/ld+json" set:html={JSON.stringify(jsonLd)} />

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] JSON-LD is injected with set:html={JSON.stringify(jsonLd)}. JSON.stringify does not escape <, so a title/description containing </script> (or similar) would break out of the <script type="application/ld+json"> tag in HTML. Content is author-controlled today, so risk is low, but this is a classic footgun if frontmatter ever includes that sequence.

Suggestion: Serialize safely, e.g. JSON.stringify(jsonLd).replace(/</g, '\\u003c'), or use a small helper that always Unicode-escapes < before set:html.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 201281e. JSON-LD is now serialized with < escaped to \\u003c before set:html, so a title/description containing </script> cannot break out of the <script type="application/ld+json"> tag.

Comment thread src/utils/agentContent.ts Outdated
}

function formatDate(date: Date): string {
return date.toISOString().split('T')[0];

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] formatDate uses date.toISOString().split('T')[0], which reports the UTC calendar day. Posts with evening offset timestamps (e.g. 2026-07-11T21:00:00-04:002026-07-12T01:00:00.000Z) would publish as the next day in markdown/llms output. Current posts happen to stay on the intended day, but the site already uses offset-aware pubDates, so this can drift.

Suggestion: Format a stable calendar date in a fixed zone (e.g. America/New_York via Intl.DateTimeFormat with timeZone + year/month/day), or document that pubDate must be UTC-midnight / date-only so UTC truncation matches author intent. Add a unit test for a late-evening offset timestamp.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 201281e. formatDate now uses Intl.DateTimeFormat with timeZone: 'America/New_York' (en-CA → YYYY-MM-DD) so late-evening offset timestamps stay on the local calendar day. Added a unit test for 2026-07-11T21:00:00-04:00.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant