Skip to content

feat(md-exports): Improve soft-404 recovery for agent .md requests - #18953

Open
sergical wants to merge 1 commit into
masterfrom
feat/md-soft-404-improvements
Open

feat(md-exports): Improve soft-404 recovery for agent .md requests#18953
sergical wants to merge 1 commit into
masterfrom
feat/md-soft-404-improvements

Conversation

@sergical

@sergical sergical commented Aug 6, 2026

Copy link
Copy Markdown
Member

DESCRIBE YOUR PR

Agent-facing .md soft-404s (the docs.md_export.not_found metric, ~6.9k hits since July 20) fall into three classes that today all get the same generic "Page Not Found" markdown. This PR gives each class a response the agent can actually act on:

  • platform-redirect (top missed path, ~330 hits/month): new md-overrides/platform-redirect.mdx explains that the page is a platform chooser with no content of its own and shows how to build /platforms/<platform>/<page>.md URLs directly, with platform and framework link lists rendered from the doctree.
  • Redirected pages (e.g. /product/sentry-mcp.md): the md-exports catch-all route now consults both redirect tables (redirects.js and the middleware legacy list) and serves a "Page Moved" body pointing at the destination — the .md URL for internal targets, the URL as-is for external ones like mcp.sentry.dev. Pattern (:path*) sources are skipped for now; the new metric attribute will show whether they matter.
  • Pages that exist but whose static export was missing (transient deploy-window gaps, e.g. the product.md burst on July 30): the route detects doctree hits and serves "Markdown Export Unavailable" with the HTML link and child pages, cached for 60s so a restored export takes over quickly.

The metric gains an outcome attribute (redirected | page_exists | unknown_path) so dashboards can separate agent-invented URLs from export-pipeline gaps. The metric name and existing attributes are unchanged.

Review notes: the middleware.ts diff is only export keywords on the two redirect arrays and the Redirect type — route.ts is the substantive change. Verified with 232 passing vitest tests (8 new covering redirect, external-destination, pattern-skip, and page-exists paths), clean tsc/ESLint/Prettier, and the override rendered through a mirror of the generate-md-exports MDX pipeline against the real doctree. On the Vercel preview, check /platform-redirect.md and /product/sentry-mcp.md.

IS YOUR CHANGE URGENT?

Help us prioritize incoming PRs by letting us know when the change needs to go live.

  • Urgent deadline (GA date, etc.):
  • Other deadline:
  • None: Not urgent, can wait up to 1 week+

SLA

  • Teamwork makes the dream work, so please add a reviewer to your PRs.
  • Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it.
    Thanks in advance for your help!

PRE-MERGE CHECKLIST

Make sure you've checked the following before merging your changes:

  • Checked Vercel preview for correctness, including links
  • PR was reviewed and approved by any necessary SMEs (subject matter experts)
  • PR was reviewed and approved by a member of the Sentry docs team

🤖 Generated with Claude Code

The docs.md_export.not_found metric shows agents hitting three distinct miss
classes that all got the same generic not-found body. Handle each:

- Add md-overrides/platform-redirect.mdx so the platform chooser (the top
  missed path, ~330 hits/month) exports guidance on building platform URLs
  directly, with platform and framework link lists.
- Teach the md-exports catch-all route to consult both redirect tables
  (redirects.js and the middleware legacy list) and point moved pages at
  their destination - .md URL for internal targets, as-is for external.
- Detect requested paths that exist in the doctree and serve a Markdown
  Export Unavailable body with a 60s cache so transient deploy-window gaps
  recover quickly.

Tag the metric with outcome (redirected | page_exists | unknown_path) so
dashboards can separate agent-invented URLs from export-pipeline gaps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
develop-docs Ready Ready Preview Aug 6, 2026 12:19am
sentry-docs Ready Ready Preview Aug 6, 2026 12:19am

Request Review

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 83b7042. Configure here.


## Frameworks

<FrameworkGroups />

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Platform override is never generated

High Severity

platform-redirect.mdx has no static HTML artifact to override, so platform-redirect.md continues falling through to the generic not-found route.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 83b7042. Configure here.

// export-pipeline gap). Very different signal from an invented URL.
outcome = 'page_exists';
hasSuggestions = !!ancestor.children?.length;
body = renderExportMissingBody(requestedPath, ancestor);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Synthetic doctree nodes appear as pages

Medium Severity

Exact doctree matches are treated as real pages without checking node.missing, so synthetic hierarchy nodes receive a false “Markdown Export Unavailable” response and page_exists metric.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 83b7042. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant