Skip to content

feat(website): llms-full.txt and raw-markdown alternates for docs - #729

Merged
ajitpratap0 merged 2 commits into
mainfrom
feat/llms-full-and-md-alternates
Oct 18, 2026
Merged

ajitpratap0 merged 2 commits into
mainfrom
feat/llms-full-and-md-alternates

Conversation

@ajitpratap0

Copy link
Copy Markdown
Owner

Summary

  • Every docs page is now served as raw markdown at /docs/<slug>.md (rewrite to a static route handler at /md/docs/[...slug]; paths with an extension skip the trailing-slash redirect).
  • Each docs page advertises it with <link rel="alternate" type="text/markdown" href="https://gosqlx.dev/docs/<slug>.md">.
  • llms-full.txt and the .md alternates share one buildDocMarkdown builder; llms.txt documents the .md convention.

Test Plan

  • npm ci && npm run typecheck && npm run lint && npm run build (lint: warnings only, no errors).
  • next start: /docs/getting-started.md and /docs/reference/api/ast.md return 200 text/markdown; unknown slug returns 404; /llms-full.txt has 42 Source: entries (all sidebar docs); docs page HTML contains the alternate link.

Breaking Changes

None.

Closes #679
Part of #634

🤖 Generated with Claude Code

https://claude.ai/code/session_01RkzVu3JY9Y6BRWfjesWts9

…l.txt

Serve every docs page as /docs/<slug>.md (rewrite to a static route handler,
exempt from the trailing-slash redirect), advertise it with
<link rel="alternate" type="text/markdown"> on each docs page, mention it in
llms.txt, and share one markdown builder with llms-full.txt (42 docs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RkzVu3JY9Y6BRWfjesWts9
@vercel

vercel Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
gosqlx-website Ready Ready Preview Oct 18, 2026 6:41am UTC

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 19s —— View job


Review of PR #729

I read the diff only. I did not build the site or run any tests, so the notes below come from reading the code.

Summary: The change is small and well factored. buildDocMarkdown is shared between llms-full.txt and the .md route, so the two outputs can't drift apart. The output of llms-full.txt should be byte-identical to before.

Looks good

  • Single shared builder (src/lib/llms.ts). The --- separator stays in the full-file loop, and the per-page output has no leading separator.
  • The route uses force-static, dynamicParams = false and generateStaticParams, so every .md page is prerendered and unknown slugs return 404.
  • The beforeFiles rewrite means the .md URLs match before the filesystem and dynamic routes.
  • alternates.types emits <link rel="alternate" type="text/markdown"> as intended.

Suggestions

  1. alternates.types href may not be absolute. The PR description says the link is https://gosqlx.dev/docs/<slug>.md. docMarkdownPath returns a relative path (/docs/<slug>.md), which only becomes absolute if metadataBase is set. Please confirm metadataBase is set in the root layout, and check the rendered HTML. Your test plan already greps for the link, so this may be fine.
  2. Rewrite pattern /docs/:slug*.md. With :slug* followed by a literal .md, path-to-regexp semantics can be surprising. For /docs/reference/api/ast.md, make sure the capture is reference/api/ast and not something with a trailing .. Your manual check of /docs/reference/api/ast.md covers this. A short note in the PR, or a smoke test for a nested slug and a slug that contains a dot, would be a cheap guard.
  3. Trailing-slash behaviour. /docs/getting-started.md/ is probably redirected or 404s, which is acceptable. Mentioning it in the llms.txt line is optional.
  4. Caching headers. MARKDOWN_HEADERS only sets Content-Type. Consider adding X-Content-Type-Options: nosniff and possibly Cache-Control. Static route handlers get default caching on most hosts, but explicit headers are cheap. Optionally add Vary/Link: <canonical>; rel="canonical" so search engines don't index the .md duplicates. Since these are plain text, X-Robots-Tag: noindex could also be worth considering.
  5. Inline llms.txt entry. The new bullet isn't a markdown link, unlike its neighbours. That's fine for documenting a convention, but some llms.txt parsers expect [name](url). A real link to an example page ([Example: getting-started](…/docs/getting-started.md)) would be friendlier to those parsers.
  6. Tests. There is no automated coverage. If the website has a test setup, a small unit test for buildDocMarkdown and docMarkdownPath would help. For example, it could check that the H1 is stripped and not duplicated. The route is covered only by your manual next start checks, which is reasonable for now.

Security

I found nothing concerning. Content comes from the repo's own docs and slugs are validated against the known set (dynamicParams = false).

No blocking issues. Nice cleanup overall.

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 13s —— View job


Review of PR #729

I read the diff and the related files. I did not build the site or run tests.

Summary: The change is small and well factored. buildDocMarkdown is shared between llms-full.txt and the .md route, so the two outputs can't drift apart. I see no blocking issues.

Checked

  • Absolute alternate link: metadataBase is set to https://gosqlx.dev in website/src/app/layout.tsx:17. The relative /docs/<slug>.md in alternates.types will therefore resolve to an absolute URL.
  • llms-full.txt output: the --- separator stays in the loop and the per-page builder has no leading separator, so the output should match the previous version.
  • Route: force-static, dynamicParams = false and generateStaticParams prerender every page. Unknown slugs return 404.
  • Rewrite: beforeFiles makes the .md URLs match before the filesystem and dynamic routes. Your next start check on the nested reference/api/ast.md path covers the :slug*.md capture.

Suggestions (non-blocking)

  1. Headers (website/src/lib/llms.ts, MARKDOWN_HEADERS): consider adding X-Content-Type-Options: nosniff. Also consider Link: <https://gosqlx.dev/docs/<slug>/>; rel="canonical" so search engines treat the .md files as alternates and not duplicates. X-Robots-Tag: noindex is a lighter option. The canonical header needs per-slug headers, so build it in the route handler.
  2. llms.txt bullet: the new "Raw markdown" line isn't a [name](url) link, unlike its neighbours. Some llms.txt parsers expect that format. A real link to an example page would be friendlier to them.
  3. Rewrite edge cases: /docs/getting-started.md/ (trailing slash) and a bare /docs/.md are untested. Both probably 404, which is fine. A quick curl for each would confirm it.
  4. Tests: there is no automated coverage. The unit-testable pieces are docMarkdownPath and buildDocMarkdown. A test could check that the H1 is stripped and not duplicated, and that the Source: line is correct. This is only worth doing if the website already has a test setup.

Security

I found nothing concerning. Content comes from the repo's own docs, and slugs are limited to the known set by dynamicParams = false.

@ajitpratap0
ajitpratap0 merged commit 3d89b53 into main Oct 18, 2026
32 checks passed
@ajitpratap0
ajitpratap0 deleted the feat/llms-full-and-md-alternates branch October 18, 2026 06:50

This branch was successfully deployed

1 active deployment
Preview — 134bb2eb Deployed Oct 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(website): llms-full.txt and raw-markdown alternates for docs [4.7]

1 participant