Skip to content

feat: LLM readability layer — llms.txt, serve-markdown, MCP discovery, and CDN caching#329

Open
Nonnyjoe wants to merge 4 commits into
mainfrom
feat/llm-readability
Open

feat: LLM readability layer — llms.txt, serve-markdown, MCP discovery, and CDN caching#329
Nonnyjoe wants to merge 4 commits into
mainfrom
feat/llm-readability

Conversation

@Nonnyjoe
Copy link
Copy Markdown
Member

@Nonnyjoe Nonnyjoe commented May 25, 2026

Summary

  • Syncs llms.txt to cover all sitemap URLs with correct .md links and AI agent instructions
  • Adds skill.md, MCP discovery endpoint (/.well-known/mcp), and a visually-hidden llms.txt link in every page body
  • Adds plugins/serve-markdown.js — serves raw .md at build time alongside HTML, handles generated-index pages, and supports Accept: text/markdown content negotiation in the dev server
  • Fixes broken internal links across several versioned doc pages
  • Adds per-file-type CDN cache headers (customHttp.yml) and Amplify rewrite rules (amplify.yml) so .md files, llms-full.txt, skill.md, and the MCP endpoint are served correctly without being intercepted by the SPA fallback

What is NOT in this PR

The Amplify Gen 2 CDK backend (CloudFront distribution + Lambda function) lives in a separate PR (feat/amplify-lambda) that will be monitored on staging before merging.

Test plan

  • https://docs.cartesi.io/llms.txt — all links end with .md and resolve
  • https://docs.cartesi.io/cartesi-rollups/overview.md — returns raw markdown directly
  • https://docs.cartesi.io/skill.md — resolves
  • https://docs.cartesi.io/.well-known/mcp — returns JSON
  • curl -H "Accept: text/markdown" https://docs.cartesi.io/cartesi-rollups/overview — returns Content-Type: text/markdown
  • HTML pages load normally in browser (no regressions)
  • Cache-Control headers correct per file type (assets immutable, html/md short TTL)

Related to #328

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant