Skip to content

docs(robots): mention /llms.txt and /feed.json in header - #435

Open
laichunpongben wants to merge 1 commit into
mainfrom
feat/robots-allowall-doc
Open

docs(robots): mention /llms.txt and /feed.json in header#435
laichunpongben wants to merge 1 commit into
mainfrom
feat/robots-allowall-doc

Conversation

@laichunpongben

Copy link
Copy Markdown
Owner

Adds a one-line note pointing AI crawlers + feed readers at the dedicated indexes:

  • /llms.txt - llmstxt.org-style markdown table-of-contents
  • /feed.json - JSON Feed 1.1 (alongside /rss.xml)

robots.txt is the first thing any crawler reads; advertising the structured indexes here saves it a discovery round-trip.

Files:

  • public/robots.txt (+1 / 0)

Generated with Claude Code

Adds a one-line note pointing AI crawlers + feed readers at the dedicated indexes:
- /llms.txt - llmstxt.org-style markdown table-of-contents
- /feed.json - JSON Feed 1.1 (alongside /rss.xml)

robots.txt is the first thing any crawler reads; advertising the structured indexes here saves it a discovery round-trip and routes traffic to the right surface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant