Skip to content

Commit 8a37fcd

Browse files
Clarify llms.txt availability and make example links clickable
Co-Authored-By: Devin Logan <[email protected]>
1 parent b083acb commit 8a37fcd

File tree

1 file changed

+18
-3
lines changed

1 file changed

+18
-3
lines changed

fern/products/docs/pages/seo/llms-txt.mdx

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,17 +9,25 @@ description: Enable tools like Cursor, GitHub Copilot, ChatGPT, and Claude to un
99

1010
Contains a lightweight summary of your documentation site with each page distilled into a one-sentence description and URL. The format is structured to help AI tools parse and understand your documentation's hierarchy and organization.
1111

12+
`llms.txt` is available at any level of your documentation hierarchy (e.g., `/llms.txt`, `/docs/llms.txt`, `/v1/api-reference/llms.txt`).
13+
1214
See an example: [docs.cohere.com/llms.txt](https://docs.cohere.com/llms.txt)
1315

1416
## `llms-full.txt`
1517

1618
Contains complete documentation content including the full text of all pages. For API documentation, this includes your complete API Reference with resolved OpenAPI specifications and SDK code examples for enabled languages. Each API endpoint page includes its full OpenAPI spec.
1719

20+
`llms-full.txt` is available at the root of your documentation site and summarizes all of your documentation.
21+
1822
When Fern detects an LLM bot accessing your documentation, it automatically serves Markdown instead of HTML, reducing token consumption by 90%+.
1923

20-
View the llms.txt files for this site:
21-
- `https://buildwithfern.com/learn/llms.txt`
22-
- `https://buildwithfern.com/learn/llms-full.txt`
24+
See an example: [docs.cohere.com/llms-full.txt](https://docs.cohere.com/llms-full.txt)
25+
26+
## View in Action
27+
28+
Check out the llms.txt files for this site:
29+
- [https://buildwithfern.com/learn/llms.txt](https://buildwithfern.com/learn/llms.txt)
30+
- [https://buildwithfern.com/learn/llms-full.txt](https://buildwithfern.com/learn/llms-full.txt)
2331

2432
<Frame>
2533
<img
@@ -34,6 +42,13 @@ View the llms.txt files for this site:
3442
The [Fern Dashboard](https://dashboard.buildwithfern.com/) provides comprehensive analytics for llms.txt usage including:
3543
- Traffic by LLM provider (Claude, ChatGPT, Cursor, etc.)
3644
- Page-level breakdowns of bot vs. human visitors for `.md` and `llms.txt` files
45+
- Performance metrics including load times, processing times, and content length
46+
- User agent detection to identify bot traffic versus browser requests
3747

3848
This visibility helps you understand LLM traffic patterns and optimize your documentation for AI consumption.
3949

50+
## Technical Details
51+
52+
The llms.txt generation system handles complex schema types including self-referencing schemas, preventing infinite recursion during content generation. For large documentation sites, the system uses optimized timeouts to ensure reliable content delivery without timing out.
53+
54+
For REST API endpoints, the markdown generation includes OpenAPI specifications in YAML format, making it easy for LLMs to understand your API structure. The same markdown generation logic is used for both serving llms.txt files and indexing content for search, ensuring consistency across your documentation platform.

0 commit comments

Comments
 (0)