feat: add @docusaurus/plugin-llms-txt#11420
Conversation
- Add plugin to automatically generate llms.txt files - Helps AI tools understand Docusaurus site structure - Includes comprehensive test suite and documentation - Follows Docusaurus plugin conventions
- Introduced a new demo page for the @docusaurus/plugin-llms-txt showcasing its features and configuration. - Updated dogfooding configuration to include the new plugin with site details and options for content inclusion.
|
Hi @guneysol! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
✅ [V2]Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
There was a problem hiding this comment.
I'm sorry if you put effort into this, but I'm going to close this PR.
It may be good enough for your site, and is a valid solution for a third-party plugin, but the implementation is too opinionated, and the feature is not designed in a way that would make sense to ship in Docusaurus core
Please keep in mind that we already have an issue to implement llms.txt, and that we usually prefer to design a feature before rushing into implementing it. Here I only have an implementation, but we don't have any design that explains the various possible choices to design this feature and their tradeoffs.
Also, be aware that other users before you wrote third-party llms.txt plugins, and submitting a PR here does not guarantee that your implementation is better than others. We may prefer other implementations, or an implementation that takes the best of everything we have seen.
My suggestion would be to publish this plugin as a third-party npm package and propose it to the community in this issue among all the other proposed solutions. We can then iterate from there and design the feature that would make sense for Docusaurus core based on all the third-party implementations we have seen.
| function getRouteType(path: string): 'doc' | 'blog' | 'page' { | ||
| if (path.includes('/docs/') || path.startsWith('/docs')) { | ||
| return 'doc'; | ||
| } | ||
| if (path.includes('/blog/') || path.startsWith('/blog')) { | ||
| return 'blog'; | ||
| } | ||
| return 'page'; | ||
| } |
There was a problem hiding this comment.
This is not something we can do. Paths should not be hardcoded, and should not fallback to "page".
Third-party plugins exist, and there's no finite number of "route types".
2 participants
feat: add @docusaurus/plugin-llms-txt
Pre-flight checklist
Motivation
This PR adds support for generating
llms.txtfiles, which provide structured context about documentation sites for AI tools and LLMs. As AI assistants become increasingly important in developer workflows, there's a growing need for documentation sites to be AI-friendly.Problem: AI tools struggle to understand the structure and navigation of documentation websites, leading to poor recommendations and missed content.
Solution: The
llms.txtstandard provides a simple, machine-readable index of site content that AI tools can use to better understand and reference documentation.Benefits:
Test Plan
Live Demo & Testing
Deploy Preview: https://deploy-preview-11420--docusaurus-2.netlify.app/
Key test links:
Verified Working Features
Comprehensive Test Coverage
Manual Testing
Example Output
Related issues/PRs
This addresses the growing ecosystem need for AI-friendly documentation sites. The implementation follows the emerging
llms.txtstandard and provides a foundation for future AI integrations with Docusaurus.Implementation highlights:
Technical Details
Plugin Features:
postBuildlifecyclenoIndexsettingNo Breaking Changes: This is a purely additive feature that requires opt-in usage.
Dependencies: Only uses existing Docusaurus packages +
fs-extraandtslib(no new security surface).Scale Testing: Successfully generates llms.txt with 1,300+ entries from the full Docusaurus website, demonstrating production readiness.