Add automated Vercel-style llms.txt generation

[fcakyon]

Adds automatic Vercel-style llms.txt file generation after site build.

Generated file: llms.txt (22K tokens vs 49K tokens in Vercel)

ultralytics-mkdocs-llmstxt.mp4

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Adds automatic llms.txt generation after MkDocs builds to make Ultralytics docs easier for LLMs to discover and consume 🤖📚

📊 Key Changes

• ✅ New plugin option: add_llms_txt (default: True) to control whether llms.txt is generated
• 🏗️ Added on_post_build() hook to generate llms.txt after the site finishes building
• 🧩 Implemented generate_llms_txt() in plugin/postprocess.py:
  • Builds a structured list of documentation pages (Vercel-style) using MkDocs nav when available
  • Falls back to scanning all Markdown files if nav isn’t provided
  • Pulls optional page descriptions from Markdown frontmatter (description)
  • Writes the result to site/llms.txt
• 🔧 Updated standalone postprocess_site() to optionally generate llms.txt via add_llms_txt (default: True)

🎯 Purpose & Impact

• 🤝 Improves LLM friendliness: provides a single, curated index of docs pages for AI tools and crawlers to reference
• 🧭 Better structure and discoverability: respects MkDocs navigation order/sections when nav is configured
• 📝 Richer context: includes per-page descriptions (when present in frontmatter) to help models choose relevant pages
• ⚙️ Low-risk, opt-out behavior: enabled by default but can be disabled via config if undesired (add_llms_txt: false) ✅

[UltralyticsAssistant]

👋 Hello @fcakyon, thank you for submitting a ultralytics/mkdocs 🚀 PR! This is an automated message, and an Ultralytics engineer will assist you shortly. 🤖✅

• ✅ Define a Purpose: Clearly explain the purpose of your fix or feature in your PR description, and link to any relevant issues. Ensure your commit messages are clear, concise, and adhere to the project's conventions.
• ✅ Synchronize with Source: Confirm your PR is synchronized with the ultralytics/mkdocs main branch. If it's behind, update it by clicking the 'Update branch' button or by running git pull and git merge main locally.
• ✅ Ensure CI Checks Pass: Verify all Ultralytics Continuous Integration (CI) checks are passing. If any checks fail, please address the issues.
• ✅ Update Documentation: Update the relevant documentation for any new or modified features.
• ✅ Add Tests: If applicable, include or update tests to cover your changes, and confirm that all tests are passing.
• ✅ Sign the CLA: Please ensure you have signed our Contributor License Agreement if this is your first Ultralytics PR by writing "I have read the CLA Document and I sign the CLA" in a new message.
• ✅ Minimize Changes: Limit your changes to the minimum necessary for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

For more guidance, please refer to our Contributing Guide. Don't hesitate to leave a comment if you have any questions. Thank you for contributing to Ultralytics! 🚀

[UltralyticsAssistant]

🔍 PR Review

_{Made with ❤️ by Ultralytics Actions}

Solid, minimal integration via on_post_build() and a clear add_llms_txt toggle. Main concerns are robustness: generate_llms_txt()’s reliance on site_dir.parent/mkdocs.yml can miss the real config/nav in common build setups, and frontmatter parsing can hard-fail on custom YAML tags. Also consider making URL conversion more path-safe and avoiding unconditional stdout prints during builds.

💬 Posted 4 inline comments

[fcakyon]

Recreating with clean git history