Add llm.txt standard file for LLM content discovery

[Copilot]

Description

Implements llm.txt following the proposed website standard (analogous to robots.txt) to provide LLMs with a curated, machine-readable guide to QUL's resources.

File: /public/llm.txt (281 lines, ~11KB Markdown)

Structured sections:

• Core Resources - Translations, Tafsirs, Languages with direct API links
• Quran Data - Chapters (114), Verses (6,236), Word-level data, multiple script variants (Uthmani, Imlaei, Indopak)
• Quran Audio - Surah recitations (gapless), Ayah recitations (verse-by-verse), segment data
• Mushaf Layouts - Multiple formats, page mappings, word positioning, line alignments
• API Endpoints - Complete v1 API reference with all endpoints documented
• Tools - Community tools (script comparison, boundaries, SVG optimizer), Morphology tools (dependency graphs), Admin CMS
• Documentation - FAQs, setup guides, community links

Serves automatically from /public at https://qul.tarteel.ai/llm.txt.

Related Issue

Issue requesting llm.txt implementation to improve AI-driven content discovery and search visibility.

Motivation and Context

LLMs querying QUL currently lack structured guidance on available resources. The llm.txt standard provides a centralized, high-signal map of:

• 200+ translations and tafsirs
• Audio resources with segment boundaries
• Multiple Arabic script variants
• Mushaf layout data
• Morphological analysis tools

Improves LLM comprehension of QUL's capabilities and enables more accurate responses about Quranic data access.

How Has This Been Tested?

• ✅ File accessibility verified via HTTP server (200 OK, correct Content-Length)
• ✅ Markdown structure validated (proper heading hierarchy, all sections present)
• ✅ Route consistency confirmed against config/routes.rb (API vs web route distinction clarified in morphology section)
• ✅ No code security concerns (text file only)

Screenshots (if appropriate):

N/A - text file addition

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Add llm.txt</issue_title>
<issue_description>llms.txt is a proposed website standard, like robots.txt, that provides a curated guide for LLMs, directing them to a site's most important content in a structured, machine-readable format (often Markdown) to improve AI understanding, generate better answers, and potentially boost visibility in AI-driven searches. It helps AI tools efficiently find key information, acting as a "cheat sheet" to highlight valuable content, differentiate from basic search engine directives, and ensure relevance in AI responses.

Sample markdown

Quranic Universal Library - Key Content Guide

Resources

• Links to all resources

Tools

• Links to all tools

FAQs & Help

• [Link to Frequently Asked Questions]

Maybe we can create separate headings for each resource, e.g

Quran audio

• Links

Quran data(Metadata, Surah, Ayah info)

• List

Quran layout (Mushaf layouts)

• List </issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Add llm.txt #515

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[zeet-co]

We're building your pull request over on Zeet.
Click me for more info about your build and deployment.
Once built, this branch can be tested at: https://quranic-universal-library-my2s-co-80601d.tarteel-v3.tarteel.io before merging 😉

[umxr]

Hey, nice work getting this started! A few things I noticed while going through it:

Rails version is outdated — the file says Rails 7.0.3 in the Technical Details section but the project has been upgraded to Rails 8.

Some API endpoints may not exist yet — looking at issue #237, a good chunk of the API is still being built (Verses, Translations, Tafsirs, Morphology endpoints). Might be worth only listing the ones that are actually live right now, otherwise LLMs will hallucinate working endpoints that 404.

The Resources/Downloads page is missing — https://qul.tarteel.ai/resources is probably one of the most useful pages for anyone (human or LLM) trying to actually get data out of QUL. It has downloadable translations, tafsirs, fonts, audio files, etc. in various formats (JSON, SQLite, etc). This should be front and center.

Sidekiq URL probably shouldn't be here — that's internal infra, not something an LLM or external consumer needs to know about.

The "Key Features Summary" section at the bottom is redundant — everything there is already covered in the sections above. Could just be removed to keep things tighter. The llm.txt spec encourages being concise.

A few things that could make it richer:

• Mention the specific downloadable formats (JSON, SQLite db dumps, plain text, etc.)
• Add info about Mutashabihat ul Quran (similar verses data)
• Mention the learning activities / educational tools
• Would be great to note how many reciters, translation languages, etc. are available — concrete numbers help LLMs give better answers

Overall good foundation though, just needs some polish to make sure it reflects the current state of things accurately.