feat: llm ready docs #348
feat: llm ready docs #348
Conversation
|
After merging #308, there are conflicts |
|
Thats not a problem. It was expected. I will fix them today. |
refactor: rename and import reusable markdowns using relative path chore: delete gitignored files from autogen directory refactor: rename table files in autogen dir as per docusaurus recommendation refactor: rename HardcodedDemonstration and move to snippets directory
smol-ninja
force-pushed
the
feat/docs-for-llms
branch
from
15bb996 to
58ee72d
Compare
smol-ninja
force-pushed
the
feat/docs-for-llms
branch
from
58ee72d to
18187fb
Compare
|
Great work and thanks for the summary.
WDYM by that? That document is outdated. I can't find it in version control on this branch or |
|
The description is now outdated so please don't review the PR right now, its still work in progress. I will update the description later. I should have marked it as a draft. Sorry about that. I will request a review when its done. |
refactor: fix PluginOptions type refactor: smol wording changes
refactor: replace bash content with CodeBlock in markdown files build: include patch number in llms plugin version
|
I have updated the description and this PR is ready to be merged. There are a few hiccups in the actual outcome of the implementation, but imo its a good start. The terrible UX is because of the difference in the URLs on which markdowns are hosted and the URLs on which the HTML pages are hosted. I have opened an issue for that in the plugin repo. Once the above issue is resolved, we can add a new page similar to https://docs.stripe.com/building-with-llms.md but until then, I will hold onto that. @PaulRBerg if you can have a quick look and re-approve the PR (or provide your thoughts, if any), that would be great. |
|
@PaulRBerg should I merge this PR? |
|
will review this weekend |
This PR addresses the following issues:
HardcodedDemonstration.tsxto_hardcoded-demonstration.mdxand move it tosnippetspropinstead of exporting multiple variables.CodeBlockinstead of typical markdown approach for encapsulating large code lines.buncommand instead ofjustcommand in guidesIntroduction
Stripe docs are very likely built on gitbook which provides "LLM-ready docs" as a native feature. All these LLM-ready docs integrations loosely (not yet widely accepted) follow the standard defined by llmstxt.org.
Docusaurus currently lacks this feature, but I came across docusaurus-plugin-llms which is mentioned on llmstxt website. Its not yet fully matured but a good start until this feature is natively provided by Docusaurus, similar to Gitbook.
Links
An expected behaviour is to host /concepts/what-is-sablier on
/concepts/what-is-sablier.md. However, the plugin does not expose markdown files through the original URLs and instead host all markdown files directly from thebuilddirectly.So, the above page would be hosted on /what-is-sablier.md.
Similarly, /guides/lockup/examples/local-environment is hosted at /configure-your-local-environment.md and for Flow it will be on /configure-your-local-environment-1.md. This is a terrible UX. As a result, one should refer to
llms.txtto find all the valid URLs.I have opened an issue in the plugin repo requesting this feature.
All links can be found at /llms.txt.
Single markdown files
For the purpose of LLM training and fine tuning, single markdown files are hosted on /llms-training-data.txt. Similarly, protocol specific single files are hosted on /llms-flow.txt and so on.