Un-rendered Navigation Links

[jamescat]

Example URL(s)

• https://developers.cloudflare.com/browser-rendering/
• https://developers.cloudflare.com/kv/

Actual Behavior

Seemingly any page that includes the "LLM resources" navigation heading / sub-list, is failing to properly render the content of the links within that section. (E.g. the "/kv/llms-full.txt" & "/workers/prompt.txt" links.)

Instead, they are merely delivered as plain text content with no rendering.

Expected Behavior

These links appear to go to URLs that are mostly Markdown content, which should presumably be being rendered into HTML as a normal part of the documentation when the link is visited.

Additional information

Sample screenshots

• Screenshot 1 -- relevant navigation links

• Screenshot 2 -- sample un-rendered content

[Oxyjun]

Hey @jamescat thanks for this issue.

The LLM resources pages are designed to be easily copied and pasted for LLM ingestion, so that users can easily load the information onto their AI tools. For this purpose, the .txt files are designed to simply be plaintext (as seen in screenshot 2).

Hope that makes sense!

[jamescat]

Hunh... You're telling me that a list of URL's like llms.txt is intended to help build out an LLM?? [shakes head] Okay, if you say so.

They all seem like basically useless content for an LLM (all links, or overly brief statements, plus some sample code), so I was assuming they'd be pages that pointed users to LLM-relevant pages from any contained links inside the text data (which I barely glanced at, I suppose), or something along those lines.

But if that's their purpose (despite the apparent lack of appropriateness, at least immediately & from my POV anyway) then... [shrug] So be it. Case closed then, I suppose.

[kodster28]

Hunh... You're telling me that a list of URL's like llms.txt is intended to help build out an LLM?? [shakes head] Okay, if you say so.

They all seem like basically useless content for an LLM (all links, or overly brief statements, plus some sample code), so I was assuming they'd be pages that pointed users to LLM-relevant pages from any contained links inside the text data (which I barely glanced at, I suppose), or something along those lines.

But if that's their purpose (despite the apparent lack of appropriateness, at least immediately & from my POV anyway) then... [shrug] So be it. Case closed then, I suppose.

Thanks for raising!

It's technically a proposal for a standard, but it's the only signal that anyone has really given to those of us in the content space. We've seen mixed results on it, tbh (only a few crawlers ever hit it) + Google explicitly says don't worry about those sorts of things, we know how to scrape content.

The unrendered bit does matter a lot though, more details in our style guide.

It's an evolving space, for sure, and a lot of it involves experimenting around and seeing what works.