[Bots] Update managed robots.txt (cloudflare#23328)

* [Bots] Update managed robots.txt

* small tweaks

* Test

* Update

Author: kodster28

src/content/docs/bots/additional-configurations/managed-robots-txt.mdx

@@ -6,33 +6,62 @@ sidebar:
   label: Managed robots.txt
 ---
 
-import { Render, Tabs, TabItem } from "~/components"
+import { Render, Tabs, TabItem } from "~/components";
 
-Protect your website or application from AI crawlers by implementing a `robots.txt` file on your domain to direct AI bot operators on what content they can and cannot scrape for AI model training. 
+Protect your website or application from AI crawlers by implementing a `robots.txt` file on your domain to direct AI bot operators on what content they can and cannot scrape for AI model training.
 
-Cloudflare's managed `robots.txt` explicitly disallows known bots engaged in scraping for AI purposes and AI agent activity. 
+Cloudflare's managed `robots.txt` explicitly disallows known bots engaged in scraping for AI purposes.
 
-AI bots are expected to follow the `robots.txt` directives. Otherwise, they risk getting banned. 
+AI bots are expected to follow the `robots.txt` directives.
 
 ## Compatibility with existing `robots.txt` files
 
-Cloudflare will independently check whether your website has an existing `robots.txt` file. 
+Cloudflare will independently check whether your website has an existing `robots.txt` file and update the behavior of this feature based on your website.
 
-- If your website already has a `robots.txt` file — verified by a HTTP `200` response — Cloudflare will prepend our managed `robots.txt` before your existing `robots.txt`, combining both into a single response. 
-- If your website does not have a `robots.txt` file, it will receive Cloudflare's managed block directives.
+### Existing robots.txt file
+
+If your website already has a `robots.txt` file — verified by a HTTP `200` response — Cloudflare will prepend our managed `robots.txt` before your existing `robots.txt`, combining both into a single response.
+
+For example, without this feature enabled, the `robots.txt` content of `crawlstop.com` would be:
+
+```txt
+User-agent: *
+Disallow: /lp
+Disallow: /feedback
+Disallow: /langtest
+
+Sitemap: https://www.crawlstop.com/sitemap.xml
+```
+
+With the managed `robots.txt` enabled, Cloudflare will prepend our managed content before your original content, resulting in what you can view at https://crawlstop.com/robots.txt.
+
+### No robots.txt file
+
+If your website does not have a `robots.txt` file, Cloudflare creates a new file with our managed block directives and serves it for you.
+
+## Implementation
 
 To implement a `robots.txt` file on your domain based on your plan:
 
 <Tabs>
-  <TabItem label="Bot Fight Mode">
-    <Render file="enable-managed-robots-txt" params={{ one: "Bot Fight Mode" }} />
-  </TabItem>
-  <TabItem label="Super Bot Fight Mode">
-    <Render file="enable-managed-robots-txt" params={{ one: "Super Bot Fight Mode" }} />
-  </TabItem>
-    <TabItem label="Bot Management for Enterprise">
-    <Render file="enable-managed-robots-txt" params={{ one: "Bot Management" }} />
-  </TabItem>
+	<TabItem label="Bot Fight Mode">
+		<Render
+			file="enable-managed-robots-txt"
+			params={{ one: "Bot Fight Mode" }}
+		/>
+	</TabItem>
+	<TabItem label="Super Bot Fight Mode">
+		<Render
+			file="enable-managed-robots-txt"
+			params={{ one: "Super Bot Fight Mode" }}
+		/>
+	</TabItem>
+	<TabItem label="Bot Management for Enterprise">
+		<Render
+			file="enable-managed-robots-txt"
+			params={{ one: "Bot Management" }}
+		/>
+	</TabItem>
 </Tabs>
 
 ## Availability