[Bots] Managed robots.txt update (cloudflare#23213)

* text edits

* changelog

* changelog

* Update src/content/docs/bots/additional-configurations/managed-robots-txt.mdx

Co-authored-by: Kate Tungusova <[email protected]>

* Update src/content/docs/bots/additional-configurations/managed-robots-txt.mdx

Co-authored-by: Kate Tungusova <[email protected]>

---------

Co-authored-by: Kate Tungusova <[email protected]>

Author: 2 people

src/content/docs/bots/additional-configurations/managed-robots-txt.mdx

@@ -16,7 +16,10 @@ AI bots are expected to follow the `robots.txt` directives. Otherwise, they risk
 
 ## Compatibility with existing `robots.txt` files
 
-Cloudflare will indepedently check whether your website has an existing `robots.txt` file. If your website already has one — verified by a HTTP `200` response — we will prepend our managed `robots.txt` before your existing `robots.txt`, combining both into a single response. 
+Cloudflare will independently check whether your website has an existing `robots.txt` file. 
+
+- If your website already has a `robots.txt` file — verified by a HTTP `200` response — Cloudflare will prepend our managed `robots.txt` before your existing `robots.txt`, combining both into a single response. 
+- If your website does not have a `robots.txt` file, it will receive Cloudflare's managed block directives.
 
 To implement a `robots.txt` file on your domain based on your plan:
 
@@ -33,4 +36,5 @@ To implement a `robots.txt` file on your domain based on your plan:
 </Tabs>
 
 ## Availability
+
 Managed `robots.txt` for AI crawlers is available on all plans.

src/content/release-notes/bots.yaml

@@ -5,6 +5,9 @@ productLink: "/bots/"
 productArea: Application security
 productAreaLink: /fundamentals/reference/changelog/security/
 entries:
+  - publish_date: "2025-07-02"
+    title: Managed robots.txt will prepend existing files
+    description: Cloudflare will prepend our managed `robots.txt` before your existing `robots.txt`, combining both into a single response.
   - publish_date: "2025-05-14"
     title: Anomaly detection events now receive a bot score of 2
     description: Events detected by the [anomaly detection engine](/bots/concepts/bot-detection-engines/#anomaly-detection-enterprise) are now given a bot score of 2.