Adding more content, matching text to current UI.

Author: Oxyjun

src/content/docs/ai-audit/features/ai-crawler-detection.mdx renamed to src/content/docs/ai-audit/features/detect-ai-crawlers.mdx

@@ -1,5 +1,5 @@
 ---
-title: AI crawler detection
+title: Detect AI crawlers
 pcx_content_type: concept
 sidebar:
   order: 2
@@ -17,6 +17,8 @@ AI Audit provides you with the following metrics to help you understand how AI c
 | Summary                               | A list of AI crawlers with the most number of crawl requests                  |
 | Most popular paths by AI crawlers     | The most popular pages crawled by AI crawlers, for each AI crawler            |
 
+The **Summary** table also enables you to [Direct AI crawlers with managed robots.txt](/bots/additional-configurations/managed-robots-txt/).
+
 ## Filter AI crawler data
 
 You can use filters to narrow the scope of your result.

src/content/docs/ai-audit/features/enforce-robots-txt.mdx

@@ -0,0 +1,37 @@
+---
+title: Enforce robots.txt
+pcx_content_type: concept
+sidebar:
+  order: 5
+---
+
+import { Steps } from "~/components";
+
+AI Audit allows you to enforce [`robots.txt`](/radar/glossary/#robotstxt) which instructs bots which webpages they can and cannot access.
+
+To enforce `robots.txt`:
+
+<Steps>
+1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/), and select your account and domain.
+2. Go to **AI Audit**.
+3. From the dropdown at the top of the page, select a specific subdomain where you wish you enforce `robots.txt`.
+4. From **Summary**, select **Enforce robots.txt policy**.
+5. From the **Enforce your robots.txt policy** page, select **Go to WAF custom rules**.
+6. From the **New custom rule** page, name your custom rule.
+   - The page will automatically populate the values for the custom rule.
+7. From **Then take action...**:
+   - For **Choose action**, select **Block**.
+   - For **With response type**, select **Default Cloudflare WAF block page**.
+8. From **Place at**:
+   - For **Select order**, select **Last**.
+9. Select **Deploy**.
+</Steps>
+
+This custom rule ensures that bots which do not respect `robots.txt` are blocked.
+
+## Related resources
+
+For more information, refer to the following resources.
+
+- [What is robots.txt? | How a robots.txt file works](https://www.cloudflare.com/en-gb/learning/bots/what-is-robots-txt/)
+- [Direct AI crawlers with managed robots.txt](/bots/additional-configurations/managed-robots-txt/)

src/content/docs/ai-audit/get-started.mdx

@@ -36,22 +36,36 @@ To use AI Audit:
 4. From the **Bot traffic** page, under **Block AI Bots**, select **Enable**.
 </Steps>
 
-## 2. Block specific AI crawlers (Enterprise plan only)
+## 2. Block specific bot categories (Enterprise plan only)
 
 Customers on the Enterprise plan can choose to only block specific AI crawlers, while allowing others.
 
 {/* prettier-ignore */}
 <Steps>
 1. Go to the **AI Audit**.
 2. From **Most Popular Paths**, select **Block Some**.
-3. From the **Security rules** page,
+3. From the **Security rules** page, select **Create rule** > **Custom rules**.
+4. Provide a name for the custom rule. For example, "Block unwanted AI crawlers".
+5. From the **Field** dropdown, select **Verified Bot Category**.
+6. From the **Value** dropdown, select the specific bot category you wish to block.
+   - You can use **And** / **Or** buttons to add additional conditions. For example, you can use multiple **Or** options to include multiple bot categories in the same rule.
+7. From the **Then take action...** section:
+   - For **Choose action**, select **Block**.
+   - For **With response type**, select **Default Cloudflare WAF block page**.
+8. From the **Place at** section:
+   - For **Select order**, select **First**.
+9. Select **Save**.
 </Steps>
 
+This custom rule will only block the AI bots which belong to the [verified bot categories](/bots/concepts/bot/verified-bots/categories/) you have included in your rule (in step 6).
+
+For more information on creating a custom WAF rule, refer to [Create a custom rule in the dashboard](/waf/custom-rules/create-dashboard/).
+
 ## 3. Review detected AI crawlers
 
 Review the AI crawlers detected on your site in the **Metrics** tab of the Cloudflare dashboard for key metrics.
 
-Refer to [AI crawler detection](/ai-audit/features/ai-crawler-detection/) for more information.
+Refer to [Detect AI crawlers](/ai-audit/features/detect-ai-crawlers/) for more information.
 
 ## Related resources