Skip to content

Commit 8c68653

Browse files
committed
Final alignment changes
1 parent f193bbd commit 8c68653

File tree

5 files changed

+31
-44
lines changed

5 files changed

+31
-44
lines changed
-24.6 KB
Loading

src/content/changelog/ai-audit/2024-09-23-ai-audit-launch.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ Every site on Cloudflare now has access to [**AI Audit**](/ai-audit/), which sum
99
You can use this data to:
1010

1111
- Understand how and how often crawlers access your site (and which content is the most popular).
12-
- Block some or all of the AI bots accessing your site.
12+
- Block specific AI bots accessing your site.
1313
- Use Cloudflare to enforce your `robots.txt` policy via an automatic WAF rule.
1414

1515
![View AI bot activity with AI Audit](~/assets/images/changelog/ai-audit/ai-audit-overview.png)

src/content/docs/ai-audit/features/analyze-ai-crawlers.mdx

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@ sidebar:
55
order: 7
66
---
77

8+
import { Tabs, TabItem } from "~/components";
9+
810
:::note[AI Audit metrics]
911
AI Audit metrics are only available for customers on a paid plan.
1012
:::
@@ -32,13 +34,27 @@ AI Audit provides you with the following metrics to help you understand how AI c
3234
| Paths | Top five most requested paths |
3335
| AI crawler operators | Top five most active AI crawler operators (by requests) |
3436

35-
## Filter the date range
37+
## Filter date range
38+
39+
You can use the date filter to choose the period of time you wish to analyze.
40+
41+
<Tabs>
42+
<TabItem label="Free plans">
43+
44+
Filter options:
45+
46+
- Past 24 hours
47+
48+
</TabItem><TabItem label="Paid plans">
49+
50+
Filter options:
3651

37-
You can use date filter to narrow the scope of your result.
52+
- Past 24 hours
53+
- Past 7 days
54+
- Past 14 days
55+
- Past month
3856

39-
- **Date range:** Filter the date range of your results. You can choose from three predetermined date ranges:
40-
- Past 7 days
41-
- Past 14 days
42-
- Past month
57+
</TabItem>
58+
</Tabs>
4359

4460
The values of the AI Audit metrics will update according to your filter.

src/content/docs/ai-audit/features/manage-ai-crawlers.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ The **AI Crawlers** tab displays a table of AI crawlers that are requesting acce
4646

4747
You can block an AI crawler to completely stop the AI crawler from scraping your webpage.
4848

49+
Use the toggles in the **Block** column to block specific AI crawlers from accessing your content.
50+
4951
## Filter AI crawler data
5052

5153
You can use filters to narrow the scope of your result.

src/content/docs/ai-audit/get-started.mdx

Lines changed: 6 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -16,43 +16,15 @@ import { Details, Render, Steps, Tabs, TabItem, GlossaryTooltip } from "~/compon
1616
This guide instructs you on how to:
1717

1818
- View AI crawlers that are interacting with pages in your domain (a [Cloudflare zone](/fundamentals/concepts/accounts-and-zones/#zones)).
19-
- Create a rule to block AI crawlers on your pages.
19+
- Use AI Audit to block individual crawlers from accessing your content.
2020

2121
## Prerequisites
2222

2323
1. Sign up for a [Cloudflare account](https://dash.cloudflare.com/sign-up/).
2424
2. [Connect your domain to Cloudflare](/fundamentals/manage-domains/add-site/).
2525
3. Make sure your domain is [proxying traffic through Cloudflare](/fundamentals/concepts/how-cloudflare-works/#cloudflare-as-a-reverse-proxy).
2626

27-
## 1. Block all AI crawlers
28-
29-
<Tabs>
30-
<TabItem label="All plans">
31-
To block all AI crawlers:
32-
33-
{/* prettier-ignore */}
34-
<Steps>
35-
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/), and select your account and domain.
36-
2. Go to **AI Audit**.
37-
3. Go to the **AI Crawlers** tab.
38-
4. From **Block AI Bots**, toggle the switch.
39-
</Steps>
40-
41-
This blocks all AI crawlers requesting access to your domain. You will also see all of the individual switches for AI crawlers automatically toggle to **Block**.
42-
43-
Note that blocking all AI crawlers using this method also enables Cloudflare's [Bot Fight Mode: Block AI bots](/bots/get-started/bot-fight-mode/#block-ai-bots).
44-
45-
<Details header="Which bots will Cloudflare block?">
46-
<Render file="list-ai-bots" product="bots" />
47-
</Details>
48-
49-
:::note
50-
For more details on how this rule interacts with other Cloudflare settings, refer to [How it works](/bots/concepts/bot/#how-it-works).
51-
:::
52-
</TabItem>
53-
</Tabs>
54-
55-
## 2. Block specific AI crawlers
27+
## 1. Block specific AI crawlers
5628

5729
<Tabs syncKey="plans">
5830
<TabItem label="Free plan">
@@ -95,13 +67,10 @@ For more information, refer to [Manage AI crawlers](/ai-audit/features/manage-ai
9567

9668
You can also create more complex rules when taking action on AI crawlers, using [Cloudflare WAF](/waf/). For more information on creating more specific rules, refer to [Create a custom rule in the dashboard](/waf/custom-rules/create-dashboard/).
9769

98-
## 3. Review detected AI crawlers (Enterprise plan only)
70+
## 2. Review detected AI crawlers
9971

100-
<Tabs syncKey="plans">
101-
<TabItem label="Free plan">
102-
AI Audit metrics are only available on paid plans. Upgrade your plan to view key metrics on AI crawlers.
103-
104-
</TabItem><TabItem label="Paid plan">
72+
<Tabs>
73+
<TabItem label="All plans">
10574

10675
Review the AI crawlers detected on your site.
10776

@@ -112,7 +81,7 @@ Review the AI crawlers detected on your site.
11281
</TabItem>
11382
</Tabs>
11483

115-
Refer to [Analyze AI crawlers](/ai-audit/features/analyze-ai-crawlers/) for more information on AI Audit metrics.
84+
Note that on free plans, the **Metrics** tab only displays metrics for the past 24 hours.
11685

11786
## Related resources
11887

0 commit comments

Comments
 (0)