You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- View AI crawlers that are interacting with pages in your domain (a [Cloudflare zone](/fundamentals/concepts/accounts-and-zones/#zones)).
19
-
-Create a rule to block AI crawlers on your pages.
19
+
-Use AI Audit to block individual crawlers from accessing your content.
20
20
21
21
## Prerequisites
22
22
23
23
1. Sign up for a [Cloudflare account](https://dash.cloudflare.com/sign-up/).
24
24
2.[Connect your domain to Cloudflare](/fundamentals/manage-domains/add-site/).
25
25
3. Make sure your domain is [proxying traffic through Cloudflare](/fundamentals/concepts/how-cloudflare-works/#cloudflare-as-a-reverse-proxy).
26
26
27
-
## 1. Block all AI crawlers
28
-
29
-
<Tabs>
30
-
<TabItemlabel="All plans">
31
-
To block all AI crawlers:
32
-
33
-
{/* prettier-ignore */}
34
-
<Steps>
35
-
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/), and select your account and domain.
36
-
2. Go to **AI Audit**.
37
-
3. Go to the **AI Crawlers** tab.
38
-
4. From **Block AI Bots**, toggle the switch.
39
-
</Steps>
40
-
41
-
This blocks all AI crawlers requesting access to your domain. You will also see all of the individual switches for AI crawlers automatically toggle to **Block**.
42
-
43
-
Note that blocking all AI crawlers using this method also enables Cloudflare's [Bot Fight Mode: Block AI bots](/bots/get-started/bot-fight-mode/#block-ai-bots).
44
-
45
-
<Detailsheader="Which bots will Cloudflare block?">
46
-
<Renderfile="list-ai-bots"product="bots" />
47
-
</Details>
48
-
49
-
:::note
50
-
For more details on how this rule interacts with other Cloudflare settings, refer to [How it works](/bots/concepts/bot/#how-it-works).
51
-
:::
52
-
</TabItem>
53
-
</Tabs>
54
-
55
-
## 2. Block specific AI crawlers
27
+
## 1. Block specific AI crawlers
56
28
57
29
<TabssyncKey="plans">
58
30
<TabItemlabel="Free plan">
@@ -95,13 +67,10 @@ For more information, refer to [Manage AI crawlers](/ai-audit/features/manage-ai
95
67
96
68
You can also create more complex rules when taking action on AI crawlers, using [Cloudflare WAF](/waf/). For more information on creating more specific rules, refer to [Create a custom rule in the dashboard](/waf/custom-rules/create-dashboard/).
97
69
98
-
## 3. Review detected AI crawlers (Enterprise plan only)
70
+
## 2. Review detected AI crawlers
99
71
100
-
<TabssyncKey="plans">
101
-
<TabItemlabel="Free plan">
102
-
AI Audit metrics are only available on paid plans. Upgrade your plan to view key metrics on AI crawlers.
103
-
104
-
</TabItem><TabItemlabel="Paid plan">
72
+
<Tabs>
73
+
<TabItemlabel="All plans">
105
74
106
75
Review the AI crawlers detected on your site.
107
76
@@ -112,7 +81,7 @@ Review the AI crawlers detected on your site.
112
81
</TabItem>
113
82
</Tabs>
114
83
115
-
Refer to [Analyze AI crawlers](/ai-audit/features/analyze-ai-crawlers/) for more information on AI Audit metrics.
84
+
Note that on free plans, the **Metrics** tab only displays metrics for the past 24 hours.
0 commit comments