Skip to content

Commit 945ce86

Browse files
[Bots] block AI bots updates (#17826)
* block ai bots updates * space * update placement and link * wording edit * Apply suggestions from code review Co-authored-by: Pedro Sousa <[email protected]> --------- Co-authored-by: Pedro Sousa <[email protected]>
1 parent a06379c commit 945ce86

File tree

6 files changed

+28
-10
lines changed

6 files changed

+28
-10
lines changed

src/content/docs/bots/get-started/biz-and-ent.mdx

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,6 @@ If you have an Enterprise subscription without Bot Management, your application
3737

3838
## Block AI bots
3939

40-
<Render file="ai-bots-definition" />
41-
4240
<Render file="block-ai-bots-enable" params={{ one: "Super Bot Fight Mode" }} />
4341

4442
:::note

src/content/docs/bots/get-started/bm-subscription.mdx

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,6 @@ If you are not seeing Bot Management enabled on your zone or if you still see **
3333

3434
## Block AI bots
3535

36-
<Render file="ai-bots-definition" />
37-
3836
<Render file="block-ai-bots-enable" params={{ one: "Bot Management" }} />
3937

4038
:::note

src/content/docs/bots/get-started/free.mdx

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,6 @@ To disable Bot Fight Mode:
2929

3030
## Block AI bots
3131

32-
<Render file="ai-bots-definition" />
33-
3432
<Render file="block-ai-bots-enable" params={{ one: "Bot Fight Mode" }} />
3533

3634
:::note

src/content/docs/bots/get-started/pro.mdx

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,6 @@ Super Bot Fight Mode is included in your Pro subscription. When enabled, the pro
3232

3333
## Block AI bots
3434

35-
<Render file="ai-bots-definition" />
36-
3735
<Render file="block-ai-bots-enable" params={{ one: "Super Bot Fight Mode" }} />
3836

3937
:::note

src/content/partials/bots/ai-bots-definition.mdx

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,30 @@
33

44
---
55

6-
You can block artificial intelligence (AI) bots, crawlers, and scrapers from scraping your website content and training large language models (LLM) to recreate it without your permission. When you enable this feature via a pre-configured managed rule, Cloudflare can detect and block verified AI bots that respect `robots.txt` and crawl rate, and do not hide their behavior from your website. The rule has also been expanded to include more signatures of AI bots that do not follow the rules.
6+
You can block artificial intelligence (AI) bots, crawlers, and scrapers from scraping your website content and training large language models (LLM) to recreate it without your permission.
7+
8+
:::note
9+
Some AI bots overlap with definitely automated bots and verified bots, the latter becoming verified AI bots.
10+
:::
11+
12+
When you enable this feature via a pre-configured managed rule, Cloudflare can detect and block verified AI bots that comply with `robots.txt` and respect crawl rates, and do not hide their behavior from your website. The rule has also been expanded to include more signatures of AI bots that do not follow the rules.
13+
14+
The rule to block AI bots takes precedence over all other Super Bot Fight Mode rules. For example, if you have enabled **Block AI bots** and **Allow verified bots**, verified AI bots will also be blocked even if you allow other verified bots on your website or application.
15+
16+
For Bot Management customers, if you have set a rule to serve managed challenges to definitely automated bots, AI bots will also be challenged because custom rules run in a phase before Super Bot Fight Mode, which is the phase when the rule to block AI bots runs.
17+
18+
This behavior remains the same if the setting for verified, definitely automated, and likely bots is set to `block` or `allow`. If you have an action to `allow` for these rules, the request is not matched to any rule and proceeds to the next ruleset phase. Similarly, if the action is set to `block`, they will be blocked in the earlier phase and do not move on to match the AI rule at all. However, when the action is `challenge`, the request matches a rule and therefore will not be matched to any rules after.
19+
20+
For self-serve non-Bot Management customers, all rules for verified, definitely automated, and likely bots run in the phase following the AI bots rule.
21+
22+
```mermaid
23+
---
24+
title: Rule phases
25+
---
26+
flowchart LR
27+
accTitle: AI bots rule phases diagram
28+
accDescr: This diagram details the phases in which AI bots rules run.
29+
A[Custom rules] --> B[Block AI bots<br>managed rule] --> C[Allow verified bots rule]
30+
```
731

832
This feature is available on all Cloudflare plans.

src/content/partials/bots/block-ai-bots-enable.mdx

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@ inputParameters: param1
55

66
import { Markdown } from "~/components"
77

8+
To block [AI bots](/bots/concepts/bot/#ai-bots):
9+
810
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/) and select your account and domain.
911
2. Go to **Security** > **Bots**.
1012
3. Select **Configure {props.one}**.
11-
4. Enable **Block AI bots**.
13+
4. Enable **Block AI bots**.

0 commit comments

Comments
 (0)