Skip to content

Commit 08dd85f

Browse files
block ai bots updates
1 parent 35b3b01 commit 08dd85f

File tree

2 files changed

+29
-2
lines changed

2 files changed

+29
-2
lines changed

src/content/partials/bots/ai-bots-definition.mdx

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,31 @@
33

44
---
55

6-
You can block artificial intelligence (AI) bots, crawlers, and scrapers from scraping your website content and training large language models (LLM) to recreate it without your permission. When you enable this feature via a pre-configured managed rule, Cloudflare can detect and block verified AI bots that respect `robots.txt` and crawl rate, and do not hide their behavior from your website. The rule has also been expanded to include more signatures of AI bots that do not follow the rules.
6+
You can block artificial intelligence (AI) bots, crawlers, and scrapers from scraping your website content and training large language models (LLM) to recreate it without your permission.
7+
8+
:::note
9+
Some AI bots overlap with definitely automated bots and verified bots, the latter becoming verified AI bots.
10+
:::
11+
12+
When you enable this feature via a pre-configured managed rule, Cloudflare can detect and block verified AI bots that comply with `robots.txt` and respect crawl rates, and do not hide their behavior from your website. The rule has also been expanded to include more signatures of AI bots that do not follow the rules.
13+
14+
The rule to block AI bots takes precedence over all other Super Bot Fight Mode rules. For example, if you have enabled **Block AI bots** and **Allow verified bots**, verified AI bots will also be blocked even if you allow other verified bots on your website or application.
15+
16+
For Bot Management customers, if you have set a rule to serve managed challenges to definitely automated bots, AI bots will also be challenged because custom rules run in a phase before Super Bot Fight Mode, which is the phase when the rule to block AI bots runs.
17+
18+
This behavior remains the same if the setting for verified, definitely automated, and likely bots is set to `block` or `allow`. If you have an action to `allow` for these rules, the request is not matched to any rule and proceeds to the next ruleset phase. Similarly, if the action is set to block, they will be blocked in the earlier phase and do not move on to match the AI rule at all. However, when the action is `challenge`, the request matches a rule and therefore will not be matched to any rules after.
19+
20+
For non-Bot Management customers, all rules for verified, definitely automated, and likely bots run in the phase following the AI bots rule.
21+
22+
```mermaid
23+
---
24+
title: Rule phases
25+
---
26+
flowchart LR
27+
accTitle: AI bots rule phases diagram
28+
accDescr: This diagram details the phases in which AI bots rules run.
29+
A[Custom rules] --> B[Block AI bots rule] --> C[Allow verified bots rule]
30+
31+
```
732

833
This feature is available on all Cloudflare plans.

src/content/partials/bots/block-ai-bots-enable.mdx

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@ inputParameters: param1
55

66
import { Markdown } from "~/components"
77

8+
To block AI bots:
9+
810
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/) and select your account and domain.
911
2. Go to **Security** > **Bots**.
1012
3. Select **Configure {props.one}**.
11-
4. Enable **Block AI bots**.
13+
4. Enable **Block AI bots**.

0 commit comments

Comments
 (0)