Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,10 +1,26 @@
---
pcx_content_type: overview
title: AI Labyrinth
sidebar:
order: 7
---

import { Render } from "~/components"
import { Render, Tabs, TabItem } from "~/components"

The AI Labyrinth adds invisible links on your webpage with specific `Nofollow` tags to block AI crawlers that do not adhere to the recommended guidelines and crawl without permission. AI crawlers that scrape your website content without permission will be stuck in a maze of never-ending links, and their details are recorded and used by all Cloudflare customers who choose to block [AI bots](/bots/concepts/bot/#ai-bots).

These links do not impact your search engine optimization (SEO) or your website's appearance, and are only seen by bots. AI bots that respect no-crawl instructions will safely ignore this honeypot.

To enable [AI Labyrinth](/bots/additional-configurations/ai-labyrinth) based on your plan:

<Tabs>
<TabItem label="Bot Fight Mode">
<Render file="ai-labyrinth-enable" params={{ one: "Bot Fight Mode" }} />
</TabItem>
<TabItem label="Super Bot Fight Mode">
<Render file="ai-labyrinth-enable" params={{ one: "Super Bot Fight Mode" }} />
</TabItem>
<TabItem label="Bot Management for Enterprise">
<Render file="ai-labyrinth-enable" params={{ one: "Bot Management" }} />
</TabItem>
</Tabs>
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
pcx_content_type: reference
title: Static resource protection
sidebar:
order: 7
order: 8

---

Expand Down
4 changes: 0 additions & 4 deletions src/content/docs/bots/get-started/bot-fight-mode.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,6 @@ To disable Bot Fight Mode:
You can view blocked AI bot traffic via [Security Analytics](/waf/analytics/security-analytics/).
:::

### Enable AI Labyrinth

<Render file="ai-labyrinth-enable" params={{ one: "Bot Fight Mode" }} />

## Visibility

You can see bot-related actions by going to **Security** > **Events**. Any requests challenged by this product will be labeled **Bot Fight Mode** in the **Service** field. This allows you to observe, analyze, and follow trends in your bot traffic over time.
Expand Down
22 changes: 12 additions & 10 deletions src/content/docs/bots/get-started/bot-management.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -47,28 +47,30 @@ Cloudflare recommends that you deploy the following basic settings and customize
You can view blocked AI bot traffic via [Security Analytics](/waf/analytics/security-analytics/).
:::

### Enable AI Labyrinth
### Enable Javascript detections

<Render file="ai-labyrinth-enable" params={{ one: "Bot Management" }} />

### Enable Javascript Detections

Enabling [JavaScript Detections](/bots/additional-configurations/javascript-detections/) validates that the browser can run JavaScript, and is stored in the `cf.bot_management.js_detection.passed` variable.
Enabling [JavaScript detections](/bots/additional-configurations/javascript-detections/) validates that the browser can run JavaScript, and is stored in the `cf.bot_management.js_detection.passed` variable.

<Render file="javascript-detections-enable" params={{ one: "Bot Management" }} />

### Deploy default templates

Cloudflare has [default templates](https://dash.cloudflare.com/?to=/:account/:zone/security/security-rules?template=bot_traffic) for definite bots, which we are very confident are automated (bot score 1) and likely bots that have many bot tells (bot score 2-29). In our templates, we recommend to allow verified bots like Google SEO crawler and access to static resources, which should be cached anyway.
Cloudflare has [default templates](https://dash.cloudflare.com/?to=/:account/:zone/security/security-rules?template=bot_traffic) for definite bots, which have a [bot score](/bots/concepts/bot-score/) of 1, and likely bots which have a bot score of 2 to 29. In our templates, Cloudflare recommends to allow verified bots such as Google SEO Crawler and access to cached static resources.

- [Definite Bots template](https://dash.cloudflare.com/?to=/:account/:zone:/security/security-rules/custom-rules/create?template=Definitely%20Bots): Targets malicious bot traffic while ignoring verified bots and routes delivering static content.

`(cf.bot_management.score eq 1 and not cf.bot_management.verified_bot and not cf.bot_management.static_resource)`
```txt wrap
(cf.bot_management.score eq 1 and not cf.bot_management.verified_bot and not cf.bot_management.static_resource)
```

- [Likely Bots template](https://dash.cloudflare.com/?to=/:account/:zone/security/security-rules/custom-rules/create?template=Likely%20Bots): Targets traffic likely to be malicious bots while ignoring verified bots and routes with static content. It may contain a small amount of non-bot traffic.

`(cf.bot_management.score ge 2 and cf.bot_management.score le 29 and not cf.bot_management.verified_bot and not cf.bot_management.static_resource)`
```txt wrap
(cf.bot_management.score ge 2 and cf.bot_management.score le 29 and not cf.bot_management.verified_bot and not cf.bot_management.static_resource)
```

- (Optional) [JavaScript detections template](https://dash.cloudflare.com/?to=/:account/:zone/security/security-rules/custom-rules/create?template=JavaScript%20Verified%20URLs): If you enabled JavaScript detections, then set up a [managed challenge](/fundamentals/security/cloudflare-challenges/#managed-challenge-recommended), make sure to add a method and URI path. JavaScript detections improves security for URLs that should only expect JavaScript-enabled clients.

`(not cf.bot_management.js_detection.passed and http.request.method eq "" and http.request.uri.path in {""})`
```txt wrap
(not cf.bot_management.js_detection.passed and http.request.method eq "" and http.request.uri.path in {""})
```
4 changes: 0 additions & 4 deletions src/content/docs/bots/get-started/super-bot-fight-mode.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,6 @@ Accounts with an Enterprise subscription but not the [Bot Management add-on](/bo
You can view blocked AI bot traffic via [Security Analytics](/waf/analytics/security-analytics/).
:::

### Enable AI Labyrinth

<Render file="ai-labyrinth-enable" params={{ one: "Super Bot Fight Mode" }} />

## Analytics

### Bot Report
Expand Down
4 changes: 0 additions & 4 deletions src/content/partials/bots/ai-labyrinth-enable.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,6 @@ inputParameters: param1

import { Markdown } from "~/components"

AI Labyrinth decreases the accuracy and wastes the resources of AI Bots by confabulating articles on your website that are not visible to users.

To enable [AI Labyrinth](/bots/additional-configurations/ai-labyrinth):

1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/) and select your account and domain.
2. Go to **Security** > **Bots**.
3. Select **Configure {props.one}**.
Expand Down
Loading