diff --git a/src/content/docs/bots/concepts/bot/index.mdx b/src/content/docs/bots/concepts/bot/index.mdx index dc99df6e16d7db2..5fd6bdf8161ee86 100644 --- a/src/content/docs/bots/concepts/bot/index.mdx +++ b/src/content/docs/bots/concepts/bot/index.mdx @@ -19,15 +19,12 @@ Bots can be used for good (chatbots, search engine crawlers) or for evil (invent For more background, refer to [What is a bot?](https://www.cloudflare.com/learning/bots/what-is-a-bot/). ::: -## Verified bots +## Verified bots and signed agents -For more information, refer to [Verified bots](/bots/concepts/bot/verified-bots/). - :::note - -The method for allowing or blocking verified bots depends on [your plan](/bots/get-started/). +The method for allowing or blocking verified bots depends on [your plan](/bots/concepts/bot/verified-bots/#availability). ::: ## AI bots diff --git a/src/content/docs/bots/concepts/bot/verified-bots/index.mdx b/src/content/docs/bots/concepts/bot/verified-bots/index.mdx index 9b2a905c60efddd..5424605da54ea54 100644 --- a/src/content/docs/bots/concepts/bot/verified-bots/index.mdx +++ b/src/content/docs/bots/concepts/bot/verified-bots/index.mdx @@ -55,10 +55,14 @@ The bot must be verified using one of the following validation methods: You can segment your verified bot traffic by its type and purpose by adding the Verified Bot Categories field `cf.verified_bot_category` as a filter criteria in [WAF Custom rules](/waf/custom-rules/), [Advanced Rate Limiting](/waf/rate-limiting-rules/), and Late Transform rules. -:::note +:::caution The Verified Bot Categories field is not compatible with legacy Firewall rules. ::: +:::note +Verified Bot Categories is available on all plans. +::: + | Name | String value | Example | | -------------------------- | ---------------------------- | ----------------------------------------------------------------------- | | Academic research | `Academic Research` | Library of Congress, TurnItInBot, Bibliothèque nationale de France | @@ -79,6 +83,10 @@ The Verified Bot Categories field is not compatible with legacy Firewall rules. | Webhooks | `Webhooks` | Payment processors, WordPress Integration tools | | Other | `Other` | | -### Availability +--- -Verified Bot Categories is available on all plans. +## Availability + +Verified bots are excluded by default when [Bot Fight Mode](/bots/get-started/bot-fight-mode/) is enabled to block definite bots. + +[Super Bot Fight Mode](/bots/get-started/super-bot-fight-mode/) and [Enterprise Bot Management](/bots/get-started/bot-management/) customers have the option to block or allow verified bots. \ No newline at end of file diff --git a/src/content/docs/bots/get-started/bot-fight-mode.mdx b/src/content/docs/bots/get-started/bot-fight-mode.mdx index 117e6f06ab56192..0d9203053c09264 100644 --- a/src/content/docs/bots/get-started/bot-fight-mode.mdx +++ b/src/content/docs/bots/get-started/bot-fight-mode.mdx @@ -21,6 +21,8 @@ Bot Fight Mode is a simple, free product that helps detect and mitigate bot traf +--- + ## Enable Bot Fight Mode To start using Bot Fight Mode: @@ -46,6 +48,8 @@ To start using Bot Fight Mode: +--- + ## Disable Bot Fight Mode If you find that **Bot Fight Mode** is causing problems with your application traffic, you may want to disable it. @@ -71,6 +75,8 @@ To disable Bot Fight Mode: +--- + ## Block AI bots Refer to [Block AI bots](/bots/additional-configurations/block-ai-bots/). @@ -80,10 +86,14 @@ Refer to [Block AI bots](/bots/additional-configurations/block-ai-bots/). You can view blocked AI bot traffic via [Security Analytics](/waf/analytics/security-analytics/). ::: +--- + ## Visibility You can see bot-related actions by going to **Security** > **Events**. Any requests challenged by this product will be labeled **Bot Fight Mode** in the **Service** field. This allows you to observe, analyze, and follow trends in your bot traffic over time. +--- + ## Limitations You cannot bypass or skip Bot Fight Mode using the _Skip_ action in WAF custom rules or using Page Rules. _Skip_, _Bypass_, and _Allow_ actions apply to rules or rulesets running on the [Ruleset Engine](/ruleset-engine/). While Super Bot Fight Mode rules are implemented in the Ruleset Engine, Bot Fight Mode checks are not. This is why you can skip Super Bot Fight Mode, but not Bot Fight Mode. If you need to skip Bot Fight Mode, consider using [Super Bot Fight Mode](/bots/get-started/super-bot-fight-mode/). diff --git a/src/content/docs/bots/get-started/bot-management.mdx b/src/content/docs/bots/get-started/bot-management.mdx index f3d16cbe2f453dd..2b12df4ba14b259 100644 --- a/src/content/docs/bots/get-started/bot-management.mdx +++ b/src/content/docs/bots/get-started/bot-management.mdx @@ -19,6 +19,8 @@ This Enterprise product provides the most flexibility to customers by: - Allowing customers to take action on this score with [WAF custom rules](/waf/custom-rules/) or [`Workers`](/workers/runtime-apis/request/#incomingrequestcfproperties). - Allowing customers to view this score in Bot Analytics or Logs. +--- + ## Enable Bot Management for Enterprise Bot Management is automatically enabled for Enterprise zones entitled with the add-on. @@ -53,6 +55,8 @@ Bot Management is automatically enabled for Enterprise zones entitled with the a If you are not seeing Bot Management enabled on your zone or if you still see **Add Bot Management** on the Cloudflare dashboard, contact your account team for the proper entitlements. ::: +--- + ## Setup Cloudflare recommends that you deploy the following basic settings and customize them according to the traffic in your zone. @@ -69,12 +73,6 @@ Refer to [Block AI bots](/bots/additional-configurations/block-ai-bots/). You can view blocked AI bot traffic via [Security Analytics](/waf/analytics/security-analytics/). ::: -### Enable JavaScript detections - -Enabling [JavaScript detections](/bots/additional-configurations/javascript-detections/) validates that the browser can run JavaScript, and is stored in the `cf.bot_management.js_detection.passed` variable. - - - ### Deploy default templates Cloudflare has [default templates](https://dash.cloudflare.com/?to=/:account/:zone/security/security-rules?template=bot_traffic) for definite bots, which have a [bot score](/bots/concepts/bot-score/) of 1, and likely bots which have a bot score of 2 to 29. In our templates, Cloudflare recommends to allow verified bots such as Google SEO Crawler and access to cached static resources. diff --git a/src/content/docs/bots/get-started/super-bot-fight-mode.mdx b/src/content/docs/bots/get-started/super-bot-fight-mode.mdx index cfa3322b69dbed2..15dc1987f9f1834 100644 --- a/src/content/docs/bots/get-started/super-bot-fight-mode.mdx +++ b/src/content/docs/bots/get-started/super-bot-fight-mode.mdx @@ -64,10 +64,10 @@ To start using Super Bot Fight Mode: If your organization also uses [Cloudflare Tunnel](/cloudflare-one/connections/connect-networks/), keep **Definitely Automated** set to **Allow**. Otherwise, tunnels might fail with a `websocket: bad handshake` error. ::: -### Conditions - +--- + ## Disable Super Bot Fight Mode If you find that **Super Bot Fight Mode** is causing problems with your application traffic, you may want to disable it. @@ -99,6 +99,8 @@ To disable Super Bot Fight Mode: +--- + ## Block AI bots Refer to [Block AI bots](/bots/additional-configurations/block-ai-bots/). @@ -108,6 +110,8 @@ Refer to [Block AI bots](/bots/additional-configurations/block-ai-bots/). You can view blocked AI bot traffic via [Security Analytics](/waf/analytics/security-analytics/). ::: +--- + ## Analytics ### Bot Report @@ -122,6 +126,8 @@ To access the **Bot Report**, go to **Security** > **Bots**. If you see a double You can see bot-related actions by going to **Security** > **Events**. Any requests challenged by this product will be labeled **Super Bot Fight Mode** in the **Service** field. This allows you to observe, analyze, and follow trends in your bot traffic over time. +--- + ## Ruleset Engine Super Bot Fight Mode runs during the `http_request_sbfm` phase of the [Ruleset Engine](/ruleset-engine/about/phases/). diff --git a/src/content/partials/bots/verified-bots.mdx b/src/content/partials/bots/verified-bots.mdx index 07711cc0fbb4da2..643ff4708ca88c8 100644 --- a/src/content/partials/bots/verified-bots.mdx +++ b/src/content/partials/bots/verified-bots.mdx @@ -3,10 +3,8 @@ --- -import { GlossaryTooltip } from "~/components" +Cloudflare maintains an internal directory of [verified bot](/bots/concepts/bot/verified-bots/) and [signed agents](/bots/concepts/bot/signed-agents/) that are associated with search engine optimization (SEO), website monitoring, and more. -Cloudflare maintains an internal list of verified bots that are associated with search engine optimization (SEO), website monitoring, and more. +You can use this directory to prevent any bot protection measures from impacting otherwise helpful bots and agents, such as search crawlers. -You can use this list to prevent any bot protection measures from impacting otherwise helpful bots, such as search crawlers. - -For a partial list of verified bots, refer to [Cloudflare Radar](https://radar.cloudflare.com/verified-bots). +For a partial list of verified bots and signed agents, refer to [Cloudflare Radar](https://radar.cloudflare.com/verified-bots).