diff --git a/src/content/docs/bots/concepts/bot/verified-bots/index.mdx b/src/content/docs/bots/concepts/bot/verified-bots/index.mdx index 6d9cbdfb62e4b5d..9524e3a00f57116 100644 --- a/src/content/docs/bots/concepts/bot/verified-bots/index.mdx +++ b/src/content/docs/bots/concepts/bot/verified-bots/index.mdx @@ -59,30 +59,179 @@ You can segment your verified bot traffic by its type and purpose by adding the The Verified Bot Categories field is not compatible with legacy Firewall rules. ::: -:::note +:::note[Availability] Verified Bot Categories is available on all plans. ::: -| Name | String value | Example | -| -------------------------- | ---------------------------- | ----------------------------------------------------------------------- | -| Academic research | `Academic Research` | Library of Congress, TurnItInBot, Bibliothèque nationale de France | -| Accessibility | `Accessibility` | Accessible Web Bot | -| Advertising or marketing | `Advertising & Marketing` | Google Adsbot | -| Aggregators | `Aggregator` | Pinterest, Indeed Jobsbot | -| AI Assistant | `AI Assistant` | Perplexity-User, DuckAssistBot | -| AI Crawler | `AI Crawler` | Google Bard, ChatGPT bot | -| AI Search | `AI Search` | OAI-SearchBot | -| Archiver | `Archiver` | Internet Archive, CommonCrawl | -| Feed fetcher | `Feed Fetcher` | RSS or Podcast feed updaters | -| Monitoring or analytics | `Monitoring & Analytics` | Uptime Monitors | -| Page preview | `Page Preview` | Facebook, Slack, Twitter, or Discord Link Preview tools | -| Search engine crawler | `Search Engine Crawler` | Googlebot, Bingbot, Yandexbot, Baidubot | -| Search engine optimization | `Search Engine Optimization` | Google Lighthouse, GT Metrix, Pingdom, AddThis | -| Security | `Security` | Vulnerability Scanners, SSL Domain Control Validation (DCV) Check Tools | -| Social media marketing | `Social Media Marketing` | Brandwatch | -| Webhooks | `Webhooks` | Payment processors, WordPress Integration tools | -| Other | `Other` | | +
+Academic research + +**String value**: `Academic Research` + +**Definition**: Gathers data for scholarly research or academic purposes. + +**Example**: Library of Congress, TurnItInBot, Bibliothèque nationale de France +
+ +
+Accessibility + +**String value**: `Accessibility` + +**Definition**: Scans websites to identify their accessibility. + +**Example**: Accessible Web Bot +
+ +
+Advertising or marketing + +**String value**: `Advertising & Marketing` + +**Definition**: Automates marketing tasks including, but not limited to, ad placement and performance tracking. + +**Example**: Google Adsbot +
+ +
+Aggregators + +**String value**: `Aggregator` + +**Definition**: Collects content from various online sources and consolidates it in one place. + +**Example**: Pinterest, Indeed Jobsbot +
+ +
+AI Assistant + +**String value**: `AI Assistant` + +**Definition**: Automated AI bot driven by user action. + +**Example**: Perplexity-User, DuckAssistBot +
+ +
+AI Crawler + +**String value**: `AI Crawler` + +**Definition**: Crawls websites for content that is used for training AI models. + +**Example**: Google Bard, ChatGPT bot +
+ +
+AI Search + +**String value**: `AI Search` + +**Definition**: Powers AI-driven search experiences. + +**Example**: OAI-SearchBot +
+ +
+Archiver + +**String value**: `Archiver` + +**Definition**: Saves snapshots of websites to preserve digital content for historical records. + +**Example**: Internet Archive, CommonCrawl +
+ +
+Feed fetcher + +**String value**: `Feed Fetcher` + +**Definition**: Retrieves updates from feeds to power readers or other applications. + +**Example**: RSS or Podcast feed updaters +
+ +
+Monitoring or analytics + +**String value**: `Monitoring & Analytics` + +**Definition**: Tracks a website's uptime, performance, and user traffic to gather key monitoring metrics. + +**Example**: Uptime Monitors +
+ +
+Page preview + +**String value**: `Page Preview` + +**Definition**: Generates previews for links shared on social media or in messaging apps. + +**Example**: Facebook, Slack, Twitter, or Discord Link Preview tools +
+ +
+Search engine crawler + +**String value**: `Search Engine Crawler` + +**Definition**: A bot that discovers and indexes web pages for search results. + +**Example**: Googlebot, Bingbot, Yandexbot, Baidubot +
+ +
+Search engine optimization + +**String value**: `Search Engine Optimization` + +**Definition**: Analyzes websites to improve their standing in search engine results pages. + +**Example**: Google Lighthouse, GT Metrix, Pingdom, AddThis +
+ +
+Security + +**String value**: `Security` + +**Definition**: Scans websites to detect security vulnerabilities and potential threats. + +**Example**: Vulnerability Scanners, SSL Domain Control Validation (DCV) Check Tools +
+ +
+Social media marketing + +**String value**: `Social Media Marketing` + +**Definition**: Manages and automates activities on social platforms. + +**Example**: Brandwatch +
+ +
+Webhooks + +**String value**: `Webhooks` + +**Definition**: An automated messenger that sends data from one application to another for specific events. + +**Example**: Payment processors, WordPress Integration tools +
+ +
+Other + +**String value**: `Other` + +**Definition**: A dedicated category for bots that do not fit into the other classifications. +
+Cloudflare reserves the right to re-assign verified bot categories if the bot's public documentation and observed behavior differ from the category listed in the bot submission form. ### Known issues