Skip to content

Commit 05ad3ee

Browse files
patriciasantaanapatriciasantaana
committed
verified bot definitions
1 parent feec1a6 commit 05ad3ee

File tree

1 file changed

+169
-20
lines changed
  • src/content/docs/bots/concepts/bot/verified-bots

1 file changed

+169
-20
lines changed

‎src/content/docs/bots/concepts/bot/verified-bots/index.mdx‎

Lines changed: 169 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -59,30 +59,179 @@ You can segment your verified bot traffic by its type and purpose by adding the
5959
The Verified Bot Categories field is not compatible with legacy Firewall rules.
6060
:::
6161

62-
:::note
62+
:::note[Availability]
6363
Verified Bot Categories is available on all plans.
6464
:::
6565

66-
| Name | String value | Example |
67-
| -------------------------- | ---------------------------- | ----------------------------------------------------------------------- |
68-
| Academic research | `Academic Research` | Library of Congress, TurnItInBot, Bibliothèque nationale de France |
69-
| Accessibility | `Accessibility` | Accessible Web Bot |
70-
| Advertising or marketing | `Advertising & Marketing` | Google Adsbot |
71-
| Aggregators | `Aggregator` | Pinterest, Indeed Jobsbot |
72-
| AI Assistant | `AI Assistant` | Perplexity-User, DuckAssistBot |
73-
| AI Crawler | `AI Crawler` | Google Bard, ChatGPT bot |
74-
| AI Search | `AI Search` | OAI-SearchBot |
75-
| Archiver | `Archiver` | Internet Archive, CommonCrawl |
76-
| Feed fetcher | `Feed Fetcher` | RSS or Podcast feed updaters |
77-
| Monitoring or analytics | `Monitoring & Analytics` | Uptime Monitors |
78-
| Page preview | `Page Preview` | Facebook, Slack, Twitter, or Discord Link Preview tools |
79-
| Search engine crawler | `Search Engine Crawler` | Googlebot, Bingbot, Yandexbot, Baidubot |
80-
| Search engine optimization | `Search Engine Optimization` | Google Lighthouse, GT Metrix, Pingdom, AddThis |
81-
| Security | `Security` | Vulnerability Scanners, SSL Domain Control Validation (DCV) Check Tools |
82-
| Social media marketing | `Social Media Marketing` | Brandwatch |
83-
| Webhooks | `Webhooks` | Payment processors, WordPress Integration tools |
84-
| Other | `Other` | |
66+
<details>
67+
<summary>Academic research</summary>
68+
69+
**String value**: `Academic Research`
70+
71+
**Definition**: Gathers data for scholarly research or academic purposes.
72+
73+
**Example**: Library of Congress, TurnItInBot, Bibliothèque nationale de France
74+
</details>
75+
76+
<details>
77+
<summary>Accessibility</summary>
78+
79+
**String value**: `Accessibility`
80+
81+
**Definition**: Scans websites to identify their accessibility.
82+
83+
**Example**: Accessible Web Bot
84+
</details>
85+
86+
<details>
87+
<summary>Advertising or marketing</summary>
88+
89+
**String value**: `Advertising & Marketing`
90+
91+
**Definition**: Automates marketing tasks including, but not limited to, ad placement and performance tracking.
92+
93+
**Example**: Google Adsbot
94+
</details>
95+
96+
<details>
97+
<summary>Aggregators</summary>
98+
99+
**String value**: `Aggregator`
100+
101+
**Definition**: Collects content from various online sources and consolidates it in one place.
102+
103+
**Example**: Pinterest, Indeed Jobsbot
104+
</details>
105+
106+
<details>
107+
<summary>AI Assistant</summary>
108+
109+
**String value**: `AI Assistant`
110+
111+
**Definition**: Automated AI bot driven by user action.
112+
113+
**Example**: Perplexity-User, DuckAssistBot
114+
</details>
115+
116+
<details>
117+
<summary>AI Crawler</summary>
118+
119+
**String value**: `AI Crawler`
120+
121+
**Definition**: Crawls websites for content that is used for training AI models.
122+
123+
**Example**: Google Bard, ChatGPT bot
124+
</details>
125+
126+
<details>
127+
<summary>AI Search</summary>
128+
129+
**String value**: `AI Search`
130+
131+
**Definition**: Powers AI-driven search experiences.
132+
133+
**Example**: OAI-SearchBot
134+
</details>
135+
136+
<details>
137+
<summary>Archiver</summary>
138+
139+
**String value**: `Archiver`
140+
141+
**Definition**: Saves snapshots of websites to preserve digital content for historical records.
142+
143+
**Example**: Internet Archive, CommonCrawl
144+
</details>
145+
146+
<details>
147+
<summary>Feed fetcher</summary>
148+
149+
**String value**: `Feed Fetcher`
150+
151+
**Definition**: Retrieves updates from feeds to power readers or other applications.
152+
153+
**Example**: RSS or Podcast feed updaters
154+
</details>
155+
156+
<details>
157+
<summary>Monitoring or analytics</summary>
158+
159+
**String value**: `Monitoring & Analytics`
160+
161+
**Definition**: Tracks a website's uptime, performance, and user traffic to gather key monitoring metrics.
162+
163+
**Example**: Uptime Monitors
164+
</details>
165+
166+
<details>
167+
<summary>Page preview</summary>
168+
169+
**String value**: `Page Preview`
170+
171+
**Definition**: Generates previews for links shared on social media or in messaging apps.
172+
173+
**Example**: Facebook, Slack, Twitter, or Discord Link Preview tools
174+
</details>
175+
176+
<details>
177+
<summary>Search engine crawler</summary>
178+
179+
**String value**: `Search Engine Crawler`
180+
181+
**Definition**: A bot that discovers and indexes web pages for search results.
182+
183+
**Example**: Googlebot, Bingbot, Yandexbot, Baidubot
184+
</details>
185+
186+
<details>
187+
<summary>Search engine optimization</summary>
188+
189+
**String value**: `Search Engine Optimization`
190+
191+
**Definition**: Analyzes websites to improve their standing in search engine results pages.
192+
193+
**Example**: Google Lighthouse, GT Metrix, Pingdom, AddThis
194+
</details>
195+
196+
<details>
197+
<summary>Security</summary>
198+
199+
**String value**: `Security`
200+
201+
**Definition**: Scans websites to detect security vulnerabilities and potential threats.
202+
203+
**Example**: Vulnerability Scanners, SSL Domain Control Validation (DCV) Check Tools
204+
</details>
205+
206+
<details>
207+
<summary>Social media marketing</summary>
208+
209+
**String value**: `Social Media Marketing`
210+
211+
**Definition**: Manages and automates activities on social platforms.
212+
213+
**Example**: Brandwatch
214+
</details>
215+
216+
<details>
217+
<summary>Webhooks</summary>
218+
219+
**String value**: `Webhooks`
220+
221+
**Definition**: An automated messenger that sends data from one application to another for specific events.
222+
223+
**Example**: Payment processors, WordPress Integration tools
224+
</details>
225+
226+
<details>
227+
<summary>Other</summary>
228+
229+
**String value**: `Other`
230+
231+
**Definition**: A dedicated category for bots that do not fit into the other classifications.
232+
</details>
85233

234+
Cloudflare reserves the right to re-assign verified bot categories if the bot's public documentation and observed behavior differ from the category listed in the bot submission form.
86235

87236
### Known issues
88237

0 commit comments

Comments
 (0)