Skip to content

Commit a4cd350

Browse files
CameronWhitesideCameron Whiteside
andauthored
Create changelog for updated AI Crawl Control metrics (#25825)
* changelog: AA: add AI Crawl Control enhanced metrics changelog entry * changelog: AA: small typo fixes to metrics changelog entry * changelog: AA: small typo fixes to metrics changelog entry * changelog: AA: improved language and semantic header levels * changelog: AA: improved get started steps --------- Co-authored-by: Cameron Whiteside <[email protected]>
1 parent 44d8689 commit a4cd350

File tree

3 files changed

+47
-0
lines changed

3 files changed

+47
-0
lines changed
68.6 KB
Loading
76.5 KB
Loading
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
title: Enhanced AI Crawl Control metrics with new drilldowns and filters
3+
description: Analyze crawler activity with requests over time charts, referrer breakdowns, and configurable filters with data export capabilities
4+
date: 2025-10-14
5+
---
6+
7+
AI Crawl Control now provides enhanced metrics and CSV data exports to help you better understand AI crawler activity across your sites.
8+
9+
## What's new
10+
11+
### Track crawler requests over time
12+
13+
Visualize crawler activity patterns over time, and group data by different dimensions:
14+
15+
- **By Crawler** — Track activity from individual AI crawlers (GPTBot, ClaudeBot, Bytespider)
16+
- **By Category** — Analyze crawler purpose or type
17+
- **By Operator** — Discover which companies (OpenAI, Anthropic, ByteDance) are crawling your site
18+
- **By Host** — Break down activity across multiple subdomains
19+
- **By Status Code** — Monitor HTTP response codes to crawlers (200s, 300s, 400s, 500s)
20+
21+
![AI Crawl Control requests over time chart with grouping tabs](~/assets/images/ai-crawl-control/ai-crawl-control-requests-over-time.png "Interactive chart showing crawler requests over time with filterable dimensions")
22+
23+
### Analyze referrer data (Paid plans)
24+
25+
Identify traffic sources with referrer analytics:
26+
27+
- View top referrers driving traffic to your site
28+
- Understand discovery patterns and content popularity from AI operators
29+
30+
![AI Crawl Control top referrers breakdown](~/assets/images/ai-crawl-control/ai-crawl-control-top-referrers.png "Bar chart showing top referrers and their respective traffic volumes")
31+
32+
### Export data
33+
34+
Download your filtered view as a CSV:
35+
36+
- Includes all applied filters and groupings
37+
- Useful for custom reporting and deeper analysis
38+
39+
## Get started
40+
41+
1. Log in to the Cloudflare dashboard, and select your account and domain.
42+
2. Go to **AI Crawl Control** > **Metrics**.
43+
3. Use the grouping tabs to explore different views of your data.
44+
4. Apply filters to focus on specific crawlers, time ranges, or response codes.
45+
5. Select **Download CSV** to export your filtered data for further analysis.
46+
47+
Learn more about [AI Crawl Control](/ai-crawl-control).

0 commit comments

Comments
 (0)