Skip to content

Commit c452863

Browse files
author
Cameron Whiteside
committed
changelog: AA: add AI Crawl Control enhanced metrics changelog entry
1 parent 3e9068f commit c452863

File tree

3 files changed

+47
-0
lines changed

3 files changed

+47
-0
lines changed
68.6 KB
Loading
76.5 KB
Loading
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
title: Enhanced AI Crawl Control metrics with new drilldowns and filters
3+
description: Analyze crawler activity with requests over time charts, referrer breakdowns, and configurable filters with data export capabilities
4+
date: 2025-10-14
5+
---
6+
7+
AI Crawl Control now provides enhanced metrics and CSV data exports to help you better understand AI crawler activity across your sites.
8+
9+
## What's new
10+
11+
#### Track crawler requests over time
12+
13+
Visualize crawler activity patterns over time, and group data by different dimensions:
14+
15+
- **By Crawler** — Track activity from individual AI crawlers (GPTBot, ClaudeBot, Bytespider)
16+
- **By Category** — Analyze crawler purpose or type
17+
- **By Operator** — See which companies (OpenAI, Anthropic, ByteDance) are crawling your site
18+
- **By Host** — Break down activity across multiple subdomains
19+
- **By Status Code** — Monitor HTTP response codes to crawlers (200s, 300s, 400s, 500s)
20+
21+
![AI Crawl Control requests over time chart with grouping tabs](~/assets/images/ai-crawl-control/ai-crawl-control-requests-over-time.png "Interactive chart showing crawler requests over time with filterable dimensions")
22+
23+
#### Analyze referrer data (Paid plans)
24+
25+
Identify traffic sources with referrer analytics:
26+
27+
- View top referrers driving traffic to your site
28+
- Understand discovery patterns and content popularity from AI operators
29+
30+
![AI Crawl Control top referrers breakdown](~/assets/images/ai-crawl-control/ai-crawl-control-top-referrers.png "Bar chart showing top referrers and their respective traffic volumes")
31+
32+
#### Export data
33+
34+
Download your filtered view as a CSV:
35+
36+
- Includes all applied filters and groupings
37+
- Useful for custom reporting and deeper analysis
38+
39+
## Getting started
40+
41+
1. Navigate to **AI Crawl Control** for any domain in your Cloudflare dashboard
42+
2. Select the **Metrics** tab
43+
3. Use the dimension tabs to explore different views of your data
44+
4. Apply filters to focus on specific crawlers, time ranges, or response codes
45+
5. Click **Export** to download your data for further analysis
46+
47+
Learn more about [AI Crawl Control](/ai-crawl-control)

0 commit comments

Comments
 (0)