Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: Enhanced AI Crawl Control metrics with new drilldowns and filters
description: Analyze crawler activity with requests over time charts, referrer breakdowns, and configurable filters with data export capabilities
date: 2025-10-14
---

AI Crawl Control now provides enhanced metrics and CSV data exports to help you better understand AI crawler activity across your sites.

## What's new

### Track crawler requests over time

Visualize crawler activity patterns over time, and group data by different dimensions:

- **By Crawler** — Track activity from individual AI crawlers (GPTBot, ClaudeBot, Bytespider)
- **By Category** — Analyze crawler purpose or type
- **By Operator** — Discover which companies (OpenAI, Anthropic, ByteDance) are crawling your site
- **By Host** — Break down activity across multiple subdomains
- **By Status Code** — Monitor HTTP response codes to crawlers (200s, 300s, 400s, 500s)

![AI Crawl Control requests over time chart with grouping tabs](~/assets/images/ai-crawl-control/ai-crawl-control-requests-over-time.png "Interactive chart showing crawler requests over time with filterable dimensions")

### Analyze referrer data (Paid plans)

Identify traffic sources with referrer analytics:

- View top referrers driving traffic to your site
- Understand discovery patterns and content popularity from AI operators

![AI Crawl Control top referrers breakdown](~/assets/images/ai-crawl-control/ai-crawl-control-top-referrers.png "Bar chart showing top referrers and their respective traffic volumes")

### Export data

Download your filtered view as a CSV:

- Includes all applied filters and groupings
- Useful for custom reporting and deeper analysis

## Get started

1. Log in to the Cloudflare dashboard, and select your account and domain.
2. Go to **AI Crawl Control** > **Metrics**.
3. Use the grouping tabs to explore different views of your data.
4. Apply filters to focus on specific crawlers, time ranges, or response codes.
5. Select **Download CSV** to export your filtered data for further analysis.

Learn more about [AI Crawl Control](/ai-crawl-control).
Loading