Skip to content

Commit 5efc592

Browse files
authored
Merge branch 'production' into patch-8
2 parents c771a06 + 99c717e commit 5efc592

File tree

61 files changed

+7364
-20
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

61 files changed

+7364
-20
lines changed

.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,6 +138,7 @@
138138
/src/content/docs/workers/observability/ @irvinebroque @mikenomitch @rohinlohe @kodster28 @cloudflare/pcx-technical-writing
139139
/src/content/docs/workers/static-assets @irvinebroque @GregBrimble @WalshyDev @kodster28 @cloudflare/deploy-config @cloudflare/pcx-technical-writing
140140
/src/content/docs/workflows/ @elithrar @celso @cloudflare/pcx-technical-writing
141+
/src/content/docs/sandbox/ @whoiskatrin @ghostwriternr @cloudflare/pcx-technical-writing @ai-agents
141142

142143
# DDoS Protection
143144

68.6 KB
Loading
76.5 KB
Loading

src/components/CopyPageButton.tsx

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -63,8 +63,7 @@ export default function CopyPageButton() {
6363
const indexMdUrl = new URL("index.md", window.location.href).toString();
6464
const prompt = `Read this page from the Cloudflare docs: ${encodeURIComponent(indexMdUrl)} and answer questions about the content.`;
6565
track("clicked copy page button", {
66-
value: "docs ai",
67-
label: vendor,
66+
value: `${vendor} ai`,
6867
});
6968
window.open(`${externalAIURL}${prompt}`, "_blank");
7069
};
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
title: Enhanced AI Crawl Control metrics with new drilldowns and filters
3+
description: Analyze crawler activity with requests over time charts, referrer breakdowns, and configurable filters with data export capabilities
4+
date: 2025-10-14
5+
---
6+
7+
AI Crawl Control now provides enhanced metrics and CSV data exports to help you better understand AI crawler activity across your sites.
8+
9+
## What's new
10+
11+
### Track crawler requests over time
12+
13+
Visualize crawler activity patterns over time, and group data by different dimensions:
14+
15+
- **By Crawler** — Track activity from individual AI crawlers (GPTBot, ClaudeBot, Bytespider)
16+
- **By Category** — Analyze crawler purpose or type
17+
- **By Operator** — Discover which companies (OpenAI, Anthropic, ByteDance) are crawling your site
18+
- **By Host** — Break down activity across multiple subdomains
19+
- **By Status Code** — Monitor HTTP response codes to crawlers (200s, 300s, 400s, 500s)
20+
21+
![AI Crawl Control requests over time chart with grouping tabs](~/assets/images/ai-crawl-control/ai-crawl-control-requests-over-time.png "Interactive chart showing crawler requests over time with filterable dimensions")
22+
23+
### Analyze referrer data (Paid plans)
24+
25+
Identify traffic sources with referrer analytics:
26+
27+
- View top referrers driving traffic to your site
28+
- Understand discovery patterns and content popularity from AI operators
29+
30+
![AI Crawl Control top referrers breakdown](~/assets/images/ai-crawl-control/ai-crawl-control-top-referrers.png "Bar chart showing top referrers and their respective traffic volumes")
31+
32+
### Export data
33+
34+
Download your filtered view as a CSV:
35+
36+
- Includes all applied filters and groupings
37+
- Useful for custom reporting and deeper analysis
38+
39+
## Get started
40+
41+
1. Log in to the Cloudflare dashboard, and select your account and domain.
42+
2. Go to **AI Crawl Control** > **Metrics**.
43+
3. Use the grouping tabs to explore different views of your data.
44+
4. Apply filters to focus on specific crawlers, time ranges, or response codes.
45+
5. Select **Download CSV** to export your filtered data for further analysis.
46+
47+
Learn more about [AI Crawl Control](/ai-crawl-control).

src/content/docs/cloudflare-one/connections/connect-networks/private-net/cloudflared/connect-cidr.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ To connect your infrastructure with Cloudflare Tunnel:
1515

1616
<Render file="tunnel/connect-private-network" product="cloudflare-one" />
1717

18-
2. In the **CIDR** tab for the tunnel, enter the IP/CIDR range that you wish to route through the tunnel (for example `10.0.0.0/8`).
18+
3. In the **CIDR** tab for the tunnel, enter the IP/CIDR range that you wish to route through the tunnel (for example `10.0.0.0/8`).
1919

2020
## 2. Set up the client
2121

src/content/docs/cloudflare-one/connections/connect-networks/use-cases/grpc.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ To establish a secure, outbound-only connection to Cloudflare:
3434

3535
<Render file="tunnel/connect-private-network" product="cloudflare-one" />
3636

37-
2. In the **CIDR** tab for the tunnel, enter the private IP or CIDR address of your server.
37+
3. In the **CIDR** tab for the tunnel, enter the private IP or CIDR address of your server.
3838

3939
## 3. Route private network IPs through WARP
4040

src/content/docs/cloudflare-one/connections/connect-networks/use-cases/rdp/rdp-warp-to-tunnel.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,9 +52,9 @@ By default, Internet Explorer will be installed and configured in [Enhanced Secu
5252

5353
<Render file="tunnel/connect-private-network" product="cloudflare-one" />
5454

55-
2. In the **CIDR** tab for the tunnel, enter the private IP or CIDR address of your server. In GCP, the server IP is the **Internal IP** of the VM instance.
55+
3. In the **CIDR** tab for the tunnel, enter the private IP or CIDR address of your server. In GCP, the server IP is the **Internal IP** of the VM instance.
5656

57-
3. (Optional) [Set up Zero Trust policies](/cloudflare-one/connections/connect-networks/private-net/cloudflared/#4-recommended-filter-network-traffic-with-gateway) to fine-tune access to your server.
57+
4. (Optional) [Set up Zero Trust policies](/cloudflare-one/connections/connect-networks/private-net/cloudflared/#4-recommended-filter-network-traffic-with-gateway) to fine-tune access to your server.
5858

5959
## 4. Set up the client
6060

src/content/docs/cloudflare-one/connections/connect-networks/use-cases/smb.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,8 @@ You can use Cloudflare Tunnel to create a secure, outbound-only connection from
2626

2727
<Render file="tunnel/connect-private-network" product="cloudflare-one" />
2828

29-
2. In the **CIDR** tab for the tunnel, enter the private IP or CIDR address of your server.
30-
3. (Optional) [Set up Zero Trust policies](/cloudflare-one/connections/connect-networks/private-net/cloudflared/#4-recommended-filter-network-traffic-with-gateway) to fine-tune access to your server.
29+
3. In the **CIDR** tab for the tunnel, enter the private IP or CIDR address of your server.
30+
4. (Optional) [Set up Zero Trust policies](/cloudflare-one/connections/connect-networks/private-net/cloudflared/#4-recommended-filter-network-traffic-with-gateway) to fine-tune access to your server.
3131

3232
### 2. Set up the client
3333

src/content/docs/cloudflare-one/connections/connect-networks/use-cases/ssh/ssh-warp-to-tunnel.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -61,9 +61,9 @@ In order to be able to establish an SSH connection, do not enable [OS Login](htt
6161

6262
<Render file="tunnel/connect-private-network" product="cloudflare-one" />
6363

64-
2. In the **CIDR** tab for the tunnel, enter the private IP or CIDR address of your server. In GCP, the server IP is the **Internal IP** of the VM instance.
64+
3. In the **CIDR** tab for the tunnel, enter the private IP or CIDR address of your server. In GCP, the server IP is the **Internal IP** of the VM instance.
6565

66-
3. (Optional) [Set up Zero Trust policies](/cloudflare-one/connections/connect-networks/private-net/cloudflared/#4-recommended-filter-network-traffic-with-gateway) to fine-tune access to your server.
66+
4. (Optional) [Set up Zero Trust policies](/cloudflare-one/connections/connect-networks/private-net/cloudflared/#4-recommended-filter-network-traffic-with-gateway) to fine-tune access to your server.
6767

6868
## 4. Set up the client
6969

0 commit comments

Comments
 (0)