Skip to content

Commit 543f9c3

Browse files
OxyjunOxyjun
committed
Iterating over feedback part 1
1 parent cacb286 commit 543f9c3

File tree

7 files changed

+72
-38
lines changed

7 files changed

+72
-38
lines changed

‎src/content/changelog/ai-audit/ai-audit-launch.mdx‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
title: AI Audit
3+
description: AI Audit is available as a beta product.
4+
products:
5+
- ai-audit
6+
date: 2024-09-23T11:00:00Z
7+
---
8+
9+
[**AI Audit**](/ai-audit/) is now available!
10+

‎src/content/docs/ai-audit/features/ai-crawler-detection.mdx‎

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -13,17 +13,27 @@ AI Audit provides you with the following metrics to help you understand how AI c
1313

1414
| Metric | Description |
1515
| ------------------------------------- | ----------------------------------------------------------------------------- |
16-
| Total AI crawler requests | The total number of requests to crawl your website, from all AI crawlers |
17-
| Blocked requests | The number of requests you have blocked (by blocking AI crawlers) |
18-
| Allowed requests | The number of requests you have allowed (by allowing AI crawlers) |
1916
| Request by AI crawlers | A graph which displays the number of crawl requests from each AI crawler |
17+
| Summary | A list of AI crawlers with the most number of crawl requests |
2018
| Most popular paths by AI crawlers | The most popular pages crawled by AI crawlers, for each AI crawler |
21-
| Hosts | The owner of the AI crawler |
22-
| Overall popular paths | The most popular pages crawled by AI crawlers, from all AI crawlers |
23-
| Most active AI crawlers by bot owners | The AI crawler owners with the highest number of requests to access your site |
2419

25-
## Filter date range
20+
## Filter AI crawler data
2621

27-
You can use the date filter to choose the period of time you wish to analyze.
22+
You can use filters to narrow the scope of your result.
2823

29-
Click on the date filter dropdown and select the range of dates. The values of the AI Audit metrics will update according to your filter.
24+
- **Provider:** Filter by the AI crawler owners.
25+
- **Bot type:** Filter by the type of the AI bot (for example, AI crawler, AI assistant, or archiver).
26+
- **Date range:** Filter the date range of your results. You can choose from three predetermined date ranges:
27+
- Past 7 days
28+
- Past 14 days
29+
- Past month
30+
31+
The values of the AI Audit metrics will update according to your filter.
32+
33+
## Filter subdomains
34+
35+
You can use the subdomain filter to narrow the scope of your result.
36+
37+
From the dropdown, select either **All subdomains**, or the specific subdomain you wish to view.
38+
39+
The values of the AI Audit metrics will update according to your filter.

‎src/content/docs/ai-audit/get-started.mdx‎

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,19 +14,25 @@ description: Learn how to set up AI Audit.
1414

1515
import { Render, Steps } from "~/components";
1616

17+
This guide instructs you through
18+
19+
- Viewing AI crawlers that are interacting with your domain.
20+
- Creating a rule to block AI crawlers on your website.
21+
1722
## Prerequisites
1823

1924
1. Sign up for a [Cloudflare account](https://dash.cloudflare.com/sign-up/).
2025
2. [Connect your domain to Cloudflare](/fundamentals/manage-domains/add-site/).
26+
3. Configure traffic to be [proxied by Cloudflare](/dns/proxy-status/).
2127

2228
## 1. Block all AI crawlers
2329

24-
To enable AI Audit:
30+
To use AI Audit:
2531

2632
<Steps>
2733
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/), and select your account and domain.
2834
2. Go to **AI Audit**.
29-
3. From **AI Crawlers** tab, toggle **Block AI Crawlers**.
35+
3. From **Most Popular Paths**, select **Block All**.
3036
4. From the **Bot traffic** page, under **Block AI Bots**, select **Enable**.
3137
</Steps>
3238

@@ -35,12 +41,21 @@ To enable AI Audit:
3541
Customers on the Enterprise plan can choose to only block specific AI crawlers, while allowing others.
3642

3743
<Steps>
38-
1. Go to the **AI Crawlers** tab.
39-
2. Toggle the switch under the **Block** column for the AI crawlers you wish to block.
44+
1. Go to the **AI Audit**.
45+
2. From **Most Popular Paths**, select **Block Some**.
46+
3. From the **Security rules** page,
4047
</Steps>
4148

4249
## 3. Review detected AI crawlers
4350

4451
Review the AI crawlers detected on your site in the **Metrics** tab of the Cloudflare dashboard for key metrics.
4552

46-
Refer to [AI crawler detection](/ai-audit/ai-crawler-detection/) for more information.
53+
Refer to [AI crawler detection](/ai-audit/features/ai-crawler-detection/) for more information.
54+
55+
## Related resources
56+
57+
Refer to the following related resources:
58+
59+
- Cloudflare blog: [Start auditing and controlling the AI models accessing your content](https://blog.cloudflare.com/nl-nl/cloudflare-ai-audit-control-ai-content-crawlers/)
60+
- Block AI crawlers that do not adhere to recommended guidelines using [Cloudflare AI Labyrinth](/bots/additional-configurations/ai-labyrinth/).
61+
- [Direct AI crawlers with managed robots.txt](/bots/additional-configurations/managed-robots-txt/).

‎src/content/docs/ai-audit/index.mdx‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ head:
1010
description: AI Audit is a tool which allows you to analyze and control how third-party AI crawlers interact with your website.
1111
---
1212

13-
import { Description, Feature, FeatureTable, Plan, LinkButton } from "~/components";
13+
import { Description, Feature, FeatureTable, Plan, LinkButton, RelatedProduct } from "~/components";
1414

1515
<Description>
1616

@@ -40,3 +40,22 @@ AI Audit is currently only available as a beta product.
4040
Displays information about AI crawlers in your domains' pages.
4141
</Feature>
4242

43+
---
44+
45+
## Related Products
46+
47+
<RelatedProduct
48+
header="Bots"
49+
href="/bots/"
50+
product="bots"
51+
>
52+
Identify and mitigate automated traffic to protect your domain from bad bots.
53+
</RelatedProduct>
54+
55+
<RelatedProduct
56+
header="Web Application Firewall"
57+
href="/waf/"
58+
product="waf"
59+
>
60+
Get automatic protection from vulnerabilities and the flexibility to create custom rules.
61+
</RelatedProduct>

‎src/content/docs/ai-audit/reference/allowed-ai-crawlers.mdx‎

Lines changed: 0 additions & 8 deletions
This file was deleted.

‎src/content/docs/ai-audit/reference/index.mdx‎

Lines changed: 0 additions & 12 deletions
This file was deleted.

‎src/content/docs/ai-audit/release-notes.mdx‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
---
22
pcx_content_type: changelog
3-
title: Release notes
3+
title: Changelog
44
release_notes_file_name:
55
- ai-audit
66
sidebar:
77
order: 100
88
---
99

10-
import { ProductReleaseNotes } from "~/components";
10+
import { ProductChangelog } from "~/components";
1111

1212
{/* <!-- Actual content lives in /src/content/release-notes/ai-audit.yaml. Update the file there for new entries to appear here. For more details, refer to https://developers.cloudflare.com/style-guide/documentation-content-strategy/content-types/changelog/#yaml-file --> */}
1313

14-
<ProductReleaseNotes />
14+
<ProductChangelog />

0 commit comments

Comments
 (0)