Skip to content

Commit b4ea3fc

Browse files
Oxyjunsdnts
authored andcommitted
Adding note on detection, adding note on PPC beta (cloudflare#23386)
1 parent 797e8ff commit b4ea3fc

File tree

1 file changed

+14
-0
lines changed

1 file changed

+14
-0
lines changed

src/content/docs/ai-audit/features/manage-ai-crawlers.mdx

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,11 +41,25 @@ The **AI Crawlers** tab displays a table of AI crawlers that are requesting acce
4141
</TabItem>
4242
</Tabs>
4343

44+
:::note[Quality of AI crawler detection]
45+
On the free plan, AI Audit identifies AI crawlers based on their [user agent strings](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent). This enables AI Audit to detect easy-to-detect (well-known) AI crawlers.
46+
47+
Upgrade your plan to enable a more thorough detection using Cloudflare's [Bot Management detection ID](/bots/reference/bot-management-variables/#ruleset-engine-fields) field.
48+
:::
49+
4450
## Take action for each AI crawler
4551

4652
<Tabs>
4753
<TabItem label="With pay per crawl">
4854

55+
:::note[Pay per crawl closed beta]
56+
Pay per crawl is currently in closed beta.
57+
58+
To find out how to join the beta program, reach out to us at [Pay per crawl signup](http://www.cloudflare.com/paypercrawl-signup/), or contact your account executive if you are an existing Enterprise customer.
59+
60+
To learn more about pay per crawl, refer to Cloudflare blog: [Introducing pay per crawl: enabling content owners to charge AI crawlers for access](https://blog.cloudflare.com/introducing-pay-per-crawl/).
61+
:::
62+
4963
For each AI crawler, you can take one of three actions: allow, charge, or block.
5064

5165
<Example title="Allow access">

0 commit comments

Comments
 (0)