Skip to content

Commit 201dfa6

Browse files
committed
Fixing headings clash
1 parent 2d21d0f commit 201dfa6

File tree

1 file changed

+29
-21
lines changed

1 file changed

+29
-21
lines changed

src/content/docs/ai-audit/features/manage-ai-crawlers.mdx

Lines changed: 29 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ sidebar:
55
order: 2
66
---
77

8-
import { Steps, GlossaryTooltip, Tabs, TabItem } from "~/components";
8+
import { Steps, GlossaryTooltip, Tabs, TabItem, Example } from "~/components";
99

1010
AI Audit enables you to take specific action for each AI crawler.
1111

@@ -44,27 +44,11 @@ The **AI Crawlers** tab displays a table of AI crawlers that are requesting acce
4444
## Take action for each AI crawler
4545

4646
<Tabs>
47-
<TabItem label="Without pay per crawl">
48-
49-
For each AI crawler, you can choose to block or allow access.
50-
51-
### Block access
52-
53-
You can block an AI crawler to completely stop the AI crawler from scraping your webpage.
54-
55-
Use the toggles in the **Block** column to block specific AI crawlers from accessing your content.
56-
57-
### Allow access
58-
59-
You can allow an AI crawler to scrape your content. Note that you can still choose to [Enforce `robots.txt`](/ai-audit/features/manage-ai-crawlers/#take-action-for-each-ai-crawler).
60-
61-
For more details on how this rule interacts with other Cloudflare settings, refer to [How it works](/bots/concepts/bot/#how-it-works).
62-
63-
</TabItem><TabItem label="With pay per crawl">
47+
<TabItem label="With pay per crawl">
6448

6549
For each AI crawler, you can take one of three actions: allow, charge, or block.
6650

67-
### Allow access
51+
<Example title="Allow access">
6852

6953
You can allow an AI crawler to scrape your content.
7054

@@ -74,19 +58,43 @@ Note that you can still choose to [Enforce `robots.txt`](/ai-audit/features/mana
7458

7559
For more details on how this rule interacts with other Cloudflare settings, refer to [How it works](/bots/concepts/bot/#how-it-works).
7660

77-
### Charge for crawl (Enterprise only)
61+
</Example>
62+
63+
<Example title="Charge for crawl (Enterprise only)">
7864

7965
You can charge the owner of the AI crawler for each crawl request.
8066

8167
From the **Actions** column, select **Charge**.
8268

8369
For more information, refer to [What is Pay Per Crawl?](/ai-audit/features/pay-per-crawl/what-is-pay-per-crawl/).
8470

85-
### Block access
71+
</Example>
72+
73+
<Example title="Block access">
8674

8775
You can block an AI crawler to completely stop the AI crawler from scraping your webpage.
8876

8977
From the **Actions** column, select **Block**.
78+
</Example>
79+
80+
</TabItem>
81+
<TabItem label="Without pay per crawl">
82+
83+
For each AI crawler, you can choose to allow or block access.
84+
85+
<Example title="Allow access">
86+
87+
You can allow an AI crawler to scrape your content. Note that you can still choose to [Enforce `robots.txt`](/ai-audit/features/manage-ai-crawlers/#take-action-for-each-ai-crawler).
88+
89+
For more details on how this rule interacts with other Cloudflare settings, refer to [How it works](/bots/concepts/bot/#how-it-works).
90+
91+
</Example>
92+
93+
<Example title="Block access">
94+
You can block an AI crawler to completely stop the AI crawler from scraping your webpage.
95+
96+
Use the toggles in the **Block** column to block specific AI crawlers from accessing your content.
97+
</Example>
9098

9199
</TabItem>
92100
</Tabs>

0 commit comments

Comments
 (0)