Skip to content

Commit 07b6134

Browse files
Update robots.txt (#25495)
* Update robots.txt * Apply suggestion from @GregBrimble Co-authored-by: Greg Brimble <[email protected]> * move below user agent --------- Co-authored-by: Greg Brimble <[email protected]>
1 parent 1eb0d92 commit 07b6134

File tree

1 file changed

+3
-5
lines changed

1 file changed

+3
-5
lines changed

public/robots.txt

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,12 +22,13 @@
2222
# answers).
2323
# ai-train: training or fine-tuning AI models.
2424

25-
# ANY RESTRICTIONS EXPRESSED VIA CONTENT-SIGNALS ARE EXPRESS
25+
# ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS
2626
# RESERVATIONS OF RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN
2727
# UNION DIRECTIVE 2019/790 ON COPYRIGHT AND RELATED RIGHTS
2828
# IN THE DIGITAL SINGLE MARKET.
2929

3030
User-agent: *
31+
Content-Signal: ai-train=yes, search=yes, ai-input=yes
3132

3233
Allow: /
3334
Disallow: /client-ip-geolocation
@@ -36,7 +37,4 @@ Disallow: /constellation
3637
Disallow: /cdn-cgi/
3738
Disallow: /email-security/
3839

39-
Content-Usage: tdm=y
40-
Content-Signal: ai-train=yes, search=yes, ai-input=yes
41-
42-
Sitemap: https://developers.cloudflare.com/sitemap-index.xml
40+
Sitemap: https://developers.cloudflare.com/sitemap-index.xml

0 commit comments

Comments
 (0)