Skip to content

Commit aae1b41

Browse files
authored
Update robots.txt to remove the * user-agent group (#13726)
1 parent 8d1c4e1 commit aae1b41

File tree

1 file changed

+1
-6
lines changed

1 file changed

+1
-6
lines changed

polaris.shopify.com/public/robots.txt

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,3 @@
1-
# Global directive for all robots
2-
User-agent: *
3-
Noai: true
4-
Noimageai: true
5-
61
# Block OpenAI's crawler
72
User-agent: GPTBot
83
Disallow: /
@@ -31,4 +26,4 @@ Disallow: /
3126
User-agent: CCBot
3227
Disallow: /
3328
User-agent: ChatGPT-User
34-
Disallow: /
29+
Disallow: /

0 commit comments

Comments
 (0)