Skip to content

Commit c2c24d8

Browse files
committed
Extend block list
1 parent 3bd6601 commit c2c24d8

File tree

1 file changed

+34
-4
lines changed

1 file changed

+34
-4
lines changed

Public/robots.txt

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,55 @@
11
Sitemap: https://swiftpackageindex.com/sitemap.xml
22

3+
User-agent: Amazonbot
4+
Disallow: /
5+
6+
User-agent: anthropic-ai
7+
Disallow: /
8+
9+
User-agent: Applebot
10+
Disallow: /
11+
312
User-agent: Bytespider
413
Disallow: /
514

6-
User-agent: GPTBot
15+
User-agent: ChatGPT-User
16+
Disallow: /
17+
18+
User-agent: Claude-Web
719
Disallow: /
820

921
User-agent: ClaudeBot
1022
Disallow: /
1123

12-
User-agent: anthropic-ai
24+
User-agent: Cohere-ai
1325
Disallow: /
1426

15-
User-agent: Applebot
27+
User-agent: Diffbot
28+
Disallow: /
29+
30+
User-Agent: FacebookBot
1631
Disallow: /
1732

1833
User-agent: Google-Extended
1934
Disallow: /
2035

21-
User-agent: Cohere-ai
36+
User-agent: GPTBot
37+
Disallow: /
38+
39+
User-agent: ImagesiftBot
40+
Disallow: /
41+
42+
User-agent: Omgili
43+
Disallow: /
44+
45+
User-agent: Omgilibot
46+
Disallow: /
47+
48+
User-agent: PerplexityBot
2249
Disallow: /
2350

2451
User-agent: Seekr
2552
Disallow: /
53+
54+
User-agent: YouBot
55+
Disallow: /

0 commit comments

Comments
 (0)