Skip to content

Commit 3548473

Browse files
author
Tom
committed
Update README
1 parent 2d23cf8 commit 3548473

File tree

1 file changed

+1
-3
lines changed

1 file changed

+1
-3
lines changed

README.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,7 @@ Robots.txt checker
66
Given a robots.txt file this library will give you a straight forward yes/no as to whether you're allowed to access
77
a given resource with a given user agent. Internally it organises the file into a tree.
88

9-
Wildcards are supported in a basic way:
10-
11-
`Disallow: /hello/*/world` will disallow `/hello/whatever/world` but won't disallow `/hello/what/who/world`
9+
Wildcards are supported.
1210

1311
If you find any bugs with this library please don't hesitate to let me know, either create an issue on GitHub or submit a pull request.
1412

0 commit comments

Comments
 (0)