We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
1 parent 2d23cf8 commit 3548473Copy full SHA for 3548473
README.md
@@ -6,9 +6,7 @@ Robots.txt checker
6
Given a robots.txt file this library will give you a straight forward yes/no as to whether you're allowed to access
7
a given resource with a given user agent. Internally it organises the file into a tree.
8
9
-Wildcards are supported in a basic way:
10
-
11
-`Disallow: /hello/*/world` will disallow `/hello/whatever/world` but won't disallow `/hello/what/who/world`
+Wildcards are supported.
12
13
If you find any bugs with this library please don't hesitate to let me know, either create an issue on GitHub or submit a pull request.
14
0 commit comments