-
My website is https://dizkaz.com, a simple news link aggregation site. Since I deployed anubis, I can't search content by using the google site search. My bot policy is bots:
- name: cloudflare-workers
headers_regex:
CF-Worker: .*
action: DENY
- name: well-known
path_regex: ^/.well-known/.*$
action: ALLOW
- name: favicon
path_regex: ^/favicon.ico$
action: ALLOW
- name: robots-txt
path_regex: ^/robots.txt$
action: ALLOW
- name: generic-browser
user_agent_regex: Mozilla
action: CHALLENGE Did this blocked the google bot? |
Beta Was this translation helpful? Give feedback.
Answered by
Xe
Aug 3, 2025
Replies: 1 comment
-
You need to explicitly allow Google: bots:
- name: cloudflare-workers
headers_regex:
CF-Worker: .*
action: DENY
- import: (data)/crawlers/googlebot.yaml # allows googlebot
- name: well-known
path_regex: ^/.well-known/.*$
action: ALLOW
- name: favicon
path_regex: ^/favicon.ico$
action: ALLOW
- name: robots-txt
path_regex: ^/robots.txt$
action: ALLOW
- name: generic-browser
user_agent_regex: Mozilla
action: CHALLENGE |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
oodzchen
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
You need to explicitly allow Google: