-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Hosts Blocklists
- Introduction
- Rules examples
- Adblock-style syntax
- /etc/hosts syntax
- Domains-only syntax
- Hostlists compiler
There are three different approaches to writing hosts blocklists:
- Adblock-style syntax - modern approach to writing filtering rules based on using a subset of the Adblock-style syntax. This way blocklists will be compatible with browser ad blockers.
- /etc/hosts syntax - the old, tried and true approach is to use the same syntax as Operation Systems use for the "hosts" files.
- domains-only syntax - a list of domain names.
If you are creating a blocklist for AdGuard Home, we recommend using the Adblock-style syntax. It has a couple of important advantages over the old-style syntax:
-
Blocklists size. Using pattern-matching allows you to have a single rule instead of hundreds of
/etc/hostsentries. - Compatibility. Your blocklist will be compatible with browser ad blockers, and it will be easier to share rules with a browser filter list.
- Extensibility. For the last decade the Adblock-style syntax has greatly evolved, and we don't see why we can't extend it even more, and provide additional features for network-wide blockers.
If you're maintaining an old-style
/etc/hostsblocklist or if you maintain multiple filter lists (regardless of which type), we provide a tool that can be used to compile blocklists for AdGuard Home. We called it Hostlist compiler and we use it ourselves to create AdGuard SDN filter.
-
||example.org^- block access to theexample.orgdomain and all its subdomains -
@@||example.org^- unblock access to theexample.orgdomain and all its subdomains -
0.0.0.0 example.org- (attention, old-style /etc/hosts syntax) blockexample.orgdomain (but NOT its subdomains) -
! Here goes a comment- just a comment -
# Also a comment- just a comment -
/REGEX/- block access to the domains matching the specified regular expression
This is a subset of the traditional Adblock-style syntax which is used by browser ad blockers.
rule = ["@@"] pattern [ "$" modifiers ]
modifiers = [modifier0, modifier1[, ...[, modifierN]]]
-
pattern— the hostname mask. Every hostname is matched against this mask. The pattern can also contain special characters, which are described below. -
@@— a marker that is used in the "exception" rules. Start your rule with this marker if you want to turn off filtering for the matching hostnames. -
modifiers— parameters that clarify the rule. They may limit the scope of the rule or even completely change the way it works.
-
*— wildcard character. It is used to represent "any set of characters". This can also be an empty string or a string of any length. -
||— matching the beginning of a hostname (and any subdomain). For instance,||example.orgmatchesexample.organdtest.example.org, but nottestexample.org. -
^— separator character mark. Unlike browser ad blocking, there's nothing to "separate" in a hostname, so the only purpose of this character is to mark the end of the hostname. -
|— a pointer to the beginning or the end of the hostname. The value depends on the character placement in the mask. For example, the ruleample.org|corresponds toexample.org, but not toexample.org.com.|examplecorresponds toexample.org, but not totest.example.
If you want even more flexibility in making rules, you can use Regular expressions instead of the default simplified matching syntax.
If you want to use a regular expression, the pattern has to look like this:
pattern = "/" regexp "/"
Any line that starts with an exclamation mark is a comment and it will be ignored by the filtering engine. Comments are usually placed above rules and used to describe what a rule does.
! This is a comment
Examples:
-
/example.*/will block hosts matching theexample.*regex. -
@@/example.*/$importantwill unblock hosts matching theexample.*regex. Note that this rule also has the$importantmodifier.
You can change the behavior of a rule by using additional modifiers. Modifiers must be located at the end of the rule after the $ character and be separated by commas.
Examples:
||example.org^$important
-
||example.org^- a matching pattern -
$- a delimiter, it signals that now modifiers start -
important- a modifier
You may want to use multiple modifiers in a rule. Separate them by commas in this case.
||example.org^$client=127.0.0.1,dnstype=A
-
||example.org^- a matching pattern -
$- a delimiter, it signals that now modifiers start -
client=127.0.0.1- client modifier with its value -
,- modifiers delimiter -
dnstype=A- dnstype modifier with its value
IMPORTANT: If a rule contains a modifier not listed in this document, the whole rule must be ignored. This way we will avoid false-positives when people are trying to use unmodified browser ad blockers' filter lists like EasyList or EasyPrivacy.
The $client modifier allows specifying clients this rule will be working for. It accepts both client names, IP addresses or CIDR ranges.
The syntax is:
$client=value1|value2|...
You can also specify "excluded" clients by adding a ~ character before the client IP or name. In this case, the rule will not be applied to this client's DNS requests.
$client=~value1
Specifying client names
Client names usually contain spaces or other special characters, that's why you should enclose the name in quotes (both double-quotes and single-quotes are supported). If the client name contains quotes, use \ to escape them. Also, you need to escape commas (,) and pipes (|).
Please note, that when specifying an "excluded" client, you must keep
~out of the quotes.
Examples
-
@@||*^$client=127.0.0.1— unblock everything for localhost -
||example.org^$client='Frank\'s laptop'— blockexample.orgfor the client namedFrank's laptoponly. Note that quote (') in the name must be escaped. -
||example.org^$client=~'Mary\'s\, John\'s\, and Boris\'s laptops'— blockexample.orgfor everyone except the client namedMary's, John's, and Boris's laptops. Note that comma (,) must be escaped as well. -
||example.org^$client=~Mom|~Dad|Kids-- blockexample.orgforKids, but not forMomandDad. This example demonstrates how to specify multiple clients in one rule. -
||example.org^$client=192.168.0.0/24-- blockexample.orgfor all clients with IP addresses in the range192.168.0.0-192.168.0.255
(Since v0.105.0.)
The $dnstype modifier allows specifying DNS request type on which this rule
will be triggered.
The syntax is:
$dnstype=value1|value2|...
$dnstype=~value1|~value2|~...
The names of the types are case-insensitive, but are validated against a set of actual DNS resource record (RR) types.
This:
$dnstype=~value1|value2
Is equivalent to this:
$dnstype=value2
Examples
-
||example.org^$dnstype=AAAA— Block IPv6 DNS requests forexample.org. -
||example.org^$dnstype=~A|~CNAME— Only allowAandCNAMEDNS requests forexample.org, block out the rest.
(Since v0.105.0.)
The $dnsrewrite response modifier allows replacing the content of the response
to the DNS request for the matching hosts.
The shorthand syntax is:
$dnsrewrite=1.2.3.4
$dnsrewrite=abcd::1234
$dnsrewrite=example.net
$dnsrewrite=REFUSED
The keywords, like REFUSED, MUST be in all caps.
The full syntax is of the form RCODE;RRTYPE;VALUE:
$dnsrewrite=NOERROR;A;1.2.3.4
$dnsrewrite=NOERROR;AAAA;abcd::1234
$dnsrewrite=NOERROR;CNAME;example.net
$dnsrewrite=REFUSED;;
The CNAME one is special because AdGuardHome will resolve the host and add
its info to the response. That is, if example.net has IP 1.2.3.4, and the
user has this in their filter rules:
||example.com^$dnsrewrite=example.net
Or:
||example.com^$dnsrewrite=NOERROR;CNAME;example.net
Then the response will be something like:
$ nslookup example.com my.adguard.local
Server: my.adguard.local
Address: 127.0.0.1#53
Non-authoritative answer:
example.com canonical name = example.net.
Name: example.net
Address: 1.2.3.4
Keyword rewrites (for example, REFUSED) take precedence over the other. Next,
the CNAME rewrite. After that, all other records's values are summed as one
response, so this:
||example.com^$dnsrewrite=NOERROR;A;1.2.3.4
||example.com^$dnsrewrite=NOERROR;A;1.2.3.5
Will result in a response with two A records.
Currently supported RR types with examples:
-
||4.3.2.1.in-addr.arpa.^$dnsrewrite=NOERROR;PTR;example.net.adds aPTRrecord for reverse DNS. Reverse DNS requests for1.2.3.4to the AdGuardHome DNS server will result inexample.net.NOTE: the IP MUST be in reverse order, and the value MUST contain a final dot. See RFC.
-
||example.com^$dnsrewrite=NOERROR;A;1.2.3.4adds anArecord with the value1.2.3.4. -
||example.com^$dnsrewrite=NOERROR;AAAA;abcd::1234adds anAAAArecord with the valueabcd::1234. -
||example.com^$dnsrewrite=NOERROR;CNAME;example.orgadds aCNAMErecord. See explanation above. -
||example.com^$dnsrewrite=NOERROR;HTTPS;32 example.com alpn=h3adds anHTTPSrecord. Only a subset of parameter values is supported: values must becontiguousand, where avalue-listis expected`, only one value is currently supported:ipv4hint=127.0.0.1 // Supported. ipv4hint="127.0.0.1" // Unsupported. ipv4hint=127.0.0.1,127.0.0.2 // Unsupported. ipv4hint="127.0.0.1,127.0.0.2" // Unsupported.This will change in the future.
-
||example.com^$dnsrewrite=NOERROR;MX;32 example.mailadds anMXrecord with precedence value32and exchange valueexample.mail. -
||example.com^$dnsrewrite=NOERROR;SVCB;32 example.com alpn=h3adds andSVCBvalue. SeeHTTPSabove. -
||example.com^$dnsrewrite=NOERROR;TXT;hello_worldadds aTXTrecord with the valuehello_world. -
||example.com^$dnsrewrite=NXDOMAIN;;responds with anNXDOMAINcode.
Exception rules remove one or all rules:
-
||example.com^$dnsrewriteremoves all DNS rewrite rules. -
||example.com^$dnsrewrite=1.2.3.4removes the DNS rewrite rule that adds anArecord with the value1.2.3.4.
The $important modifier applied to a rule increases its priority over any other rule without $important modifier. Even over basic exception rules.
Example 1:
||example.org^$important
@@||example.org^
||example.org^$important will block all requests despite the exception rule.
Example 2:
||example.org^$important
@@||example.org^$important
Now the exception rule also has the $important modifier so it will prevail.
The rules with the $badfilter modifier disable other basic rules to which they refer. It means that the text of the disabled rule should match the text of the $badfilter rule (without the badfilter modifier).
Examples:
-
||example.com$badfilterdisables||example.com -
@@||example.org^$badfilterdisables@@||example.org^
$ctag modifier allows to block domains only for specific types of DNS clients. You can assign tags to clients in AdGuardHome UI. In the future we plan to assign tags automatically by analyzing the behavior of each client.
The syntax is:
$ctag=value1|value2|...
If one of client's tags matches the $ctag values - this rule applies to this client.
The syntax for exclusion is:
$ctag=~value1|~value2|...
If one of client's tags matches the exclusion $ctag values - this rule doesn't apply to this client.
Examples:
-
||example.org^$ctag=device_pc|device_phone- blockexample.orgfor clients tagged asdevice_pcordevice_phone -
||example.org^$ctag=~device_phone- blockexample.orgfor all clients except those tagged asdevice_phone
The list of allowed tags:
By device type:
-
device_audio- Audio device -
device_camera- Camera device -
device_gameconsole- Game Console -
device_laptop- Laptop -
device_nas- NAS (Network-attached Storage) -
device_other- Other device -
device_pc- PC -
device_phone- Phone -
device_printer- Printer -
device_securityalarm- Security alarm -
device_tablet- Tablet -
device_tv- TV
By Operating System:
-
os_android- Android -
os_ios- iOS -
os_linux- Linux -
os_macos- Macos -
os_other- Other OS -
os_windows- Windows
By user group:
-
user_admin- Administrator -
user_child- Child -
user_regular- Regular user
For each host a single line should be present with the following information:
IP_address canonical_hostname [aliases...]
Fields of the entry are separated by any number of blanks and/or tab characters.
Text from the # character until the end of the line is a comment and is ignored.
Example:
# This is a comment
Hostnames may contain only alphanumeric characters, minus signs (-), and periods (.). They must begin with an alphabetic character and end with an alphanumeric character. Optional aliases provide for name changes, alternate spellings, shorter hostnames, or generic hostnames (for example, localhost).
Examples:
127.0.0.1 example.org foo
127.0.0.1 example.com
Please note, that the
IP_addressvalue is ignored by most of the DNS filtering software.
This is just a list of domain names, one name per line.
Example:
example.com
example.org
If a string is not a valid domain (e.g. *.example.org), AdGuard Home will consider it to be an adblock-style rule.
If you are maintaining a blocklist and use different sources in it, Hostlists compiler may be useful to you.
This is a simple tool that makes it easier to compile a hosts blocklist compatible with AdGuard Home or any other AdGuard product with DNS filtering.
What it's capable of:
- Compile a single blocklist from multiple sources.
- Exclude the rules you don't need.
- Cleanup the resulting list: deduplicate, remove invalid rules, compress the list.