Add support for special use domains

[ruihildt]

It would be nice to be able to parse special use domains and recognize them as such. Right now it's not the case.

Maybe with a isSpecialUse boolean?

See: https://datatracker.ietf.org/doc/html/rfc6761

[remusao]

Hi @ruihildt, I am not very familiar with special use domains. Could you share some examples of expected behavior?

Thanks,

[ruihildt]

I'm working on a webextension which is dynamically proxying based on URL, and right now if I use isPrivate and isIcann, most of these domains are not recognized.

I have actually tried to understand if they should be falling under isPrivate, but didn't find relevant documentation.

I'll extract the relevant part of the spec for each special domain below.
It's unclear to me whether tldts would fall under Application software or Name resolution APIs and libraries (or if this is even relevant).

In any case, I think it would be nice to be able to parse these when using tldts.

---

6.1. Domain Name Reservation Considerations for Private Addresses

The private-address [RFC1918] reverse-mapping domains listed below,
and any names falling within those domains, are Special-Use Domain
Names:

 **10.in-addr.arpa.      21.172.in-addr.arpa.  26.172.in-addr.arpa.
 16.172.in-addr.arpa.  22.172.in-addr.arpa.  27.172.in-addr.arpa.
 17.172.in-addr.arpa.  30.172.in-addr.arpa.  28.172.in-addr.arpa.
 18.172.in-addr.arpa.  23.172.in-addr.arpa.  29.172.in-addr.arpa.
 19.172.in-addr.arpa.  24.172.in-addr.arpa.  31.172.in-addr.arpa.
 20.172.in-addr.arpa.  25.172.in-addr.arpa.  168.192.in-addr.arpa.**

These domains, and any names falling within these domains, are
special in the following ways:

1. Users are free to use these names as they would any other
   reverse-mapping names. However, since there is no central
   authority responsible for use of private addresses, users SHOULD
   be aware that these names are likely to yield different results
   on different networks.

2. Application software SHOULD NOT recognize these names as special,
   and SHOULD use these names as they would other reverse-mapping
   names.

3. Name resolution APIs and libraries SHOULD NOT recognize these
   names as special and SHOULD NOT treat them differently. Name
   resolution APIs SHOULD send queries for these names to their
   configured caching DNS server(s).

---

6.2. Domain Name Reservation Considerations for "test."

The domain "test.", and any names falling within ".test.", are
special in the following ways:

1. Users are free to use these test names as they would any other
   domain names. However, since there is no central authority
   responsible for use of test names, users SHOULD be aware that
   these names are likely to yield different results on different
   networks.

2. Application software SHOULD NOT recognize test names as special,
   and SHOULD use test names as they would other domain names.

3. Name resolution APIs and libraries SHOULD NOT recognize test
   names as special and SHOULD NOT treat them differently. Name
   resolution APIs SHOULD send queries for test names to their
   configured caching DNS server(s).

---

6.3. Domain Name Reservation Considerations for "localhost."

The domain "localhost." and any names falling within ".localhost."
are special in the following ways:

1. Users are free to use localhost names as they would any other
   domain names. Users may assume that IPv4 and IPv6 address
   queries for localhost names will always resolve to the respective
   IP loopback address.

2. Application software MAY recognize localhost names as special, or
   MAY pass them to name resolution APIs as they would for other
   domain names.

3. Name resolution APIs and libraries SHOULD recognize localhost
   names as special and SHOULD always return the IP loopback address
   for address queries and negative responses for all other query
   types. Name resolution APIs SHOULD NOT send queries for
   localhost names to their configured caching DNS server(s).

---

6.4. Domain Name Reservation Considerations for "invalid."

The domain "invalid." and any names falling within ".invalid." are
special in the ways listed below. In the text below, the term
"invalid" is used in quotes to signify such names, as opposed to
names that may be invalid for other reasons (e.g., being too long).

1. Users are free to use "invalid" names as they would any other
   domain names. Users MAY assume that queries for "invalid" names
   will always return NXDOMAIN responses.

2. Application software MAY recognize "invalid" names as special or
   MAY pass them to name resolution APIs as they would for other
   domain names.

3. Name resolution APIs and libraries SHOULD recognize "invalid"
   names as special and SHOULD always return immediate negative
   responses. Name resolution APIs SHOULD NOT send queries for
   "invalid" names to their configured caching DNS server(s).

---

6.5. Domain Name Reservation Considerations for Example Domains

The domains "example.", "example.com.", "example.net.",
"example.org.", and any names falling within those domains, are
special in the following ways:

1. Users SHOULD understand that example names are reserved for use
   in documentation.

2. Application software SHOULD NOT recognize example names as
   special and SHOULD use example names as they would other domain
   names.

3. Name resolution APIs and libraries SHOULD NOT recognize example
   names as special and SHOULD NOT treat them differently. Name
   resolution APIs SHOULD send queries for example names to their
   configured caching DNS server(s).

---

While internal. is actually not part of the spec yet. Here's the working draft: https://datatracker.ietf.org/doc/html/draft-davies-internal-tld-02

[remusao]

Hi @ruihildt and sorry for not getting back to you sooner. Thanks for the documentation on these special use domains. If this issue is still relevant to you, would you be able to provide a set of test cases with current (as well as expected) tldts output on a bunch of these domains/URLs? That'd be useful. Also if you have any extra insights on what change we would need to make on tldts to handle these, it'd be great!