100.64.0.0/10 is routable address space

[osterman]

It was raised by @Nuru that 100.64.0.0/10 is actually reserved address space. This also happens to be the kops default fornonMasqueradeCIDR: 100.64.0.0/10. So this has been in use for by kops as the default for around 2 years, and since a lot of people are using it, it should be reasonably well vetted and supported. Just be aware that it could be a bad idea to use it. The address is routable, and it routes to the carrier's NAT pool, so if it ever gets out of the cluster it will cause weird and potentially very hard-to-diagnose problems.

[Nuru]

My preferred CIDR is 172.24.0.0/13, which leaves 172.16.0.0/16 for naive users, 172.17.0.0/16 for Docker (it's default), and 172.18.0.0/16 for Docker Compose, and still gives kops 16 networks of 64k addresses each to use.

References:

• https://tools.ietf.org/html/rfc6598
  • https://tools.ietf.org/html/rfc6598#section-4
• Inappropriate use of RFC6598 addresses kubernetes/kops#2075
  • Inappropriate use of RFC6598 addresses kubernetes/kops#2075 (comment)
• Error using mysql-agent with Kops oracle/mysql-operator#190
• nonMasqueradeCIDR DoS kubernetes/kops#1458
• https://docs.docker.com/v17.09/engine/userguide/networking/#the-default-bridge-network