Skip to content

Commit c04be8c

Browse files
Apply suggestions from code review
Co-authored-by: Erin Schaffer <[email protected]>
1 parent f354a1a commit c04be8c

File tree

1 file changed

+25
-29
lines changed

1 file changed

+25
-29
lines changed

articles/aks/concepts-network-legacy-cni.md

Lines changed: 25 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -13,57 +13,53 @@ ms.custom: fasttrack-edit
1313

1414
## Prerequisites
1515

16-
For both Azure CNI Node Subnet and Kubenet, the following prerequisites are required:
16+
The following prerequisites are required for Azure CNI Node Subnet and kubenet:
1717

18-
- The virtual network for the AKS cluster must allow outbound internet connectivity.
19-
- AKS clusters can't use `169.254.0.0/16`, `172.30.0.0/16`, `172.31.0.0/16`, or `192.0.2.0/24` for the Kubernetes service address range, pod address range, or cluster virtual network address range.
20-
- The cluster identity used by the AKS cluster must have at least [Network Contributor](../role-based-access-control/built-in-roles.md#network-contributor) permissions on the subnet within your virtual network. If you wish to define a [custom role](../role-based-access-control/custom-roles.md) instead of using the built-in Network Contributor role, the following permissions are required:
18+
* The virtual network for the AKS cluster must allow outbound internet connectivity.
19+
* AKS clusters can't use `169.254.0.0/16`, `172.30.0.0/16`, `172.31.0.0/16`, or `192.0.2.0/24` for the Kubernetes service address range, pod address range, or cluster virtual network address range.
20+
* The cluster identity used by the AKS cluster must have at least [Network Contributor](../role-based-access-control/built-in-roles.md#network-contributor) permissions on the subnet within the virtual network. If you want to define a [custom role](../role-based-access-control/custom-roles.md) instead of using the built-in Network Contributor role, the following permissions are required:
2121

2222
- `Microsoft.Network/virtualNetworks/subnets/join/action`
2323
- `Microsoft.Network/virtualNetworks/subnets/read`
2424
- `Microsoft.Authorization/roleAssignments/write`
2525

26-
- The subnet assigned to the AKS node pool can't be a [delegated subnet](../virtual-network/subnet-delegation-overview.md).
27-
- AKS doesn't apply Network Security Groups (NSGs) to its subnet and doesn't modify any of the NSGs associated with that subnet. If you provide your own subnet and add NSGs associated with that subnet, you must ensure the security rules in the NSGs allow traffic within the node CIDR range. For more information, see [Network security groups][aks-network-nsg].
26+
* The subnet assigned to the AKS node pool can't be a [delegated subnet](../virtual-network/subnet-delegation-overview.md).
27+
- AKS doesn't apply Network Security Groups (NSGs) to its subnet and doesn't modify any of the NSGs associated with that subnet. If you provide your own subnet and add NSGs associated with that subnet, make sure the security rules in the NSGs allow traffic within the node CIDR range. For more information, see [Network security groups][aks-network-nsg].
2828

29-
> [!WARNING]
30-
> To use Windows Server node pools, you must use Azure CNI. The kubenet network model isn't available for Windows Server containers.
29+
> [!NOTE]
30+
> kubenet isn't available for Windows Server containers. To use Windows Server node pools, you need to use Azure CNI.
3131
32-
## Azure CNI NodeSubnet
32+
## Azure CNI Node Subnet
3333

34-
By default, AKS clusters use [kubenet][kubenet] and create a virtual network and subnet. With _kubenet_, nodes get an IP address from a virtual network subnet. Network address translation (NAT) is then configured on the nodes, and pods receive an IP address "hidden" behind the node IP. This approach reduces the number of IP addresses that you need to reserve in your network space for pods to use.
3534
With [Azure Container Networking Interface (CNI)][cni-networking], every pod gets an IP address from the subnet and can be accessed directly. Systems in the same virtual network as the AKS cluster see the pod IP as the source address for any traffic from the pod. Systems outside the AKS cluster virtual network see the node IP as the source address for any traffic from the pod. These IP addresses must be unique across your network space and must be planned in advance. Each node has a configuration parameter for the maximum number of pods that it supports. The equivalent number of IP addresses per node are then reserved up front for that node. This approach requires more planning, and often leads to IP address exhaustion or the need to rebuild clusters in a larger subnet as your application demands grow.
3635

37-
## Deployment parameters
36+
### Deployment parameters
3837

3938
When you create an AKS cluster, the following parameters are configurable for Azure CNI networking:
4039

41-
**Virtual network**: The virtual network into which you want to deploy the Kubernetes cluster. If you want to create a new virtual network for your cluster, select _Create new_ and follow the steps in the _Create virtual network_ section. If you want to select an existing virtual network, make sure it's in the same location and Azure subscription as your Kubernetes cluster. For information about the limits and quotas for an Azure virtual network, see [Azure subscription and service limits, quotas, and constraints](../azure-resource-manager/management/azure-subscription-service-limits.md#azure-resource-manager-virtual-networking-limits).
40+
**Virtual network**: The virtual network into which you want to deploy the Kubernetes cluster. You can create a new virtual network or use an existing one. If you want to use an existing virtual network, make sure it's in the same location and Azure subscription as your Kubernetes cluster. For information about the limits and quotas for an Azure virtual network, see [Azure subscription and service limits, quotas, and constraints](../azure-resource-manager/management/azure-subscription-service-limits.md#azure-resource-manager-virtual-networking-limits).
4241

43-
**Subnet**: The subnet within the virtual network where you want to deploy the cluster. If you want to create a new subnet in the virtual network for your cluster, select _Create new_ and follow the steps in the _Create subnet_ section. For hybrid connectivity, the address range shouldn't overlap with any other virtual networks in your environment.
42+
**Subnet**: The subnet within the virtual network where you want to deploy the cluster. You can add new subnets into the virtual network during the cluster creation process. For hybrid connectivity, the address range shouldn't overlap with any other virtual networks in your environment.
4443

4544
**Azure Network Plugin**: When Azure network plugin is used, the internal LoadBalancer service with "externalTrafficPolicy=Local" can't be accessed from VMs with an IP in clusterCIDR that doesn't belong to AKS cluster.
4645

4746
**Kubernetes service address range**: This parameter is the set of virtual IPs that Kubernetes assigns to internal [services][services] in your cluster. This range can't be updated after you create your cluster. You can use any private address range that satisfies the following requirements:
4847

49-
- Must not be within the virtual network IP address range of your cluster
50-
- Must not overlap with any other virtual networks with which the cluster virtual network peers
51-
- Must not overlap with any on-premises IPs
52-
- Must not be within the ranges `169.254.0.0/16`, `172.30.0.0/16`, `172.31.0.0/16`, or `192.0.2.0/24`
48+
- Must not be within the virtual network IP address range of your cluster.
49+
- Must not overlap with any other virtual networks with which the cluster virtual network peers.
50+
- Must not overlap with any on-premises IPs.
51+
- Must not be within the ranges `169.254.0.0/16`, `172.30.0.0/16`, `172.31.0.0/16`, or `192.0.2.0/24`.
5352

54-
Although it's technically possible to specify a service address range within the same virtual network as your cluster, doing so isn't recommended. Unpredictable behavior can result if overlapping IP ranges are used. For more information, see the [FAQ](#azure-cni-pod-subnet-frequently-asked-questions) section of this article. For more information on Kubernetes services, see [Services][services] in the Kubernetes documentation.
53+
While it's possible to specify a service address range within the same virtual network as your cluster, we don't recommend it. Overlapping IP ranges can result in unpredictable behavior. For more information, see the [FAQ](#faq). For more information on Kubernetes services, see [Services][services] in the Kubernetes documentation.
5554

5655
**Kubernetes DNS service IP address**: The IP address for the cluster's DNS service. This address must be within the *Kubernetes service address range*. Don't use the first IP address in your address range. The first address in your subnet range is used for the *kubernetes.default.svc.cluster.local* address.
5756

5857
## Kubenet
5958

6059
AKS clusters use kubenet and create an Azure virtual network and subnet for you by default. With kubenet, nodes get an IP address from the Azure virtual network subnet. Pods receive an IP address from a logically different address space to the Azure virtual network subnet of the nodes. Network address translation (NAT) is then configured so the pods can reach resources on the Azure virtual network. The source IP address of the traffic is NAT'd to the node's primary IP address. This approach greatly reduces the number of IP addresses you need to reserve in your network space for pods to use.
6160

62-
With Azure Container Networking Interface (CNI), every pod gets an IP address from the subnet and can be accessed directly. These IP addresses must be planned in advance and unique across your network space. Each node has a configuration parameter for the maximum number of pods it supports. The equivalent number of IP addresses per node are then reserved up front for that node. This approach requires more planning, and often leads to IP address exhaustion or the need to rebuild clusters in a larger subnet as your application demands grow. You can configure the maximum pods deployable to a node at cluster creation time or when creating new node pools. If you don't specify `maxPods` when creating new node pools, you receive a default value of _110_ for kubenet.
63-
64-
## Before you begin
61+
You can configure the maximum pods deployable to a node at cluster creation time or when creating new node pools. If you don't specify `maxPods` when creating new node pools, you receive a default value of _110_ for kubenet.
6562

66-
You need the Azure CLI version 2.0.65 or later installed and configured. Run `az --version` to find the version. If you need to install or upgrade, see [Install Azure CLI][install-azure-cli].
6763

6864
## Overview of kubenet networking with your own subnet
6965

@@ -93,25 +89,25 @@ With _Azure CNI_, each pod receives an IP address in the IP subnet and can commu
9389
> [!NOTE]
9490
> Some of the system pods such as **konnectivity** within the cluster use the host node IP address rather than an IP from the overlay address space. The system pods will only use the node IP and not an IP address from the virtual network.
9591
96-
### IP address availability and exhaustion
92+
## IP address availability and exhaustion
9793

98-
A common issue with _Azure CNI_ is that the assigned IP address range is too small to then add more nodes when you scale or upgrade a cluster. The network team also might not be able to issue a large enough IP address range to support your expected application demands.
94+
A common issue with _Azure CNI_ is that the assigned IP address range is too small to then add more nodes when you scale or upgrade a cluster. The network team also might not be able to issue a large enough IP address range to support your expected application demands. As a compromise, you can create an AKS cluster that uses _kubenet_ and connect to an existing virtual network subnet. This approach lets the nodes receive defined IP addresses without the need to reserve a large number of IP addresses up front for any potential pods that could run in the cluster.
9995

100-
As a compromise, you can create an AKS cluster that uses _kubenet_ and connect to an existing virtual network subnet. This approach lets the nodes receive defined IP addresses without the need to reserve a large number of IP addresses up front for any potential pods that could run in the cluster. With *kubenet*, you can use a much smaller IP address range and support large clusters and application demands. For example, with a */27* IP address range on your subnet, you can run a 20-25 node cluster with enough room to scale or upgrade. This cluster size can support up to *2,200-2,750* pods (with a default maximum of 110 pods per node). The maximum number of pods per node that you can configure with *kubenet* in AKS is 250.
96+
With kubenet, you can use a much smaller IP address range and support large clusters and application demands. For example, with a */27* IP address range on your subnet, you can run a 20-25 node cluster with enough room to scale or upgrade. This cluster size can support up to *2,200-2,750* pods (with a default maximum of 110 pods per node). The maximum number of pods per node that you can configure with kubenet in AKS is 250.
10197

10298
The following basic calculations compare the difference in network models:
10399

104100
- **kubenet**: A simple _/24_ IP address range can support up to _251_ nodes in the cluster. Each Azure virtual network subnet reserves the first three IP addresses for management operations. This node count can support up to _27,610_ pods, with a default maximum of 110 pods per node.
105-
- **Azure CNI**: That same basic _/24_ subnet range can only support a maximum of _8_ nodes in the cluster. This node count can only support up to _240_ pods, with a default maximum of 30 pods per node).
101+
- **Azure CNI**: That same basic _/24_ subnet range can only support a maximum of _8_ nodes in the cluster. This node count can only support up to _240_ pods, with a default maximum of 30 pods per node.
106102

107103
> [!NOTE]
108104
> These maximums don't take into account upgrade or scale operations. In practice, you can't run the maximum number of nodes the subnet IP address range supports. You must leave some IP addresses available for scaling or upgrading operations.
109105
110-
### Virtual network peering and ExpressRoute connections
106+
## Virtual network peering and ExpressRoute connections
111107

112-
To provide on-premises connectivity, both _kubenet_ and Azure-CNI_ network approaches can use [Azure virtual network peering][vnet-peering] or [ExpressRoute connections][express-route]. Plan your IP address ranges carefully to prevent overlap and incorrect traffic routing. For example, many on-premises networks use a *10.0.0.0/8* address range that's advertised over the ExpressRoute connection. We recommend creating your AKS clusters in Azure virtual network subnets outside this address range, such as _172.16.0.0/16_.
108+
You can use [Azure virtual network peering][vnet-peering] or [ExpressRoute connections][express-route] with _Azure CNI_ and _kubenet_ to provide on-premises connectivity. Make sure you plan your IP addresses carefully to prevent overlap and incorrect traffic routing. For example, many on-premises networks use a *10.0.0.0/8* address range that's advertised over the ExpressRoute connection. We recommend creating your AKS clusters in Azure virtual network subnets outside of this address range, such as *172.16.0.0/16*.
113109

114-
For more information to help you decide which network model to use, see [Compare network models and their support scope][network-comparisons].
110+
For more information, see [Compare network models and their support scopes][network-comparisons].
115111

116112
## Azure CNI Pod Subnet frequently asked questions
117113

0 commit comments

Comments
 (0)