CNI overlay

Author: greg-lindsay

articles/application-gateway/for-containers/alb-controller-release-notes.md

@@ -5,7 +5,7 @@ services: application-gateway
 author: greg-lindsay
 ms.service: azure-appgw-for-containers
 ms.topic: release-notes
-ms.date: 2/15/2025
+ms.date: 3/24/2025
 ms.author: greglin
 ---
 
@@ -26,7 +26,8 @@ Instructions for new or existing deployments of ALB Controller are found in the
 
 | ALB Controller Version | Gateway API Version | Kubernetes Version | Release Notes |
 | ---------------------- | ------------------- | ------------------ | ------------- |
-| 1.4.14 | v1.1.1 | v1.26, v1.27, v1.28, v1.29, v1.30 | Bug fixes, including [issue #4086](https://github.com/Azure/AKS/issues/4086)  |
+| 1.5.1 | v1.1.1 | v1.26, v1.27, v1.28, v1.29, v1.30 | Support for Azure CNI Overlay |
+
 ## Release history
 
 | ALB Controller Version | Gateway API Version | Kubernetes Version | Release Notes |

articles/application-gateway/for-containers/container-networking.md

@@ -0,0 +1,88 @@
+---
+title: Container Networking with Azure Application Gateway for Containers
+description: Learn how Azure Application Gateway for Containers works with different container networking interfaces.
+services: application gateway
+author: greg-lindsay
+ms.service: azure-appgw-for-containers
+ms.topic: concept-article
+ms.date: 3/24/2025
+ms.author: greglin
+---
+
+# Container Networking with Application Gateway for Containers
+
+## Overview
+
+Kubernetes uses Container Networking Interface (CNI) plugins to manage networking in Kubernetes clusters. CNIs are responsible for assigning IP addresses to pods, network routing between pods, Kubernetes Service routing, and more.
+
+Azure Kubernetes Service (AKS) uses two main networking models: **overlay** network and **flat** network.
+
+* **Overlay networks**:
+  * Conserve VNet IP address space by using logically separate CIDR ranges for pods.
+  * Maximum cluster scale support.
+  * Simple IP address management.
+  
+* **Flat networks**:
+  * Pods get full VNet connectivity and can be directly reached via their private IP address from connected networks.
+  * Require large, nonfragmented VNet IP address space.
+
+When choosing a networking model, consider the use cases for each CNI plugin and the type of network model it uses:
+
+| CNI plugin | Networking model | Use case highlights |
+|-------------|----------------------|-----------------------|
+| **Azure CNI Overlay** | Overlay | - Best for VNET IP conservation<br/>- Max node count supported by API Server + 250 pods per node<br/>- Simpler configuration<br/> -No direct external pod IP access |
+| **Azure CNI Pod Subnet** | Flat | - Direct external pod access<br/>- Modes for efficient VNet IP usage _or_ large cluster scale support(Preview) |
+| **Azure CNI Node Subnet** | Flat | - Direct external pod access<br/>- Simpler configuration <br/>- Limited scale <br/>- Inefficient use of VNet IPs |
+
+When provisioning Application Gateway for Containers into a cluster that has CNI Overlay or CNI enabled, Application Gateway for Containers automatically detects the intended network configuration. There are no changes needed in Gateway or Ingress API configuration to specify CNI Overlay or CNI.
+
+## CNI Overlay and Application Gateway for Containers
+
+![Diagram depicting traffic from the Internet ingressing into Application Gateway for Containers and being sent to backend pods in an overlay network in AKS.](./media/container-networking/application-gateway-for-containers-cni-overlay.svg)
+
+A separate routing domain is created in the Azure Networking stack for the pod's private CIDR space, which creates an Overlay network for direct communication between pods. When Application Gateway for Containers is provisioned, the Overlay routing domain is further extended to the Application Gateway for Containers subnet, allowing proxying of requests from Application Gateway for Containers directly to pods.
+
+Application Gateway for Containers supports Azure Network Policies, Calico, and Cilium Kubernetes network policies running within the cluster.
+
+> [!IMPORTANT]
+> Application Gateway for Containers with CNI Overlay is in PREVIEW.<br>
+> See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.
+
+### Limitations
+
+* ALB Controller: You must be running version 1.5.0 or greater to take advantage of CNI Overlay.
+* Subnet Size: The Application Gateway for Containers subnet must be a /24 prefix; only one deployment is supported per subnet. A larger or smaller prefix isn't supported.
+* Regional VNet Peering: Application Gateway for Containers deployed in a virtual network in region A and the AKS cluster nodes in a virtual network in region A is not supported.
+* Global VNet Peering: Application Gateway for Containers deployed in a virtual network in region A and the AKS cluster nodes in a virtual network in region B is not supported.
+
+## CNI and Application Gateway for Containers
+
+Application Gateway for Containers supports various deployments of Azure CNI running within your Kubernetes cluster.
+
+* Azure CNI for dynamic IP allocation
+* Azure CNI for static block allocation
+
+In both cases, Application Gateway for Containers supports Azure Network Policies, Calico, and Cilium Kubernetes network policies running within the cluster.
+
+## Kubenet and Application Gateway for Containers
+
+Kubenet isn't supported by Application Gateway for Containers. If using Kubenet, we recommend [upgrading to CNI Overlay](/azure/aks/upgrade-aks-ipam-and-dataplane#kubenet-cluster-upgrade).
+
+## FAQ
+
+Q: Can I upgrade an existing cluster with Application Gateway for Containers from CNI to CNI Overlay?
+
+A: Yes, upgrade of the AKS cluster from CNI to CNI Overlay and Application Gateway for Containers automatically detects the change. It is recommended to schedule this during a maintenance window as it may take a few minutes post-cluster upgrade to detect and configure support for CNI Overlay.
+
+>[!WARNING]
+> Ensure the Application Gateway for Containers subnet is a /24 prior to upgrading. Upgrading from CNI to CNI Overlay with a larger subnet (i.e. /23) will lead to an outage and require the Application Gateway for Containers subnet to be recreated with a /24 subnet size.
+
+Q: Can I upgrade an existing cluster with Kubenet to CNI Overlay?
+
+A: Yes, however, installation of Application Gateway for Containers on a cluster with Kubenet isn't supported. Install Application Gateway for Containers post-upgrade to CNI Overlay.
+
+## Next Steps
+
+* [Deploy ALB Controller](quickstart-deploy-application-gateway-for-containers-alb-controller.md?tabs=install-helm-windows)
+* [Application Gateway for Containers components](application-gateway-for-containers-components.md)
+* [Upgrade AKS to CNI Overlay](/azure/aks/upgrade-aks-ipam-and-dataplane#upgrade-an-existing-cluster-to-azure-cni-overlay)

articles/application-gateway/for-containers/faq.yml

@@ -6,7 +6,7 @@ metadata:
   author: greg-lindsay
   ms.service: azure-appgw-for-containers
   ms.topic: concept-article
-  ms.date: 02/27/2024
+  ms.date: 3/24/2025
   ms.author: greglin
 
 title: Frequently asked questions about Application Gateway for Containers
@@ -19,19 +19,29 @@ sections:
         answer: Application Gateway for Containers offers various layer 7 load-balancing capabilities for your container applications. This service is highly available, scalable, and fully managed by Azure.
 
       - question: Where does Application Gateway for Containers store customer data?
-        answer: Application Gateway for Containers stores and processes data in the region of its deployed resource(s). Configuration data may be replicated to its region pair, where applicable, for resiliency.
+        answer: Application Gateway for Containers stores and processes data in the region of its deployed resources. Configuration data may be replicated to its region pair, where applicable, for resiliency.
 
       - question: How does Application Gateway for Containers handle routine maintenance?
         answer: Routine maintenance and updates are designed to not be service impacting and require no customer intervention. For updates that might break existing configurations or cause existing product functionality changes, notifications are published via [Azure Service Health](/azure/service-health/alerts-activity-log-service-notifications-portal). These notifications are also sent via email to subscription service administrators.
 
       - question: Does Application Gateway for Containers support FIPS?
-        answer: Yes, Application Gateway for Containers can run in a FIPS 140-2 approved mode of operation, commonly referred to as "FIPS mode". FIPS mode calls a FIPS 140-2 validated cryptographic module that ensures FIPS-compliant algorithms for encryption, hashing, and signing are used. Please open a support case via the Azure portal to request FIPS mode to be enabled if required. 
+        answer: Yes, Application Gateway for Containers can run in a FIPS 140-2 approved mode of operation, commonly referred to as **FIPS mode**. FIPS mode calls a FIPS 140-2 validated cryptographic module that ensures FIPS-compliant algorithms for encryption, hashing, and signing are used. If necessary, open a support case via the Azure portal to request that FIPS mode be enabled. 
+
+      - question: Does Application Gateway for Containers support Kubenet?
+        answer: No, Application Gateway for Containers doesn't support Kubenet in favor of [CNI Overlay](/azure/aks/azure-cni-overlay). [Learn more](/azure/aks/upgrade-aks-ipam-and-dataplane) about migrating from Kubenet to CNI Overlay.
 
   - name: Performance
     questions:
       - question: How does Application Gateway for Containers support high availability and scalability?
         answer: Application Gateway for Containers automatically ensures underlying components are spread across availability zones for increased resiliency, if the Azure region supports it.  If the region doesn't support zones, fault domains and update domains be used to help mitigate impact during planned maintainence and unexpected failures.
-          
+                 >[!WARNING]
+                 > Ensure the Application Gateway for Containers subnet is a /24 prior to upgrading. Upgrading from CNI to CNI Overlay with a larger subnet (i.e. /23) will lead to an outage and require the Application Gateway for Containers subnet to be recreated with a /24 subnet size.
+
+  - name: Configuration - AKS
+    questions:
+      - question: Can I upgrade an existing AKS cluster with Application Gateway for Containers from CNI to CNI Overlay?
+        answer: Yes. Upgrade of the AKS cluster from CNI to CNI Overlay and Application Gateway for Containers automatically detects the change. It is recommended to schedule this during a maintenance window as it may take a few minutes post-cluster upgrade to detect and configure support for CNI Overlay.
+        
   - name: Configuration - TLS
     questions:
       - question: Does Application Gateway for Containers support reencryption (end-to-end encryption) of traffic to backend targets?
@@ -43,16 +53,16 @@ sections:
   - name: Configuration - ALB Controller
     questions:
       - question: Is it supported to install both Application Gateway Ingress Controller (AGIC) and ALB Controller in the same Kubernetes cluster?
-        answer: Yes, both Application Gateway Ingress Controller (AGIC) and ALB Controller may run at the same time in the same Kubernetes cluster. Updates to AGIC or to ALB Controller won't interfere with each other. 
+        answer: Yes, both Application Gateway Ingress Controller (AGIC) and ALB Controller may run at the same time in the same Kubernetes cluster. Updates to AGIC or to ALB Controller don't interfere with each other. 
 
       - question: Is it supported to install multiple ALB Controllers on the same Kubernetes cluster?
-        answer: It's possible to install multiple ALB Controllers on the same cluster, however this isn't recommended or supported as it's not possible to partition gateways, routes, services, etc.
+        answer: It's possible to install multiple ALB Controllers on the same cluster, however this option isn't recommended or supported as it's not possible to partition gateways, routes, services, etc.
 
       - question: Is it supported to increase the number of pods/replicas for ALB Controller?
-        answer: No, user defined replica count is not supported.  ALB Controller is automatically provisioned with at least two replicas in active/passive configuration to enable high availability.
+        answer: No, user defined replica count is not supported. ALB Controller is automatically provisioned with at least two replicas in active/passive configuration to enable high availability.
 
       - question: Is it supported to use the same managed identity with multiple ALB Controllers?
         answer: No.  Each ALB controller must use its own unique managed identity.
 
       - question: Can I share the same frontend resource between multiple Gateway and/or Ingress resources in Kubernetes?
-        answer: No. A frontend should be unique to a single Ingress or Gateway resource. Multiple hostnames and routes can be defined in a given Gateway or Ingress resource to eliminate the need for numerous frontend resources.
+        answer: No. A frontend should be unique to a single Ingress or Gateway resource. Multiple hostnames and routes can be defined in a given Gateway or Ingress resource to eliminate the need for numerous frontend resources.