Merge branch 'main' into release-preview-aml-cli-v2-refresh

Author: v-alje

articles/active-directory/governance/TOC.yml

@@ -30,8 +30,6 @@
       items:
       - name: Manage access to resources - Microsoft Graph
         href: /graph/tutorial-access-package-api?toc=/azure/active-directory/governance/toc.json&bc=/azure/active-directory/governance/breadcrumb/toc.json
-      - name: Manage access to resources - PowerShell
-        href: /powershell/microsoftgraph/tutorial-entitlement-management?view=graph-powershell-beta
       - name: Review access to Microsoft 365 groups - Microsoft Graph
         href: /graph/tutorial-accessreviews-m365group
       - name: Review access to security groups - Microsoft Graph
@@ -156,4 +154,4 @@
   - name: Access reviews - Microsoft Graph API
     href: /graph/api/resources/accessreviewsv2-overview
   - name: Entitlement management - Microsoft Graph API
-    href: /graph/api/resources/entitlementmanagement-overview
+    href: /graph/api/resources/entitlementmanagement-overview

articles/aks/TOC.yml

@@ -317,6 +317,8 @@
               href: configure-kubenet-dual-stack.md
             - name: Use Azure-CNI
               href: configure-azure-cni.md
+        - name: Bring your own CNI
+          href: use-byo-cni.md
         - name: Create an internal load balancer
           href: internal-lb.md
         - name: Use a Standard Load Balancer

articles/aks/azure-disk-volume.md

@@ -3,7 +3,7 @@ title: Create a static volume for pods in Azure Kubernetes Service (AKS)
 description: Learn how to manually create a volume with Azure disks for use with a pod in Azure Kubernetes Service (AKS)
 services: container-service
 ms.topic: article
-ms.date: 03/29/2019
+ms.date: 04/01/2019
 
 
 #Customer intent: As a developer, I want to learn how to manually create and attach storage to a specific pod in AKS.
@@ -69,6 +69,7 @@ spec:
   accessModes:
     - ReadWriteOnce
   persistentVolumeReclaimPolicy: Retain
+  storageClassName: managed-csi
   csi:
     driver: disk.csi.azure.com
     readOnly: false
@@ -91,7 +92,7 @@ spec:
     requests:
       storage: 100Gi
   volumeName: pv-azuredisk
-  storageClassName: ""
+  storageClassName: managed-csi
 ```
 
 Use the `kubectl` commands to create the *PersistentVolume* and *PersistentVolumeClaim*.

articles/aks/azure-files-csi.md

@@ -3,7 +3,7 @@ title: Use Container Storage Interface (CSI) drivers for Azure Files on Azure Ku
 description: Learn how to use the Container Storage Interface (CSI) drivers for Azure Files in an Azure Kubernetes Service (AKS) cluster.
 services: container-service
 ms.topic: article
-ms.date: 03/24/2021
+ms.date: 04/01/2021
 author: palma21
 
 ---
@@ -39,6 +39,7 @@ A storage class is used to define how an Azure Files share is created. A storage
 * **Standard_GRS**: Standard geo-redundant storage
 * **Standard_ZRS**: Standard zone-redundant storage
 * **Standard_RAGRS**: Standard read-access geo-redundant storage
+* **Standard_RAGZRS**: Standard read-access geo-zone-redundant storage
 * **Premium_LRS**: Premium locally redundant storage
 * **Premium_ZRS**: Premium zone-redundant storage
 

articles/aks/azure-files-volume.md

@@ -4,7 +4,7 @@ titleSuffix: Azure Kubernetes Service
 description: Learn how to manually create a volume with Azure Files for use with multiple concurrent pods in Azure Kubernetes Service (AKS)
 services: container-service
 ms.topic: article
-ms.date: 03/9/2022
+ms.date: 04/1/2022
 
 
 #Customer intent: As a developer, I want to learn how to manually create and attach storage using Azure Files to a pod in AKS.
@@ -124,6 +124,7 @@ spec:
   accessModes:
     - ReadWriteMany
   persistentVolumeReclaimPolicy: Retain
+  storageClassName: azurefile-csi
   csi:
     driver: file.csi.azure.com
     readOnly: false
@@ -155,7 +156,7 @@ metadata:
 spec:
   accessModes:
     - ReadWriteMany
-  storageClassName: ""
+  storageClassName: azurefile-csi
   volumeName: azurefile
   resources:
     requests:

articles/aks/ingress-tls.md

@@ -567,14 +567,13 @@ spec:
   - host: hello-world-ingress.MY_CUSTOM_DOMAIN
     http:
       paths:
-      - path:
+      - path: /static(/|$)(.*)
         pathType: Prefix
         backend:
           service:
             name: aks-helloworld-one
             port: 
               number: 80
-        path: /static(/|$)(.*)
 ```
 
 Create the ingress resource using the `kubectl apply` command.

articles/aks/support-policies.md

@@ -28,7 +28,7 @@ Microsoft manages and monitors the following components through the control pane
 * Kubelet or Kubernetes API servers
 * Etcd or a compatible key-value store, providing Quality of Service (QoS), scalability, and runtime
 * DNS services (for example, kube-dns or CoreDNS)
-* Kubernetes proxy or networking
+* Kubernetes proxy or networking (except when [BYOCNI](use-byo-cni.md) is used)
 * Any additional addon or system component running in the kube-system namespace
 
 AKS isn't a Platform-as-a-Service (PaaS) solution. Some components, such as agent nodes, have *shared responsibility*, where users must help maintain the AKS cluster. User input is required, for example, to apply an agent node operating system (OS) security patch.
@@ -52,9 +52,9 @@ Microsoft provides technical support for the following examples:
 * Connectivity to all Kubernetes components that the Kubernetes service provides and supports, such as the API server.
 * Management, uptime, QoS, and operations of Kubernetes control plane services (Kubernetes control plane, API server, etcd, and coreDNS, for example).
 * Etcd data store. Support includes automated, transparent backups of all etcd data every 30 minutes for disaster planning and cluster state restoration. These backups aren't directly available to you or any users. They ensure data reliability and consistency. On-demand rollback or restore is not supported as a feature.
-* Any integration points in the Azure cloud provider driver for Kubernetes. These include integrations into other Azure services such as load balancers, persistent volumes, or networking (Kubernetes and Azure CNI).
+* Any integration points in the Azure cloud provider driver for Kubernetes. These include integrations into other Azure services such as load balancers, persistent volumes, or networking (Kubernetes and Azure CNI, except when [BYOCNI](use-byo-cni.md) is in use).
 * Questions or issues about customization of control plane components such as the Kubernetes API server, etcd, and coreDNS.
-* Issues about networking, such as Azure CNI, kubenet, or other network access and functionality issues. Issues could include DNS resolution, packet loss, routing, and so on. Microsoft supports various networking scenarios:
+* Issues about networking, such as Azure CNI, kubenet, or other network access and functionality issues, except when [BYOCNI](use-byo-cni.md) is in use. Issues could include DNS resolution, packet loss, routing, and so on. Microsoft supports various networking scenarios:
   * Kubenet and Azure CNI using managed VNETs or with custom (bring your own) subnets.
   * Connectivity to other Azure services and applications
   * Ingress controllers and ingress or load balancer configurations
@@ -75,6 +75,7 @@ Microsoft doesn't provide technical support for the following examples:
   > Microsoft can provide best-effort support for third-party open-source projects such as Helm. Where the third-party open-source tool integrates with the Kubernetes Azure cloud provider or other AKS-specific bugs, Microsoft supports examples and applications from Microsoft documentation.
 * Third-party closed-source software. This software can include security scanning tools and networking devices or software.
 * Network customizations other than the ones listed in the [AKS documentation](./index.yml).
+* Custom or 3rd-party CNI plugins used in [BYOCNI](use-byo-cni.md) mode.
 
 
 ## AKS support coverage for agent nodes

articles/aks/use-byo-cni.md

@@ -0,0 +1,214 @@
+---
+title: Bring your own Container Network Interface (CNI) plugin (preview)
+description: Learn how to utilize Azure Kubernetes Service with your own Container Network Interface (CNI) plugin
+services: container-service
+ms.topic: article
+ms.date: 3/30/2022
+---
+
+# Bring your own Container Network Interface (CNI) plugin with Azure Kubernetes Service (AKS) (PREVIEW)
+
+Kubernetes does not provide a network interface system by default; this functionality is provided by [network plugins][kubernetes-cni]. Azure Kubernetes Service provides several supported CNI plugins. Documentation for supported plugins can be found from the [networking concepts page][aks-network-concepts].
+
+While the supported plugins meet most networking needs in Kubernetes, advanced users of AKS may desire to utilize the same CNI plugin used in on-premises Kubernetes environments or to make use of specific advanced functionality available in other CNI plugins.
+
+This article shows how to deploy an AKS cluster with no CNI plugin pre-installed, which allows for installation of any third-party CNI plugin that works in Azure.
+
+[!INCLUDE [preview features callout](./includes/preview/preview-callout.md)]
+
+## Support
+
+BYOCNI has support implications - Microsoft support will not be able to assist with CNI-related issues in clusters deployed with BYOCNI. For example, CNI-related issues would cover most east/west (pod to pod) traffic, along with `kubectl proxy` and similar commands. If CNI-related support is desired, a supported AKS network plugin can be used or support could be procured for the BYOCNI plugin from a third-party vendor.
+
+Support will still be provided for non-CNI-related issues.
+
+## Prerequisites
+
+* For ARM/Bicep, use at least template version 2022-01-02-preview
+* For Azure CLI, use at least version 0.5.55 of the `aks-preview` extension
+* The virtual network for the AKS cluster must allow outbound internet connectivity.
+* AKS clusters may not use `169.254.0.0/16`, `172.30.0.0/16`, `172.31.0.0/16`, or `192.0.2.0/24` for the Kubernetes service address range, pod address range, or cluster virtual network address range.
+* The cluster identity used by the AKS cluster must have at least [Network Contributor](../role-based-access-control/built-in-roles.md#network-contributor) permissions on the subnet within your virtual network. If you wish to define a [custom role](../role-based-access-control/custom-roles.md) instead of using the built-in Network Contributor role, the following permissions are required:
+  * `Microsoft.Network/virtualNetworks/subnets/join/action`
+  * `Microsoft.Network/virtualNetworks/subnets/read`
+* The subnet assigned to the AKS node pool cannot be a [delegated subnet](../virtual-network/subnet-delegation-overview.md).
+* AKS doesn't apply Network Security Groups (NSGs) to its subnet and will not modify any of the NSGs associated with that subnet. If you provide your own subnet and add NSGs associated with that subnet, you must ensure the security rules in the NSGs allow traffic within the node CIDR range. For more details, see [Network security groups][aks-network-nsg].
+
+## Cluster creation steps
+
+### Install the aks-preview CLI extension
+
+```azurecli-interactive
+# Install the aks-preview extension
+az extension add --name aks-preview
+
+# Update the extension to make sure you have the latest version installed
+az extension update --name aks-preview
+```
+
+### Deploy a cluster
+
+# [Azure CLI](#tab/azure-cli)
+
+Deploying a BYOCNI cluster requires passing the `--network-plugin` parameter with the parameter value of `none`.
+
+1. First, create a resource group to create the cluster in:
+    ```azurecli-interactive
+    az group create -l <Region> -n <ResourceGroupName>
+    ```
+
+1. Then create the cluster itself:
+    ```azurecli-interactive
+    az aks create -l <Region> -g <ResourceGroupName> -n <ClusterName> --network-plugin none
+    ```
+
+# [Azure Resource Manager](#tab/azure-resource-manager)
+
+When using an Azure Resource Manager template to deploy, pass `none` to the `networkPlugin` parameter to the `networkProfile` object. See the [Azure Resource Manager template documentation][deploy-arm-template] for help with deploying this template, if needed.
+
+```json
+{
+  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
+  "contentVersion": "1.0.0.0",
+  "parameters": {
+    "clusterName": {
+      "type": "string",
+      "defaultValue": "aksbyocni"
+    },
+    "location": {
+      "type": "string",
+      "defaultValue": "[resourceGroup().location]"
+    },
+    "kubernetesVersion": {
+      "type": "string",
+      "defaultValue": "1.22"
+    },
+    "nodeCount": {
+      "type": "int",
+      "defaultValue": 3
+    },
+    "nodeSize": {
+      "type": "string",
+      "defaultValue": "Standard_B2ms"
+    }
+  },
+  "resources": [
+    {
+      "type": "Microsoft.ContainerService/managedClusters",
+      "apiVersion": "2022-02-02-preview",
+      "name": "[parameters('clusterName')]",
+      "location": "[parameters('location')]",
+      "identity": {
+        "type": "SystemAssigned"
+      },
+      "properties": {
+        "agentPoolProfiles": [
+          {
+            "name": "nodepool1",
+            "count": "[parameters('nodeCount')]",
+            "mode": "System",
+            "vmSize": "[parameters('nodeSize')]"
+          }
+        ],
+        "dnsPrefix": "[parameters('clusterName')]",
+        "kubernetesVersion": "[parameters('kubernetesVersion')]",
+        "networkProfile": {
+          "networkPlugin": "none"
+        }
+      }
+    }
+  ]
+}
+```
+
+# [Bicep](#tab/bicep)
+
+When using a Bicep template to deploy, pass `none` to the `networkPlugin` parameter to the `networkProfile` object. See the [Bicep template documentation][deploy-bicep-template] for help with deploying this template, if needed.
+
+```bicep
+param clusterName string = 'aksbyocni'
+param location string = resourceGroup().location
+param kubernetesVersion string = '1.22'
+param nodeCount int = 3
+param nodeSize string = 'Standard_B2ms'
+
+resource aksCluster 'Microsoft.ContainerService/managedClusters@2022-02-02-preview' = {
+  name: clusterName
+  location: location
+  identity: {
+    type: 'SystemAssigned'
+  }
+  properties: {
+    agentPoolProfiles: [
+      {
+        name: 'nodepool1'
+        count: nodeCount
+        mode: 'System'
+        vmSize: nodeSize
+      }
+    ]
+    dnsPrefix: clusterName
+    kubernetesVersion: kubernetesVersion
+    networkProfile: {
+      networkPlugin: 'none'
+    }
+  }
+}
+```
+
+### Deploy a CNI plugin
+
+When AKS provisioning completes, the cluster will be online, but all of the nodes will be in a `NotReady` state:
+
+```bash
+$ kubectl get nodes
+NAME                                STATUS     ROLES   AGE    VERSION
+aks-nodepool1-23902496-vmss000000   NotReady   agent   6m9s   v1.21.9
+
+$ kubectl get node -o custom-columns='NAME:.metadata.name,STATUS:.status.conditions[?(@.type=="Ready")].message'
+NAME                                STATUS
+aks-nodepool1-23902496-vmss000000   container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
+```
+
+At this point, the cluster is ready for installation of a CNI plugin.
+
+---
+## Next steps
+
+Learn more about networking in AKS in the following articles:
+
+* [Use a static IP address with the Azure Kubernetes Service (AKS) load balancer](static-ip.md)
+* [Use an internal load balancer with Azure Container Service (AKS)](internal-lb.md)
+
+* [Create a basic ingress controller with external network connectivity][aks-ingress-basic]
+* [Enable the HTTP application routing add-on][aks-http-app-routing]
+* [Create an ingress controller that uses an internal, private network and IP address][aks-ingress-internal]
+* [Create an ingress controller with a dynamic public IP and configure Let's Encrypt to automatically generate TLS certificates][aks-ingress-tls]
+* [Create an ingress controller with a static public IP and configure Let's Encrypt to automatically generate TLS certificates][aks-ingress-static-tls]
+
+<!-- LINKS - External -->
+[kubernetes-cni]: https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/
+[cni-networking]: https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md
+[kubenet]: https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/#kubenet
+
+<!-- LINKS - Internal -->
+[az-aks-create]: /cli/azure/aks#az_aks_create
+[aks-ssh]: ssh.md
+[ManagedClusterAgentPoolProfile]: /azure/templates/microsoft.containerservice/managedclusters#managedclusteragentpoolprofile-object
+[aks-network-concepts]: concepts-network.md
+[aks-network-nsg]: concepts-network.md#network-security-groups
+[aks-ingress-basic]: ingress-basic.md
+[aks-ingress-tls]: ingress-tls.md
+[aks-ingress-static-tls]: ingress-static-ip.md
+[aks-http-app-routing]: http-application-routing.md
+[aks-ingress-internal]: ingress-internal-ip.md
+[az-extension-add]: /cli/azure/extension#az_extension_add
+[az-extension-update]: /cli/azure/extension#az_extension_update
+[az-feature-register]: /cli/azure/feature#az_feature_register
+[az-feature-list]: /cli/azure/feature#az_feature_list
+[az-provider-register]: /cli/azure/provider#az_provider_register
+[network-policy]: use-network-policies.md
+[nodepool-upgrade]: use-multiple-node-pools.md#upgrade-a-node-pool
+[network-comparisons]: concepts-network.md#compare-network-models
+[system-node-pools]: use-system-pools.md
+[prerequisites]: configure-azure-cni.md#prerequisites