Merge branch 'main' into release-azure-policy-versioning

Author: v-alje

.openpublishing.redirection.json

@@ -640,6 +640,11 @@
             "redirect_url": "/azure/azure-maps/supported-languages",
             "redirect_document_id": true
         },
+        {
+            "source_path_from_root": "/articles/azure-maps/tutorial-geofence.md",
+            "redirect_url": "/azure/azure-maps/",
+            "redirect_document_id": true
+        },
         {
             "source_path_from_root": "/articles/azure-maps/search-categories.md",
             "redirect_url": "/azure/azure-maps/supported-search-categories",

articles/ai-studio/concepts/rbac-ai-studio.md

@@ -27,7 +27,7 @@ In this article, you learn how to manage access (authorization) to an Azure AI S
 
 In the Azure AI Studio, there are two levels of access: the hub and the project. The hub is home to the infrastructure (including virtual network setup, customer-managed keys, managed identities, and policies) and where you configure your Azure AI services. Hub access can allow you to modify the infrastructure, create new hubs, and create projects. Projects are a subset of the hub that act as workspaces that allow you to build and deploy AI systems. Within a project you can develop flows, deploy models, and manage project assets. Project access lets you develop AI end-to-end while taking advantage of the infrastructure setup on the hub.
 
-:::image type="content" source="../media/concepts/azureai-hub-project-relationship.png" alt-text="Diagram of the relationship between AI Studio resources." lightbox="../media/concepts/azureai-hub-project-relationship.png":::
+:::image type="content" source="../media/concepts/resource-provider-connected-resources.svg" alt-text="Diagram of the relationship between AI Studio resources.":::
 
 One of the key benefits of the hub and project relationship is that developers can create their own projects that inherit the hub security settings. You might also have developers who are contributors to a project, and can't create new projects.
 

articles/aks/azure-blob-csi.md

@@ -4,7 +4,7 @@ description: Learn how to use the Container Storage Interface (CSI) driver for A
 ms.topic: article
 ms.custom:
 ms.subservice: aks-storage
-ms.date: 11/24/2023
+ms.date: 06/24/2024
 author: tamram
 ms.author: tamram
 
@@ -141,7 +141,7 @@ To have a storage volume persist for your workload, you can use a StatefulSet. T
             "kubernetes.io/os": linux
           containers:
             - name: statefulset-blob-nfs
-              image: mcr.microsoft.com/oss/nginx/nginx:1.19.5
+              image: mcr.microsoft.com/oss/nginx/nginx:1.22
               volumeMounts:
                 - name: persistent-storage
                   mountPath: /mnt/blob
@@ -190,7 +190,7 @@ To have a storage volume persist for your workload, you can use a StatefulSet. T
             "kubernetes.io/os": linux
           containers:
             - name: statefulset-blob
-              image: mcr.microsoft.com/oss/nginx/nginx:1.19.5
+              image: mcr.microsoft.com/oss/nginx/nginx:1.22
               volumeMounts:
                 - name: persistent-storage
                   mountPath: /mnt/blob

articles/aks/istio-deploy-addon.md

@@ -34,7 +34,7 @@ export LOCATION=<location>
 
 ## Install Istio add-on
 
-This section includes steps to install the Istio add-on during cluster creation or enable for an existing cluster using the Azure CLI. If you want to install the add-on using Bicep, see [install an AKS cluster with the Istio service mesh add-on using Bicep][install-aks-cluster-istio-bicep]. To learn more about the Bicep resource definition for an AKS cluster, see [Bicep managedCluster reference][bicep-aks-resource-definition].
+This section includes steps to install the Istio add-on during cluster creation or enable for an existing cluster using the Azure CLI. If you want to install the add-on using Bicep, see the guide for [installing an AKS cluster with the Istio service mesh add-on using Bicep][install-aks-cluster-istio-bicep]. To learn more about the Bicep resource definition for an AKS cluster, see [Bicep managedCluster reference][bicep-aks-resource-definition].
 
 ### Revision selection
 
@@ -98,12 +98,13 @@ Confirm the `istiod` pod has a status of `Running`. For example:
 
 ```
 NAME                               READY   STATUS    RESTARTS   AGE
-istiod-asm-1-18-74f7f7c46c-xfdtl   1/1     Running   0          2m
+istiod-asm-1-18-74f7f7c46c-xfdtl   2/2     Running   0          2m
+istiod-asm-1-18-74f7f7c46c-4nt2v   2/2     Running   0          2m
 ```
 
 ## Enable sidecar injection
 
-To automatically install sidecar to any new pods, you will need to annotate your namespaces with the revision label corresponding to the control plane revision currently installed.
+To automatically install sidecar to any new pods, you need to annotate your namespaces with the revision label corresponding to the control plane revision currently installed.
 
 If you're unsure which revision is installed, use:
 
@@ -204,7 +205,7 @@ reviews-v2-7d79d5bd5d-8zzqd       2/2     Running   0          2m41s
 reviews-v3-7dbcdcbc56-m8dph       2/2     Running   0          2m41s
 ```
 
-Confirm that all the pods have status of `Running` with 2 containers in the `READY` column. The second container (`istio-proxy`) added to each pod is the Envoy sidecar injected by Istio, and the other is the application container.
+Confirm that all the pods have status of `Running` with two containers in the `READY` column. The second container (`istio-proxy`) added to each pod is the Envoy sidecar injected by Istio, and the other is the application container.
 
 To test this sample application against ingress, check out [next-steps](#next-steps).
 
@@ -240,6 +241,8 @@ az group delete --name ${RESOURCE_GROUP} --yes --no-wait
 ## Next steps
 
 * [Deploy external or internal ingresses for Istio service mesh add-on][istio-deploy-ingress]
+* [Scale istiod and ingress gateway HPA][istio-scaling-guide]
+
 
 <!--- External Links --->
 [install-aks-cluster-istio-bicep]: https://github.com/Azure-Samples/aks-istio-addon-bicep
@@ -255,4 +258,4 @@ az group delete --name ${RESOURCE_GROUP} --yes --no-wait
 [istio-deploy-ingress]: istio-deploy-ingress.md
 [az-aks-mesh-get-revisions]: /cli/azure/aks/mesh#az-aks-mesh-get-revisions(aks-preview)
 [bicep-aks-resource-definition]: /azure/templates/microsoft.containerservice/managedclusters
-
+[istio-scaling-guide]: istio-scale.md#scaling

articles/aks/istio-deploy-ingress.md

@@ -239,7 +239,8 @@ az group delete --name ${RESOURCE_GROUP} --yes --no-wait
 ## Next steps
 
 * [Secure ingress gateway for Istio service mesh add-on][istio-secure-gateway]
+* [Scale ingress gateway HPA][istio-scaling-guide]
 
 [istio-deploy-addon]: istio-deploy-addon.md
 [istio-secure-gateway]: istio-secure-gateway.md
-
+[istio-scaling-guide]: istio-scale.md#scaling

articles/aks/istio-scale.md

@@ -1,15 +1,15 @@
 ---
-title: Istio service mesh AKS add-on performance
-description: Istio service mesh AKS add-on performance
+title: Istio service mesh Azure Kubernetes Service add-on performance and scaling
+description: Istio service mesh Azure Kubernetes Service add-on performance and scaling
 ms.topic: article
 ms.custom:
 ms.service: azure-kubernetes-service
 ms.date: 03/19/2024
 ms.author: shalierxia
 ---
 
-# Istio service mesh add-on performance
-The Istio-based service mesh add-on is logically split into a control plane (`istiod`) and a data plane. The data plane is composed of Envoy sidecar proxies inside workload pods. Istiod manages and configures these Envoy proxies. This article presents the performance of both the control and data plane for revision asm-1-19, including resource consumption, sidecar capacity, and latency overhead. Additionally, it provides suggestions for addressing potential strain on resources during periods of heavy load. 
+# Istio service mesh add-on performance and scaling
+The Istio-based service mesh add-on is logically split into a control plane (`istiod`) and a data plane. The data plane is composed of Envoy sidecar proxies inside workload pods. Istiod manages and configures these Envoy proxies. This article presents the performance of both the control and data plane for revision asm-1-19, including resource consumption, sidecar capacity, and latency overhead. Additionally, it provides suggestions for addressing potential strain on resources during periods of heavy load. This article also covers how to customize scaling for the control plane and gateways. 
 
 ## Control plane performance
 [Istiod’s CPU and memory requirements][control-plane-performance] correlate with the rate of deployment and configuration changes and the number of proxies connected. The scenarios tested were:
@@ -20,7 +20,7 @@ The Istio-based service mesh add-on is logically split into a control plane (`is
 #### Test specifications
 - One `istiod` instance with default settings
 - Horizontal pod autoscaling disabled
-- Tested with two network plugins: Azure CNI Overlay and Azure CNI Overlay with Cilium [ (recommended network plugins for large scale clusters) ](/azure/aks/azure-cni-overlay?tabs=kubectl#choosing-a-network-model-to-use)
+- Tested with two network plugins: Azure Container Networking Interface (CNI) Overlay and Azure CNI Overlay with Cilium [ (recommended network plugins for large scale clusters) ](/azure/aks/azure-cni-overlay?tabs=kubectl#choosing-a-network-model-to-use)
 - Node SKU: Standard D16 v3 (16 vCPU, 64-GB memory)
 - Kubernetes version: 1.28.5
 - Istio revision: asm-1-19
@@ -70,43 +70,71 @@ The [ClusterLoader2 framework][clusterloader2] was used to determine the maximum
 ## Data plane performance
 Various factors impact [sidecar performance][data-plane-performance] such as request size, number of proxy worker threads, and number of client connections. Additionally, any request flowing through the mesh traverses the client-side proxy and then the server-side proxy. Therefore, latency and resource consumption are measured to determine the data plane performance.
 
-[Fortio][fortio] was used to create the load. The test was conducted with the [Istio benchmark repository][istio-benchmark] that was modified for use with the add-on.
+[`Fortio`][fortio] was used to create the load. The test was conducted with the [Istio benchmark repository][istio-benchmark] that was modified for use with the add-on.
 
 #### Test specifications
 - Tested with two network plugins: Azure CNI Overlay and Azure CNI Overlay with Cilium [ (recommended network plugins for large scale clusters) ](/azure/aks/azure-cni-overlay?tabs=kubectl#choosing-a-network-model-to-use)
 - Node SKU: Standard D16 v5 (16 vCPU, 64-GB memory)
 - Kubernetes version: 1.28.5
 - Two proxy workers
 - 1-KB payload
-- 1000 QPS at varying client connections
-- `http/1.1` protocol and mutual TLS enabled
+- 1,000 Queries per second (QPS) at varying client connections
+- `http/1.1` protocol and mutual Transport Layer Security (TLS) enabled
 - 26 data points collected
 
 #### CPU and memory
-The memory and CPU usage for both the client and server proxy for 16 client connections and 1000 QPS across all network plugin scenarios is roughly 0.4 vCPU and 72 MB. 
+The memory and CPU usage for both the client and server proxy for 16 client connections and 1,000 QPS across all network plugin scenarios is roughly 0.4 vCPU and 72 MB. 
 
 #### Latency
 The sidecar Envoy proxy collects raw telemetry data after responding to a client, which doesn't directly affect the request's total processing time. However, this process delays the start of handling the next request, contributing to queue wait times and influencing average and tail latencies. Depending on the traffic pattern, the actual tail latency varies. 
 
-The following evaluates the impact of adding sidecar proxies to the data path, showcasing the P90 and P99 latency.
+The following results evaluate the impact of adding sidecar proxies to the data path, showcasing the P90 and P99 latency.
 
 | Azure CNI Overlay |Azure CNI Overlay with Cilium |
 |:-------------------------:|:-------------------------:|
 [ ![Diagram that compares P99 latency for Azure CNI Overlay.](./media/aks-istio-addon/latency-box-plot/overlay-azure-p99.png) ](./media/aks-istio-addon/latency-box-plot/overlay-azure-p99.png#lightbox) |  [ ![Diagram that compares P99 latency for Azure CNI Overlay with Cilium.](./media/aks-istio-addon/latency-box-plot/overlay-cilium-p99.png) ](./media/aks-istio-addon/latency-box-plot/overlay-cilium-p99.png#lightbox)
 [ ![Diagram that compares P90 latency for Azure CNI Overlay.](./media/aks-istio-addon/latency-box-plot/overlay-azure-p90.png) ](./media/aks-istio-addon/latency-box-plot/overlay-azure-p90.png#lightbox)  |  [ ![Diagram that compares P90 latency for Azure CNI Overlay with Cilium.](./media/aks-istio-addon/latency-box-plot/overlay-cilium-p90.png) ](./media/aks-istio-addon/latency-box-plot/overlay-cilium-p90.png#lightbox)
 
+## Scaling 
+
+### Horizontal pod autoscaling
+
+[Horizontal pod autoscaling (HPA)][hpa] is enabled for the `istiod` and ingress gateway pods. The default configurations for `istiod` and the gateways are:
+- Min Replicas: 2
+- Max Replicas: 5
+- CPU Utilization: 80%
+
+> [!NOTE]
+> To prevent conflicts with the `PodDisruptionBudget`, the add-on does not allow setting the `minReplicas` below the initial default of `2`.
+
+The following are the `istiod` and ingress gateway HPA resources:
+```console
+NAMESPACE           NAME                                         REFERENCE
+aks-istio-ingress   aks-istio-ingressgateway-external-asm-1-19   Deployment/aks-istio-ingressgateway-external-asm-1-19
+
+aks-istio-ingress   aks-istio-ingressgateway-internal-asm-1-19   Deployment/aks-istio-ingressgateway-internal-asm-1-19
+
+aks-istio-system    istiod-asm-1-19                              Deployment/istiod-asm-1-19
+```
+
+The HPA configuration can be modified through patches and direct edits. Example:
+
+```bash
+kubectl patch hpa aks-istio-ingressgateway-external-asm-1-19 -n aks-istio-ingress --type merge --patch '{"spec": {"minReplicas": 3, "maxReplicas": 6}}'
+```
+
 ## Service entry
-Istio's ServiceEntry custom resource definition enables adding other services into the Istio’s internal service registry. A [ServiceEntry][serviceentry] allows services already in the mesh to route or access the services specified. However, the configuration of multiple ServiceEntries with the `resolution` field set to DNS can cause a [heavy load on DNS servers][understanding-dns]. The following suggestions can help reduce the load:
+Istio's ServiceEntry custom resource definition enables adding other services into the Istio’s internal service registry. A [ServiceEntry][serviceentry] allows services already in the mesh to route or access the services specified. However, the configuration of multiple ServiceEntries with the `resolution` field set to DNS can cause a [heavy load on Domain Name System (DNS) servers][understanding-dns]. The following suggestions can help reduce the load:
 
 - Switch to `resolution: NONE` to avoid proxy DNS lookups entirely. Suitable for most use cases.
 - Increase TTL (Time To Live) if you control the domains being resolved.
 - Limit the ServiceEntry scope with `exportTo`.
 
-
 [control-plane-performance]: https://istio.io/latest/docs/ops/deployment/performance-and-scalability/#control-plane-performance
 [data-plane-performance]: https://istio.io/latest/docs/ops/deployment/performance-and-scalability/#data-plane-performance
 [clusterloader2]: https://github.com/kubernetes/perf-tests/tree/master/clusterloader2#clusterloader
 [fortio]: https://fortio.org/
 [istio-benchmark]: https://github.com/istio/tools/tree/master/perf/benchmark#istio-performance-benchmarking
 [serviceentry]: https://istio.io/latest/docs/reference/config/networking/service-entry/
 [understanding-dns]: https://preliminary.istio.io/latest/docs/ops/configuration/traffic-management/dns/#proxy-dns-resolution
+[hpa]: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/

articles/aks/monitor-aks.md

@@ -178,9 +178,9 @@ When you [enable collection of Prometheus metrics](#integrations) for your clust
 
  Level | Alerts |
 |:---|:---|
-| Pod level | KubePodCrashLooping<br>Job didn't complete in time<br>Pod container restarted in last 1 hour<br>Ready state of pods is less than 80%<br>Number of pods in failed state are greater than 0<br>KubePodNotReadyByController<br>KubeStatefulSetGenerationMismatch<br>KubeJobNotCompleted<br>KubeJobFailed<br>Average CPU usage per container is greater than 95%<br>Average Memory usage per container is greater than 95%<br>KubeletPodStartUpLatencyHigh  |
-| Cluster level | Average PV usage is greater than 80%<br>KubeDeploymentReplicasMismatch<br>KubeStatefulSetReplicasMismatch<br>KubeHpaReplicasMismatch<br>KubeHpaMaxedOut<br>KubeCPUQuotaOvercommit<br>KubeMemoryQuotaOvercommit<br>KubeVersionMismatch<br>KubeClientErrors<br>CPUThrottlingHigh<br>KubePersistentVolumeFillingUp<br>KubePersistentVolumeInodesFillingUp<br>KubePersistentVolumeErrors |
-| Node level | Average node CPU utilization is greater than 80%<br>Working set memory for a node is greater than 80%<br>Number of OOM killed containers is greater than 0<br>KubeNodeUnreachable<br>KubeNodeNotReady<br>KubeNodeReadinessFlapping<br>KubeContainerWaiting<br>KubeDaemonSetNotScheduled<br>KubeDaemonSetMisScheduled<br>KubeletPlegDurationHigh<br>KubeletServerCertificateExpiration<br>KubeletClientCertificateRenewalErrors<br>KubeletServerCertificateRenewalErrors<br>KubeQuotaAlmostFull<br>KubeQuotaFullyUsed<br>KubeQuotaExceeded |
+| Cluster level | KubeCPUQuotaOvercommit<br>KubeMemoryQuotaOvercommit<br>KubeContainerOOMKilledCount<br>KubeClientErrors<br>KubePersistentVolumeFillingUp<br>KubePersistentVolumeInodesFillingUp<br>KubePersistentVolumeErrors<br>KubeContainerWaiting<br>KubeDaemonSetNotScheduled<br>KubeDaemonSetMisScheduled<br>KubeQuotaAlmostFull |
+| Node level | KubeNodeUnreachable<br>KubeNodeReadinessFlapping |
+| Pod level | KubePVUsageHigh<br>KubeDeploymentReplicasMismatch<br>KubeStatefulSetReplicasMismatch<br>KubeHpaReplicasMismatch<br>KubeHpaMaxedOut<br>KubePodCrashLooping<br>KubeJobStale<br>KubePodContainerRestart<br>KubePodReadyStateLow<br>KubePodFailedState<br>KubePodNotReadyByController<br>KubeStatefulSetGenerationMismatch<br>KubeJobFailed<br>KubeContainerAverageCPUHigh<br>KubeContainerAverageMemoryHigh<br>KubeletPodStartUpLatencyHigh |
 
 
 
@@ -214,4 +214,4 @@ When the [Network Observability](/azure/aks/network-observability-overview) add-
 
 <!-- Add additional links. You can change the wording of these and add more if useful.   -->
 
-- See [Monitoring AKS data reference](monitor-aks-reference.md) for a reference of the metrics, logs, and other important values created by AKS.
+- See [Monitoring AKS data reference](monitor-aks-reference.md) for a reference of the metrics, logs, and other important values created by AKS.