Merge pull request #300695 from craigshoemaker/aca/gpu-types-updates

[Container Apps] Update: GPU types -> clarify messaging

Author: v-dirichards

articles/container-apps/gpu-serverless-overview.md

@@ -8,7 +8,7 @@ ms.custom:
   - ignite-2024
   - build-2025
 ms.topic: how-to
-ms.date: 05/19/2025
+ms.date: 06/02/2025
 ms.author: cshoe
 ---
 

articles/container-apps/gpu-types.md

@@ -4,15 +4,15 @@ description: Learn to how select the most appropriate GPU type for your containe
 services: container-apps
 author: craigshoemaker
 ms.service: azure-container-apps
-ms.topic: how-to
-ms.date: 03/18/2025
+ms.topic: conceptual
+ms.date: 06/02/2025
 ms.author: cshoe
 ai-usage: ai-generated
 ---
 
 # Comparing GPU types in Azure Container Apps
 
-Azure Container Apps supports serverless GPU acceleration (preview), enabling compute-intensive machine learning, and AI workloads in containerized environments. This capability allows you to use GPU hardware without managing the underlying infrastructure, following the serverless model that defines Container Apps.
+Azure Container Apps supports serverless GPU acceleration, enabling compute-intensive machine learning, and AI workloads in containerized environments. This capability allows you to use GPU hardware without managing the underlying infrastructure, following the serverless model that defines Container Apps.
 
 This article compares the Nvidia T4 and A100 GPU options available in Azure Container Apps. Understanding the technical differences between these GPU types is important as you optimize your containerized applications for performance, cost-efficiency, and workload requirements.
 
@@ -22,25 +22,30 @@ The fundamental differences between T4 and A100 GPU types involve the amount of
 
 | GPU type | Description |
 |---|---|
-| T4 | Delivers cost-effective acceleration ideal for inference workloads and mainstream AI applications. The GPU is built on the Turing architecture, which provides sufficient computational power for most production inference scenarios. |
-| A100 | Features performance advantages for demanding workloads that require maximum computational power. The [massive memory capacity](#specs) helps you work with large language models, complex computer vision applications, or scientific simulations that wouldn't fit in the T4's more limited memory. |
+| T4 | Delivers cost-effective acceleration ideal for inference workloads and mainstream AI applications. |
+| A100 | Features performance advantages for demanding workloads that require maximum computational power. The [extended memory capacity](#specs) helps you work with large language models, complex computer vision applications, or scientific simulations that wouldn't fit in the T4's more limited memory. |
 
 The following table provides a comparison of the technical specifications between the NVIDIA T4 and NVIDIA A100 GPUs available in Azure Container Apps. These specifications highlight the key hardware differences, performance capabilities, and optimal use cases for each GPU type.
 
 <a name="specs"></a>
 
 | Specification | NVIDIA T4 | NVIDIA A100 |
 |---------------|-----------|-------------|
-| **Memory** | 16GB VRAM | 40GB or 80GB HBM2/HBM2e |
+| **GPU Memory** | 16GB VRAM | 80GB HBM2/HBM2e |
 | **Architecture** | Turing | Ampere |
-| **Power Consumption** | 70W TDP | Higher (400W for SXM variant) |
-| **Precision Support** | FP32, FP16 | TF32, FP32, FP16, BFLOAT16, INT8, INT4 |
-| **Training Performance** | Limited for modern deep learning | Up to 20x faster than T4 for large models |
 | **Inference Performance** | Cost-effective for smaller models | Substantially higher, especially for large models |
-| **Special Features** | - | MIG technology (up to seven isolated instances), NVLink |
-| **Optimal Model Size** | Small models (<5GB) | Medium to large models (>5GB) |
+| **Optimal Model Size** | Small models (<10GB) | Medium to large models (>10GB) |
 | **Best Use Cases** | Cost-effective inference, mainstream AI applications | Training workloads, large models, complex computer vision, scientific simulations |
-| **Scalability** | Limited multi-GPU scaling | Better multi-GPU scaling with NVLink |
+
+## Select a GPU type
+
+Choosing between the T4 and A100 GPUs requires careful consideration of several key factors. The primary workload type should guide the initial decision: for inference-focused workloads, especially with smaller models, the T4 often provides sufficient performance at a more attractive price point. For training-intensive workloads or inference with large models, the A100's superior performance becomes more valuable and often necessary.
+
+Model size and complexity represent another critical decision factor. For small models (under 5GB), the T4's 16GB memory is typically adequate. For medium-sized models (5-15GB) consider testing on both GPU types to determine the optimal cost vs. performance for your situation. Large models (over 15GB) often require the A100's expanded memory capacity and bandwidth.
+
+Evaluate your performance requirements carefully. For baseline acceleration needs, the T4 provides a good balance of performance and cost. For maximum performance in demanding applications, the A100 delivers superior results especially for large-scale AI and  high-performance computing workloads. Latency-sensitive applications benefit from the A100's higher compute capability and memory bandwidth, which reduce processing time.
+
+If you begin using a T4 GPU and then later decide to move to an A100, then request a quota capacity adjustment.
 
 ## Differences between GPU types
 
@@ -52,30 +57,14 @@ For inference workloads, choosing between T4 and A100 depends on several factors
 
 The T4 provides the most cost-effective inference acceleration, particularly when deploying smaller models. The A100, however, delivers substantially higher inference performance, especially for large models, where it can perform faster than the T4 GPU.
 
-When looking to scale, the T4 often provides better cost-performance ratio, while the A100 excels in scenarios requiring maximum performance. The A100 type is specially suited for large models or when using MIG to serve multiple inference workloads simultaneously.
+When looking to scale, the T4 often provides better cost-performance ratio, while the A100 excels in scenarios requiring maximum performance. The A100 type is specially suited for large models.
 
 ### Training workloads
 
 For AI training workloads, the difference between these GPUs becomes even more pronounced. The T4, while capable of handling small model training, faces significant limitations for modern deep learning training.
 
 The A100 is overwhelmingly superior for training workloads, delivering up to 20 times better performance for large models compared to the T4. The substantially larger memory capacity (40 GB or 80GB) enables training of larger models without the need for complex model parallelism techniques in many cases. The A100's higher memory bandwidth also significantly accelerates data loading during training, reducing overall training time.
 
-### Mixed precision and specialized workloads
-
-The capabilities for mixed precision and specialized compute formats differ significantly between these GPUs. The T4 supports FP32 and FP16 precision operations, providing reasonable acceleration for mixed precision workloads. However, its support for specialized formats is limited compared to the A100.
-
-The A100 offers comprehensive support for a wide range of precision formats, including TF32, FP32, FP16, BFLOAT16, INT8, and INT4. Since the A100 uses TensorFloat-32 (TF32), this GPU provides the mathematical accuracy of FP32 while delivering higher performance.
-
-For workloads that benefit from mixed precision or require specialized formats, the A100 offers significant advantages in terms of both performance and flexibility.
-
-## Selecting a GPU type
-
-Choosing between the T4 and A100 GPUs requires careful consideration of several key factors. The primary workload type should guide the initial decision: for inference-focused workloads, especially with smaller models, the T4 often provides sufficient performance at a more attractive price point. For training-intensive workloads or inference with large models, the A100's superior performance becomes more valuable and often necessary.
-
-Model size and complexity represent another critical decision factor. For small models (under 5GB), the T4's 16GB memory is typically adequate. For medium-sized models (5-15GB) consider testing on both GPU types to determine the optimal cost vs. performance for your situation. Large models (over 15GB) often require the A100's expanded memory capacity and bandwidth.
-
-Evaluate your performance requirements carefully. For baseline acceleration needs, the T4 provides a good balance of performance and cost. For maximum performance in demanding applications, the A100 delivers superior results especially for large-scale AI and  high-performance computing workloads. Latency-sensitive applications benefit from the A100's higher compute capability and memory bandwidth, which reduce processing time.
-
 ## Special considerations
 
 Keep in mind the following exceptions when you're selecting a GPU type: