Update container FAQ concurrency

Author: ut-karsh

articles/ai-services/speech-service/speech-container-faq.yml

@@ -150,57 +150,48 @@ sections:
           
           
       - question: |
-          What are the recommended resources, CPU and RAM; for 50 concurrent requests?
+          What are the recommended CPU and RAM resources?
         answer: |
-          How many concurrent requests will a 4-core, 8-GB RAM handle? If we have to serve for example, 50 concurrent requests, how many Core and RAM is recommended?
-          
-          At real-time, 8 with our latest `en-US`, so we recommend using more docker containers beyond six concurrent requests. Beyond 16 cores it becomes nonuniform memory access (NUMA) node sensitive. The following table describes the minimum and recommended allocation of resources for each Speech container.
+        
+          The following table describes the minimum and recommended allocation of resources for each Speech container.
 
           # [Speech-to-text](#tab/stt)
-
-          The following table describes the minimum and recommended allocation of resources
           
-          | Container | Minimum | Recommended | Speech Model |
-          |-----------|---------|-------------|--------------|
-          | Speech to text | 4 core, 4-GB memory | 8 core, 8-GB memory | +4 to 8 GB memory |
+          | Container | Minimum | Recommended | Speech Model | Concurrency Limit |
+          |-----------|---------|-------------|--------------|-------------------|
+          | Speech to text | 4 core, 4-GB memory | 8 core, 8-GB memory | +4 to 8 GB memory | 2 sessions per core |
           
           > [!NOTE]
           > The minimum and recommended allocations are based on Docker limits, *not* the host machine resources.
           > For example, speech to text containers memory map portions of a large language model. We recommend that the entire file should fit in memory. You need to add an extra 4 to 8 GB to load the speech models (see above table).
           > Also, the first run of either container might take longer because models are being paged into memory.
           
           # [Custom speech-to-text](#tab/cstt)
-
-          The following table describes the minimum and recommended allocation of resources
           
-          | Container | Minimum | Recommended | Speech Model |
-          |-----------|---------|-------------|--------------|
-          | Custom speech to text | 4 core, 4-GB memory | 8 core, 8-GB memory | +4 to 8 GB memory |
+          | Container | Minimum | Recommended | Speech Model | Concurrency Limit |
+          |-----------|---------|-------------|--------------|-------------------|
+          | Custom speech to text | 4 core, 4-GB memory | 8 core, 8-GB memory | +4 to 8 GB memory | 2 sessions per core |
           
           > [!NOTE]
           > The minimum and recommended allocations are based on Docker limits, *not* the host machine resources.
           > For example, speech to text containers memory map portions of a large language model. We recommend that the entire file should fit in memory. You need to add an extra 4 to 8 GB to load the speech models (see above table).
           > Also, the first run of either container might take longer because models are being paged into memory.
           
           # [Neural Text-to-speech](#tab/tts)
-
-          The following table describes the minimum and recommended allocation of resources
           
-          | Container | Minimum | Recommended |
-          |-----------|---------|-------------|
+          | Container | Minimum | Recommended | Concurrency Limit |
+          |-----------|---------|-------------|-------------------| 5 requests per 8 core CPU |
           | Neural text to speech | 6 core, 12-GB memory | 8 core, 16-GB memory |
           
           > [!NOTE]
           > The minimum and recommended allocations are based on Docker limits, *not* the host machine resources.
           > Also, the first run of either container might take longer because models are being paged into memory.
           
           # [Speech language identification](#tab/lid)
-
-          The following table describes the minimum and recommended allocation of resources
           
-          | Container | Minimum | Recommended |
-          |-----------|---------|-------------|
-          | Speech language identification | 1 core, 1-GB memory | 1 core, 1-GB memory |
+          | Container | Minimum | Recommended | Concurrency Limit |
+          |-----------|---------|-------------|-------------------|
+          | Speech language identification | 1 core, 1-GB memory | 1 core, 1-GB memory | 2 sessions per core |
           
           > [!NOTE]
           > The minimum and recommended allocations are based on Docker limits, *not* the host machine resources.
@@ -210,12 +201,13 @@ sections:
 
           - Each core must be at least 2.6 GHz or faster.
           - For files, the throttling will be in the Speech SDK, at 2x. The first 5 seconds of audio aren't throttled.
-          - The decoder is capable of doing about 2-3x real-time. For this, the overall CPU usage will be close to two cores for a single recognition. That's why we don't recommend keeping more than two active connections, per container instance. The extreme side would be to put about 10 decoders at 2x real-time in an eight-core machine like `DS13_V2`. For the container version 1.3 and later, there's a param you could try setting `-e DECODER_MAX_COUNT=20`.
-          - For microphone, it is at 1x real-time. The overall usage should be at about one core for a single recognition.
+          - For speech to text, the decoder is capable of doing about 2-3x real-time. That's why we don't recommend keeping more than two active connections per core. The extreme side would be to put about 10 decoders at 2x real-time in an eight-core machine like `DS13_V2`. For the container version 1.3 and later, there's a param you could try setting `-e DECODER_MAX_COUNT=20`.
+          - For microphone, speech to text happens at 1x real-time. The overall usage should be at about one core for a single recognition.
+          - Beyond 16 cores the system becomes nonuniform memory access (NUMA) node sensitive.
          
-          Consider the total number of hours of audio you have. If the number is large, to improve reliability and availability, we suggest running more instances of containers, either on a single box or on multiple boxes, behind a load balancer. Orchestration could be done using Kubernetes (K8S) and Helm, or with Docker compose.
+          Consider the total number of hours of audio you have, and your expected time to complete the task. If the number is large, to improve reliability and availability, we suggest running more instances of containers, either on a single box or on multiple boxes, behind a load balancer. Orchestration could be done using Kubernetes (K8S) and Helm, or with Docker compose.
          
-          As an example, to handle 1000 hours/24 hours, we have tried setting up 3-4 VMs, with 10 instances/decoders per VM.
+          As an example, we have performed speech to text on 1000 hours of audio within 24 hours with 4-5 VMs and 10 instances/decoders per VM.
 
       - question: |
           Does the Speech container support punctuation?