Merge pull request #836 from MicrosoftDocs/main

10/16 11:00 AM IST Publish

Author: PhilKang0704

articles/ai-services/encryption/cognitive-services-encryption-keys-portal.md

@@ -2,13 +2,13 @@
 title: Customer-Managed Keys for Azure AI services
 titleSuffix: Azure AI services
 description: Learn about using customer-managed keys to improve data security with Azure AI services.
-author: deeikele
+author: PatrickFarley
 ms.service: azure-ai-services
 ms.custom:
   - ignite-2023
 ms.topic: conceptual
 ms.date: 11/15/2023
-ms.author: deeikele
+ms.author: pafarley
 ---
 
 # Customer-managed keys for encryption

articles/ai-services/speech-service/how-to-custom-voice-training-data.md

@@ -55,14 +55,14 @@ Follow these guidelines when preparing audio.
 | -------- | ----- |
 | File format | RIFF (.wav), grouped into a .zip file |
 | File name | File name characters supported by Windows OS, with .wav extension.<br>The characters `\ / : * ? " < > \|` aren't allowed. <br>It can't start or end with a space, and can't start with a dot. <br>No duplicate file names allowed. |
-| Sampling rate	| When you create a custom neural voice, 24,000 Hz is required. |
+| Sampling rate	| 24 KHz and higher required when creating a custom neural voice. |
 | Sample format | PCM, at least 16-bit |
 | Audio length | Shorter than 15 seconds |
 | Archive format | .zip |
 | Maximum archive size | 2048 MB |
 
 > [!NOTE]
-> The default sampling rate for a custom neural voice is 24,000 Hz. Audio files with a sampling rate lower than 16,000 Hz will be rejected. If a .zip file contains .wav files with different sample rates, only those equal to or higher than 16,000 Hz will be imported. Your audio files with a sampling rate higher than 16,000 Hz and lower than 24,000 Hz will be up-sampled to 24,000 Hz to train a neural voice. It's recommended that you should use a sample rate of 24,000 Hz for your training data.
+> The default sampling rate for a custom neural voice is 24 KHz. Audio files with a sampling rate lower than 16,000 Hz will be rejected. If a .zip file contains .wav files with different sample rates, only those equal to or higher than 16,000 Hz will be imported. Your audio files with a sampling rate higher than 16,000 Hz and lower than 24 KHz will be up-sampled to 24 KHz to train a neural voice. It's recommended that you should use a sample rate of 24 KHz and higher for your training data.
 
 ### Transcription data for Individual utterances + matching transcript
 
@@ -104,14 +104,14 @@ Follow these guidelines when preparing audio for segmentation.
 | -------- | ----- |
 | File format | RIFF (.wav) or .mp3, grouped into a .zip file |
 | File name	|  File name characters supported by Windows OS, with .wav extension. <br>The characters `\ / : * ? " < > \|` aren't allowed. <br>It can't start or end with a space, and can't start with a dot. <br>No duplicate file names allowed. |
-| Sampling rate	| When you create a custom neural voice, 24,000 Hz is required. |
+| Sampling rate	| 24 KHz and higher required when creating a custom neural voice. |
 | Sample format |RIFF(.wav): PCM, at least 16-bit.<br/><br/>mp3: At least 256 KBps bit rate.|
 | Audio length | Longer than 20 seconds |
 | Archive format | .zip |
 | Maximum archive size | 2048 MB, at most 1,000 audio files included |
 
 > [!NOTE]
-> The default sampling rate for a custom neural voice is 24,000 Hz. Audio files with a sampling rate lower than 16,000 Hz will be rejected. Your audio files with a sampling rate higher than 16,000 Hz and lower than 24,000 Hz will be up-sampled to 24,000 Hz to train a neural voice. It's recommended that you should use a sample rate of 24,000 Hz for your training data.
+> The default sampling rate for a custom neural voice is 24 KHz. Audio files with a sampling rate lower than 16,000 Hz will be rejected. Your audio files with a sampling rate higher than 16,000 Hz and lower than 24 KHz will be up-sampled to 24 KHz to train a neural voice. It's recommended that you should use a sample rate of 24 KHz and higher for your training data.
 
 All audio files should be grouped into a zip file. It's OK to put .wav files and .mp3 files into the same zip file. For example, you can upload a 45-second audio file named 'kingstory.wav' and a 200-second long audio file named 'queenstory.mp3' in the same zip file. All .mp3 files will be transformed into the .wav format after processing.
 
@@ -147,14 +147,14 @@ Follow these guidelines when preparing audio.
 | -------- | ----- |
 | File format | RIFF (.wav) or .mp3, grouped into a .zip file |
 | File name |  File name characters supported by Windows OS, with .wav extension. <br>The characters `\ / : * ? " < > \|` aren't allowed. <br>It can't start or end with a space, and can't start with a dot. <br>No duplicate file names allowed. |
-| Sampling rate	| When you create a custom neural voice, 24,000 Hz is required. |
+| Sampling rate	| 24 KHz and higher required when creating a custom neural voice. |
 | Sample format |RIFF(.wav): PCM, at least 16-bit<br>mp3: At least 256 KBps bit rate.|
 | Audio length | No limit |
 | Archive format | .zip |
 | Maximum archive size | 2048 MB, at most 1,000 audio files included |
 
 > [!NOTE]
-> The default sampling rate for a custom neural voice is 24,000 Hz. Your audio files with a sampling rate higher than 16,000 Hz and lower than 24,000 Hz will be up-sampled to 24,000 Hz to train a neural voice. It's recommended that you should use a sample rate of 24,000 Hz for your training data.
+> The default sampling rate for a custom neural voice is 24 KHz. Your audio files with a sampling rate higher than 16,000 Hz and lower than 24 KHz will be up-sampled to 24 KHz to train a neural voice. It's recommended that you should use a sample rate of 24 KHz and higher for your training data.
 
 All audio files should be grouped into a zip file. Once your dataset is successfully uploaded, the Speech service helps you segment the audio file into utterances based on our speech batch transcription service. Unique IDs are assigned to the segmented utterances automatically. Matching transcripts are generated through speech recognition. All .mp3 files will be transformed into the .wav format after processing. You can check the segmented utterances and the matching transcripts by downloading the dataset.
 

articles/ai-services/speech-service/record-custom-voice-samples.md

@@ -336,15 +336,15 @@ Take regular breaks and provide a beverage to help your voice talent keep their
 
 ### After the session
 
-Modern recording studios run on computers. At the end of the session, you receive one or more audio files, not a tape. These files are probably WAV or AIFF format in CD quality (44.1 KHz 16-bit) or better. 24 KHz 16-bit is common and desirable. The default sampling rate for a custom neural voice is 24 KHz.  It's recommended that you should use a sample rate of 24 KHz for your training data. Higher sampling rates, such as 96 KHz, aren't usually needed.
+Modern recording studios run on computers. At the end of the session, you receive one or more audio files, not a tape. These files are probably WAV or AIFF format in CD quality (44.1 KHz 16-bit) or better. 24 KHz 16-bit is common and desirable. The default sampling rate for a custom neural voice is 24 KHz.  It's recommended that you should use a sample rate of 24 KHz and higher for your training data. Higher sampling rates, such as 96 KHz, aren't usually needed.
 
 Speech Studio requires each provided utterance to be in its own file. Each audio file delivered by the studio contains multiple utterances. So the primary post-production task is to split up the recordings and prepare them for submission. The recording engineer might have placed markers in the file (or provided a separate cue sheet) to indicate where each utterance starts.
 
 Use your notes to find the exact takes you want, and then use a sound editing utility, such as [Avid Pro Tools](https://www.avid.com/en/pro-tools), [Adobe Audition](https://www.adobe.com/products/audition.html), or the free [Audacity](https://www.audacityteam.org/), to copy each utterance into a new file.
 
 Listen to each file carefully. At this stage, you can edit out small unwanted sounds that you missed during recording, like a slight lip smack before a line, but be careful not to remove any actual speech. If you can't fix a file, remove it from your dataset and note that you've done so.
 
-Convert each file to 16 bits and a sample rate of 24 KHz before saving and if you recorded the studio chatter, remove the second channel. Save each file in WAV format, naming the files with the utterance number from your script.
+Convert each file to 16 bits and a sample rate of 24 KHz and higher before saving and if you recorded the studio chatter, remove the second channel. Save each file in WAV format, naming the files with the utterance number from your script.
 
 Finally, create the transcript that associates each WAV file with a text version of the corresponding utterance. [Train your voice model](./professional-voice-train-voice.md) includes details of the required format. You can copy the text directly from your script. Then create a Zip file of the WAV files and the text transcript.
 

articles/ai-services/speech-service/speech-container-faq.yml

@@ -55,7 +55,7 @@ sections:
           
           > 2 (clusters) * 5 (VMs per cluster) * $0.3528/hour * 365 (days) * 24 (hours) = $31 K / year
           
-          When mapping to physical machine, a general estimation is 1 vCPU = 1 Physical CPU Core. In reality, 1vCPU is more powerful than a single core.
+          When mapping to a physical machine, a general estimation is 1 vCPU = 1 Physical CPU Core. In reality, 1vCPU is more powerful than a single core.
           
           For on-premises, all of these extra factors come into play:
           
@@ -150,57 +150,66 @@ sections:
           
           
       - question: |
-          What are the recommended resources, CPU and RAM; for 50 concurrent requests?
+          What are the recommended CPU and RAM resources for Speech containers?
         answer: |
-          How many concurrent requests will a 4-core, 8-GB RAM handle? If we have to serve for example, 50 concurrent requests, how many Core and RAM is recommended?
-          
-          At real-time, 8 with our latest `en-US`, so we recommend using more docker containers beyond six concurrent requests. Beyond 16 cores it becomes nonuniform memory access (NUMA) node sensitive. The following table describes the minimum and recommended allocation of resources for each Speech container.
 
-          # [Speech-to-text](#tab/stt)
+          # [Speech to text](#tab/stt)
 
-          The following table describes the minimum and recommended allocation of resources
+          The following table describes the minimum and recommended allocation of resources for Speech to text.
           
-          | Container | Minimum | Recommended | Speech Model |
-          |-----------|---------|-------------|--------------|
-          | Speech to text | 4 core, 4-GB memory | 8 core, 8-GB memory | +4 to 8 GB memory |
+          | Container | Minimum | Recommended | Speech Model | Concurrency Limit |
+          |-----------|---------|-------------|--------------|-------------------|
+          | Speech to text | 4 core, 4-GB memory | 8 core, 8-GB memory | +4 to 8 GB memory | 2 sessions per core |
           
           > [!NOTE]
           > The minimum and recommended allocations are based on Docker limits, *not* the host machine resources.
           > For example, speech to text containers memory map portions of a large language model. We recommend that the entire file should fit in memory. You need to add an extra 4 to 8 GB to load the speech models (see above table).
           > Also, the first run of either container might take longer because models are being paged into memory.
+
+          - For files, the throttling will be in the Speech SDK, at 2x. The first 5 seconds of audio aren't throttled.
+          - For speech to text, the decoder is capable of doing about 2-3x real-time. That's why we don't recommend keeping more than two active connections per core. The extreme side would be to put about 10 decoders at 2x real-time in an eight-core machine like `DS13_V2`. For the container version 1.3 and later, there's a param you could try setting `-e DECODER_MAX_COUNT=20`.
+          - For microphone, speech to text happens at 1x real-time. The overall usage should be at about one core for a single recognition.
+          - Beyond 16 cores the system becomes nonuniform memory access (NUMA) node sensitive.
           
-          # [Custom speech-to-text](#tab/cstt)
+          # [Custom speech to text](#tab/cstt)
 
-          The following table describes the minimum and recommended allocation of resources
+          The following table describes the minimum and recommended allocation of resources for Custom speech to text.
           
-          | Container | Minimum | Recommended | Speech Model |
-          |-----------|---------|-------------|--------------|
-          | Custom speech to text | 4 core, 4-GB memory | 8 core, 8-GB memory | +4 to 8 GB memory |
+          | Container | Minimum | Recommended | Speech Model | Concurrency Limit |
+          |-----------|---------|-------------|--------------|-------------------|
+          | Custom speech to text | 4 core, 4-GB memory | 8 core, 8-GB memory | +4 to 8 GB memory | 2 sessions per core |
           
           > [!NOTE]
           > The minimum and recommended allocations are based on Docker limits, *not* the host machine resources.
           > For example, speech to text containers memory map portions of a large language model. We recommend that the entire file should fit in memory. You need to add an extra 4 to 8 GB to load the speech models (see above table).
           > Also, the first run of either container might take longer because models are being paged into memory.
+
+          - For files, the throttling will be in the Speech SDK, at 2x. The first 5 seconds of audio aren't throttled.
+          - For speech to text, the decoder is capable of doing about 2-3x real-time. That's why we don't recommend keeping more than two active connections per core. The extreme side would be to put about 10 decoders at 2x real-time in an eight-core machine like `DS13_V2`. For the container version 1.3 and later, there's a param you could try setting `-e DECODER_MAX_COUNT=20`.
+          - For microphone, speech to text happens at 1x real-time. The overall usage should be at about one core for a single recognition.
+          - Beyond 16 cores the system becomes nonuniform memory access (NUMA) node sensitive.
           
           # [Neural Text-to-speech](#tab/tts)
 
-          The following table describes the minimum and recommended allocation of resources
+          The following table describes the minimum and recommended allocation of resources for Neural text to speech.
           
-          | Container | Minimum | Recommended |
-          |-----------|---------|-------------|
-          | Neural text to speech | 6 core, 12-GB memory | 8 core, 16-GB memory |
+          | Container | Minimum | Recommended | Concurrency Limit |
+          |-----------|---------|-------------|-------------------|
+          | Neural text to speech | 6 core, 12-GB memory | 8 core, 16-GB memory \n 8 core, 24-GB memory (multilingual voices) | 5 requests per 8 core CPU |
           
           > [!NOTE]
           > The minimum and recommended allocations are based on Docker limits, *not* the host machine resources.
           > Also, the first run of either container might take longer because models are being paged into memory.
+
+          - Real-time performance varies depending on concurrency. With a concurrency of 1, an NTTS container instance can achieve 10x real-time performance. However, when concurrency increases to 5, real-time performance drops to 3x or lower. We recommended sending less than 5 concurrent requests in one container. Start more containers for increased concurrency.
           
           # [Speech language identification](#tab/lid)
 
-          The following table describes the minimum and recommended allocation of resources
+          The following table describes the minimum and recommended allocation of resources for Language Identification.
           
-          | Container | Minimum | Recommended |
-          |-----------|---------|-------------|
-          | Speech language identification | 1 core, 1-GB memory | 1 core, 1-GB memory |
+          | Container | Minimum | Recommended | Concurrency Limit |
+          |-----------|---------|-------------|-------------------|
+          | Speech language identification | 1 core, 1-GB memory | 1 core, 1-GB memory | 2 sessions per core |
           
           > [!NOTE]
           > The minimum and recommended allocations are based on Docker limits, *not* the host machine resources.
@@ -209,13 +218,10 @@ sections:
           ***
 
           - Each core must be at least 2.6 GHz or faster.
-          - For files, the throttling will be in the Speech SDK, at 2x. The first 5 seconds of audio aren't throttled.
-          - The decoder is capable of doing about 2-3x real-time. For this, the overall CPU usage will be close to two cores for a single recognition. That's why we don't recommend keeping more than two active connections, per container instance. The extreme side would be to put about 10 decoders at 2x real-time in an eight-core machine like `DS13_V2`. For the container version 1.3 and later, there's a param you could try setting `-e DECODER_MAX_COUNT=20`.
-          - For microphone, it is at 1x real-time. The overall usage should be at about one core for a single recognition.
          
-          Consider the total number of hours of audio you have. If the number is large, to improve reliability and availability, we suggest running more instances of containers, either on a single box or on multiple boxes, behind a load balancer. Orchestration could be done using Kubernetes (K8S) and Helm, or with Docker compose.
+          Consider the total number of hours of audio you have, and your expected time to complete the task. If the number is large, to improve reliability and availability, we suggest running more instances of containers, either on a single box or on multiple boxes, behind a load balancer. Orchestration could be done using Kubernetes (K8S) and Helm, or with Docker compose.
          
-          As an example, to handle 1000 hours/24 hours, we have tried setting up 3-4 VMs, with 10 instances/decoders per VM.
+          As an example, we have performed speech to text on 1000 hours of audio within 24 hours with 4-5 VMs and 10 instances/decoders per VM.
 
       - question: |
           Does the Speech container support punctuation?