draft for text to speech basics

Author: Trevor Bye

articles/cognitive-services/Speech-Service/includes/how-to/text-to-speech-basics/text-to-speech-basics-csharp.md

@@ -0,0 +1,267 @@
+---
+author: trevorbye
+ms.service: cognitive-services
+ms.topic: include
+ms.date: 03/25/2020
+ms.author: trbye
+---
+
+## Prerequisites
+
+This article assumes that you have an Azure account and Speech service subscription. If you don't have an account and subscription, [try the Speech service for free](../../../get-started.md).
+
+## Install the Speech SDK
+
+Before you can do anything, you'll need to install the Speech SDK. Depending on your platform, use the following instructions:
+
+* <a href="https://docs.microsoft.com/azure/cognitive-services/speech-service/quickstarts/setup-platform?tabs=dotnet&pivots=programming-language-csharp" target="_blank">.NET Framework <span class="docon docon-navigate-external x-hidden-focus"></span></a>
+* <a href="https://docs.microsoft.com/azure/cognitive-services/speech-service/quickstarts/setup-platform?tabs=dotnetcore&pivots=programming-language-csharp" target="_blank">.NET Core <span class="docon docon-navigate-external x-hidden-focus"></span></a>
+* <a href="https://docs.microsoft.com/azure/cognitive-services/speech-service/quickstarts/setup-platform?tabs=unity&pivots=programming-language-csharp" target="_blank">Unity <span class="docon docon-navigate-external x-hidden-focus"></span></a>
+* <a href="https://docs.microsoft.com/azure/cognitive-services/speech-service/quickstarts/setup-platform?tabs=uwps&pivots=programming-language-csharp" target="_blank">UWP <span class="docon docon-navigate-external x-hidden-focus"></span></a>
+* <a href="https://docs.microsoft.com/azure/cognitive-services/speech-service/quickstarts/setup-platform?tabs=xaml&pivots=programming-language-csharp" target="_blank">Xamarin <span class="docon docon-navigate-external x-hidden-focus"></span></a>
+
+## Import dependencies
+
+To run the examples in this article, include the following `using` statements at the top of your script.
+
+```csharp
+using System;
+using Microsoft.CognitiveServices.Speech;
+using Microsoft.CognitiveServices.Speech.Audio;
+using System.Threading.Tasks;
+using System.Net;
+using System.IO;
+using System.Text;
+using System.Xml.Linq;
+```
+
+## Create a speech configuration
+
+To call the Speech service using the Speech SDK, you need to create a [`SpeechConfig`](https://docs.microsoft.com/dotnet/api/microsoft.cognitiveservices.speech.speechconfig?view=azure-dotnet). This class includes information about your subscription, like your key and associated region, endpoint, host, or authorization token.
+
+> [!NOTE]
+> Regardless of whether you're performing speech recognition, speech synthesis, translation, or intent recognition, you'll always create a configuration.
+
+There are a few ways that you can initialize a [`SpeechConfig`](https://docs.microsoft.com/dotnet/api/microsoft.cognitiveservices.speech.speechconfig?view=azure-dotnet):
+
+* With a subscription: pass in a key and the associated region.
+* With an endpoint: pass in a Speech service endpoint. A key or authorization token is optional.
+* With a host: pass in a host address. A key or authorization token is optional.
+* With an authorization token: pass in an authorization token and the associated region.
+
+In this example, you create a [`SpeechConfig`](https://docs.microsoft.com/dotnet/api/microsoft.cognitiveservices.speech.speechconfig?view=azure-dotnet) using a subscription key and region. You also create some basic boilerplate code to use for the rest of this article, which you modify for different customizations.
+
+```csharp
+public class Program 
+{
+    public static async Task SynthesizeAudioAsync() 
+    {
+        var config = SpeechConfig.FromSubscription("YourSubscriptionKey", "YourServiceRegion");
+    }
+
+    static void Main(string[] args)
+    {
+        SynthesizeAudioAsync().Wait();
+    }
+}
+```
+
+## Synthesize speech to a file
+
+Next, you create a [`SpeechSynthesizer`](https://docs.microsoft.com/dotnet/api/microsoft.cognitiveservices.speech.speechsynthesizer?view=azure-dotnet) object, which executes text-to-speech conversions and outputs to speakers, files, or other output streams. The [`SpeechSynthesizer`](https://docs.microsoft.com/dotnet/api/microsoft.cognitiveservices.speech.speechsynthesizer?view=azure-dotnet) accepts as params the [`SpeechConfig`](https://docs.microsoft.com/dotnet/api/microsoft.cognitiveservices.speech.speechconfig?view=azure-dotnet) object created in the previous step, and an [`AudioConfig`](https://docs.microsoft.com/dotnet/api/microsoft.cognitiveservices.speech.audio.audioconfig?view=azure-dotnet) object that specifies how output results should be handled.
+
+To start, create an `AudioConfig` to automatically write the output to a `.wav` file, using the `FromWavFileOutput()` function, and wrap it in a `using` block. 
+
+```csharp
+public static async Task SynthesizeAudioAsync() 
+{
+    var config = SpeechConfig.FromSubscription("YourSubscriptionKey", "YourServiceRegion");
+    using (var fileOutput = AudioConfig.FromWavFileOutput("path/to/write/file.wav"))
+    {
+    }
+}
+```
+
+Next, inside the `using` block you just created, create a nested `using` block and initialize the `SpeechSynthesizer`. Pass your `config` object and the `fileOutput` object as params. Then, executing speech synthesis and writing to a file is as simple as running `SpeakTextAsync()` with a string of text.
+
+```csharp
+public static async Task SynthesizeAudioAsync() 
+{
+    var config = SpeechConfig.FromSubscription("YourSubscriptionKey", "YourServiceRegion");
+    using (var audioConfig = AudioConfig.FromWavFileOutput("path/to/write/file.wav"))
+    {
+        using (var synthesizer = new SpeechSynthesizer(config, audioConfig))
+        {
+            await synthesizer.SpeakTextAsync("A simple test to write to a file.");
+        }
+    }
+}
+```
+
+Run the program, and a synthesized `.wav` file is written to the location you specified. This is a good example of the most basic usage, but next you look at customizing output and handling the output response as an in-memory stream for working with custom scenarios.
+
+## Get result as an in-memory stream
+
+For many scenarios in speech application development, you likely need the resulting audio data as an in-memory stream rather than directly writing to a file. This will allow you to build custom behavior including:
+
+* Abstract the resulting byte array as a seek-able stream for custom downstream services.
+* Integrate the result with other API's or services.
+* Modify the audio data, write custom `.wav` headers, etc.
+
+It's simple to make this change from the previous example. First, remove the `AudioConfig` block, as you will manage the output behavior manually from this point onward for increased control. Then pass `null` for the `AudioConfig` in the `SpeechSynthesizer` constructor.
+
+This time, you save the result to a [`SpeechSynthesisResult`](https://docs.microsoft.com/dotnet/api/microsoft.cognitiveservices.speech.speechsynthesisresult?view=azure-dotnet) variable. The `AudioData` property contains a `byte []` of the output data. Simply grab the `byte []` and write it to a new `MemoryStream`. From here you can implement any custom behavior using the resulting output, but in this example you write to a file manually.
+
+```csharp
+public static async Task SynthesizeAudioAsync() 
+{
+    var config = SpeechConfig.FromSubscription("YourSubscriptionKey", "YourServiceRegion");
+    using (var synthesizer = new SpeechSynthesizer(config, null))
+    {
+        var result = await synthesizer.SpeakTextAsync("Getting the response as a memory stream.");
+        MemoryStream stream = new MemoryStream();
+        stream.Write(result.AudioData);
+
+        FileStream fs = File.Create("path/to/write/file.wav");
+        stream.WriteTo(fs);
+        fs.Close();
+        stream.Close();
+    }
+}
+```
+
+## Customize audio format
+
+The following section shows how to customize audio output attributes including:
+
+* Audio file type
+* Sample-rate
+* Bit-depth
+
+To change the audio format, you use the `SetSpeechSynthesisOutputFormat()` function on the `SpeechConfig` object. This function expects an `enum` of type [`SpeechSynthesisOutputFormat`](https://docs.microsoft.com/dotnet/api/microsoft.cognitiveservices.speech.speechsynthesisoutputformat?view=azure-dotnet), which you use to select the output format. See the reference docs for a [list of audio formats](https://docs.microsoft.com/dotnet/api/microsoft.cognitiveservices.speech.speechsynthesisoutputformat?view=azure-dotnet) that are available.
+
+There are various options for different file types depending on your requirements. In this example, you specify a high-fidelity raw format `Raw24Khz16BitMonoPcm`, which requires writing the `.wav` headers manually. If you choose a format such as `Audio24Khz96KBitRateMonoMp3`, you would not need to write headers.
+
+First, create a function `WriteWavHeader()` to write the necessary audio metadata to the front of your `MemoryStream`. Since `Raw24Khz16BitMonoPcm` is a raw audio format, you need to write standardized audio file headers so that other software knows information like the number of channels, sample rate, and bit depth when your file is played.
+
+```csharp
+private static void WriteWavHeader(MemoryStream stream, bool isFloatingPoint, ushort channelCount, ushort bitDepth, int sampleRate, int totalSampleCount)
+    {
+        stream.Position = 0;
+        stream.Write(Encoding.ASCII.GetBytes("RIFF"), 0, 4);
+        stream.Write(BitConverter.GetBytes(((bitDepth / 8) * totalSampleCount) + 36), 0, 4);
+        stream.Write(Encoding.ASCII.GetBytes("WAVE"), 0, 4);
+        stream.Write(Encoding.ASCII.GetBytes("fmt "), 0, 4);
+        stream.Write(BitConverter.GetBytes(16), 0, 4);
+
+        // audio format (floating point (3) or PCM (1)). Any other format indicates compression.
+        stream.Write(BitConverter.GetBytes((ushort)(isFloatingPoint ? 3 : 1)), 0, 2);
+
+        stream.Write(BitConverter.GetBytes(channelCount), 0, 2);
+        stream.Write(BitConverter.GetBytes(sampleRate), 0, 4);
+        stream.Write(BitConverter.GetBytes(sampleRate * channelCount * (bitDepth / 8)), 0, 4);
+        stream.Write(BitConverter.GetBytes((ushort)channelCount * (bitDepth / 8)), 0, 2);
+        stream.Write(BitConverter.GetBytes(bitDepth), 0, 2);
+        stream.Write(Encoding.ASCII.GetBytes("data"), 0, 4);
+        stream.Write(BitConverter.GetBytes((bitDepth / 8) * totalSampleCount), 0, 4);
+    }
+```
+
+Next, set the `SpeechSynthesisOutputFormat` on the `SpeechConfig` object. Similar to the example in the previous section, you write the `byte []` from the result to a `MemoryStream`, but first you must write the custom `.wav` headers for the chosen file type. Use the function you created above, passing the memory stream by reference. For the other params, the number of **channels** is 1 (mono), the **bit-depth** is 16, the **sample-rate** is 24,000 (24Khz), and the **total samples** is the length of the raw `byte []` from the `SpeechSynthesisResult`.
+
+```csharp
+public static async Task SynthesizeAudioAsync() 
+{
+    var config = SpeechConfig.FromSubscription("YourSubscriptionKey", "YourServiceRegion");
+    config.SetSpeechSynthesisOutputFormat(SpeechSynthesisOutputFormat.Raw24Khz16BitMonoPcm);
+
+    using (var synthesizer = new SpeechSynthesizer(config, null))
+    {
+        var result = await synthesizer.SpeakTextAsync("Customizing audio output.");
+        MemoryStream stream = new MemoryStream();
+        // first write the headers to the front of the stream
+        WriteWavHeader(stream, false, 1, 16, 24000, result.AudioData.Length);
+        stream.Write(result.AudioData);
+
+        FileStream fs = File.Create("path/to/write/file.wav");
+        stream.WriteTo(fs);
+        fs.Close();
+        stream.Close();
+    }
+}
+```
+
+Running your program again will write a custom-formatted `.wav` file to the specified path.
+
+## Use SSML to customize speech characteristics
+
+Speech Synthesis Markup Language (SSML) allows you to fine-tune the pitch, pronunciation, speaking rate, volume, and more of the text-to-speech output by submitting your requests from an XML schema. This section shows a few practical usage examples, but for a more detailed guide, see the [SSML how-to article](../../../speech-synthesis-markup.md).
+
+To start using SSML for customization, you make a simple change that switches the voice.
+First, create a new XML file for the SSML config in your root project directory, in this example `ssml.xml`. The root element is always `<speak>`, and wrapping the text in a `<voice>` element allows you to change the voice using the `name` param. This example changes the voice to a male English (UK) voice. See the [full list](https://docs.microsoft.com/azure/cognitive-services/speech-service/language-support#standard-voices) of supported standard voices for additional options.
+
+```xml
+<speak version="1.0" xmlns="https://www.w3.org/2001/10/synthesis" xml:lang="en-US">
+  <voice name="en-GB-George-Apollo">
+    When you're on the motorway, it's a good idea to use a sat nav.
+  </voice>
+</speak>
+```
+
+Next, you need to change the speech synthesis request to reference your XML file. The request is mostly the same, but instead of using the `SpeakTextAsync()` function, you use `SpeakSsmlAsync()`. This function expects an XML string, so you first load your SSML config as a string using `XDocument.Load()`. From here, the result object is exactly the same as previous examples.
+
+> [!NOTE]
+> If you're using Visual Studio, your build config likely will not find your XML file by default. To fix this, right click the XML file and 
+> select **Properties**. Change **Build Action** to *Content*, and change **Copy to Output Directory** to *Copy always*.
+
+```csharp
+public static async Task SynthesizeAudioAsync() 
+{
+    var config = SpeechConfig.FromSubscription("YourSubscriptionKey", "YourServiceRegion");
+    using (var synthesizer = new SpeechSynthesizer(config, null))
+    {
+        string ssml = XDocument.Load(@"./ssml.xml").ToString();
+        var result = await synthesizer.SpeakSsmlAsync(ssml);
+
+        MemoryStream stream = new MemoryStream();
+        stream.Write(result.AudioData);
+
+        FileStream fs = File.Create("path/to/write/file.wav");
+        stream.WriteTo(fs);
+        fs.Close();
+        stream.Close();
+    }
+}
+```
+
+The output works, but there a few simple additional changes you can make to help it sound more natural. The overall speaking speed is a little too fast, so we'll add a `<prosody>` tag and reduce the speed to **90%** of the default rate. Additionally, the pause after the comma in the sentence is a little too short and unnatural sounding. To fix this issues, add a `<break>` tag to delay the speech, and set the time param to **200ms**. Re-run the synthesis to see how these customizations affected the output.
+
+```xml
+<speak version="1.0" xmlns="https://www.w3.org/2001/10/synthesis" xml:lang="en-US">
+  <voice name="en-GB-George-Apollo">
+    <prosody rate="0.9">
+      When you're on the motorway,<break time="200ms"/> it's a good idea to use a sat-nav.
+    </prosody>
+  </voice>
+</speak>
+```
+
+### Neural voices
+
+Neural voices are speech synthesis algorithms powered by deep neural networks. When using a neural voice, synthesized speech is nearly indistinguishable from the human recordings. With the human-like natural prosody and clear articulation of words, neural voices significantly reduce listening fatigue when users interact with AI systems.
+
+To switch to a neural voice, change the `name` to one of the [neural voice options](https://docs.microsoft.com/azure/cognitive-services/speech-service/language-support#neural-voices). Then, add an XML namespace for `mstts`, and wrap your text in the `<mstts:express-as>` tag. Use the `style` param to customize the speaking style. This example uses `cheerful`, but try setting it to `customerservice` or `chat` to see the difference in speaking style.
+
+> [!IMPORTANT]
+> Neural voices are **only** supported for Speech resources created in *East US*, *South East Asia*, and *West Europe* regions.
+
+```xml
+<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xmlns:mstts="https://www.w3.org/2001/mstts" xml:lang="en-US">
+  <voice name="en-US-AriaNeural">
+    <mstts:express-as style="cheerful">
+      This is awesome!
+    </mstts:express-as>
+  </voice>
+</speak>
+```
+