Merge pull request #113338 from xiaojul/jun_work

[Cog Svcs] update text and screenshots in custom command quickstarts

Author: Court72

articles/cognitive-services/Speech-Service/how-to-custom-speech-commands-fulfill-sdk.md

@@ -3,14 +3,14 @@ title: How to fulfill commands from a client with the Speech SDK
 titleSuffix: Azure Cognitive Services
 description: In this article, we explain how to handle Custom Commands activities on a client with the Speech SDK.
 services: cognitive-services
-author: don-d-kim
-manager: yetian
+author: trevorbye
+manager: nitinme
 
 ms.service: cognitive-services
 ms.subservice: speech-service
 ms.topic: conceptual
-ms.date: 03/12/2020
-ms.author: donkim
+ms.date: 05/04/2020
+ms.author: trbye
 ---
 
 # Fulfill commands from a client with the Speech SDK (Preview)
@@ -23,22 +23,22 @@ In this article, you'll:
 - Receive and visualize the custom JSON payload contents from a C# UWP Speech SDK client application
 
 ## Prerequisites
-
-- [Visual Studio 2019](https://visualstudio.microsoft.com/downloads/)
-- An Azure subscription key for Speech service
-  - [Get one for free](get-started.md) or create it on the [Azure portal](https://portal.azure.com)
-- A previously created Custom Commands app
-  - [Quickstart: Create a Custom Command with Parameters (Preview)](./quickstart-custom-speech-commands-create-parameters.md)
-- A Speech SDK enabled client application
-  - [Quickstart: Connect to a Custom Command application with the Speech SDK (Preview)](./quickstart-custom-speech-commands-speech-sdk.md)
+> [!div class = "checklist"]
+> * [Visual Studio 2019](https://visualstudio.microsoft.com/downloads/)
+> * An Azure subscription key for Speech service:
+[Get one for free](get-started.md) or create it on the [Azure portal](https://portal.azure.com)
+> * A previously created Custom Commands app:
+[Quickstart: Create a Custom Command with Parameters (Preview)](./quickstart-custom-speech-commands-create-parameters.md)
+> * A Speech SDK enabled client application:
+[Quickstart: Connect to a Custom Command application with the Speech SDK (Preview)](./quickstart-custom-speech-commands-speech-sdk.md)
 
 ## Optional: Get started fast
 
 This article describes, step by step, how to make a client application to talk to your Custom Commands application. If you prefer to dive right in, the complete, ready-to-compile source code used in this article is available in the [Speech SDK Samples](https://aka.ms/csspeech/samples).
 
 ## Fulfill with JSON payload
 
-1. Open your previously created Custom Commands application from the [Speech Studio](https://speech.microsoft.com/)
+1. Open the Custom Commands application you previously created from [Quickstarts: Create a custom command with parameters](./quickstart-custom-speech-commands-create-parameters.md)
 1. Check the **Completion Rules** section to make sure you have the previously created rule that responds back to the user
 1. To send a payload directly to the client, create a new rule with a Send Activity action
 
@@ -51,9 +51,7 @@ This article describes, step by step, how to make a client application to talk t
    | Conditions | Required Parameter - `OnOff` and `SubjectDevice` | Conditions that determine when the rule can run |
    | Actions | `SendActivity` (see below) | The action to take when the rule condition is true |
 
-   > [!div class="mx-imgBorder"]
-   > ![Send Activity payload](media/custom-speech-commands/fulfill-sdk-send-activity-action.png)
-
+1. Copy the JSON below to **Activity content**
    ```json
    {
      "type": "event",
@@ -62,12 +60,14 @@ This article describes, step by step, how to make a client application to talk t
      "device": "{SubjectDevice}"
    }
    ```
+   > [!div class="mx-imgBorder"]
+   > ![Send Activity payload](media/custom-speech-commands/fulfill-sdk-send-activity-action.png)
 
 ## Create visuals for device on or off state
 
-In [Quickstart: Connect to a Custom Command application with the Speech SDK (Preview)](./quickstart-custom-speech-commands-speech-sdk.md) you created a Speech SDK client application that handled commands such as `turn on the tv`, `turn off the fan`. Now add some visuals so you can see the result of those commands.
+In [Quickstart: Connect to a Custom Command application with the Speech SDK](./quickstart-custom-speech-commands-speech-sdk.md), you created a Speech SDK client application that handled commands such as `turn on the tv`, `turn off the fan`. With some visuals added, you can see the result of those commands.
 
-Add labeled boxes with text indicating **On** or **Off** using the following XML added to `MainPage.xaml.cs`
+Add labeled boxes with text indicating **On** or **Off** using the following XML added to `MainPage.xaml`
 
 ```xml
 <StackPanel Orientation="Horizontal" HorizontalAlignment="Center" Margin="20">
@@ -87,36 +87,57 @@ Add labeled boxes with text indicating **On** or **Off** using the following XML
 ```
 
 ## Handle customizable payload
+### Add reference libraries
 
-Now that you've created a JSON payload, you can add a reference to the [JSON.NET](https://www.newtonsoft.com/json) library to handle deserialization.
+Since you've created a JSON payload, you need to add a reference to the [JSON.NET](https://www.newtonsoft.com/json) library to handle deserialization.
+- Right-client your solution.
+- Choose **Manage NuGet Packages for Solution**, Select **Install** 
+- Search for **Newtonsoft.json** in the update list, Update **Microsoft.NETCore.UniversalWindowsPlatform** to newest version
 
 > [!div class="mx-imgBorder"]
 > ![Send Activity payload](media/custom-speech-commands/fulfill-sdk-json-nuget.png)
 
-In `InitializeDialogServiceConnector` add the following to your `ActivityReceived` event handler. The additional code will extract the payload from the activity and change the visual state of the tv or fan accordingly.
+In `MainPage.xaml.cs', add
+- `using Newtonsoft.Json;` 
+- `using Windows.ApplicationModel.Core;`
+
+### Handle received payload
+
+In `InitializeDialogServiceConnector`, replace the `ActivityReceived` event handler with following code. The modified `ActivityReceived` event handler will extract the payload from the activity and change the visual state of the tv or fan accordingly.
 
 ```C#
 connector.ActivityReceived += async (sender, activityReceivedEventArgs) =>
 {
     NotifyUser($"Activity received, hasAudio={activityReceivedEventArgs.HasAudio} activity={activityReceivedEventArgs.Activity}");
 
     dynamic activity = JsonConvert.DeserializeObject(activityReceivedEventArgs.Activity);
+    var name = activity?.name != null ? activity.name.ToString() : string.Empty;
 
-    if(activity?.name == "SetDeviceState")
+    if (name.Equals("UpdateDeviceState"))
     {
-        var state = activity?.state;
-        var device = activity?.device;
-        switch(device)
+        Debug.WriteLine("Here");
+        var state = activity?.device != null ? activity.state.ToString() : string.Empty;
+        var device = activity?.device != null ? activity.device.ToString() : string.Empty;
+
+        if (state.Equals("on") || state.Equals("off"))
         {
-            case "tv":
-                State_TV.Text = state;
-                break;
-            case "fan":
-                State_Fan.Text = state;
-                break;
-            default:
-                NotifyUser($"Received request to set unsupported device {device} to {state}");
-                break;
+            switch (device)
+            {
+                case "tv":
+                    await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(
+                        CoreDispatcherPriority.Normal, () => { State_TV.Text = state; });
+                    break;
+                case "fan":
+                    await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(
+                        CoreDispatcherPriority.Normal, () => { State_Fan.Text = state; });
+                    break;
+                default:
+                    NotifyUser($"Received request to set unsupported device {device} to {state}");
+                    break;
+            }
+        }
+        else { 
+            NotifyUser($"Received request to set unsupported state {state}");
         }
     }
 
@@ -134,6 +155,8 @@ connector.ActivityReceived += async (sender, activityReceivedEventArgs) =>
 1. Select the Talk button
 1. Say `turn on the tv`
 1. The visual state of the tv should change to "On"
+   > [!div class="mx-imgBorder"]
+   > ![Send Activity payload](media/custom-speech-commands/fulfill-sdk-turn-on-tv.png)
 
 ## Next steps
 

articles/cognitive-services/Speech-Service/media/custom-speech-commands/fulfill-sdk-completion-rule.png

@@ -30,7 +30,8 @@ In this article, we'll select a custom output voice for the application we creat
 > ![Sample Sentences with parameters](media/custom-speech-commands/select-custom-voice.png)
 
 > [!NOTE]
-> Custom voices can be created from the Custom Voice project page. Select the **Speech Studio** link, then **Custom Voice** to get started.
+> - For **Public voices**, **Neural types** are only available for specific regions. To check availability, see [standard and neural voices by region/endpoint](https://docs.microsoft.com/azure/cognitive-services/speech-service/regions#standard-and-neural-voices).
+> - For **Custom voices**, they can be created from the Custom Voice project page. See [Get Started with Custom Voice](./how-to-custom-voice.md).
 
 Now the application will respond in the selected voice, instead of the default voice.
 

articles/cognitive-services/Speech-Service/media/custom-speech-commands/fulfill-sdk-publish-application.png

@@ -19,32 +19,34 @@ After creating a hosted Custom Commands application, you can begin talking to it
 In this article, you'll:
 
 - Publish a Custom Commands application and get an application identifier (App ID)
-- Create a client app using the Speech SDK to allow you to talk to your Custom Commands application
+- Create a Universal Windows Platform (UWP) client app using the Speech SDK to allow you to talk to your Custom Commands application
 
 ## Prerequisites
 
 A Custom Commands application is required to complete this article. If you haven't created a Custom Commands application yet, you can do so in these previous quickstarts:
-
-- [Quickstart: Create a Custom Command (Preview)](./quickstart-custom-speech-commands-create-new.md)
-- [Quickstart: Create a Custom Command with Parameters (Preview)](./quickstart-custom-speech-commands-create-parameters.md)
+> [!div class = "checklist"]
+> * [Quickstart: Create a Custom Command (Preview)](./quickstart-custom-speech-commands-create-new.md)
+> * [Quickstart: Create a Custom Command with Parameters (Preview)](./quickstart-custom-speech-commands-create-parameters.md)
 
 You'll also need:
-
-- [Visual Studio 2019](https://visualstudio.microsoft.com/downloads/)
-- An Azure subscription key for Speech Services. [Get one for free](get-started.md) or create it on the [Azure portal](https://portal.azure.com)
+> [!div class = "checklist"]
+> * [Visual Studio 2019](https://visualstudio.microsoft.com/downloads/)
+> * An Azure subscription key for Speech Services. [Get one for free](get-started.md) or create it on the [Azure portal](https://portal.azure.com)
+> * [Enable your device for development](https://docs.microsoft.com/windows/uwp/get-started/enable-your-device-for-development)
 
 ## Optional: Get started fast
 
 This quickstart describes, step by step, how to make a client application to connect to your Custom Commands app. If you prefer to dive right in, the complete, ready-to-compile source code used in this quickstart is available in the [Speech SDK Samples](https://aka.ms/csspeech/samples) under the `quickstart` folder.
 
 ## Step 1: Publish Custom Commands application
 
-1. Open your [previously created Custom Commands application](./quickstart-custom-speech-commands-create-new.md) and select **Publish**
+1. Open your [previously created Custom Commands application (Preview)](./quickstart-custom-speech-commands-create-new.md) and select **Publish**
 
    > [!div class="mx-imgBorder"]
    > ![Publish application](media/custom-speech-commands/fulfill-sdk-publish-application.png)
 
 1. Copy the App ID from the publish notification for later use
+1. Copy the Speech Resource Key for later use
 
 ## Step 2: Create a Visual Studio project
 
@@ -124,7 +126,7 @@ Add the code-behind source as follows:
 
 1. In **Solution Explorer**, open the code-behind source file `MainPage.xaml.cs` (grouped under `MainPage.xaml`)
 
-1. Replace the file's contents with the following code:
+1. Replace the file's contents with the following code: 
 
    ```csharp
    using Microsoft.CognitiveServices.Speech;
@@ -293,6 +295,11 @@ Add the code-behind source as follows:
        }
    }
    ```
+    > [!NOTE]
+    > If you see error: "The type 'Object' is defined in an assembly that is not referenced"
+    > 1. Right-client your solution.
+    > 1. Choose **Manage NuGet Packages for Solution**, Select **Updates** 
+    > 1. If you see **Microsoft.NETCore.UniversalWindowsPlatform** in the update list, Update **Microsoft.NETCore.UniversalWindowsPlatform** to newest version
 
 1. Add the following code to the method body of `InitializeDialogServiceConnector`
 
@@ -414,3 +421,6 @@ Add the code-behind source as follows:
 > [!div class="nextstepaction"]
 > [How to: Fulfill commands on the client with the Speech SDK (preview)](./how-to-custom-speech-commands-fulfill-sdk.md)
 > [How To: Add validations to Custom Command parameters (Preview)](./how-to-custom-speech-commands-validations.md)
+
+## Sample source code
+Check out our client sample codes at [GitHub-VoiceAssistant](https://github.com/Azure-Samples/Cognitive-Services-Voice-Assistant)