Merge pull request #229628 from eric-urban/eur/intent-recognition

denrea · web-flow · commit 0418c152cb2e · 2023-03-06T16:03:44.000-08:00
clu updates
diff --git a/articles/cognitive-services/Speech-Service/includes/quickstarts/intent-recognition-clu/cpp.md b/articles/cognitive-services/Speech-Service/includes/quickstarts/intent-recognition-clu/cpp.md
@@ -94,10 +94,6 @@ Follow these steps to create a new console application and install the Speech SD
         {
             std::cout << "RECOGNIZED: Text=" << result->Text << std::endl;
             std::cout << "  Intent Id: " << result->IntentId << std::endl;
-    
-            // There is a known issue with the LanguageUnderstandingServiceResponse_JsonResult 
-            // property when used with CLU in the Speech SDK version 1.25. 
-            // The following should return JSON in a future release.
             std::cout << "  Intent Service JSON: " << result->Properties.GetProperty(PropertyId::LanguageUnderstandingServiceResponse_JsonResult) << std::endl;
         }
         else if (result->Reason == ResultReason::RecognizedSpeech)
@@ -152,14 +148,58 @@ Follow these steps to create a new console application and install the Speech SD
 Speak into your microphone when prompted. What you speak should be output as text: 
 
 ```console
-Say something ...
-RECOGNIZED: Text=Go ahead and delete the e-mail.
-    Intent Id: Delete.
-    Language Understanding JSON: 
+Speak into your microphone.
+RECOGNIZED: Text=Turn on the lights.
+    Intent Id: HomeAutomation.TurnOn.
+    Language Understanding JSON: {"kind":"ConversationResult","result":{"query":"turn on the lights","prediction":{"topIntent":"HomeAutomation.TurnOn","projectKind":"Conversation","intents":[{"category":"HomeAutomation.TurnOn","confidenceScore":0.97712576},{"category":"HomeAutomation.TurnOff","confidenceScore":0.8431633},{"category":"None","confidenceScore":0.782861}],"entities":[{"category":"HomeAutomation.DeviceType","text":"lights","offset":12,"length":6,"confidenceScore":1,"extraInformation":[{"extraInformationKind":"ListKey","key":"light"}]}]}}}.
 ```
 
-> [NOTE]
-> There is a known issue with the LanguageUnderstandingServiceResponse_JsonResult property when used with CLU in the Speech SDK version 1.25. You can get detailed JSON output in a future release. Via JSON, the intents are returned in the probability order of most likely to least likely. For example, the `topIntent` might be `Delete` with a confidence score of 0.95413816 (95.41%). The second most likely intent might be `Cancel` with a confidence score of 0.8985081 (89.85%).
+> [!NOTE]
+> Support for the JSON response for CLU via the LanguageUnderstandingServiceResponse_JsonResult property was added in the Speech SDK version 1.26.
+
+The intents are returned in the probability order of most likely to least likely. Here's a formatted version of the JSON output where the `topIntent` is `HomeAutomation.TurnOn` with a confidence score of 0.97712576 (97.71%). The second most likely intent might be `HomeAutomation.TurnOff` with a confidence score of 0.8985081 (84.31%).
+
+```json
+{
+  "kind": "ConversationResult",
+  "result": {
+    "query": "turn on the lights",
+    "prediction": {
+      "topIntent": "HomeAutomation.TurnOn",
+      "projectKind": "Conversation",
+      "intents": [
+        {
+          "category": "HomeAutomation.TurnOn",
+          "confidenceScore": 0.97712576
+        },
+        {
+          "category": "HomeAutomation.TurnOff",
+          "confidenceScore": 0.8431633
+        },
+        {
+          "category": "None",
+          "confidenceScore": 0.782861
+        }
+      ],
+      "entities": [
+        {
+          "category": "HomeAutomation.DeviceType",
+          "text": "lights",
+          "offset": 12,
+          "length": 6,
+          "confidenceScore": 1,
+          "extraInformation": [
+            {
+              "extraInformationKind": "ListKey",
+              "key": "light"
+            }
+          ]
+        }
+      ]
+    }
+  }
+}
+```
 
 ## Remarks
 Now that you've completed the quickstart, here are some additional considerations:
diff --git a/articles/cognitive-services/Speech-Service/includes/quickstarts/intent-recognition-clu/csharp.md b/articles/cognitive-services/Speech-Service/includes/quickstarts/intent-recognition-clu/csharp.md
@@ -89,10 +89,6 @@ Follow these steps to create a new console application and install the Speech SD
                 {
                     Console.WriteLine($"RECOGNIZED: Text={recognitionResult.Text}");
                     Console.WriteLine($"    Intent Id: {recognitionResult.IntentId}.");
-                    
-                    // There is a known issue with the LanguageUnderstandingServiceResponse_JsonResult 
-                    // property when used with CLU in the Speech SDK version 1.25. 
-                    // The following should return JSON in a future release.
                     Console.WriteLine($"    Language Understanding JSON: {recognitionResult.Properties.GetProperty(PropertyId.LanguageUnderstandingServiceResponse_JsonResult)}.");
                 }
                 else if (recognitionResult.Reason == ResultReason.RecognizedSpeech)
@@ -136,14 +132,58 @@ dotnet run
 Speak into your microphone when prompted. What you speak should be output as text: 
 
 ```console
-Say something ...
-RECOGNIZED: Text=Go ahead and delete the e-mail.
-    Intent Id: Delete.
-    Language Understanding JSON: 
+Speak into your microphone.
+RECOGNIZED: Text=Turn on the lights.
+    Intent Id: HomeAutomation.TurnOn.
+    Language Understanding JSON: {"kind":"ConversationResult","result":{"query":"turn on the lights","prediction":{"topIntent":"HomeAutomation.TurnOn","projectKind":"Conversation","intents":[{"category":"HomeAutomation.TurnOn","confidenceScore":0.97712576},{"category":"HomeAutomation.TurnOff","confidenceScore":0.8431633},{"category":"None","confidenceScore":0.782861}],"entities":[{"category":"HomeAutomation.DeviceType","text":"lights","offset":12,"length":6,"confidenceScore":1,"extraInformation":[{"extraInformationKind":"ListKey","key":"light"}]}]}}}.
 ```
 
-> [NOTE]
-> There is a known issue with the LanguageUnderstandingServiceResponse_JsonResult property when used with CLU in the Speech SDK version 1.25. You can get detailed JSON output in a future release. Via JSON, the intents are returned in the probability order of most likely to least likely. For example, the `topIntent` might be `Delete` with a confidence score of 0.95413816 (95.41%). The second most likely intent might be `Cancel` with a confidence score of 0.8985081 (89.85%).
+> [!NOTE]
+> Support for the JSON response for CLU via the LanguageUnderstandingServiceResponse_JsonResult property was added in the Speech SDK version 1.26.
+
+The intents are returned in the probability order of most likely to least likely. Here's a formatted version of the JSON output where the `topIntent` is `HomeAutomation.TurnOn` with a confidence score of 0.97712576 (97.71%). The second most likely intent might be `HomeAutomation.TurnOff` with a confidence score of 0.8985081 (84.31%).
+
+```json
+{
+  "kind": "ConversationResult",
+  "result": {
+    "query": "turn on the lights",
+    "prediction": {
+      "topIntent": "HomeAutomation.TurnOn",
+      "projectKind": "Conversation",
+      "intents": [
+        {
+          "category": "HomeAutomation.TurnOn",
+          "confidenceScore": 0.97712576
+        },
+        {
+          "category": "HomeAutomation.TurnOff",
+          "confidenceScore": 0.8431633
+        },
+        {
+          "category": "None",
+          "confidenceScore": 0.782861
+        }
+      ],
+      "entities": [
+        {
+          "category": "HomeAutomation.DeviceType",
+          "text": "lights",
+          "offset": 12,
+          "length": 6,
+          "confidenceScore": 1,
+          "extraInformation": [
+            {
+              "extraInformationKind": "ListKey",
+              "key": "light"
+            }
+          ]
+        }
+      ]
+    }
+  }
+}
+```
 
 ## Remarks
 Now that you've completed the quickstart, here are some additional considerations:
diff --git a/articles/cognitive-services/Speech-Service/includes/quickstarts/intent-recognition-clu/deploy-clu-model.md b/articles/cognitive-services/Speech-Service/includes/quickstarts/intent-recognition-clu/deploy-clu-model.md
@@ -13,7 +13,7 @@ Go to the [Language Studio](https://aka.ms/languageStudio) and sign in with your
 
 ### Create a conversational language understanding project
 
-For this quickstart, you can download [this sample project](https://go.microsoft.com/fwlink/?linkid=2196152) and import it. This project can predict the intended commands from user input, such as: reading emails, deleting emails, and attaching a document to an email. 
+For this quickstart, you can download [this sample home automation project](https://github.com/Azure-Samples/cognitive-services-sample-data-files/blob/master/language-service/CLU/HomeAutomationDemo.json) and import it. This project can predict the intended commands from user input, such as turning lights on and off. 
 
 [!INCLUDE [Import project](../../../../language-service/conversational-language-understanding/includes/language-studio/import-project.md)]
 
diff --git a/articles/cognitive-services/Speech-Service/intent-recognition.md b/articles/cognitive-services/Speech-Service/intent-recognition.md
@@ -13,11 +13,11 @@ keywords: intent recognition
 
 # What is intent recognition?
 
-In this overview, you will learn about the benefits and capabilities of intent recognition. The Cognitive Services Speech SDK provides two ways to recognize intents, both described below. An intent is something the user wants to do: book a flight, check the weather, or make a call. Using intent recognition, your applications, tools, and devices can determine what the user wants to initiate or do based on options you define in the Intent Recognizer or LUIS.
+In this overview, you will learn about the benefits and capabilities of intent recognition. The Cognitive Services Speech SDK provides two ways to recognize intents, both described below. An intent is something the user wants to do: book a flight, check the weather, or make a call. Using intent recognition, your applications, tools, and devices can determine what the user wants to initiate or do based on options you define in the Intent Recognizer or Conversational Language Understanding (CLU) model.
 
 ## Pattern matching
 
-The Speech SDK provides an embedded pattern matcher that you can use to recognize intents in a very strict way. This is useful for when you need a quick offline solution. This works especially well when the user is going to be trained in some way or can be expected to use specific phrases to trigger intents. For example: "Go to floor seven", or "Turn on the lamp" etc. It is recommended to start here and if it no longer meets your needs, switch to using LUIS or a combination of the two. 
+The Speech SDK provides an embedded pattern matcher that you can use to recognize intents in a very strict way. This is useful for when you need a quick offline solution. This works especially well when the user is going to be trained in some way or can be expected to use specific phrases to trigger intents. For example: "Go to floor seven", or "Turn on the lamp" etc. It is recommended to start here and if it no longer meets your needs, switch to using [CLU](#conversational-language-understanding) or a combination of the two. 
 
 Use pattern matching if: 
 * You're only interested in matching strictly what the user said. These patterns match more aggressively than [conversational language understanding (CLU)](/azure/cognitive-services/language-service/conversational-language-understanding/overview).
