Skip to content

Commit dd523c0

Browse files
GitHubber17GitHubber17
authored
Merge pull request #98285 from encorona-ms/encorona-one-step-correction
[CogSvcs] Adding how to guides for confirmations, one-step correction and minor docs improvements for custom commands
2 parents 3ca98e1 + d24221a commit dd523c0

11 files changed

+244
-59
lines changed

‎articles/cognitive-services/Speech-Service/custom-commands.md

Lines changed: 10 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Custom Commands (Preview) - Speech service
33
titleSuffix: Azure Cognitive Services
4-
description: An overview of the features, capabilities, and restrictions for Custom Commands (Preview), a solution for creating voice assistants.
4+
description: An overview of the features, capabilities, and restrictions for Custom Commands (Preview), a solution for creating voice applications.
55
services: cognitive-services
66
author: trrwilson
77
manager: nitinme
@@ -14,9 +14,9 @@ ms.author: travisw
1414

1515
# Custom Commands (Preview)
1616

17-
[Voice assistants](voice-assistants.md) listen to users and take an action in response, often speaking back. They use [speech-to-text](speech-to-text.md) to transcribe the user's speech, then take action on the natural language understanding of the text. This action frequently includes spoken output from the assistant generated with [text-to-speech](text-to-speech.md). Devices connect to assistants with the Speech SDK’s `DialogServiceConnector` object.
17+
Voice applications such as [Voice assistants](voice-assistants.md) listen to users and take an action in response, often speaking back. They use [speech-to-text](speech-to-text.md) to transcribe the user's speech, then take action on the natural language understanding of the text. This action frequently includes spoken output from the assistant generated with [text-to-speech](text-to-speech.md). Devices connect to assistants with the Speech SDK’s `DialogServiceConnector` object.
1818

19-
**Custom Commands (Preview)** is a streamlined solution for creating a voice assistant. It provides a unified authoring experience, an automatic hosting model, and relatively lower complexity versus other assistant creation options like [Direct Line Speech](direct-line-speech.md). This simplification, however, comes with a reduction in flexibility. So, Custom Commands (Preview) is best suited for task completion or command-and-control scenarios. It's particularly well-matched for Internet of Things (IoT) and headless devices.
19+
**Custom Commands (Preview)** is a streamlined solution for creating voice applications. It provides a unified authoring experience, an automatic hosting model, and relatively lower complexity versus other options like [Direct Line Speech](direct-line-speech.md). This simplification, however, comes with a reduction in flexibility. So, Custom Commands (Preview) is best suited for task completion or command-and-control scenarios. It's particularly well-matched for Internet of Things (IoT) and headless devices.
2020

2121
For complex conversational interaction and integration with other solutions like the [Virtual Assistant Solution and Enterprise Template](https://docs.microsoft.com/azure/bot-service/bot-builder-enterprise-template-overview) you're encouraged to use Direct Line Speech.
2222

@@ -26,7 +26,7 @@ Good candidates for Custom Commands (Preview) have a fixed vocabulary with well-
2626

2727
## Getting started with Custom Commands (Preview)
2828

29-
The first step for using Custom Commands (Preview) to make a voice assistant is to [get a speech subscription key](get-started.md) and access the Custom Commands (Preview) Builder on the [Speech Studio](https://speech.microsoft.com). From there, you can author a new Custom Commands (Preview) Application and publish it, after which an on-device application can communicate with it using the Speech SDK.
29+
The first step for using Custom Commands (Preview) to make a voice application is to [get a speech subscription key](get-started.md) and access the Custom Commands (Preview) Builder on the [Speech Studio](https://speech.microsoft.com). From there, you can author a new Custom Commands (Preview) Application and publish it, after which an on-device application can communicate with it using the Speech SDK.
3030

3131
![Authoring flow for Custom Commands (Preview)](media/voice-assistants/custom-commands-flow.png "The Custom Commands (Preview) authoring flow")
3232

@@ -36,24 +36,15 @@ We offer quickstarts designed to have you running code in less than 10 minutes.
3636
* [Create a Custom Commands (Preview) application with parameters](quickstart-custom-speech-commands-create-parameters.md)
3737
* [Connect to a Custom Commands (Preview) application with the Speech SDK, C#](quickstart-custom-speech-commands-speech-sdk.md)
3838

39-
## Sample code
39+
Once you are done with the quickstarts, explore our how-tos.
4040

41-
Sample code for creating a voice assistant with Custom Commands (Preview) is available on GitHub.
42-
43-
* [Voice assistant samples (SDK)](https://aka.ms/csspeech/samples)
44-
45-
## Customization
46-
47-
Voice assistants built using the Speech service can use the full range of customization options available for [speech-to-text](speech-to-text.md), [text-to-speech](text-to-speech.md), and [custom keyword selection](speech-devices-sdk-create-kws.md).
48-
49-
> [!NOTE]
50-
> Customization options vary by language/locale (see [Supported languages](supported-languages.md)).
51-
52-
## Reference docs
53-
54-
* [Speech SDK](speech-sdk-reference.md)
41+
- [Add validations to Custom Command parameters](./how-to-custom-speech-commands-validations.md)
42+
- [Fulfill Commands on the client with the Speech SDK](./how-to-custom-speech-commands-fulfill-sdk.md)
43+
- [Add a confirmation to a Custom Command](./how-to-custom-speech-commands-confirmations.md)
44+
- [Add a one-step correction to a Custom Command](./how-to-custom-speech-commands-one-step-correction.md)
5545

5646
## Next steps
5747

5848
* [Get a Speech service subscription key for free](get-started.md)
49+
* [Go to the Speech Studio to try out Custom Commands](https://speech.microsoft.com)
5950
* [Get the Speech SDK](speech-sdk.md)

‎articles/cognitive-services/Speech-Service/how-to-custom-speech-commands-confirmations.md

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
---
2+
title: 'How to: Add a confirmation to a custom command (Preview)'
3+
titleSuffix: Azure Cognitive Services
4+
description: In this article, how to implements confirmations for a command in Custom Commands.
5+
services: cognitive-services
6+
author: encorona-ms
7+
manager: yetian
8+
ms.service: cognitive-services
9+
ms.subservice: speech-service
10+
ms.topic: conceptual
11+
ms.date: 12/05/2019
12+
ms.author: encorona
13+
---
14+
15+
# How To: Add a confirmation to a Custom Command (Preview)
16+
17+
In this article, you'll learn how to add a confirmation to a command.
18+
19+
## Prerequisites
20+
21+
You must have completed the steps in the following articles:
22+
23+
- [Quickstart: Create a Custom Command (Preview)](./quickstart-custom-speech-commands-create-new.md)
24+
- [Quickstart: Create a Custom Command with Parameters (Preview)](./quickstart-custom-speech-commands-create-parameters.md)
25+
26+
## Create a SetAlarm command
27+
28+
To demonstrate validations, let's create a new Command allowing the user to set an alarm.
29+
30+
1. Open your previously created Custom Commands application in [Speech Studio](https://speech.microsoft.com/)
31+
1. Create a new Command **SetAlarm**
32+
1. Add a parameter called DateTime
33+
34+
| Setting | Suggested value | Description |
35+
| ----------------- | ---------------------------------------------------------| ------------------------------------------------------------------------------------------------ |
36+
| Name | DateTime | A descriptive name for your Command parameter |
37+
| Required | true | Checkbox indicating whether a value for this parameter is required before completing the Command |
38+
| Response template | "- What time?" | A prompt to ask for the value of this parameter when it isn't known |
39+
| Type | DateTime | The type of parameter, such as Number, String, or Date Time |
40+
| Date Defaults | If date is missing use today | |
41+
| Time Defaults | If time is missing use start of day | |
42+
43+
1. Add some sample sentences
44+
45+
```
46+
set an alarm for {DateTime}
47+
set alarm {DateTime}
48+
alarm for {DateTime}
49+
```
50+
51+
1. Add a Completion rule to confirm result
52+
53+
| Setting | Suggested value | Description |
54+
| ---------- | ------------------------------------------------------- | -------------------------------------------------- |
55+
| Rule Name | Set alarm | A name describing the purpose of the rule |
56+
| Actions | SpeechResponse - "- Ok, alarm set for {DateTime}" | The action to take when the rule condition is true |
57+
58+
## Try it out
59+
60+
Select the Test panel and try a few interactions.
61+
62+
- Input: Set alarm for tomorrow at noon
63+
- Output: "Ok, alarm set for 12/06/2019 12:00:00"
64+
65+
- Input: Set an alarm
66+
- Output: "What time?"
67+
- Input: 5pm
68+
- Output: "Ok, alarm set for 12/05/2019 17:00:00"
69+
70+
## Add the advanced rules for confirmation
71+
72+
1. Add an advanced rule for confirmation.
73+
74+
This rule will ask the user to confirm the date and time of the alarm and is expecting a confirmation (yes/no) for the next turn.
75+
76+
| Setting | Suggested value | Description |
77+
| --------------------- | -------------------------------------------------------------------------------- | -------------------------------------------------- |
78+
| Rule Name | Confirm date time | A name describing the purpose of the rule |
79+
| Conditions | Required Parameter - DateTime | Conditions that determine when the rule can run |
80+
| Actions | SpeechResponse - "- Are you sure you want to set an alarm for {DateTime}?" | The action to take when the rule condition is true |
81+
| State after execution | Wait for input | State for the user after the turn |
82+
| Expectations | Confirmation | Expectation for the next turn |
83+
84+
1. Add an advanced rule to handle a successful confirmation (user said yes)
85+
86+
| Setting | Suggested value | Description |
87+
| --------------------- | -------------------------------------------------------------------------------- | -------------------------------------------------- |
88+
| Rule Name | Accepted confirmation | A name describing the purpose of the rule |
89+
| Conditions | SuccessfulConfirmation & Required Parameter - DateTime | Conditions that determine when the rule can run |
90+
| State after execution | Ready for Completion | State of the user after the turn |
91+
92+
1. Add an advanced rule to handle a confirmation denied (user said no)
93+
94+
| Setting | Suggested value | Description |
95+
| --------------------- | -------------------------------------------------------------------------------- | -------------------------------------------------- |
96+
| Rule Name | Denied confirm | A name describing the purpose of the rule |
97+
| Conditions | DeniedConfirmation & Required Parameter - DateTime | Conditions that determine when the rule can run |
98+
| Actions | ClearParameter - DateTime & SpeechResponse - "- No problem, what time then?" | The action to take when the rule condition is true |
99+
| State after execution | Wait for input | State of the user after the turn |
100+
| Expectations | ElicitParameters - DateTime | Expectation for the next turn |
101+
102+
## Try it out
103+
104+
Select the Test panel and try a few interactions.
105+
106+
- Input: Set alarm for tomorrow at noon
107+
- Output: "Are you sure you want to set an alarm for 12/07/2019 12:00:00?"
108+
- Input: No
109+
- Output: "No problem, what time then?"
110+
- Input: 5pm
111+
- Output: "Are you sure you want to set an alarm for 12/06/2019 17:00:00?"
112+
- Input: Yes
113+
- Output: "Ok, alarm set for 12/06/2019 17:00:00"
114+
115+
## Next steps
116+
117+
> [!div class="nextstepaction"]
118+
> [How To: Add a one-step correction to a Custom Command (Preview)](./how-to-custom-speech-commands-one-step-correction.md)

‎articles/cognitive-services/Speech-Service/how-to-custom-speech-commands-fulfill-sdk.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: How to fulfill Custom Commands on the client with the Speech SDK
33
titleSuffix: Azure Cognitive Services
4-
description: In this article, handle Custom Commands activities on client with the Speech SDK
4+
description: In this article, we explain how to handle Custom Commands activities on client with the Speech SDK.
55
services: cognitive-services
66
author: donkim
77
manager: yetian

‎articles/cognitive-services/Speech-Service/how-to-custom-speech-commands-one-step-correction.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
---
2+
title: 'How to: Add a one-step correction to a custom command (Preview) - Speech service'
3+
titleSuffix: Azure Cognitive Services
4+
description: In this article, we explain how to implement one-step corrections for a command in Custom Commands.
5+
services: cognitive-services
6+
author: encorona-ms
7+
manager: yetian
8+
ms.service: cognitive-services
9+
ms.subservice: speech-service
10+
ms.topic: conceptual
11+
ms.date: 12/05/2019
12+
ms.author: encorona
13+
---
14+
15+
# How To: Add a one-step correction to a Custom Command (Preview)
16+
17+
In this article, you'll learn how to add one-step confirmation to a command.
18+
19+
One-step correction is used to update a command that was just completed.
20+
21+
I.e. if you just set up an alarm, you can change your mind and update the time of the alarm.
22+
23+
- Input: Set alarm for tomorrow at noon
24+
- Output: "Ok, alarm set for 12/06/2019 12:00:00"
25+
- Input: No, tomorrow at 1pm
26+
- Output: "Ok
27+
28+
Keep in mind that this implies that you as a developer have a mechanism to update the alarm in your backend application.
29+
30+
## Prerequisites
31+
32+
You must have completed the steps in the following articles:
33+
34+
- [Quickstart: Create a Custom Command (Preview)](./quickstart-custom-speech-commands-create-new.md)
35+
- [Quickstart: Create a Custom Command with Parameters (Preview)](./quickstart-custom-speech-commands-create-parameters.md)
36+
- [How To: Add a confirmation to a Custom Command (Preview)](./how-to-custom-speech-commands-confirmations.md)
37+
38+
## Add the advanced rules for one-step correction
39+
40+
To demonstrate one-step correction, let's extend the **SetAlarm** command created in the [Confirmations How To](./how-to-custom-speech-commands-confirmations.md).
41+
42+
1. Add an advanced rule to update the previous alarm.
43+
44+
This rule will ask the user to confirm the date and time of the alarm and is expecting a confirmation (yes/no) for the next turn.
45+
46+
| Setting | Suggested value | Description |
47+
| --------------------- | ---------------------------------------------------------------- | -------------------------------------------------- |
48+
| Rule Name | Update previous alarm | A name describing the purpose of the rule |
49+
| Conditions | UpdateLastCommand & Required Parameter - DateTime | Conditions that determine when the rule can run |
50+
| Actions | SpeechResponse - "- Updating previous alarm to {DateTime}" | The action to take when the rule condition is true |
51+
| State after execution | Complete command | State of the user after the turn |
52+
53+
1. Move the rule you just created to the top of advanced rules (scroll over the rule in the panel and click the UP arrow).
54+
> [!div class="mx-imgBorder"]
55+
> ![Add a range validation](media/custom-speech-commands/one-step-correction-rules.png)
56+
57+
> [!NOTE]
58+
> In a real application, in the Actions section of this rule you'll also send back an activity to the client or call an HTTP endpoint to update the alarm in your system.
59+
60+
## Try it out
61+
62+
Select the Test panel and try a few interactions.
63+
64+
- Input: Set alarm for tomorrow at noon
65+
- Output: "Are you sure you want to set an alarm for 12/07/2019 12:00:00?"
66+
- Input: Yes
67+
- Output: "Ok, alarm set for 12/07/2019 12:00:00"
68+
- Input: No, tomorrow at 1pm
69+
- Output: "Updating previous alarm to 12/07/2019 13:00:00"

0 commit comments

Comments
 (0)