Skip to content

Commit 9d2aa35

Browse files
authored
Merge pull request #211807 from valindrae/valindrae-patch-recognize
Valindrae patch recognize
2 parents 60b5704 + 07b9c16 commit 9d2aa35

File tree

10 files changed

+539
-34
lines changed

10 files changed

+539
-34
lines changed

articles/communication-services/concepts/voice-video-calling/call-automation.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ The following list presents the set of features that are currently available in
4545
| | Reject an incoming call | ✔️ | ✔️ |
4646
| Mid-call scenarios | Add one or more endpoints to an existing call | ✔️ | ✔️ |
4747
| | Play Audio from an audio file | ✔️ | ✔️ |
48+
| | Recognize user input through DTMF | ✔️ | ✔️ |
4849
| | Remove one or more endpoints from an existing call| ✔️ | ✔️ |
4950
| | Blind Transfer* a call to another endpoint | ✔️ | ✔️ |
5051
| | Hang up a call (remove the call leg) | ✔️ | ✔️ |
@@ -91,7 +92,9 @@ These actions can be performed on the calls that are answered or placed using Ca
9192

9293
**Add/Remove participant(s)** – One or more participants can be added in a single request with each participant being a variation of supported destination endpoints. A web hook callback is sent for every participant successfully added to the call.
9394

94-
**Play** - When your application answers a call or places an outbound call, you can play an audio prompt for the caller. This audio can be looped if needed in scenarios like playing hold music. To learn more, view our [quickstart](../../quickstarts/voice-video-calling/play-action.md)
95+
**Play** - When your application answers a call or places an outbound call, you can play an audio prompt for the caller. This audio can be looped if needed in scenarios like playing hold music. To learn more, view our [concepts](./play-action.md) and [quickstart](../../quickstarts/voice-video-calling/play-action.md).
96+
97+
**Recognize input** - After your application has played an audio prompt, you can request user input to drive business logic and navigation in your application. To learn more, view our [concepts](./recognize-action.md) and [quickstart](../../quickstarts/voice-video-calling/Recognize-Action.md).
9598

9699
**Transfer** – When your application answers a call or places an outbound call to an endpoint, that endpoint can be transferred to another destination endpoint. Transferring a 1:1 call will remove your application's ability to control the call using the Call Automation SDKs.
97100

@@ -132,6 +135,8 @@ The Call Automation events are sent to the web hook callback URI specified when
132135
| ParticipantUpdated | The status of a participant changed while your application’s call leg was connected to a call |
133136
| PlayCompleted| Your application successfully played the audio file provided |
134137
| PlayFailed| Your application failed to play audio |
138+
| RecognizeCompleted | Recognition of user input was successfully completed |
139+
| RecognizeFailed | Recognition of user input was unsuccessful <br/><br/>*to learn more about recognize action events view our [quickstart](../../quickstarts/voice-video-calling/Recognize-Action.md)*|
135140

136141
## Known Issues
137142

53.5 KB
Loading
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
title: Recognize action
3+
description: Conceptual information about using Recognize action with Call Automation.
4+
author: Kunaal
5+
ms.service: azure-communication-services
6+
ms.topic: include
7+
ms.date: 09/16/2022
8+
ms.author: kpunjabi
9+
ms.custom: private_preview
10+
---
11+
12+
# Recognize action overview
13+
14+
> [!IMPORTANT]
15+
> Functionality described on this document is currently in private preview. Private preview includes access to SDKs and documentation for testing purposes that are not yet available publicly.
16+
> Apply to become an early adopter by filling out the form for [preview access to Azure Communication Services](https://aka.ms/ACS-EarlyAdopter).
17+
18+
With the Recognize action developers will be able to enhance their IVR or contact center applications to recognize user input. One of the most common scenarios of recognition is to play a message and request user input. This input is received in the form of DTMF (input via the digits on their calling device) which then allows the application to navigate the user to the next action.
19+
20+
**DTMF**
21+
Dual-tone multifrequency (DTMF) recognition is the process of understanding tones/sounds generated by a telephone when a number is pressed. Equipment at the receiving end listening for the specific tone then converts them into commands. These commands generally signal user intent when navigating a menu in an IVR scenario or in some cases can be used to capture important information that the user needs to provide via their phones keypad.
22+
23+
**DTMF events and their associated tones**
24+
25+
|Event|Tone|
26+
| --- |--|
27+
|0|Zero|
28+
|1|One|
29+
|2|Two|
30+
|3|Three|
31+
|4|Four|
32+
|5|Five|
33+
|6|Six|
34+
|7|Seven|
35+
|8|Eight|
36+
|9|Nine|
37+
|A|A|
38+
|B|B|
39+
|C|C|
40+
|D|D|
41+
|*|Asterisk|
42+
|#|Pound|
43+
44+
## Common use cases
45+
46+
The recognize action can be used for many reasons, below are a few examples of how developers can use the recognize action in their application.
47+
48+
### Improve user journey with self-service prompts
49+
50+
- **Users can control the call** - By enabling input recognition you allow the caller to navigate your IVR menu and provide information that can be used to resolve their query.
51+
- **Gather user information** - By enabling input recognition your application can gather input from the callers. This can be information such as account numbers, credit card information, etc.
52+
53+
### Interrupt audio prompts
54+
55+
**User can exit from an IVR menu and speak to a human agent** - With DTMF interruption your application can allow users to exit interrupt the flow of the IVR menu and be able to chat to a human agent.
56+
57+
58+
## How the Recognize action workflow looks
59+
60+
![Recognize Action](./media/recognize-action-flow.png)
61+
62+
## What's coming up next for Recognize action
63+
64+
As we invest more into this functionality, we recommend developers sign up to our TAP program that allows you to get early access to the newest feature releases. Over the coming months the recognize action will add in new capabilities that use our integration with Azure Cognitive Services to provide AI capabilities such as Speech-To-Text. With these, you can improve customer interactions and recognize voice inputs from participants on the call.
65+
66+
## Next steps
67+
68+
- Check out the [Recognize action quickstart](../../quickstarts/voice-video-calling/recognize-action.md) to learn more.

articles/communication-services/quickstarts/voice-video-calling/callflows-for-customer-interactions.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,5 +39,6 @@ If you want to clean up and remove a Communication Services subscription, you ca
3939

4040
## Next steps
4141
- Learn more about [Call Automation](../../concepts/voice-video-calling/call-automation.md) and its features.
42-
- Learn how to [manage inbound telephony calls](../telephony/Manage-Inbound-Calls.md) with Call Automation.
43-
- Learn more about [Play action](../../concepts/voice-video-calling/Play-Action.md).
42+
- Learn how to [manage inbound telephony calls](../telephony/manage-inbound-calls.md) with Call Automation.
43+
- Learn more about [Play action](../../concepts/voice-video-calling/play-action.md).
44+
- Learn more about [Recognize action](../../concepts/voice-video-calling/recognize-action.md).
Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
---
2+
title: include file
3+
description: C# recognize action quickstart
4+
services: azure-communication-services
5+
author: Kunaal
6+
ms.service: azure-communication-services
7+
ms.subservice: azure-communication-services
8+
ms.date: 09/16/2022
9+
ms.topic: include
10+
ms.topic: include file
11+
ms.author: kpunjabi
12+
---
13+
14+
## Prerequisites
15+
- Azure account with an active subscription, for details see [Create an account for free.](https://azure.microsoft.com/free/)
16+
- Azure Communication Services resource. See [Create an Azure Communication Services resource](../../../create-communication-resource.md?tabs=windows&pivots=platform-azp)
17+
- Create a new web service application using the [Call Automation SDK](../../Callflows-for-customer-interactions.md).
18+
- The latest [.NET library](https://dotnet.microsoft.com/download/dotnet-core) for your operating system.
19+
- [Apache Maven](https://maven.apache.org/download.cgi).
20+
- Obtain the NuGet package from the [Azure SDK Dev Feed](https://github.com/Azure/azure-sdk-for-net/blob/main/CONTRIBUTING.md#nuget-package-dev-feed)
21+
22+
## Technical specifications
23+
24+
The following parameters are available to customize the Recognize function:
25+
26+
| Parameter | Type|Default (if not specified) | Description | Required or Optional |
27+
| ------- |--| ------------------------ | --------- | ------------------ |
28+
| Prompt <br/><br/> *(for details on Play action, refer to [this quickstart](../../play-action.md))* | FileSource | Not set |This will be the message you wish to play before recognizing input. | Optional |
29+
| InterToneTimeout | TimeSpan | 2 seconds <br/><br/>**Min:** 1 second <br/>**Max:** 60 seconds | Limit in seconds that ACS will wait for the caller to press another digit (inter-digit timeout). | Optional |
30+
| InitialSilenceTimeout | TimeSpan | 5seconds<br/><br/>**Min:** 0 seconds <br/>**Max:** 300 seconds | How long recognize action will wait for input before considering it a timeout. | Optional |
31+
| MaxTonesToCollect | Integer | No default<br/><br/>**Min:** 1|Number of digits a developer expects as input from the participant.| Required |
32+
| StopTones |IEnumeration\<DtmfTone\> | Not set | The digit participants can press to escape out of a batch DTMF event. | Optional |
33+
| InterruptPrompt | Bool | True | If the participant has the ability to interrupt the playMessage by pressing a digit. | Optional |
34+
| InterruptCallMediaOperation | Bool | True | If this flag is set it will interrupt the current call media operation if any (for example if play is going on) and initiate recognize. | Optional |
35+
| OperationContext | String | Not set | String that developers can pass mid action, useful for allowing developers to store context about the events they receive. | Optional |
36+
37+
## Create a new C# application
38+
39+
In the console window of your operating system, use the `dotnet` command to create a new web application.
40+
41+
```console
42+
dotnet new web -n MyApplication
43+
```
44+
45+
## Install the NuGet package
46+
47+
During the preview phase, the NuGet package can be obtained by configuring your package manager to use the Azure SDK Dev Feed from [here](https://github.com/Azure/azure-sdk-for-net/blob/main/CONTRIBUTING.md#nuget-package-dev-feed)
48+
49+
## Obtain your connection string
50+
51+
From the Azure portal, locate your Communication Service resource and click on the Keys section to obtain your connection string.
52+
53+
:::image type="content" source="./../../media/call-automation/Key.png" alt-text="Screenshot of Communication Services resource page on portal to access keys":::
54+
55+
## Establish a call
56+
57+
By this point you should be familiar with starting calls, if you need to learn more about how to start a call view our [quickstart](../../Callflows-for-customer-interactions.md). In this instance, we'll answer an incoming call.
58+
59+
## Call the recognize action
60+
61+
When your application answers the call, you can provide information about recognizing participant input and playing a prompt.
62+
63+
``` csharp
64+
var targetParticipant = new PhoneNumberIdentifier("+1XXXXXXXXXXX");
65+
var recognizeOptions = new CallMediaRecognizeDtmfOptions(targetParticipant, maxTonesToCollect)
66+
{
67+
InterruptCallMediaOperation = true,
68+
InitialSilenceTimeout = TimeSpan.FromSeconds(30),
69+
Prompt = new FileSource(new System.Uri("file://path/to/file")),
70+
InterToneTimeout = TimeSpan.FromSeconds(5),
71+
InterruptPrompt = true,
72+
StopTones = new DtmfTone[] { DtmfTone.Pound },
73+
};
74+
await _callConnection.GetCallMedia().StartRecognizingAsync(recognizeOptions).ConfigureAwait(false);
75+
76+
```
77+
78+
**Note:** If parameters aren't set, the defaults will be applied where possible.
79+
80+
## Receiving recognize event updates
81+
82+
Developers can subscribe to the *RecognizeCompleted* and *RecognizeFailed* events on the webhook callback they registered for the call to create business logic in their application for determining next steps when one of the aforementioned events occurs.
83+
84+
Example of *RecognizeCompleted* event:
85+
``` json
86+
[
87+
{
88+
"id": "e9cf1c71-f119-48db-86ca-4f2530a2004d",
89+
"source": "calling/callConnections/411f0b00-d97f-49ad-a6ff-3f8c05dc64d7/RecognizeCompleted",
90+
"type": "Microsoft.Communication.RecognizeCompleted",
91+
"data": {
92+
"eventSource": "calling/callConnections/411f0b00-d97f-49ad-a6ff-3f8c05dc64d7/RecognizeCompleted",
93+
"operationContext": "267e33a9-c28e-4ecf-a33e-b3abd9526e32",
94+
"resultInformation": {
95+
"code": 200,
96+
"subCode": 8531,
97+
"message": "Action completed, max digits received."
98+
},
99+
"recognitionType": "dtmf",
100+
"collectTonesResult": {
101+
"tones": [
102+
"nine",
103+
"eight",
104+
"zero",
105+
"five",
106+
"two"
107+
]
108+
},
109+
"callConnectionId": "411f0b00-d97f-49ad-a6ff-3f8c05dc64d7",
110+
"serverCallId": "aHR0cHM6Ly9hcGkuZmxpZ2h0cHJveHkuc2t5cGUuY29tL2FwaS92Mi9jcC9jb252LXVzZWEyLTAxLmNvbnYuc2t5cGUuY29tL2NvbnYvQzNuT3lkY3E0VTZCV0gtcG1GNmc1Zz9pPTQmZT02Mzc5ODYwMDMzNDQ2MTA5MzM=",
111+
"correlationId": "53be6977-d832-4c42-8527-fb2aa4a78b74"
112+
},
113+
"time": "2022-09-13T00:55:08.2240104+00:00",
114+
"specversion": "1.0",
115+
"datacontenttype": "application/json",
116+
"subject": "calling/callConnections/411f0b00-d97f-49ad-a6ff-3f8c05dc64d7/RecognizeCompleted"
117+
}
118+
]
119+
```
120+
121+
Example of how you can deserialize the *RecognizeCompleted* event:
122+
``` csharp
123+
app.MapPost("<WEB_HOOK_ENDPOINT>", async (
124+
[FromBody] CloudEvent[] cloudEvents,
125+
[FromRoute] string contextId) =>{
126+
foreach (var cloudEvent in cloudEvents)
127+
{
128+
CallAutomationEventBase @event = CallAutomationEventParser.Parse(cloudEvent);
129+
If (@event is RecognizeCompleted recognizeCompleted)
130+
{
131+
// Access to the Collected Tones
132+
foreach(DtmfTone tone in recognizeCompleted.CollectTonesResult.Tones) {
133+
// work on each of the Dtmf tones.
134+
}
135+
}
136+
```
137+
138+
Example of *RecognizeFailed* event:
139+
140+
``` json
141+
[
142+
{
143+
"id": "47d9cb04-7039-427b-af50-aebdd94db054",
144+
"source": "calling/callConnections/411f0b00-bb72-4d5b-9524-ae1c29713335/RecognizeFailed",
145+
"type": "Microsoft.Communication.RecognizeFailed",
146+
"data": {
147+
"eventSource": "calling/callConnections/411f0b00-bb72-4d5b-9524-ae1c29713335/RecognizeFailed",
148+
"operationContext": "267e33a9-c28e-4ecf-a33e-b3abd9526e32",
149+
"resultInformation": {
150+
"code": 500,
151+
"subCode": 8511,
152+
"message": "Action failed, encountered failure while trying to play the prompt."
153+
},
154+
"callConnectionId": "411f0b00-bb72-4d5b-9524-ae1c29713335",
155+
"serverCallId": "aHR0cHM6Ly9hcGkuZmxpZ2h0cHJveHkuc2t5cGUuY29tL2FwaS92Mi9jcC9jb252LXVzZWEyLTAxLmNvbnYuc2t5cGUuY29tL2NvbnYvQzNuT3lkY3E0VTZCV0gtcG1GNmc1Zz9pPTQmZT02Mzc5ODYwMDMzNDQ2MTA5MzM=",
156+
"correlationId": "53be6977-d832-4c42-8527-fb2aa4a78b74"
157+
},
158+
"time": "2022-09-13T00:55:37.0660233+00:00",
159+
"specversion": "1.0",
160+
"datacontenttype": "application/json",
161+
"subject": "calling/callConnections/411f0b00-bb72-4d5b-9524-ae1c29713335/RecognizeFailed"
162+
}
163+
]
164+
165+
```
166+
167+
Example of how you can deserialize the *RecognizeFailed* event:
168+
169+
``` csharp
170+
app.MapPost("<WEB_HOOK_ENDPOINT>", async (
171+
[FromBody] CloudEvent[] cloudEvents,
172+
[FromRoute] string contextId) =>{
173+
foreach (var cloudEvent in cloudEvents)
174+
{
175+
CallAutomationEventBase @event = CallAutomationEventParser.Parse(cloudEvent);
176+
If (@event is RecognizeFailed recognizeFailed)
177+
{
178+
Log.error($”Recognize failed due to: {recognizeFailed.ResultInformation.Message}”);
179+
180+
}
181+
}
182+
```

0 commit comments

Comments
 (0)