You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cognitive-services/Speech-Service/how-to-windows-voice-assistants-get-started.md
+4-5Lines changed: 4 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,7 +26,7 @@ To start developing a voice assistant for Windows, you will need to mak
26
26
27
27
## Obtain resources from Microsoft
28
28
29
-
Some resources necessary for a completely customized voice agent on Windows will require resources from Microsoft. The [UWP Voice Assistant Sample](windows-voice-assistants-sample-info.md) provides sample versions of these resources for initial development and testing, so this section is unnecessary for initial development.
29
+
Some resources necessary for a completely customized voice agent on Windows will require resources from Microsoft. The [UWP Voice Assistant Sample](windows-voice-assistants-faq.md#the-uwp-voice-assistant-sample) provides sample versions of these resources for initial development and testing, so this section is unnecessary for initial development.
30
30
31
31
-**Keyword model:** Voice activation requires a keyword model from Microsoft in the form of a .bin file. Note that the .bin file provided in the UWP Voice Assistant Sample is trained on the keyword "Contoso".
32
32
-**Limited Access Feature Token:** Since the ConversationalAgent APIs provide access to microphone audio, they are protected under Limited Access Feature restrictions. To use a Limited Access Feature, you will need to obtain a Limited Access Feature token connected to the package identity of your application from Microsoft.
@@ -47,14 +47,13 @@ The following provides the requirements to create a simple dialog service using
47
47
48
48
## Try out the sample app
49
49
50
-
With the subscription key from your Speech Services subscription and bot id from your echo bot, you are ready to try out the [UWP Voice Assistant sample](windows-voice-assistants-sample-info.md). Follow the instructions in the readme to run the app and enter your credentials.
50
+
With the subscription key from your Speech Services subscription and bot id from your echo bot, you are ready to try out the [UWP Voice Assistant sample](windows-voice-assistants-faq.md#the-uwp-voice-assistant-sample.md). Follow the instructions in the readme to run the app and enter your credentials.
51
51
52
52
## Create your own voice assistant for Windows
53
53
54
54
Once you have received your Limited Access Feature token and bin file from Microsoft, you can begin on your own voice assistant on Windows.
55
55
56
56
## Next steps
57
57
58
-
- Learn more about voice activation
59
-
- Read the implementation guide for the proper steps to enable voice activation, above lock activation, and more.
60
-
- Visit the "docs" folder in the UWP Voice Assistant sample for additional resources and a walk-through of the code.
58
+
> [!div class="nextstepaction"]
59
+
> [Read the voice assistant implementation guide](windows-voice-assistants-implementation-guide.md)
Copy file name to clipboardExpand all lines: articles/cognitive-services/Speech-Service/windows-voice-assistants-faq.md
+21-5Lines changed: 21 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,13 +13,29 @@ ms.date: 04/15/2020
13
13
ms.author: travisw
14
14
---
15
15
16
-
# Frequently asked questions
16
+
# Samples and FAQs
17
17
18
-
## How do I contact Microsoft for resources like Limited Access Feature tokens and keyword model files?
18
+
## The UWP Voice Assistant Sample
19
19
20
+
The UWP Voice Assistant Sample is a voice assistant on Windows that serves to
20
21
22
+
- demonstrate the use of the Windows ConversationalAgent APIs
23
+
- display best practices for a voice agent on Windows
24
+
- provide an adaptable app and reusable components for your MVP agent application
25
+
- show how Direct Line Speech, along with Bot Framework or Custom Commands, can be used together with the Windows ConversationalAgent APIs for an end-to-end voice agent experience
21
26
22
-
## My app is showing in a small window when I activate it by voice. How can I transition from the compact view to a full application window?
27
+
The documentation provided with the sample app walks through the code path of voice activation and agent management, from the prerequisites of startup through proper cleanup.
28
+
29
+
> [!div class="nextstepaction"]
30
+
> [Visit the Github repo for the UWP Sample](https://aka.ms/MVA/sample)
31
+
32
+
## Frequently asked questions
33
+
34
+
### How do I contact Microsoft for resources like Limited Access Feature tokens and keyword model files?
35
+
36
+
37
+
38
+
### My app is showing in a small window when I activate it by voice. How can I transition from the compact view to a full application window?
23
39
24
40
When your application is first activated by voice, it is started in a compact view. Please read the [Design guidance for voice activation preview](windows-voice-assistants-best-practices.md#design-guidance-for-voice-activation-preview) for guidance on the different views and transitions between them for voice assistants on Windows.
25
41
@@ -28,10 +44,10 @@ To make the transition from compact view to full app view, use the appView API `
## Why are voice assistant features on Windows only enabled for UWP applications?
47
+
###Why are voice assistant features on Windows only enabled for UWP applications?
32
48
33
49
Given the privacy risks associated with features like voice activation, the features of the UWP platform are necessary allow the voice assistant features on Windows to be sufficiently secure.
34
50
35
-
## The UWP Voice Assistant Sample uses Direct Line Speech. Do I have to use Direct Line Speech for my voice assistant on Windows?
51
+
###The UWP Voice Assistant Sample uses Direct Line Speech. Do I have to use Direct Line Speech for my voice assistant on Windows?
36
52
37
53
The UWP Sample Application was developed using Direct Line Speech and the Speech Services SDK as a demonstration of how to use a dialog service with the Windows Conversational Agent capability. However, you can use any service for local and cloud keyword verification, speech-to-text conversion, bot dialog, and text-to-speech conversion.
Copy file name to clipboardExpand all lines: articles/cognitive-services/Speech-Service/windows-voice-assistants-overview.md
+42-9Lines changed: 42 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,15 +17,48 @@ ms.author: travisw
17
17
18
18
On Windows 10 version 2004 and up, voice assistant applications can take advantage of the Windows ConversationalAgent APIs to achieve a complete voice-enabled assistant experience.
19
19
20
-
| Feature | Details |
21
-
|----------|----------|
22
-
|Voice Activation | A voice agent can be activated by a spoken keyword, even when the agent application is closed.
23
-
|Above Lock Activation | Voice agents can be launched with voice activation while the user's computer is locked, even if the lid is closed.
24
-
|Multiple Agent Management | If multiple voice agents are installed, Windows will manage them properly and notify each voice assistant if they are interrupted or deactivated.
25
-
|Privacy Management | A set of voice-activation privacy settings gives users control of voice activation and above lock activation on a per-app basis.
20
+
## Voice Assistant Features
26
21
27
-
## Getting Started
22
+
Voice agent applications can be activated by a spoken keyword to enable a hands-free, voice driven experience. Voice activation works when the application is closed and when the screen is locked.
23
+
24
+
In addition, Windows provides a set of voice-activation privacy settings that gives users control of voice activation and above lock activation on a per-app basis.
25
+
26
+
After voice activation, Windows will manage multiple active agents properly and notify each voice assistant if they are interrupted or deactivated. This allows applications to manage interruptions and other inter-agent events properly.
27
+
28
+
## How does voice activation work?
29
+
30
+
### Agent Activation Runtime
31
+
32
+
The Agent Activation Runtime (AAR) is the ongoing process in Windows that manages application activation on a spoken keyword or button press. It starts with Windows as long as there is at least one application on the system that is registered with the system. Applications interact with AAR through the ConversationalAgent APIs in the Windows SDK.
33
+
34
+
### Registration
35
+
36
+
The first time a voice activated application is run, it registers its app id and keyword information through the ConversationalAgent APIs. AAR registers all configurations in the global mapping with the hardware or software keyword spotter on the system, allowing them to detect the application's keyword. The application also [registers with the Background Service](https://docs.microsoft.com/windows/uwp/launch-resume/register-a-background-task).
37
+
38
+
Note that this means an application cannot be activated by voice until it has been run once and registration has been allowed to complete.
39
+
40
+
### Activation
41
+
42
+
When the user speaks a keyword, the software or hardware keyword spotter on the system notifies AAR that a keyword has been detected, providing a keyword ID. AAR in turn sends a request to BackgroundService to start the application with the corresponding application ID.
43
+
44
+
### Receiving an activation
45
+
46
+
Upon receiving the request from AAR, the Background Service launches the application. The application receives a signal through the OnBackgroundActivated life cycle method in `App.xaml.cs` with a unique event argument. This tells the application that it was activated by AAR and should act accordingly.
47
+
48
+
### Keyword verification
49
+
50
+
The keyword spotter that triggered the application to start has achieved low power consumption by simplifying the keyword model. This allows the keyword spotter to be "always on" without a high power impact, but it also means the keyword spotter will likely have a high number of "false accepts" where it detects a keyword even though no keyword was spoken. This is why the voice activation system launches the application in the background: to give the application a chance to verify that the keyword was spoken before interrupting the user's current session. AAR saves the audio since a few seconds before the keyword was spotted and makes it accessible to the application. The application can use this to run a more reliable keyword spotter on the same audio.
51
+
52
+
### Foreground Activation
53
+
54
+
If the application successfully verifies the keyword, it can make a request to be shown in the foreground and display UI.
55
+
56
+
### Voice activation while app is active
57
+
58
+
AAR still signals active applications when their keyword is spoken. Rather than signalling through the life cycle method in `App.xaml.cs`, though, it signals through an event in the ConversationalAgent APIs.
59
+
60
+
## Next steps
28
61
29
62
-**Review the design guidelines:** Our [design guidelines](windows-voice-assistants-best-practices.md) lay out the key work required to provide the best possible experiences for voice activation on Windows 10.
30
-
-**Visit the Getting Started Guide:** Start [here](how-to-windows-voice-assistants-get-started.md) for the steps to begin implementing voice assistants on Windows, from setting your development environment through an introduction to the UWP Voice Assistant Sample.
31
-
-**Try out the sample app**: To experience these capabilities firsthand, visit the [UWP Voice Assistant Sample](windows-voice-assistants-sample-info.md) page and follow the steps to get the sample client running.
63
+
-**Visit the Getting Started page:** Start [here](how-to-windows-voice-assistants-get-started.md) for the steps to begin implementing voice assistants on Windows, from setting your development environment through an introduction to implementation guide.
64
+
-**Try out the sample app**: To experience these capabilities firsthand, visit the [UWP Voice Assistant Sample](windows-voice-assistants-faq.md#the-uwp-voice-assistant-sample) page and follow the steps to get the sample client running.
0 commit comments