You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Renaming article to 'Handle Microsoft Copilot hardware key state changes', removing PressAndHoldAndRelease references, and de-emphasizing audio recording scenario.
Copy file name to clipboardExpand all lines: hub/apps/develop/windows-integration/copilot-key-state.md
+5-26Lines changed: 5 additions & 26 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
title: Microsoft Copilot hardware key PressAndHoldAndRelease
2
+
title: Handle Microsoft Copilot hardware key state changes
3
3
description: Learn how to register to be activated and receive notifications when the Microsoft Copilot hardware key or Windows + C is pressed.
4
4
ms.topic: article
5
5
ms.date: 10/25/2024
@@ -8,15 +8,9 @@ ms.localizationpriority: medium
8
8
9
9
10
10
11
-
# Microsoft Copilot hardware key PressAndHoldAndRelease
11
+
# Handle Microsoft Copilot hardware key state changes
12
12
13
-
This article describes how apps can register to be activated and receive notifications when the Microsoft Copilot hardware key or Windows + C is pressed. This feature enables the following user scenario.
14
-
15
-
1. The user presses the Microsoft Copilot Hardware key or Windows + C and holds it for the system-defined 700 ms time window.
16
-
1. The system launches the Copilot Key provider app, letting the app know that the user is pressing and holding the key.
17
-
1. The app begins recording audio and shows a window indicating to the user that audio is being recorded.
18
-
1. The user speaks and then releases the key.
19
-
1. The app is notified that the key has been released, prompting it to process the recorded speech and taking additional actions based on the user's words.
13
+
This article describes how apps can register to be activated and receive notifications when the Microsoft Copilot hardware key or Windows + C is pressed, pressed and held, and released. This feature enables apps to perform different actions depending on which key state change is detected. For example, and app may perform normal activation when the key is single-pressed, but record audio or take a screenshot when the key is pressed and held..
20
14
21
15
This feature extends the features of a basic Microsoft Copilot hardware key provider, which simply registers to be launched when the hardware key is pressed. For more information, see [Microsoft Copilot hardware key providers](microsoft-copliot-key-provider.md).
22
16
@@ -83,7 +77,7 @@ public MainWindow(string state)
83
77
84
78
## Register for URI activation
85
79
86
-
The system launches Microsoft Copilot hardware key providers that implement PressAndHoldAndRelease using URI activation. Register a launch protocol by adding the [uap:Protocol](/uwp/schemas/appxpackage/uapmanifestschema/element-uap-protocol) element to your app manifest. For more information about how to register as the default handler for a URI scheme, see [Handle URI activation](/windows/apps/develop/launch/handle-uri-activation).
80
+
The system launches Microsoft Copilot hardware key providers using URI activation. Register a launch protocol by adding the [uap:Protocol](/uwp/schemas/appxpackage/uapmanifestschema/element-uap-protocol) element to your app manifest. For more information about how to register as the default handler for a URI scheme, see [Handle URI activation](/windows/apps/develop/launch/handle-uri-activation).
87
81
88
82
The following example shows the **uap:Extension** registering the URI scheme "myapp-copilothotkey".
89
83
@@ -105,7 +99,7 @@ The following example shows the **uap:Extension** registering the URI scheme "my
105
99
106
100
## Microsoft Copilot hardware key app extension
107
101
108
-
An app must be packaged in order to register as a Microsoft Copilot hardware key provider. For information on app packaging, see [An overview of Package Identity in Windows app](/windows/apps/desktop/modernize/package-identity-overview). Microsoft Copilot hardware key providers declare their registration information within the [uap3:AppExtension](/uwp/schemas/appxpackage/uapmanifestschema/element-uap3-appextension-manual). The **Name** attribute of the extension must be set to "com.microsoft.windows.copilotkeyprovider". To support the PressAndHoldAndRelease feature, apps must provide some additional entries to their **uap3:AppExtension** declaration.
102
+
An app must be packaged in order to register as a Microsoft Copilot hardware key provider. For information on app packaging, see [An overview of Package Identity in Windows app](/windows/apps/desktop/modernize/package-identity-overview). Microsoft Copilot hardware key providers declare their registration information within the [uap3:AppExtension](/uwp/schemas/appxpackage/uapmanifestschema/element-uap3-appextension-manual). The **Name** attribute of the extension must be set to "com.microsoft.windows.copilotkeyprovider". To support the key state changes, apps must provide some additional entries to their **uap3:AppExtension** declaration.
109
103
110
104
Inside of the **uap3:AppExtension** element, add a [uap3:Properties](/uwp/schemas/appxpackage/uapmanifestschema/element-uap3-properties-manual) element with child elements **PressAndHoldStart** and **PressAndHoldStart**. The contents of these elements should be the URI of the protocol scheme registered in the manifest in the previous step. The query string arguments specify whether the URI is being launched because the user pressed and held the hot key or because the user released the hot key. The app uses these query string values during app activation to determine the correct action to take.
This example shown in this walkthrough only demonstrates handling activation based on a change in the Windows Copilot hardware key pressed state. Since the purpose of the PressAndHoldAndRelease is to provide a voice chat-style interface, your app needs to implement some more sophisticated behavior.
274
-
275
-
### Recording audio
276
-
277
-
Apps will typically begin recording audio when the button state changes to *PressAndHoldStart* and stop recording when the state changes to *PressAndHoldEnd*. For information about recording audio in a Windows app, see [Basic photo, video, and audio capture with MediaCapture in a WinUI 3 app](/windows/apps/develop/camera/basic-photo-capture) or [Audio/Video Capture in Media Foundation](/windows/win32/medfound/audio-video-capture-in-media-foundation).
278
-
279
-
### Processing speech
280
-
281
-
To determine the user intent from the recorded audio, apps may need to use a speech to text processing. There are many ways to do this from a Windows app, including using local AI models or using a web service such as [Azure AI Speech service](/azure/ai-services/speech-service). For more information, see [Speech to text documentation](/azure/ai-services/speech-service/index-speech-to-text).
282
-
283
-
### Status UI
284
-
285
-
**TBD - I'm not sure of the requirements around UI for PressAndHoldAndRelease. It would be nice to provide some UI guidance and/or link to docs for the best practices for the relevant UI features (e.g. showing and hiding a window)**
0 commit comments