Merge pull request #3855 from sostel/sostel-patch-1

Eye Tracking documentation kick off

Author: keveleigh

Documentation/EyeTracking/EyeTracking_BasicSetup.md

@@ -0,0 +1,81 @@
+# Getting started with Eye Tracking in MRTK
+This page covers how to set up your Unity MRTK scene to use Eye Tracking in your app. 
+The following assumes you are starting out with a fresh new scene.
+Alternatively, you can check out our already configured Eye Tracking examples: 
+[MRTK Eye Tracking Examples](EyeTracking_ExamplesOverview.md).
+
+
+### Setting up the scene
+Set up the _MixedRealityToolkit_ by simply clicking _'Mixed Reality Toolkit -> Configure…'_ in the menu bar.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_configure.png)
+
+
+### Setting up the MRTK profiles required for Eye Tracking
+After setting up your MRTK scene, you will be asked to choose a profile for MRTK. 
+You can simply select _DefaultMixedRealityToolkitConfigurationProfile_ and then select the _'Copy & Customize'_ option.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_configprofile.png)
+
+
+### Create an "Eye Gaze Data Provider"
+- Navigate to the _'Input System Profile'_ in your MRTK main profile.
+
+- To edit the default one ( _'DefaultMixedRealityInputSystemProfile'_ ), click the _'Clone'_ button next to it. 
+
+- Double click on your new input profile and select _'+ Add Data Provider'_. 
+
+- Create a new data provider: 
+    - Under **Type** select _'Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input'_ -> _'WindowsMixedRealityEyeGazeDataProvider'_
+    
+    - For **Platform(s)** select _'Windows Universal'_.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_eyes_dataprovider.png)
+
+
+### Enabling Eye Tracking in the GazeProvider
+In HoloLens v1, head gaze was used as primary pointing technique. 
+While head gaze is still available via the _GazeProvider_ in MRTK which is attached to your [Camera](https://docs.unity3d.com/ScriptReference/Camera.html), you can check to use eye gaze instead by ticking the _'Prefer Eye Tracking'_ checkbox as shown in the screenshot below.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_eyes_gazeprovider.png)
+
+
+### Simulating Eye Tracking in the Unity Editor
+You can simulate Eye Tracking input in the Unity Editor to ensure that events are correctly triggered before deploying the app to your HoloLens 2.
+The eye gaze signal is simulated by simply using the camera's location as eye gaze origin and the camera's forward vector as eye gaze direction.
+While this is great for initial testing, please note that it is not a good imitation for rapid eye movements. 
+For this, it is better to ensure frequent tests of your eye-based interactions on the HoloLens 2.
+
+1. **Enable simulated Eye Tracking**: 
+    - Navigate to your main _'MRTK Configuration Profile'_ -> _'Input System Profile'_ -> _'Data Providers'_ -> _'Input Simulation Service'_.
+    - Check the _'Simulate Eye Position'_ checkbox.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_eyes_simulate.png)
+    
+2. **Disable default head gaze cursor**: 
+In general, we recommend to avoid showing an eye gaze cursor or if you insist on showing one to make it _very_ subtle.
+Check out our [eye gaze cursor tutorial](EyeTracking_Cursor.md) for more information on how to best handle it.
+We do recommend to hide the default head gaze cursor that is attached to the MRTK gaze pointer profile by default.
+    - Navigate to your main _'MRTK Configuration Profile'_ -> _'Input System Profile'_ -> _'PointerSettings.PointerProfile'_
+    - At the bottom of the _'PointerProfile'_, you should assign an invisible cursor prefab to the _'GazeCursor'_. If you downloaded the MRTK Examples folder, you can simply reference the included -'EyeGazeCursor'_ prefab.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_eyes_gazesettings.png)
+
+### Accessing eye gaze data
+Now that your scene is set up to use Eye Tracking, let's take a look at how to access it in your scripts: 
+[Accessing Eye Tracking Data in your Unity Script](EyeTracking_EyeGazeProvider.md).
+
+ 
+### Testing your Unity app on a HoloLens 2
+Building your app with Eye Tracking should be similar to how you would compile other HoloLens 2 MRTK apps. 
+The only difference is that the *'Gaze Input'* capability is unfortunately not yet supported by Unity under 'Player Settings -> Publishing Settings -> Capabilities'. 
+To use Eye Tracking on your HoloLens 2 device, you need to manually edit the package manifest that is part of your build Visual Studio project.
+So, follow these steps:
+1. Build your Unity project as you would normally do for _HoloLens 2_.
+2. Open your compiled Visual Studio project and then open the _'Package.appxmanifest'_ in your solution.
+3. Make sure to tick the _'Gaze Input'_ checkbox under _Capabilities_.
+
+![Enabling Gaze Input in Visual Studio](../../External/ReadMeImages/EyeTracking/mrtk_et_gazeinput.jpg)
+
+---
+[Back to "Eye Tracking in the MixedRealityToolkit"](EyeTracking_Main.md)

Documentation/EyeTracking/EyeTracking_Cursor.md

@@ -0,0 +1,8 @@
+# To show or not to show an eye gaze cursor
+
+This page covers design guidelines on when and how to use an eye gaze cursor. 
+
+<!-- TODO: Add more infos -- >
+
+---
+[Back to "Eye Tracking in the MixedRealityToolkit"](EyeTracking_Main.md)

Documentation/EyeTracking/EyeTracking_ExamplesOverview.md

@@ -0,0 +1,56 @@
+# Eye Tracking Examples in MRTK
+This page covers how to get quickly started with using Eye Tracking in MRTK by building on our provided MRTK example package.
+The samples let you experience one of our new magical input capabilities: **Eye Tracking**! 
+The demo include a number of different use cases for seamlessly combining information about what you are looking at with **Voice** and **Hand** input. 
+This enables users to quickly and effortlessly select and move holograms across their view simply by looking at a target and saying _'Select'_ or performing a hand gesture. 
+The demos also include an example for eye-gaze-directed scroll, pan and zoom of text and images on a slate. 
+Finally, an example is provided for recording and visualizing the user's visual attention on a 2D slate.
+
+
+## Overview of MRTK Eye Tracking Samples
+
+[**Eye-Supported Target Selection**](EyeTracking_TargetSelection.md)
+
+This tutorial showcases the ease of accessing eye gaze data to select targets. 
+It includes an example for subtle yet powerful feedback to provide confidence to the user that a target is focused while not being overwhelming.
+In addition, we showcase a simple example of smart notifications that automatically disappear after being read. 
+
+**Summary**: Fast and effortless target selections using a combination of Eyes, Voice and Hand input.
+
+<br>
+
+
+[**Eye-Supported Navigation**](EyeTracking_Navigation.md)
+
+Imagine you are reading some information on a distant display or your e-reader and when you reach the end of the displayed text, the text automatically scrolls up to reveal more content. 
+Or how about magically zooming directly toward where you were looking at? 
+These are some of the examples showcased in this tutorial about eye-supported navigation.
+In addition, we added an example for hands-free rotation of 3D holograms by making them automatically rotate based on your current focus. 
+
+**Summary**: Scroll, Pan, Zoom, 3D Rotation using a combination of Eyes, Voice and Hand input.
+
+<br>
+
+
+[**Eye-Supported Positioning**](EyeTracking_Positioning.md)
+
+In this tutorial, we extend an input scenario called [Put that there](https://youtu.be/CbIn8p4_4CQ) dating back to research from the MIT Media Lab in the early 1980's with eye, hand and voice input.
+The idea is simple: Benefit from your eyes for fast target selection and positioning. 
+Simply look at a hologram and say _'put this'_, look over where you want to place it and say _'there!'_. 
+For more precisely positioning your hologram, you can use additional input from your hands, voice or controllers. 
+
+**Summary**: Positioning holograms using Eyes, Voice and Hand input (*drag-and-drop*). Eye-supported sliders using Eyes+Hands. 
+
+<br>
+
+
+[**Visualization of Visual Attention**](EyeTracking_Visualization.md)
+
+Information about where users look at is an immensely powerful tool to assess usability of a design and to identify problems in efficient work streams. 
+In this tutorial, we discuss different eye tracking visualizations and how they fit different needs. 
+We provide basic examples for logging and loading Eye Tracking data and examples for how to visualize them. 
+
+**Summary**: Two-dimensional attention map (heatmaps) on slates. Recording & replaying Eye Tracking data.
+
+---
+[Back to "Eye Tracking in the MixedRealityToolkit"](EyeTracking_Main.md)

Documentation/EyeTracking/EyeTracking_EyeGazeProvider.md

@@ -0,0 +1,53 @@
+# Accessing Eye Tracking Data in your Unity Script
+
+The following assumes that you followed the steps for setting up Eye Tracking in your MRTK scene (see [Basic MRTK Setup to use Eye Tracking](EyeTracking_BasicSetup.md)).
+
+To access Eye Tracking data in your MonoBehaviour scripts is easy! Simply use *MixedRealityToolkit.InputSystem.EyeGazeProvider*.
+
+## MixedRealityToolkit.InputSystem.EyeGazeProvider
+While the *MixedRealityToolkit.InputSystem.EyeGazeProvider* provides several helpful variables, the key ones for Eye Tracking input are the following:
+
+- **UseEyeTracking**: 
+True if Eye Tracking hardware is available and the user has given permission to use Eye Tracking in the app. 
+
+- **IsEyeGazeValid**: 
+Indicates whether the current Eye Tracking data is valid. 
+It may be invalid due to exceeded timeout (should be robust to the user blinking though) or lack of tracking hardware or permissions.
+
+- **GazeOrigin**: 
+Origin of the gaze ray. 
+Please note that this will return the *head* gaze origin if 'IsEyeGazeValid' is false.
+
+- **GazeDirection**:
+Direction of the gaze ray. 
+This will return the *head* gaze direction if 'IsEyeGazeValid' is false.
+
+- **HitInfo**, **HitPosition**, **HitNormal**, etc.:
+Information about the currently gazed at target. 
+Again, if 'IsEyeGazeValid' is false, this will be based on the user's *head* gaze.
+
+
+## Examples for using MixedRealityToolkit.InputSystem.EyeGazeProvider
+Here is an example from the 
+[FollowEyeGaze.cs](xref:Microsoft.MixedReality.Toolkit.Examples.Demos.EyeTracking.FollowEyeGaze):
+
+- Get the point of a hologram that the user is looking at:
+```csharp
+// Show the object at the hit position of the user's eye gaze ray with the target.
+gameObject.transform.position = MixedRealityToolkit.InputSystem.EyeGazeProvider.HitPosition; 
+```
+
+
+
+- Showing a visual asset at a fixed distance from where the user is currently looking:
+```csharp
+// If no target is hit, show the object at a default distance along the gaze ray.
+gameObject.transform.position = 
+MixedRealityToolkit.InputSystem.EyeGazeProvider.GazeOrigin + 
+MixedRealityToolkit.InputSystem.EyeGazeProvider.GazeDirection.normalized * defaultDistanceInMeters;
+```
+
+
+
+---
+[Back to "Eye Tracking in the MixedRealityToolkit"](EyeTracking_Main.md)

Documentation/EyeTracking/EyeTracking_Main.md

@@ -0,0 +1,10 @@
+![Eye Tracking in MRTK](../../External/ReadMeImages/EyeTracking/mrtk_et_compilation.png)
+# Eye Tracking in the Mixed Reality Toolkit
+
+The [Mixed Reality Toolkit](https://github.com/Microsoft/MixedRealityToolkit-Unity) (MRTK) supports amongst others _'HoloLens 2'_ which offers an exciting and powerful new input: Eye Tracking! 
+Eye Tracking enables users to quickly and effortlessly engage with holograms across their view and can make your system smarter by better identifying a user's intention.
+New to Eye Tracking? No problem! We have created a number of videos, tutorials and samples to get you started! 
+1. [Getting started with Eye Tracking in MRTK](EyeTracking_BasicSetup.md)
+
+2. [Building on the MRTK Eye Tracking samples](EyeTracking_ExamplesOverview.md)
+---