Merge pull request #3865 from Microsoft/mrtk_development

Eye tracking and style guide docs mrtk_dev -> rc1_stabilization

Author: keveleigh

CONTRIBUTING.md

@@ -47,7 +47,7 @@ To get started, simply follow these steps
 4. Make **small and frequent** commits that include tests which could be a unity scene showing usage of your feature.
 5. Make sure that all the tests continue to pass.
 6. Follow the [Coding Guidelines](CodingGuidelines.md).
-7. Ensure the code and feature(s) are documented as described in the [Documentation Guidelines](DocumentationGuide.md).
+7. Ensure the code and feature(s) are documented as described in the [Documentation Guidelines](Documentation/DocumentationGuide.md).
 8. Ensure the code works as intended on all [platforms](#supported-platforms).
     - For Windows UWP projects, your code must be [WACK compliant](https://developer.microsoft.com/en-us/windows/develop/app-certification-kit). To do this, generate a Visual Studio solution, right click on project; Store -> Create App Packages. Follow the prompts and run WACK tests. Make sure they all succeed.
 9. Update the [documentation](#contributing) with additional information as needed.

Documentation/DevDocGuide.md

@@ -1,4 +1,4 @@
-# Developer Documentation Guide
+# Developer portal generation guide
 
 MRTK uses [docfx](https://dotnet.github.io/docfx/index.html) to generate html documentation out of triple slash comments in code and .md files in the MRTK repository. Docfx documentation generation is automatically triggered by CI (soon) on completed PRs in the mrtk_development branch.
 The current state of the developer documentation can be found on the [MRTK github.io page](https://microsoft.github.io/MixedRealityToolkit-Unity/)
@@ -129,6 +129,7 @@ The versioning system can also be used for showing doc versions from other dev b
   * External crefs - As long as there's no xref service available (and listed in the docfx build file) crefs to external libraries won't work. If you still want to link to a specific external symbol that doesn't have xref service but an online api documentation you can use a href instead. Example: linking to EditorPrefs of Unity: <see href="https://docs.unity3d.com/ScriptReference/EditorPrefs.html">EditorPrefs</see>
 
 ## See also
-* [DocFX](https://dotnet.github.io/docfx/index.html)
+* [MRTK documentation guide](DocumentationGuide.md)
 * [MRTK developer documentation on github.io](https://microsoft.github.io/MixedRealityToolkit-Unity/)
+* [DocFX](https://dotnet.github.io/docfx/index.html)
 

Documentation/DocsToDo.md

@@ -0,0 +1,11 @@
+---
+title: Documentation TODO list
+description: 
+author: 
+ms.author: 
+ms.date:
+keywords: 
+---
+
+# Documentation TODO list
+

Documentation/DocumentationGuide.md

@@ -0,0 +1,81 @@
+# Getting started with Eye Tracking in MRTK
+This page covers how to set up your Unity MRTK scene to use Eye Tracking in your app. 
+The following assumes you are starting out with a fresh new scene.
+Alternatively, you can check out our already configured Eye Tracking examples: 
+[MRTK Eye Tracking Examples](EyeTracking_ExamplesOverview.md).
+
+
+### Setting up the scene
+Set up the _MixedRealityToolkit_ by simply clicking _'Mixed Reality Toolkit -> Configure…'_ in the menu bar.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_configure.png)
+
+
+### Setting up the MRTK profiles required for Eye Tracking
+After setting up your MRTK scene, you will be asked to choose a profile for MRTK. 
+You can simply select _DefaultMixedRealityToolkitConfigurationProfile_ and then select the _'Copy & Customize'_ option.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_configprofile.png)
+
+
+### Create an "Eye Gaze Data Provider"
+- Navigate to the _'Input System Profile'_ in your MRTK main profile.
+
+- To edit the default one ( _'DefaultMixedRealityInputSystemProfile'_ ), click the _'Clone'_ button next to it. 
+
+- Double click on your new input profile and select _'+ Add Data Provider'_. 
+
+- Create a new data provider: 
+    - Under **Type** select _'Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input'_ -> _'WindowsMixedRealityEyeGazeDataProvider'_
+    
+    - For **Platform(s)** select _'Windows Universal'_.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_eyes_dataprovider.png)
+
+
+### Enabling Eye Tracking in the GazeProvider
+In HoloLens v1, head gaze was used as primary pointing technique. 
+While head gaze is still available via the _GazeProvider_ in MRTK which is attached to your [Camera](https://docs.unity3d.com/ScriptReference/Camera.html), you can check to use eye gaze instead by ticking the _'Prefer Eye Tracking'_ checkbox as shown in the screenshot below.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_eyes_gazeprovider.png)
+
+
+### Simulating Eye Tracking in the Unity Editor
+You can simulate Eye Tracking input in the Unity Editor to ensure that events are correctly triggered before deploying the app to your HoloLens 2.
+The eye gaze signal is simulated by simply using the camera's location as eye gaze origin and the camera's forward vector as eye gaze direction.
+While this is great for initial testing, please note that it is not a good imitation for rapid eye movements. 
+For this, it is better to ensure frequent tests of your eye-based interactions on the HoloLens 2.
+
+1. **Enable simulated Eye Tracking**: 
+    - Navigate to your main _'MRTK Configuration Profile'_ -> _'Input System Profile'_ -> _'Data Providers'_ -> _'Input Simulation Service'_.
+    - Check the _'Simulate Eye Position'_ checkbox.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_eyes_simulate.png)
+    
+2. **Disable default head gaze cursor**: 
+In general, we recommend to avoid showing an eye gaze cursor or if you insist on showing one to make it _very_ subtle.
+Check out our [eye gaze cursor tutorial](EyeTracking_Cursor.md) for more information on how to best handle it.
+We do recommend to hide the default head gaze cursor that is attached to the MRTK gaze pointer profile by default.
+    - Navigate to your main _'MRTK Configuration Profile'_ -> _'Input System Profile'_ -> _'PointerSettings.PointerProfile'_
+    - At the bottom of the _'PointerProfile'_, you should assign an invisible cursor prefab to the _'GazeCursor'_. If you downloaded the MRTK Examples folder, you can simply reference the included -'EyeGazeCursor'_ prefab.
+
+![MRTK](../../External/ReadMeImages/EyeTracking/mrtk_setup_eyes_gazesettings.png)
+
+### Accessing eye gaze data
+Now that your scene is set up to use Eye Tracking, let's take a look at how to access it in your scripts: 
+[Accessing Eye Tracking Data in your Unity Script](EyeTracking_EyeGazeProvider.md).
+
+ 
+### Testing your Unity app on a HoloLens 2
+Building your app with Eye Tracking should be similar to how you would compile other HoloLens 2 MRTK apps. 
+The only difference is that the *'Gaze Input'* capability is unfortunately not yet supported by Unity under 'Player Settings -> Publishing Settings -> Capabilities'. 
+To use Eye Tracking on your HoloLens 2 device, you need to manually edit the package manifest that is part of your build Visual Studio project.
+So, follow these steps:
+1. Build your Unity project as you would normally do for _HoloLens 2_.
+2. Open your compiled Visual Studio project and then open the _'Package.appxmanifest'_ in your solution.
+3. Make sure to tick the _'Gaze Input'_ checkbox under _Capabilities_.
+
+![Enabling Gaze Input in Visual Studio](../../External/ReadMeImages/EyeTracking/mrtk_et_gazeinput.jpg)
+
+---
+[Back to "Eye Tracking in the MixedRealityToolkit"](EyeTracking_Main.md)

Documentation/EyeTracking/EyeTracking_BasicSetup.md

@@ -0,0 +1,8 @@
+# To show or not to show an eye gaze cursor
+
+This page covers design guidelines on when and how to use an eye gaze cursor. 
+
+<!-- TODO: Add more infos -- >
+
+---
+[Back to "Eye Tracking in the MixedRealityToolkit"](EyeTracking_Main.md)

Documentation/EyeTracking/EyeTracking_Cursor.md

@@ -0,0 +1,56 @@
+# Eye Tracking Examples in MRTK
+This page covers how to get quickly started with using Eye Tracking in MRTK by building on our provided MRTK example package.
+The samples let you experience one of our new magical input capabilities: **Eye Tracking**! 
+The demo include a number of different use cases for seamlessly combining information about what you are looking at with **Voice** and **Hand** input. 
+This enables users to quickly and effortlessly select and move holograms across their view simply by looking at a target and saying _'Select'_ or performing a hand gesture. 
+The demos also include an example for eye-gaze-directed scroll, pan and zoom of text and images on a slate. 
+Finally, an example is provided for recording and visualizing the user's visual attention on a 2D slate.
+
+
+## Overview of MRTK Eye Tracking Samples
+
+[**Eye-Supported Target Selection**](EyeTracking_TargetSelection.md)
+
+This tutorial showcases the ease of accessing eye gaze data to select targets. 
+It includes an example for subtle yet powerful feedback to provide confidence to the user that a target is focused while not being overwhelming.
+In addition, we showcase a simple example of smart notifications that automatically disappear after being read. 
+
+**Summary**: Fast and effortless target selections using a combination of Eyes, Voice and Hand input.
+
+<br>
+
+
+[**Eye-Supported Navigation**](EyeTracking_Navigation.md)
+
+Imagine you are reading some information on a distant display or your e-reader and when you reach the end of the displayed text, the text automatically scrolls up to reveal more content. 
+Or how about magically zooming directly toward where you were looking at? 
+These are some of the examples showcased in this tutorial about eye-supported navigation.
+In addition, we added an example for hands-free rotation of 3D holograms by making them automatically rotate based on your current focus. 
+
+**Summary**: Scroll, Pan, Zoom, 3D Rotation using a combination of Eyes, Voice and Hand input.
+
+<br>
+
+
+[**Eye-Supported Positioning**](EyeTracking_Positioning.md)
+
+In this tutorial, we extend an input scenario called [Put that there](https://youtu.be/CbIn8p4_4CQ) dating back to research from the MIT Media Lab in the early 1980's with eye, hand and voice input.
+The idea is simple: Benefit from your eyes for fast target selection and positioning. 
+Simply look at a hologram and say _'put this'_, look over where you want to place it and say _'there!'_. 
+For more precisely positioning your hologram, you can use additional input from your hands, voice or controllers. 
+
+**Summary**: Positioning holograms using Eyes, Voice and Hand input (*drag-and-drop*). Eye-supported sliders using Eyes+Hands. 
+
+<br>
+
+
+[**Visualization of Visual Attention**](EyeTracking_Visualization.md)
+
+Information about where users look at is an immensely powerful tool to assess usability of a design and to identify problems in efficient work streams. 
+In this tutorial, we discuss different eye tracking visualizations and how they fit different needs. 
+We provide basic examples for logging and loading Eye Tracking data and examples for how to visualize them. 
+
+**Summary**: Two-dimensional attention map (heatmaps) on slates. Recording & replaying Eye Tracking data.
+
+---
+[Back to "Eye Tracking in the MixedRealityToolkit"](EyeTracking_Main.md)

Documentation/EyeTracking/EyeTracking_ExamplesOverview.md

@@ -0,0 +1,53 @@
+# Accessing Eye Tracking Data in your Unity Script
+
+The following assumes that you followed the steps for setting up Eye Tracking in your MRTK scene (see [Basic MRTK Setup to use Eye Tracking](EyeTracking_BasicSetup.md)).
+
+To access Eye Tracking data in your MonoBehaviour scripts is easy! Simply use *MixedRealityToolkit.InputSystem.EyeGazeProvider*.
+
+## MixedRealityToolkit.InputSystem.EyeGazeProvider
+While the *MixedRealityToolkit.InputSystem.EyeGazeProvider* provides several helpful variables, the key ones for Eye Tracking input are the following:
+
+- **UseEyeTracking**: 
+True if Eye Tracking hardware is available and the user has given permission to use Eye Tracking in the app. 
+
+- **IsEyeGazeValid**: 
+Indicates whether the current Eye Tracking data is valid. 
+It may be invalid due to exceeded timeout (should be robust to the user blinking though) or lack of tracking hardware or permissions.
+
+- **GazeOrigin**: 
+Origin of the gaze ray. 
+Please note that this will return the *head* gaze origin if 'IsEyeGazeValid' is false.
+
+- **GazeDirection**:
+Direction of the gaze ray. 
+This will return the *head* gaze direction if 'IsEyeGazeValid' is false.
+
+- **HitInfo**, **HitPosition**, **HitNormal**, etc.:
+Information about the currently gazed at target. 
+Again, if 'IsEyeGazeValid' is false, this will be based on the user's *head* gaze.
+
+
+## Examples for using MixedRealityToolkit.InputSystem.EyeGazeProvider
+Here is an example from the 
+[FollowEyeGaze.cs](xref:Microsoft.MixedReality.Toolkit.Examples.Demos.EyeTracking.FollowEyeGaze):
+
+- Get the point of a hologram that the user is looking at:
+```csharp
+// Show the object at the hit position of the user's eye gaze ray with the target.
+gameObject.transform.position = MixedRealityToolkit.InputSystem.EyeGazeProvider.HitPosition; 
+```
+
+
+
+- Showing a visual asset at a fixed distance from where the user is currently looking:
+```csharp
+// If no target is hit, show the object at a default distance along the gaze ray.
+gameObject.transform.position = 
+MixedRealityToolkit.InputSystem.EyeGazeProvider.GazeOrigin + 
+MixedRealityToolkit.InputSystem.EyeGazeProvider.GazeDirection.normalized * defaultDistanceInMeters;
+```
+
+
+
+---
+[Back to "Eye Tracking in the MixedRealityToolkit"](EyeTracking_Main.md)

Documentation/EyeTracking/EyeTracking_EyeGazeProvider.md

@@ -0,0 +1,10 @@
+![Eye Tracking in MRTK](../../External/ReadMeImages/EyeTracking/mrtk_et_compilation.png)
+# Eye Tracking in the Mixed Reality Toolkit
+
+The [Mixed Reality Toolkit](https://github.com/Microsoft/MixedRealityToolkit-Unity) (MRTK) supports amongst others _'HoloLens 2'_ which offers an exciting and powerful new input: Eye Tracking! 
+Eye Tracking enables users to quickly and effortlessly engage with holograms across their view and can make your system smarter by better identifying a user's intention.
+New to Eye Tracking? No problem! We have created a number of videos, tutorials and samples to get you started! 
+1. [Getting started with Eye Tracking in MRTK](EyeTracking_BasicSetup.md)
+
+2. [Building on the MRTK Eye Tracking samples](EyeTracking_ExamplesOverview.md)
+---