Adding support for hand based object manipulation

Holograms 211 contains a relatively simple implementation of hand based
object manipulation.  That implementation uses the world space deltas
to move holograms exactly as the hand moves.  This works fine in some
cases, but becomes limiting in more sophisticated manipulation
scenarios.

The implementation in this change is taken directly from 3D Viewer
and ActionGrams and supports more sophisticated movement scenarios
which combine both gesture and gaze.  For example, with this new
implementation if you manipulate a hologram and then turn your head
the hologram will move with your head, while still allowing you to make
adjustments with your hands.  If you manipulate a hologram and then
walk somewhere else the hologram will move with you as long as you
maintain the gesture.  This allows for users to make large adjustments
using their gaze and position, while making smaller adjustments with
their hands, resulting in more intuitive and effective manipulation.

Author: Mike Thomas

Assets/HoloToolkit/Input/README.md

@@ -72,10 +72,24 @@ Stabilize the user's gaze to account for head jitter.
 **StabilityVarianceWeight** Stability variance weight multiplier factor.
 
 #### GestureManager.cs
-GestureManager creates a gesture recognizer and signs up for a tap gesture. When a tap gesture is detected, GestureManager uses GazeManager to find the game object.
-GestureManager then sends a message to that game object. 
+GestureManager provides access to several different input gestures, including Tap and Manipulation. 
 
-It also has an **OverrideFocusedObject** which lets you send gesture input to a specific object by overriding the gaze.
+When a tap gesture is detected, GestureManager uses GazeManager to find the game object.
+GestureManager then sends a message to that game object.  It also has an **OverrideFocusedObject** which lets you send gesture input to a specific object by overriding the gaze.
+
+Using Manipulation requires subscribing to the ManipulationStarted events and then querying information about the manipulation gesture via ManipulationOffset and ManipulationHandPosition.  See GestureManipulator for an example.
+
+#### GestureManipulator.cs
+A component for moving an object via the GestureManager manipulation gesture.
+
+When an active GestureManipulator component is attached to a GameObject it will subscribe 
+to GestureManager's manipulation gestures, and move the GameObject when a ManipulationGesture occurs. 
+If the GestureManipulator is disabled it will not respond to any manipulation gestures. 
+ 
+This means that if multiple GestureManipulators are active in a given scene when a manipulation 
+gesture is performed, all the relevant GameObjects will be moved.  If the desired behavior is that only 
+a single object be moved at a time, it is recommended that objects which should not be moved disable 
+their GestureManipulators, then re-enable them when necessary (e.g. the object is focused). 
 
 #### HandGuidance.cs
 Show a GameObject when a gesturing hand is about to lose tracking.

Assets/HoloToolkit/Input/Scripts/GestureManager.cs

@@ -3,17 +3,40 @@
 
 using UnityEngine;
 using UnityEngine.VR.WSA.Input;
+using System.Collections.Generic;
 
 namespace HoloToolkit.Unity
 {
     /// <summary>
-    /// GestureManager creates a gesture recognizer and signs up for a tap gesture.
+    /// GestureManager provides access to several different input gestures, including
+    /// Tap and Manipulation.
+    /// </summary>
+    /// <remarks>
     /// When a tap gesture is detected, GestureManager uses GazeManager to find the game object.
     /// GestureManager then sends a message to that game object.
-    /// </summary>
+    /// 
+    /// Using Manipulation requires subscribing the the ManipulationStarted events and then querying
+    /// information about the manipulation gesture via ManipulationOffset and ManipulationHandPosition
+    /// </remarks>
     [RequireComponent(typeof(GazeManager))]
     public partial class GestureManager : Singleton<GestureManager>
     {
+        /// <summary>
+        /// Occurs when a manipulation gesture has started
+        /// </summary>
+        public System.Action ManipulationStarted;
+
+        /// <summary>
+        /// Occurs when a manipulation gesture ended as a result of user input
+        /// </summary>
+        public System.Action ManipulationCompleted;
+
+        /// <summary>
+        /// Occurs when a manipulated gesture ended as a result of some other condition.
+        /// (e.g. the hand being used for the gesture is no longer visible).
+        /// </summary>
+        public System.Action ManipulationCanceled;
+
         /// <summary>
         /// Key to press in the editor to select the currently gazed hologram
         /// </summary>
@@ -24,66 +47,180 @@ public partial class GestureManager : Singleton<GestureManager>
         /// set the override focused object.
         /// If its null, then the gazed at object will be selected.
         /// </summary>
-        public GameObject OverrideFocusedObject
-        {
-            get; set;
-        }
-
+        public GameObject OverrideFocusedObject { get; set; }
+        
         /// <summary>
         /// Gets the currently focused object, or null if none.
         /// </summary>
-        public GameObject FocusedObject
+        public GameObject FocusedObject { get; private set; }
+
+        /// <summary>
+        /// Whether or not a manipulation gesture is currently in progress
+        /// </summary>
+        public bool ManipulationInProgress { get; private set; }
+
+        /// <summary>
+        /// The offset of the hand from its position at the beginning of 
+        /// the currently active manipulation gesture, in world space.  Not valid if
+        /// a manipulation gesture is not in progress
+        /// </summary>
+        public Vector3 ManipulationOffset { get; private set; }
+
+        /// <summary>
+        /// The world space position of the hand being used for the current manipulation gesture.  Not valid
+        /// if a manipulation gesture is not in progress.
+        /// </summary>
+        public Vector3 ManipulationHandPosition
         {
-            get { return focusedObject; }
+            get
+            {
+                Vector3 handPosition = Vector3.zero;
+                currentHandState.properties.location.TryGetPosition(out handPosition);
+                return handPosition;
+            }
         }
 
         private GestureRecognizer gestureRecognizer;
-        private GameObject focusedObject;
+        // We use a separate manipulation recognizer here because the tap gesture recognizer cancels
+        // capturing gestures whenever the GazeManager focus changes, which is not the behavior
+        // we want for manipulation
+        private GestureRecognizer manipulationRecognizer;
+
+        private bool HandPressed { get { return pressedHands.Count > 0; } }
+        private HashSet<uint> pressedHands = new HashSet<uint>();
+
+        private InteractionSourceState currentHandState;
 
         void Start()
         {
+            InteractionManager.SourcePressed += InteractionManager_SourcePressed;
+            InteractionManager.SourceReleased += InteractionManager_SourceReleased;
+            InteractionManager.SourceUpdated += InteractionManager_SourceUpdated;
+            InteractionManager.SourceLost += InteractionManager_SourceLost;
+
             // Create a new GestureRecognizer. Sign up for tapped events.
             gestureRecognizer = new GestureRecognizer();
             gestureRecognizer.SetRecognizableGestures(GestureSettings.Tap);
 
+            manipulationRecognizer = new GestureRecognizer();
+            manipulationRecognizer.SetRecognizableGestures(GestureSettings.ManipulationTranslate);
+
             gestureRecognizer.TappedEvent += GestureRecognizer_TappedEvent;
 
+            manipulationRecognizer.ManipulationStartedEvent += ManipulationRecognizer_ManipulationStartedEvent;
+            manipulationRecognizer.ManipulationUpdatedEvent += ManipulationRecognizer_ManipulationUpdatedEvent;
+            manipulationRecognizer.ManipulationCompletedEvent += ManipulationRecognizer_ManipulationCompletedEvent;
+            manipulationRecognizer.ManipulationCanceledEvent += ManipulationRecognizer_ManipulationCanceledEvent;
+
             // Start looking for gestures.
             gestureRecognizer.StartCapturingGestures();
+            manipulationRecognizer.StartCapturingGestures();
         }
 
-        private void OnTap()
+
+
+        private void InteractionManager_SourcePressed(InteractionSourceState state)
+        {
+            if (!HandPressed)
+            {
+                currentHandState = state;
+            }
+
+            pressedHands.Add(state.source.id);
+        }
+
+        private void InteractionManager_SourceUpdated(InteractionSourceState state)
         {
-            if (focusedObject != null)
+            if (HandPressed && state.source.id == currentHandState.source.id)
             {
-                focusedObject.SendMessage("OnSelect");
+                currentHandState = state;
             }
         }
 
+        private void InteractionManager_SourceReleased(InteractionSourceState state)
+        {
+            pressedHands.Remove(state.source.id);
+        }
+
+        private void InteractionManager_SourceLost(InteractionSourceState state)
+        {
+            pressedHands.Remove(state.source.id);
+        }
+
         private void GestureRecognizer_TappedEvent(InteractionSourceKind source, int tapCount, Ray headRay)
         {
             OnTap();
         }
 
+        private void OnTap()
+        {
+            if (FocusedObject != null)
+            {
+                FocusedObject.SendMessage("OnSelect");
+            }
+        }
+
+        private void ManipulationRecognizer_ManipulationStartedEvent(InteractionSourceKind source, Vector3 cumulativeDelta, Ray headRay)
+        {
+            // Don't start another manipulation gesture if one is already underway
+            if (!ManipulationInProgress)
+            {
+                OnManipulation(true, cumulativeDelta);
+                if (ManipulationStarted != null)
+                {
+                    ManipulationStarted();
+                }
+            }
+        }
+
+        private void ManipulationRecognizer_ManipulationUpdatedEvent(InteractionSourceKind source, Vector3 cumulativeDelta, Ray headRay)
+        {
+            OnManipulation(true, cumulativeDelta);
+        }
+
+        private void ManipulationRecognizer_ManipulationCompletedEvent(InteractionSourceKind source, Vector3 cumulativeDelta, Ray headRay)
+        {
+            OnManipulation(false, cumulativeDelta);
+            if (ManipulationCompleted != null)
+            {
+                ManipulationCompleted();
+            }
+        }
+
+        private void ManipulationRecognizer_ManipulationCanceledEvent(InteractionSourceKind source, Vector3 cumulativeDelta, Ray headRay)
+        {
+            OnManipulation(false, cumulativeDelta);
+            if (ManipulationCanceled != null)
+            {
+                ManipulationCanceled();
+            }
+        }
+
+        private void OnManipulation(bool inProgress, Vector3 offset)
+        {
+            ManipulationInProgress = inProgress;
+            ManipulationOffset = offset;
+        }
+
         void LateUpdate()
         {
-            GameObject oldFocusedObject = focusedObject;
+            GameObject oldFocusedObject = FocusedObject;
 
             if (GazeManager.Instance.Hit &&
                 OverrideFocusedObject == null &&
                 GazeManager.Instance.HitInfo.collider != null)
             {
                 // If gaze hits a hologram, set the focused object to that game object.
                 // Also if the caller has not decided to override the focused object.
-                focusedObject = GazeManager.Instance.HitInfo.collider.gameObject;
+                FocusedObject = GazeManager.Instance.HitInfo.collider.gameObject;
             }
             else
             {
                 // If our gaze doesn't hit a hologram, set the focused object to null or override focused object.
-                focusedObject = OverrideFocusedObject;
+                FocusedObject = OverrideFocusedObject;
             }
 
-            if (focusedObject != oldFocusedObject)
+            if (FocusedObject != oldFocusedObject)
             {
                 // If the currently focused object doesn't match the old focused object, cancel the current gesture.
                 // Start looking for new gestures.  This is to prevent applying gestures from one hologram to another.
@@ -103,6 +240,17 @@ void OnDestroy()
         {
             gestureRecognizer.StopCapturingGestures();
             gestureRecognizer.TappedEvent -= GestureRecognizer_TappedEvent;
+
+            manipulationRecognizer.StopCapturingGestures();
+            manipulationRecognizer.ManipulationStartedEvent -= ManipulationRecognizer_ManipulationStartedEvent;
+            manipulationRecognizer.ManipulationUpdatedEvent -= ManipulationRecognizer_ManipulationUpdatedEvent;
+            manipulationRecognizer.ManipulationCompletedEvent -= ManipulationRecognizer_ManipulationCompletedEvent;
+            manipulationRecognizer.ManipulationCanceledEvent -= ManipulationRecognizer_ManipulationCanceledEvent;
+
+            InteractionManager.SourcePressed -= InteractionManager_SourcePressed;
+            InteractionManager.SourceReleased -= InteractionManager_SourceReleased;
+            InteractionManager.SourceUpdated -= InteractionManager_SourceUpdated;
+            InteractionManager.SourceLost -= InteractionManager_SourceLost;
         }
     }
 }