Eye tracking support

[j7j7]

Hi I was wondering if you could create a variant of the Nova Input Manager XR for Eye Tracking. (to be used on Quest pro)
I've tried a few times, and can get it working on a standard game object , but can't seem to get it to work with a Nova Interactable.
I can get a sphere gameobject to follow my gaze

I then created this script based on a simplified hands script.

However, what I want to be able to do is when it collides with a UIblock/interactable - i want it to trigger the same as the 'hover' function - not select. I want to then have another method to carry out the select function - either by pressing a button on the controller, or pinch function.

Any help/scripts would be appreciated.

using Nova;
using System;
using UnityEngine;

public class XRHandsInputManager : MonoBehaviour
{
    public static float ColliderRadius;
    public static Vector3 SpherePosition;

    [Tooltip("A sphere collider used for interaction detection")]
    public SphereCollider interactionSphere;

    private void Update()
    {
        SpherePosition = interactionSphere.transform.position;
        ColliderRadius = interactionSphere.radius;
        
        Interaction.Point(interactionSphere, 1);
    }
}

[j7j7]

Ok, ive got my game working with eye tracking now.
I decided to 'cheat' and use the normal ray that was working for the quest pro hitting a normal plane. which is sat on top of the UIBlock (without a mesh)
As I had developed a variant of the 'button selector' for Nova anyway- I was able to duplicate the onhover,ontrigger etc events ... and 'call' them if the eye-tracked objects is 'hit', so it does the same thing. and i didn't have to re-write all the buttons and options.

It worked ... and feels incredible to experience... just looking at a button and it hovers - feels like magic. So I now get why people are praising Apple Vision Pro so much ... because the UX is good.

It would be good to do this 'natively' with Nova interactions... to save having to have these duplicate 'planes' - even if not being rendered. Imagine the AppleXRdemo that's on here working with eye tracking .. :)

[adgray1]

Funny you should mention it :)

We had actually been working on just that when this request came in last week, but it admittedly took us a bit longer than anticipated to land on a solution that seemed reliable enough for what we wanted to deliver. The eye tracking data we get out of the Quest Pro is pretty noisy and doesn't always adjust with subtle eye movements, at least that was the case in our testing.

After some tuning, iterating, and setting expectations (e.g. small targets that are too close together won't work due to lack of eye ray precision), I think we managed to land on something we're mostly happy with* - with the asterisk being we only have one Quest Pro device on hand for testing.

VisionOSDemo.mp4

We published an APK too, if you want to just sideload it and try it for yourself! The video linked in the release description won't be available until tomorrow, but it's the same clip as above.

And if you want to try pulling the functionality into your own project, we recommend taking a look at the EyeTrackAndPinchInput object in the VisionOSDemo scene (full path: Assets\AppleXRConcept\VisionOSDemo.unity).

And please let us know what you think, especially if you run into issues regarding point, click, or drag reliability or have trouble using the setup in your own project! We would really appreciate the extra set of eyes (pun intended 😄).