Doesn't work on Meta Quest devices

[FluorescentHallucinogen]

As announced at Meta Connect 2022, Meta Quest Browser has added support for WebXR mixed reality services on Meta Quest 2 and Meta Quest Pro headsets. See https://developer.oculus.com/documentation/web/webxr-mixed-reality/ for more info.

Demos https://immersive-web.github.io/webxr-samples/immersive-ar-session.html and https://aframe-light-and-shadow.glitch.me work like a charm in Meta Quest Browser on my Meta Quest 2 headset.

But https://modelviewer.dev/examples/augmentedreality/ demos unfortunately don't work.

Steps to reproduce:

1. Open https://modelviewer.dev/examples/augmentedreality/ in Meta Quest Browser.
2. Click "View in your space" button.

Result:

1. The "View in your space" button just disappears after clicking.
2. The following errors are printed in the Meta Quest Browser console (I've used Chrome DevTools Remote debug to see the logs):

model-viewer.js:56006 Attempting to present in AR with WebXR...
model-viewer.js:47888 Feature 'hit-test' is not permitted by permissions policy
resolveARSession @ model-viewer.js:47888
present @ model-viewer.js:47948
await in present (async)
[enterARWithWebXR] @ model-viewer.js:56012
await in [enterARWithWebXR] (async)
activateAR @ model-viewer.js:55952
<computed> @ model-viewer.js:55882
model-viewer.js:47888 Feature 'dom-overlay' is not supported for mode: immersive-ar
resolveARSession @ model-viewer.js:47888
present @ model-viewer.js:47948
await in present (async)
[enterARWithWebXR] @ model-viewer.js:56012
await in [enterARWithWebXR] (async)
activateAR @ model-viewer.js:55952
<computed> @ model-viewer.js:55882
model-viewer.js:47888 Feature 'light-estimation' is not supported for mode: immersive-ar
resolveARSession @ model-viewer.js:47888
present @ model-viewer.js:47948
await in present (async)
[enterARWithWebXR] @ model-viewer.js:56012
await in [enterARWithWebXR] (async)
activateAR @ model-viewer.js:55952
<computed> @ model-viewer.js:55882
model-viewer.js:56015 Error while trying to present in AR with WebXR
[enterARWithWebXR] @ model-viewer.js:56015
await in [enterARWithWebXR] (async)
activateAR @ model-viewer.js:55952
<computed> @ model-viewer.js:55882
model-viewer.js:56016 DOMException: Failed to execute 'requestSession' on 'XRSystem': The session request contains invalid requiredFeatures and could not be fullfilled.
    at iu.resolveARSession (https://modelviewer.dev/node_modules/@google/model-viewer/dist/model-viewer.min.js:615:2125)
    at iu.present (https://modelviewer.dev/node_modules/@google/model-viewer/dist/model-viewer.min.js:615:3289)
    at async [enterARWithWebXR] (https://modelviewer.dev/node_modules/@google/model-viewer/dist/model-viewer.min.js:1125:20592)
    at async s.activateAR (https://modelviewer.dev/node_modules/@google/model-viewer/dist/model-viewer.min.js:1125:19539)
[enterARWithWebXR] @ model-viewer.js:56016
await in [enterARWithWebXR] (async)
activateAR @ model-viewer.js:55952
<computed> @ model-viewer.js:55882
model-viewer.js:47998 Error while trying to end WebXR AR session
stopPresenting @ model-viewer.js:47998
[enterARWithWebXR] @ model-viewer.js:56017
await in [enterARWithWebXR] (async)
activateAR @ model-viewer.js:55952
<computed> @ model-viewer.js:55882
model-viewer.js:47999 TypeError: Cannot read properties of null (reading 'end')
    at iu.stopPresenting (model-viewer.js:47994:39)
    at [enterARWithWebXR] (model-viewer.js:56017:50)
    at async s.activateAR (model-viewer.js:55951:22)
stopPresenting @ model-viewer.js:47999
[enterARWithWebXR] @ model-viewer.js:56017
await in [enterARWithWebXR] (async)
activateAR @ model-viewer.js:55952
<computed> @ model-viewer.js:55882
model-viewer.js:56019 Falling back to next ar-mode
[enterARWithWebXR] @ model-viewer.js:56019
await in [enterARWithWebXR] (async)
activateAR @ model-viewer.js:55952
<computed> @ model-viewer.js:55882
model-viewer.js:55958 No AR Mode can be activated. This is probably due to missing configuration or device capabilities
activateAR @ model-viewer.js:55958
[enterARWithWebXR] @ model-viewer.js:56021
await in [enterARWithWebXR] (async)
activateAR @ model-viewer.js:55952
<computed> @ model-viewer.js:55882

[FluorescentHallucinogen]

If I understand it correctly, the problem is in the 'hit-test', 'dom-overlay', and 'light-estimation' features.

But 'hit-test' is the only required feature (that produces error), 'dom-overlay' and 'light-estimation' are optional features (that produce warnings).

And feature 'hit-test' is not permitted by permissions policy.

[elalish]

I'd recommend Three.js - seems generically useful. You'll also get better feedback from their larger community.

[cabanier]

I'd recommend Three.js - seems generically useful. You'll also get better feedback from their larger community.

Yes, model-viewer is designed on top of three.js so it's unlikely that we will pull in another framework for this feature.

[FluorescentHallucinogen]

I've opened mrdoob/three.js#25619.

[FluorescentHallucinogen]

@elalish @cabanier I've implemented a simpler (without bounding box transform controls) solution to the problem. PTAL at #4643. ;)

The code is based on https://github.com/mrdoob/three.js/blob/master/examples/webxr_xr_dragging.html.

It works! :)

But I need some help with:

1. For some reason the controller models are small and are not in the right place, but far away from me, next to the 3D model. See https://youtu.be/PXChXTSLBJE. I assume it's because of the scene resizing.
2. I've used any and @ts-ignore in multiple places in the code. Trying to use the correct types leads to more and more TypeScript errors. :(

The pull request allows edits by maintainers.

After that, I plan to implement model resizing using two controllers/hands. ;)

[cabanier]

For some reason the controller models are small and are not in the right place, but far away from me, next to the 3D model. See https://youtu.be/PXChXTSLBJE. I assume it's because of the scene resizing.

I agree. There must be an additional resize. Are you sure your controller object are attached on the top node? Is there scaling applied to that node?