docs: pitfalls, faw, anchors tutorial, dom overlay tutorial, hit test tutorial, store tutorial

Author: bbohlender

README.md

@@ -6,7 +6,6 @@
 <h3 align="center">Turn any R3F app into an interactive immersive experience.</h3>
 <br/>
 
-
 <p align="center">
   <a href="https://npmjs.com/package/@react-three/xr" target="_blank">
     <img src="https://img.shields.io/npm/v/@react-three/xr?style=flat&colorA=000000&colorB=000000" alt="NPM" />
@@ -29,7 +28,7 @@ npm install three @react-three/fiber @react-three/xr@latest
 ## What does it look like?
 
 | A simple scene with a mesh that toggles its material color between `"red"` and `"blue"` when clicked through touching or pointing. | ![recording of interacting with the code below](./docs/getting-started/basic-example.gif) |
-|-|-|
+| ---------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
 
 ```tsx
 import { Canvas } from '@react-three/fiber'
@@ -40,17 +39,19 @@ const store = createXRStore()
 
 export function App() {
   const [red, setRed] = useState(false)
-  return <>
-    <button onClick={() => store.enterAR()}>Enter AR</button>
-    <Canvas>
-      <XR store={store}>
-        <mesh pointerEventsType={{ deny: 'grab' }} onClick={() => setRed(!red)} position={[0, 1, -1]}>
-          <boxGeometry />
-          <meshBasicMaterial color={red ? 'red' : 'blue'} />
-        </mesh>
-      </XR>
-    </Canvas>
-  </>
+  return (
+    <>
+      <button onClick={() => store.enterAR()}>Enter AR</button>
+      <Canvas>
+        <XR store={store}>
+          <mesh pointerEventsType={{ deny: 'grab' }} onClick={() => setRed(!red)} position={[0, 1, -1]}>
+            <boxGeometry />
+            <meshBasicMaterial color={red ? 'red' : 'blue'} />
+          </mesh>
+        </XR>
+      </Canvas>
+    </>
+  )
 }
 ```
 
@@ -64,24 +65,25 @@ export function App() {
 
 ## Tutorials
 
+- 💾 [Store](https://docs.pmnd.rs/xr/tutorials/store)
 - 👌 [Interactions](https://docs.pmnd.rs/xr/tutorials/interactions)
 - 🔧 [Options](https://docs.pmnd.rs/xr/tutorials/options)
 - 🧊 [Object Detection](https://docs.pmnd.rs/xr/tutorials/object-detection)
 - ✴ [Origin](https://docs.pmnd.rs/xr/tutorials/origin)
 - 🪄 [Teleport](https://docs.pmnd.rs/xr/tutorials/teleport)
 - 🕹️ [Gamepad](https://docs.pmnd.rs/xr/tutorials/gamepad)
 - 🎮 [Custom Controller/Hands/...](https://docs.pmnd.rs/xr/tutorials/custom-inputs)
+- ⚓️ [Anchors](https://docs.pmnd.rs/xr/tutorials/anchors)
+- 📱 [Dom Overlays](https://docs.pmnd.rs/xr/tutorials/dom-overlay)
+- 🎯 [Hit Test](https://docs.pmnd.rs/xr/tutorials/hit-test)
 - ⛨ [Guards](https://docs.pmnd.rs/xr/tutorials/guards)
 
 ## Roadmap
 
 - 🤳 XR Gestures
 - ➕ Multimodal
-- ⚓️ Anchors
 - 📺 Layers
-- 📱 Dom Overlays
 - 🕺 Tracked Body
-- 🎯 Hit Test
 - ↕ react-three/controls
 
 ## Migration guides

docs/advanced/pitfalls.md

@@ -10,9 +10,34 @@ In contrast to non-immersive 3D applications, the camera transformation in MR/VR
 
 ## Reading the camera position in XR
 
-When using @react-three/xr, the useThree hook will return the XR camera when in XR. Therefore, the returned camera and the `getWorldPosition` function can be used to get the world position of the xr camera, as well as the normal camera, when not in XR. 
+When using @react-three/xr, the useThree hook will return the XR camera when in XR. Therefore, the returned camera and the `getWorldPosition` function can be used to get the world position of the xr camera, as well as the normal camera, when not in XR.
 
 ```tsx
-const camera = useThree(state => state.camera)
+const camera = useThree((state) => state.camera)
 useFrame(() => camera.getWorldPosition(target))
-```
+```
+
+## Missing Https
+
+If you are trying to enter the AR or VR modus and nothing is happening, make sure that you are accessing the website using `https://`.
+In case you are using vite, we recommend using the `@vitejs/plugin-basic-ssl` to try out your vite application on your device while developing.
+
+## Missing XR component
+
+If you made sure that the website is accessed using `https://` and still nothing happens when executing `enterAR` or `enterVR`, it is likely that the `<XR>` component is missing. Be sure to add the `<XR>` component directly into the `<Canvas>` and make sure both the `<Canvas>` and the `<XR>` component are present when the button is pressed.
+
+## Entering while loading content
+
+If you cannot enter the VR or AR experience while the assets in your scene are loading, make sure to place a suspense boundary around your scene. With this setup, the `<XR>` component stays mounted while your scene loads.
+
+```tsx
+<Canvas>
+  <XR>
+    <Suspense>... your scene</Suspense>
+  </XR>
+</Canvas>
+```
+
+## XRSpace
+
+If you are placing `<XRSpace>` components outside of the `<XROrigin>` while changing the transformation of the `<XROrigin>` (e.g. by setting `<XROrigin position={[0,1,0]} />`), the elements rendered inside of the `<XRSpace>` will not be transformed with the origin. If the transformations of the origin should be applied to the `<XRSpace>`, make sure to place those components inside the `<XROrigin>`. Not placing `<XRSpace>` components into the `<XROrigin>` can be useful in scenarios where you want to move the `<XROrigin>` independently from the `<XRSpace>`. For instance, building a virtual elevator where your actual room is duplicated into the x-axis so that you can use the elevator to travel between multiple instances of your room.

docs/getting-started/all-components.md

@@ -60,6 +60,10 @@ Component that positions its children in the provided space.
 
 Component that allows to declare its children as teleport targets.
 
+### `XRHitTest`
+
+Component that allows to emit hit tests from its position in the scene graph.
+
 ## Pointer
 
 The core interaction concept is based on (touch/grab/ray) pointers.

docs/getting-started/all-hooks.md

@@ -26,6 +26,27 @@ Hook for checking if a session mode is supported.
 
 Hook for getting the session visibility state.
 
+### `useXRHitTestSource`
+
+Hook for creating a hit test source originating from the provided object or xr space.
+
+### `useXRHitTest`
+
+Hook that allows to set up a continous hit test originating from the provided object or xr space.
+
+### `useXRRequestHitTest`
+
+Hook that returns a function to request a single hit test.
+
+
+### `useRequestXRAnchor`
+
+Hook that returns a function that allows to request a xr anchor.
+
+### `useXRAnchor`
+
+Hook for requesting and storing a single xr anchor.
+
 ## Space
 
 @react-three/xr provides several hooks to synchronize the space provided by WebXR with the scene.
@@ -59,10 +80,14 @@ Hook for creating a grab pointer.
 
 Hook for creating a ray pointer.
 
-### `usePointerXRSessionEvent`
+### `usePointerXRInputSourceEvents`
 
 Hook for binding the XR session events such as `selectstart` to the provided pointer down/up events.
 
+### `useXRInputSourceEvent`
+
+Hook for listening to xr input source events.
+
 ## Inputs
 
 Building custom inputs requires access and bindings to the events. The following hook allow to access the inputs state and bind function to events.

docs/getting-started/development-setup.md

@@ -0,0 +1,42 @@
+---
+title: Development Setup
+description: Setup your development environment for building XR experiences.
+nav: 2
+---
+
+Building WebXR experiences has all the advantages of building for the web, with a lot of tool chains, debugging tools, and resources available. 
+Because there are so many tools available, it is often hard to choose a development setup. Therefore, we would like to recommend you a development setup for developing with `react-three/xr`.
+
+## 1. Build tool: [vite](https://vitejs.dev/)
+
+Vite is easy to set up and has all the features we need. We recommend using the packages `@vitejs/plugin-react` to enable hot module reloading and `@vitejs/plugin-basic-ssl` to enable `https`, which is required for any WebXR experience.
+
+A basic `vite.config.ts` would look like this:
+
+```ts
+import { defineConfig } from 'vite'
+import react from '@vitejs/plugin-react'
+import basicSsl from '@vitejs/plugin-basic-ssl'
+
+export default defineConfig({
+  plugins: [react(), basicSsl()],
+})
+```
+
+## 2. emulator: [iwer/devui](https://github.com/meta-quest/immersive-web-emulation-runtime/tree/main/devui)
+
+Developing WebXR experiences often requires testing WebXR-specific features, which either require an actual device or an emulator. An emulator allows testing without a specific device and without continuously switching a headset on and off.
+
+`react-three/xr` includes the [iwer/devui](https://github.com/meta-quest/immersive-web-emulation-runtime/tree/main/devui) emulator out of the box. The emulator builds on [IWER by meta](https://github.com/meta-quest/immersive-web-emulation-runtime/) and adds a easy to use overlay on top of your application. The emulator is activated if no WebXR support is detected on `localhost` or by pressing `Window/Command + Alt/Option + E`.
+
+![iwer/devui](./emulator.gif)
+
+The existing [Immersive Web Emulator](https://chromewebstore.google.com/detail/immersive-web-emulator/cgffilbpcibhmcfbgggfhfolhkfbhmik) and the previous [WebXR API Emulator](https://chromewebstore.google.com/detail/webxr-api-emulator/mjddjgeghkdijejnciaefnkjmkafnnje?hl=de) are not supported because they do not comply to the WebXR spec. If you have one of those installed and active, please turn them off, as they will prevent activiting the built-in emulator. 
+
+Another supported alternative is the Apple Vision Pro Simulator.
+
+### 3. ADB
+
+Sometimes, a WebXR experience works floorless in the emulator but fails on the actual device. In this case we recommend using Android Device Bridge (ADB) to debug your experience from another device. For example, to remotely debug a WebXR experience on a Meta Quest 3, the Meta Quest 3 must be in developer mode, and remote debugging must be enabled. Then, the Meta Quest 3 must be connected to a PC via a USB cable with ADB installed. With this setup, the Chrome browser can now be used to access `chrome://inspect` to inspect specific tabs on the device remotely.
+
+If you are using the Apple Vision Pro, there are similar features to remotely debug a WebXR experience on the Apple Vision Pro from a Safari browser on another device. 

docs/getting-started/emulator.gif

@@ -0,0 +1,26 @@
+---
+title: FAQ
+description: Frequently asked questions about react-three/xr.
+nav: 5
+---
+
+## How can I exit an XR session?
+
+```ts
+store.getState().session?.end()
+```
+
+## WebGPU
+
+WebGPU is finding its way to more and more devices. However, AR and VR devices do not yet implement WebGPU for WebXR, which requires the [WebXR-WebGPU-Binding](https://github.com/immersive-web/WebXR-WebGPU-Binding/blob/main/explainer.md). Therefore, WebGPU is not yet usable for WebXR in general.
+
+## How can I put HTML in my XR scene?
+
+If you are targeting only handheld AR experiences (e.g., for smartphones), you can use dom overlay. Here's a [tutorial for using XRDomOverlays](../tutorials/dom-overlay.md) in your `react-three/xr` experience.
+
+For non-handheld VR and AR experiences, you can use [react-three/uikit](https://github.com/pmndrs/uikit), which renders user interfaces directly inside the 3D scene and is aligned with HTML and CSS concepts.
+
+## Does it work on iOS?
+
+WebXR for VR experiences is supported on Safari for Apple Vision Pro. 
+WebXR is not supported on iOS Safari yet. The alternative is to use products such as [Variant Launch](https://launch.variant3d.com/), which allow to build WebXR experiences for iOS.

docs/getting-started/faq.md

@@ -46,24 +46,25 @@ export function App() {
 
 ## Tutorials
 
+- 💾 [Store](../tutorials/store.md)
 - 👌 [Interactions](../tutorials/interactions.md)
 - 🔧 [Options](../tutorials/options.md)
 - 🧊 [Object Detection](../tutorials/object-detection.md)
 - ✴ [Origin](../tutorials/origin.md)
 - 🪄 [Teleport](../tutorials/teleport.md)
 - 🕹️ [Gamepad](../tutorials/gamepad.md)
 - 🎮 [Custom Controller/Hands/...](../tutorials/custom-inputs.md)
+- ⚓️ [Anchors](../tutorials/anchors.md)
+- 📱 [Dom Overlay](../tutorials/dom-overlay.md)
+- 🎯 [Hit Test](../tutorials/hit-test.md)
 - ⛨ [Guards](../tutorials/guards.md)
 
 ## Roadmap
 
 - 🤳 XR Gestures
 - ➕ Multimodal
-- ⚓️ Anchors
 - 📺 Layers
-- 📱 Dom Overlays
 - 🕺 Tracked Body
-- 🎯 Hit Test
 - ↕ react-three/controls
 
 ## Migration guides

docs/getting-started/introduction.md

@@ -0,0 +1,57 @@
+---
+title: Anchors
+description: How to create and manage anchors in your AR experience?
+nav: 12
+---
+
+Anchors allow to anchor virtual objects into the physical world in AR experiences. `react-three/xr` offers a multitude of ways to create and manage anchors. A simple solution is `useXRAnchor`, which works similarly to `useState` as it returns the current anchor and a function to request a new anchor as a tuple.
+
+```tsx
+const [anchor, requestAnchor] = useXRAnchor()
+```
+
+With the `requestAnchor` function, we can request an anchor relative to the `"world"`, a `"space"`, or a `"hitTestResult"`
+
+```tsx
+requestAnchor({ relativeTo: "space", space: ... })
+```
+
+Once the anchor is created, the `useXRAnchor` hook exposes it as `anchor`. We can now use this `anchor` to put content into it using the `<XRSpace>` component. 
+
+```tsx
+<XRSpace space={anchor.anchorSpace}>
+ ...your content
+</XRSpace>
+```
+
+The following example shows a `Anchor` component that uses the `useXRAnchor` hook and the `XRSpace` component to anchor a Box to the position of the right hand or controller when the respective hand or controller is selected (pinch/trigger).
+
+```tsx
+export function Anchor() {
+  const [anchor, requestAnchor] = useXRAnchor()
+  const controllerState = useXRControllerState('right')
+  const handState = useXRHandState('right')
+  const inputSource = controllerState?.inputSource ?? handState?.inputSource
+  useXRInputSourceEvent(
+    inputSource,
+    'select',
+    async () => {
+      if (inputSource == null) {
+        return
+ }
+      requestAnchor({ relativeTo: 'space', space: inputSource.targetRaySpace })
+ },
+ [requestAnchor, inputSource],
+ )
+  if (anchor == null) {
+    return null
+ }
+  return (
+    <XRSpace space={anchor.anchorSpace}>
+      <mesh scale={0.1}>
+        <boxGeometry />
+      </mesh>
+    </XRSpace>
+ )
+}
+```

docs/tutorials/anchors.md

@@ -0,0 +1,47 @@
+---
+title: Dom Overlay
+description: How to add HTML elements for hand-held AR experiences with Dom overlay?
+nav: 13
+---
+
+For hand-held AR experiences, such as those using a Smartphone, WebXR offers the dom overlay capability, allowing developers to use HTML code overlayed over the experience. In case scene 3D overlays or overlays in non-handheld AR/VR experiences are needed, check out [pmndrs/uikit](https://github.com/pmndrs/uikit).
+
+We can add dom overlay content to an experience using the `XRDomOverlay` component, which allows to write html code inside it. This HTML code will be overlayed over the hand-held AR experience.
+
+```tsx
+<XRDomOverlay
+  style={{ width: '100%', height: '100%', display: 'flex', alignItems: 'center', justifyContent: 'center' }}
+>
+  <div style={{ backgroundColor: 'red', padding: '1rem 2rem' }}>Hello World</div>
+</XRDomOverlay>
+```
+
+The following shows the complete code for a simple AR experience with a `Hello World` button that can toggle its color when clicked on. 
+
+```tsx
+const store = createXRStore()
+
+export function App() {
+  const [bool, setBool] = useState(false)
+  return (
+    <>
+      <button onClick={() => store.enterAR()}>Enter AR</button>
+      <Canvas>
+        <XR store={store}>
+          <ambientLight />
+          <XRDomOverlay
+            style={{ width: '100%', height: '100%', display: 'flex', alignItems: 'center', justifyContent: 'center' }}
+          >
+            <div
+              style={{ backgroundColor: bool ? 'red' : 'green', padding: '1rem 2rem' }}
+              onClick={() => setBool((b) => !b)}
+            >
+ Hello World
+            </div>
+          </XRDomOverlay>
+        </XR>
+      </Canvas>
+    </>
+ )
+}
+```