Skip to content

Commit 2cc3989

Browse files
zjm-metameta-codesync[bot]
authored andcommitted
feat(examples): Add depth occlusion demo
Summary: Add example app demonstrating depth-based occlusion in AR. Creates occludable spheres, cubes, and cylinders that are hidden when passing behind real-world surfaces. Includes distance-grabbable interaction so objects can be moved to test occlusion from different positions. Reviewed By: cabanier Differential Revision: D93502318 fbshipit-source-id: 4adcd6b8f1283d7b6ab36638c35e7ef7418a509b
1 parent 1508ab8 commit 2cc3989

16 files changed

+2506
-0
lines changed
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Dependencies
2+
node_modules/
3+
.pnpm-debug.log*
4+
5+
# Build outputs
6+
dist/
7+
*.tsbuildinfo
8+
9+
# Environment files
10+
.env
11+
.env.local
12+
.env.production
13+
14+
# IDE files
15+
.vscode/
16+
.idea/
17+
*.swp
18+
*.swo
19+
20+
# OS files
21+
.DS_Store
22+
Thumbs.db
23+
24+
# Logs
25+
*.log
26+
npm-debug.log*
27+
yarn-debug.log*
28+
yarn-error.log*
29+
pnpm-debug.log*
30+
31+
# Runtime data
32+
pids
33+
*.pid
34+
*.seed
35+
*.pid.lock
36+
37+
# Coverage directory used by tools like istanbul
38+
coverage/
39+
*.lcov
40+
41+
# Temporary files
42+
.tmp/
43+
.temp/
44+
45+
# Editor directories and files
46+
.vscode/*
47+
!.vscode/extensions.json
48+
.idea
49+
*.suo
50+
*.ntvs*
51+
*.njsproj
52+
*.sln
53+
*.sw?
54+
55+
# Local SSL certificates (auto-generated)
56+
.cert/
57+
58+
# Meta Spatial Editor exports
59+
public/gltf/generated/
60+
public/glxf/
61+
public/ui/

examples/depth-occlusion/README.md

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
# Depth Occlusion Example
2+
3+
This example demonstrates depth-based occlusion in the Immersive Web SDK (IWSDK). Virtual objects are hidden when they pass behind real-world surfaces, creating a more realistic AR experience.
4+
5+
## What This Example Shows
6+
7+
- **Depth Sensing**: Configuring WebXR depth sensing with GPU-optimized float32 depth data
8+
- **Depth Occlusion**: Using the `DepthOccludable` component to automatically occlude virtual objects behind real-world geometry
9+
- **DepthSensingSystem**: Registering and configuring the system that drives depth data retrieval and occlusion rendering
10+
- **Interactive Objects**: Combining occlusion with distance grabbing and spatial anchors
11+
12+
### What's in the Example
13+
14+
The example creates three occludable primitives in an AR scene:
15+
16+
1. **Red sphere** at `(0, 0.8, -0.8)` with `DepthOccludable`
17+
2. **Green cube** at `(-0.4, 0.8, -0.6)` with `DepthOccludable`
18+
3. **Blue cylinder** at `(0.4, 0.8, -0.6)` without `DepthOccludable` (for comparison)
19+
20+
All objects are distance-grabbable and anchored, so you can move them around and observe occlusion in action. The `OcclusionDemoSystem` also applies a gentle rotation animation to occludable entities.
21+
22+
## Project Structure
23+
24+
```
25+
depth-occlusion/
26+
├── src/
27+
│ └── index.js # OcclusionDemoSystem and world setup
28+
├── index.html # HTML entry point
29+
├── vite.config.js # Vite configuration with HTTPS
30+
└── package.json # Dependencies
31+
```
32+
33+
## Quick Start
34+
35+
### Prerequisites
36+
37+
- Node.js 20.19.0+ and pnpm
38+
- An AR-capable device with depth sensing support (e.g., Meta Quest 3)
39+
40+
### Installation
41+
42+
```bash
43+
cd depth-occlusion
44+
pnpm install
45+
```
46+
47+
### Development
48+
49+
```bash
50+
# Start development server with HTTPS
51+
pnpm dev
52+
53+
# Build for production
54+
pnpm build
55+
56+
# Preview production build
57+
pnpm preview
58+
```
59+
60+
## How It Works
61+
62+
### 1. Configure Depth Sensing
63+
64+
Enable depth sensing in the WebXR session features:
65+
66+
```javascript
67+
World.create(document.getElementById('scene-container'), {
68+
xr: {
69+
sessionMode: SessionMode.ImmersiveAR,
70+
referenceSpace: ReferenceSpaceType.Unbounded,
71+
features: {
72+
depthSensing: {
73+
required: true,
74+
usage: 'gpu-optimized',
75+
format: 'float32',
76+
},
77+
hitTest: { required: true },
78+
anchors: { required: true },
79+
unbounded: { required: true },
80+
},
81+
},
82+
features: {
83+
grabbing: true,
84+
},
85+
});
86+
```
87+
88+
### 2. Register the DepthSensingSystem
89+
90+
```javascript
91+
world.registerSystem(DepthSensingSystem, {
92+
enabled: true,
93+
enableDepthTexture: true,
94+
enableOcclusion: true,
95+
useFloat32: true,
96+
});
97+
world.registerComponent(DepthOccludable);
98+
```
99+
100+
### 3. Mark Entities as Occludable
101+
102+
Add the `DepthOccludable` component to any entity that should be hidden behind real-world surfaces:
103+
104+
```javascript
105+
const entity = world.createTransformEntity(mesh);
106+
entity.addComponent(DepthOccludable);
107+
```
108+
109+
The `DepthSensingSystem` automatically injects occlusion shader code into the entity's materials when the component is added, and removes it when the component is removed. No manual shader setup is required.
110+
111+
### Depth Sensing Options
112+
113+
| Option | Type | Default | Description |
114+
|---|---|---|---|
115+
| `enabled` | boolean | `true` | Enable/disable the depth sensing system |
116+
| `enableDepthTexture` | boolean | `true` | Enable depth texture generation |
117+
| `enableOcclusion` | boolean | `true` | Enable occlusion rendering |
118+
| `useFloat32` | boolean | `true` | Use float32 depth values (vs luminance-alpha) |
119+
120+
### Depth Sensing Modes
121+
122+
The `depthSensing.usage` field controls how depth data is accessed:
123+
124+
- **`gpu-optimized`**: Depth data stays on the GPU as a texture array. Lower latency, preferred for occlusion.
125+
- **`cpu-optimized`**: Depth data is copied to CPU memory as a `DataTexture`. Useful if you need to read depth values on the CPU.
126+
127+
## Testing on Device
128+
129+
1. Start the dev server: `pnpm dev`
130+
2. On your Quest device, open the browser and navigate to `https://YOUR_LOCAL_IP:8081`
131+
3. Accept the self-signed certificate warning
132+
4. Enter AR mode
133+
5. Move the virtual objects behind real-world surfaces to see occlusion
134+
135+
## License
136+
137+
This project is licensed under the MIT License - see the LICENSE file for details.
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
<!--
2+
Copyright (c) Meta Platforms, Inc. and affiliates.
3+
4+
This source code is licensed under the MIT license found in the
5+
LICENSE file in the root directory of this source tree.
6+
-->
7+
8+
<!doctype html>
9+
<html>
10+
<head>
11+
<meta charset="utf-8" />
12+
<meta
13+
name="viewport"
14+
content="width=device-width, initial-scale=1, shrink-to-fit=no"
15+
/>
16+
<title>Depth Occlusion Demo</title>
17+
</head>
18+
<body style="margin: 0">
19+
<div id="scene-container"></div>
20+
<script type="module" src="/src/index.js"></script>
21+
</body>
22+
</html>
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{
2+
"name": "@iwsdk/depth-occlusion",
3+
"version": "0.1.0",
4+
"private": true,
5+
"type": "module",
6+
"scripts": {
7+
"dev": "vite",
8+
"build": "vite build",
9+
"preview": "vite preview",
10+
"fresh:install": "npx rimraf package-lock.json node_modules && npm install",
11+
"fresh:ci": "npx rimraf node_modules && npm ci",
12+
"fresh:dev": "npm run fresh:install && npm run dev",
13+
"fresh:build": "npm run fresh:install && npm run build"
14+
},
15+
"dependencies": {
16+
"@iwsdk/core": "file:../../packages/core/iwsdk-core.tgz",
17+
"patch-package": "^8.0.1",
18+
"three": "npm:super-three@0.181.0"
19+
},
20+
"devDependencies": {
21+
"@iwsdk/vite-plugin-gltf-optimizer": "file:../../packages/vite-plugin-gltf-optimizer/iwsdk-vite-plugin-gltf-optimizer.tgz",
22+
"@iwsdk/vite-plugin-iwer": "file:../../packages/vite-plugin-iwer/iwsdk-vite-plugin-iwer.tgz",
23+
"@iwsdk/vite-plugin-metaspatial": "file:../../packages/vite-plugin-metaspatial/iwsdk-vite-plugin-metaspatial.tgz",
24+
"@iwsdk/vite-plugin-uikitml": "file:../../packages/vite-plugin-uikitml/iwsdk-vite-plugin-uikitml.tgz",
25+
"vite": "^7.1.4",
26+
"vite-plugin-mkcert": "^1.17.0"
27+
}
28+
}
1.07 MB
Loading
883 KB
Loading
634 KB
Loading

examples/depth-occlusion/public/gltf/plantSansevieria/plantSansevieria.gltf

Lines changed: 161 additions & 0 deletions
Large diffs are not rendered by default.
45.9 KB
Loading
32.1 KB
Loading

0 commit comments

Comments
 (0)