3D tiles are not updated during VR session

[afdmulder]

Question

I start a VR session with a specific errorTarget value. As a result, I initially see only a few 3D tiles which is fine.
When I move my avatar towards an adjacent tile, I expect that tile to be loaded and displayed.
However, that does not happen. The adjacent tiles are only shown after I interrupt or pause the VR session.

Question:
Is this behavior expected or am I missing something?

Supplemental Data

Steps to reproduce behaviour:

1. Start VR session
2. Navigate to the an adjacent tile
   3 .Wait for a moment: the adjacent tile is not shown
3. interrupt the VR session
4. the tile(s) will show up

Example video:
https://3dcitytour.nl/support/3dcitytour_support.mp4

Example environment:
https://3dcitytour.nl/support

Library Version

0.4.17

Three.js Version

0.160.1

[gkjohnson]

Hi @afdmulder - without simple demo code it's hard to say what may exactly be going on but I recommend taking a look at the vr example which demonstrates a assigning a custom scheduling callback for tile loads so they can be handled in the relevant request animation from loop. WebXR uses a separate rAF loop from the one on window (which is used for scheduling) and the tiles renderer otherwise has no way of knowing which render loop to use without assigning a custom callback.

[afdmulder]

Hi Garrett, thanks for your reply and tips. In fact I based my code on the VR example.
In mains.js/reinstantiateTiles() I add the tilesets and I apply schedulingCallback.
In the mains.js/animate() I call handleTasks like you do in the VR example.

However, it looks like tasks are not added during the VR session. As soon as I pause the VR session, the numbers of tasks increases and the tasks are processed.

My test repository and branch
https://github.com/afdmulder/3DCityTour/tree/feature/vr_rendering

[afdmulder]

I've noticed that the 3D tile update process does work if I leave the web browser open (after pausing my session) as a kind of standby screen. I can still navigate in the VR world then. But as soon as I resume the VR session the update proces holds

See screenshot.

Device Oculus Meta Quest 2

[gkjohnson]

My test repository and branch
https://github.com/afdmulder/3DCityTour/tree/feature/vr_rendering

Unfortunately I can't review large scale projects or repos for problems like this - it's not sustainable with the amount of questions I get. I generally have to ask for small reproductions of the problem ideally confined to a single file or two.

[afdmulder]

Hi, that's makes fully sense. I've been thinking about it...
In your vr example I replaced the url by mine 'https://www.3drotterdam.nl/datasource-data/d0c755ef-ba1b-43fc-bc44-24fc7bb152ce/tileset.json'.

Although the 3D tiles are flipped and float in the air, their behavior is the same.
If I point to a building in VR, I teleport there.
Initially, however, you see simplified geometry. As soon as you pause/interrupt the VR session, the detail is loaded.

The small video below is based on your VR.html with the tileset I mentioned above.
https://3dcitytour.nl/support/3dcitytour_support2.mp4
Regards

[afdmulder]

I realized that I had made a small modification to B3DMLoader.js in order to be able to read the tileset at 'https://www.3drotterdam.nl/datasource-data/d0c755ef-ba1b-43fc-bc44-24fc7bb152ce/tileset.json'.
I had to add a loop inside the loader.parse function.

loader.parse(gltfBuffer, workingPath, model => {
				const { batchTable, featureTable } = b3dm;
				const adjustmentTransform = this.adjustmentTransform;

				// can have multiple scenes
				for (const scene of model.scenes) {

					const rtcCenter = featureTable.getData('RTC_CENTER');
					if (rtcCenter) {
						scene.position.x += rtcCenter[0];
						scene.position.y += rtcCenter[1];
						scene.position.z += rtcCenter[2];
					}

					// Transformatie toepassen
					scene.updateMatrix();
					scene.matrix.multiply(adjustmentTransform);
					scene.matrix.decompose(scene.position, scene.quaternion, scene.scale);

					model.batchTable = batchTable;
					model.featureTable = featureTable;
					model.scene = scene; // overschrijf default scene met de juiste

				}

				resolve(model);

			}, reject);