Passing external camera used in external see to r3f causes external scene to be unexpectedly scaled

[trusktr]

I've managed to reproduce one of the issues I have in a rather large application of mine. The large application isn't open source, but basically there is a non-react-three-fiber Three.js scene in the app, and I am rendering supplemental scenes in small parts of the UI (f.e. think of mini maps, or similar). So I have a plain Three.js scene that takes up most of the screen, and I decided to try and introduce r3f for the supplemental scenes.

One issue I have is that when I pass the main scene's camera to the supplemental react-three-fiber scenes with sharedCanvasContext.setDefaultCamera() so that I can render them with the same viewpoint as the main scene, it somehow causes the main scene to be distorted, or vice versa, as if some form of scale is being applied and it stretches one or the other.

I've managed to reproduce this issue. In the following sandbox, the small scene at the top has a red box in the middle. It should look like a square. But after a brief moment, you will see that the box is stretched.

https://codesandbox.io/s/r3f-mouse-forked-ev9bg?file=/src/index.js

If you go to the very bottom of index.js and comment out the ReactDOM.render() call, so that react-three-fiber will not be used, you will notice that non-r3f scene works fine and the box is not stretched.

For some reason, when we render the r3f scene also, then it distorts the main scene.

For reference, here it is without the r3f scenes and the plain-Three scene is fine:

https://codesandbox.io/s/r3f-mouse-forked-748g8?file=/src/index.js

Any ideas why?

[trusktr]

Aha! I see what the issue is.

r3f modifies the passed-in camera's aspect ratio here and updateDefaultCamera can be set to false to prevent that.

As an alternative, maybe we can adjust the camera's aspect before each scene renders. How may we do that?

[trusktr]

As an alternative, maybe we can adjust the camera's aspect before each scene renders. How may we do that?

Yep, that does the trick!

In my non-r3f scene's render function:

function loop() {
  // ...
  theSameCamera.aspect = width / height
  theSameCamera.updateProjectionMatrix()
  renderer.render(scene, theSameCamera)
}

In my r3f code:

const {aspect} = useThree()
useFrame(() => {
  theSameCamera.aspect = aspect
  theSameCamera.updateProjectionMatrix()
})

Although this works, it has the unfrotunate side-effect of causing my invalidateFrameLoop={true} option to be ignored, and now there is a continual render loop.

@drcmda It would be great if useFrame would only fire before the scene will re-render, but not force a game loop. Or maybe a new thing like useInvalidatedFrame, that will only fire any time the scene will re-render (whether it is because there's game loop, or invalidate() is making a render happen).

in other words, I want to do something before any time the scene will re-render, but I don't want this hook to cause a game loop. Would be great to keep the fans from spinning. :D

[drcmda]

useFrame itself shouldn't cause render, when your canvas is set to invalidate frames it must only render when there is a need unless there is a new bug im not aware of. but as you can see here https://codesandbox.io/s/perlin-cubes-forked-wtp9t it seems to work. that loop goes to sleep if orbitcontrols stops rotating.

you can orchestrate useFrame with an index:

useFrame(() => {
  // is called first
}, 1)

useFrame(() => {
  // called second
}, 2)

useFrame((state) => {
  // finally, render
  state.gl.render(state.scene, state.camera)
}, 3)