useFrame but with slow and asynchronous calculation.

[antran22]

I'm trying to use react-three-fiber with a deep learning model. My code goes somehow like this:

const  calculatePosition = async (model, video) => {
  // some heavy calculations that take about 50-100ms
  return position
};

export default function Component(props) {
  const meshRef = useRef();

  useFrame(() => {
      calculatePosition().then(([x, y, z]) => {
        meshRef.current.position.x = x;
        meshRef.current.position.y = y;
        meshRef.current.position.z = z;
      });
  });

  return (
    <mesh ref={meshRef}>
      <sphereBufferGeometry attach="geometry" args={[0.7, 250, 250]} />
      <meshBasicMaterial color={0xff0000} attach="material" />
    </mesh>
  );

The expected render speed is about 10-20fps, however this performs way worse than expectation. Also I kinda have a feeling this isn't what should be done. I'm a newb in this field so can someone point me on the right direction?

[stephencorwin]

You should look into worker threads so that it doesn't block the main thread. This will keep your FPS at 60 while doing any calculations.

[antran22]

Even if I moved the calculation function into a new worker thread, I would still need to update the ThreeJS object asynchronously would'nt I?
So it would go something like this?

  useFrame(() => {
      calculateWorker.postMessage("calculatePlease");
      calculateWorker.onmessage = ([x, y, z]) => {
        meshRef.current.position.set(x, y, z);
      };
  });

My question would be: will updating ThreeJS objects asynchronously in useFrame affect the rendering process?

[stephencorwin]

You wouldn't post a message 60 times a second in a useFrame. The onmessage handler would likely be in a useEffect. Neither postMessage or onmessage should ever be in a useFrame.

Think about it this way. useFrame fires are 60 fps. Do I need to assign a new handler every frame? No. I just need to assign a handler once and then wait for the result.

useEffect(() => {
  calculateWorker.postMessage("calculatePlease");
  calculateWorker.onmessage = ([x, y, z]) => {
    meshRef.current.position.set(x, y, z);
    meshRef.geometry.attributes.position.needsUpdate = true; // some variation of this
  };
}, [])

I believe that should work without useFrame at all. However, you should look into useUpdate. It might be necessary in order to update imperatively. https://github.com/pmndrs/react-three-fiber/blob/master/api.md#useUpdate

[drcmda]

@stephencorwin 100% on point. this is not a use case for useFrame. but it's similar. every time the worker spits a new value, you mutate the view. this is also how use-cannon works.

[antran22]

thank you guys for your support