Hight performance render

[sofly]

Hello guys! I need your help.
There is a canvas 640x480 (307200 pixels). I need to render for example 100000 points 30 times per second. What is the best way to do it?

[wcandillon]

I think using a shader would be the fastest way.

[sofly]

I will give you more details. I use react-native-camera-vision and use useFrameProcessor. FrameProcessor gives me a frame, my custom plugin gives me a lot of points, that need to be rendered on UI above video component. Frame processor and custom plugin give me result each 30ms. I use sharedValue to store an array of pixels from the frame processor. My idea was to render these points via skia imperative API, but I didn't find it. I see only one solution, in the main JS thread I need to use useEffect with setTimeout that will run every 30ms to take last data from sharedValue and use setState to render current mask.

Just need to learn how to use SkSL.

@wcandillon what do you think about it? If you have any better ideas, could you please help me?

import React, {
  useState,
  useMemo,
  memo,
  useEffect,
  MutableRefObject,
} from 'react';

import { Canvas, Shader, Fill, Skia } from '@shopify/react-native-skia';

import { IlluminationData } from './types';

interface Props {
  illuminationDataRef: MutableRefObject<{ data: IlluminationData }>;
}

export const FaceMask = memo(({ illuminationDataRef }: Props) => {
  const [data, setData] = useState<IlluminationData | null>(null);

  useEffect(() => {
    let timeoutId: NodeJS.Timeout;

    const run = () => {
      timeoutId = setTimeout(() => {
        setData(illuminationDataRef.current.data);
        run();
      }, 33);
    };

    run();

    return () => {
      clearTimeout(timeoutId);
    };
  }, [illuminationDataRef]);

  const canvasStyle = useMemo(
    () => (data ? { width: data.width, height: data.height } : null),
    [!!data],
  );

  return (
    data && (
      <Canvas style={canvasStyle}>
        <Fill>
          <Shader source={Skia.RuntimeEffect.Make(`... code here ...`)!} />
        </Fill>
      </Canvas>
    )
  );
});

[wcandillon]

I think that's good. When dealing with so many elements, the key to success is to avoid the cost of interfacing between RN and Skia. So if you send everything in a single call (in a uniform?) this would be the right strategy.

[sofly]

@wcandillon do you know if it's possible to pass dynamic count of vectors to shader?

[wcandillon]

Yes you can, this is the syntax:

// an array of 32 float elements
uniform float[32] myArray;