Rasterize contents of canvas for reuse

[FlawTECH]

Hi, I need to draw a lot of rects on a canvas over time, and obviously this quickly uses too much memory.

On the web, you can easily do your canvasCtx.fillRect(...) calls and rasterize the entire thing by calling canvasCtx.drawImage(canvas, 0, 0) and call it a day.

I wanted to try using an <Image image={rasterImg} />, and make rasterImg a shared value since I will work with that in a useFrameCallback (the data points need to be updated quite frequently).

I have two questions:

• How do I conditionally render this Image ? Obviously, putting { rasterImg.value && ()} makes reanimated complain that .value cannot be used in JSX
• Am I thinking right, here ? It feels very convoluted for what I actually want to achieve

Sorry if these seem like noob questions and feel free to redirect me to proper documentation if I've missed any.

[MatiPl01]

Hey @FlawTECH!

Explanation

You are trying to use the wrong stuff for what you are trying to achieve. The Image component (imported from React Native, as I suppose you are doing) supports only static images and you cannot simply update the image via a SharedValue. You cannot also pass a SharedValue as this component doesn't know how to handle SharedValue. Passing the value read from the SharedValue via the .value is even worse as it reads only the value available while the React's render is happening and doesn't respond to any further updates of the value.

Suggested solution

I think that you would be able to achieve what you want with the React Native Skia (https://shopify.github.io/react-native-skia/docs/). I am not sure about that but Skia should be able to handle this in some way.

Answers to the questions

How do I conditionally render this Image ? Obviously, putting { rasterImg.value && ()} makes reanimated complain that .value cannot be used in JSX

In general, conditional renders should be performed only via the React's state. If you want to hide or show the image component, use the useState hook and update the values stored in state, based on which you decide whether to render the image.

Am I thinking right, here ? It feels very convoluted for what I actually want to achieve

It is a bit complex, but as I suggested, I think you would be able to do this with RN Skia. It is compatible with Reanimated so performance of updates should be good.

[FlawTECH]

Thank you for the detailed reply !

Sorry for the mix up, I was actually referring to an SkImage from Skia (the component is called "Image" and I forgot to check the property type); the "Image" component used is also imported from Skia. The reason I want to render the component conditionally is because this SkImage is null on initialization:

import { Image, SkImage } from '@shopify/react-native-skia';
import { useSharedValue } from 'react-native-reanimated';

const rasterImg = useShared<SkImage>(null ? default blank image ?)`
...
<Image image={rasterImg} rect={imageRect} />
              ^
              not happy

So, in trying to conditionally render the (Sk)Image: I understand that using state makes more sense for that, but I'm also updating the rasterImg (SkImage) in an useFrameCallback very frequently. From what I read, changing it to a state variable would force me to make each update in a runOnJs. This would probably affect performance badly, so I can't really do that.

For now, I see 2 options:

• Make a "fake" blank image on initial render, which is what I do for now:

const rasterImg = useSharedValue<SkImage>(
  (() => {
    const surface = Skia.Surface.Make(1, 1)!;
    return surface.makeImageSnapshot();
  })()
);

Not too happy with this solution since surface can be undefined. Don't know the conditions for that, though. Maybe there is a better way ? Also makes an ugly square

• Tie a state variable to the shared value used only for conditional rendering. Linter is not too happy about that, though, since it has no way to know one guards the other (and I don't like seeing red in my files 🙂 )

Regarding my other question (and also the reason I'm using an SkImage in the first place), here is the gist of the algorithm:

1. Grab previous snapshot of pixels drawn (if any)
2. Shift a few pixels to the left (it's sort of like a timeline)
3. Draw rects (SkRect) on the now blank space on the right side (we're talking ~768 rects per iteration)
4. Take snapshot
5. Go to step 1 (in next useFrameCallback)

or the code, roughly:

frameCallback.current = useFrameCallback((frameInfo: FrameInfo) => {
  if (...) {} // bunch of checks to prevent too frequent updates
  
  const surface = Skia.Surface.MakeOffscreen(imageRect.value.width, imageRect.value.height);
  const canvas = surface?.getCanvas();

  // Step 1 & 2
  if (rasterImg.value && dataToRender.length > 0) {
    const paint = Skia.Paint();
    const shiftX = timeInPixels;
    canvas?.drawImage(rasterImg.value, -shiftX, 0, paint);
  }

  for (let i = 0; i < dataToRender.length; i++) {
    // draw ~768 rects (Step 3)
    const paint = Skia.Paint();
    paint.setColor(Skia.Color(getColor(dataToRender[i].color)));
    canvas?.drawRect(Skia.XYWHRect(someWidth, someHeight, someX, someY), paint);
  }

  // Step 4
  const snapshot = surface?.makeImageSnapshot();
  if (snapshot) {
    rasterImg.value = snapshot;
  }
}

This works but the performance is abysmal (~65-70ms). I did some checks however and the longest operation is drawing the rects, not snapshotting / restoring the picture. If you or someone else has any idea to make the draw calls faster, I'm interested.

Apologies for the long comment 🙂

[MatiPl01]

So, in trying to conditionally render the (Sk)Image: I understand that using state makes more sense for that, but I'm also updating the rasterImg (SkImage) in an useFrameCallback very frequently. From what I read, changing it to a state variable would force me to make each update in a runOnJs.

Can you get the null value later on in the frame callback or is that just an initial value? If that's just an initial value, then I'd suggest to implement your second proposed solution:

Tie a state variable to the shared value used only for conditional rendering. Linter is not too happy about that, though, since it has no way to know one guards the other (and I don't like seeing red in my files 🙂)

I think that the following code should work with no issues. Can you tell me what linter issues can you see while you use this approach?

const [show, setShow] = useState(false);
const isShown = useSharedValue(false);

and then update both at once in the frame callback only if not initalized yet:

if (!isShown.value) { // guarantees to change state just once
  isShown.value = true;
  runOnJS(setShow)(true);
}

This works but the performance is abysmal (~65-70ms). I did some checks however and the longest operation is drawing the rects, not snapshotting / restoring the picture. If you or someone else has any idea to make the draw calls faster, I'm interested.

Mhm, I am not that much experienced with RN Skia so I can't tell if you should expect better performance while drawing that many rects and don't know which other optimization I can suggest.

[FlawTECH]

I must have made a mistake somewhere else since I don't see the issue with the SkImage being null anymore.
The Image component simply seems to ... accept a null shared value ?

In any case, I added the safeguard as per your suggestion, since I think it's probably better to show something else while it loads, and again, no more errors. I must have screwed up somewhere else and accidentally fixed it 🙂

In any case, thanks again for your help. I managed to reduce the draw calls to about ~20ms per iteration, which I think is pretty good. I'll bug the folks on the Skia project if I need it to be faster !