Mutate multiple assets in parallel

[Droggelbecher]

I am trying to make a custom material with custom shader. The shader gets as input multiple textures (stored as Image assets) and displays them in a certain way.
For the way I'd like to use this I would like to modify both textures regularly to communicate with the shader.

The asset system lets me mutably borrow one of the images at a time but not more which makes this awkward to use: I cant really write a wrapper structure that neatly hides away the implementation detail of using 2 or more textures under the hood and given a user easy access to modify the necessary data.

Is there some nice way to (temprarily, within the scope of a system) get mutable references to multiple Images from eg Assets at the same time?

[nicopap]

You have two options

1. Use Assets::remove to get ownership of the image asset temporarily, then use Assets::set to reinsert it back.
2. Use a closure or a trait. The end-user API becomes honestly unpleasant, but as a general rule, inversion of control is very useful to work around lifetime limitations in rust.

trait ModifyImage {
    fn modify_1(&mut self, image: &mut Image);
    fn modify_2(&mut self, image: &mut Image);
}
fn modificator(
    assets: &mut Assets<Image>,
    handle1: &Handle<Image>,
    handle2: &Handle<Image>,
    modificator: &mut impl ModifyImage,
) {
    let Some(image1) = assets.get_mut(handle1) else { return; };
    modificator.modify_1(image1);
    let Some(image2) = assets.get_mut(handle2) else { return; };
    modificator.modify_2(image2);
}
fn with_remove(
    assets: &mut Assets<Image>,
    handle1: &Handle<Image>,
    handle2: &Handle<Image>,
) {
    let Some(mut image1) = assets.remove(handle1) else { return; };
    let Some(mut image2) = assets.remove(handle2) else {
        assets.set(handle1, image1);
        return;
    };
    some_function_to_modify_two_images(&mut image1, &mut image2);
    assets.set(handle1, image1);
    assets.set(handle2, image2);
}

[Droggelbecher]

Thanks for the elaborate answer @nicopap !
I had thought about the modificator approach but as you say it makes the end-user API a bit unpleasant.

For the remove approach: How would that affect the transfer of the texture data between CPU & GPU?
I.e. assume the images are large and only a few pixels are modified:

• Is there generally some mechanism somewhere in bevy or "below" that would update only the part that has changed, or would the whole texture be transferred to GPU for every single change?
• If yes, would I be making this impossible with the remove approach (because it has to be considered a completely new image)?

[nicopap]

Doing remove and set here is fairly cheap, since Image by itself isn't much data, it contains a Vec<u8> which is a reference to the whole image, and moving around the Image shouldn't move around the data itself.

The whole image will be re-uploaded even if just mutating the image with .get_mut(). I'm not exactly an expert on this, but taking a quick glance at crates/bevy_render/src/render_asset.rs, it seems to be where this is affected.

In fact, here is an important bit: Just calling get_mut will set the "modified" bit for the image, so make sure to not call it when you don't modify the image.

In all honesty, I encourage you to avoid modifying images this way, it's very inefficient. The Image api is also not ideal for "set pixels" kind of work. An option is to do this update only in a system with a run condition.

Maybe this is premature optimization though? How many images do you expect to update this way? Dozens? Hundreds? Thousnds? Each frame? Every minutes? I don't know. But I don't expect this to become a problem with less than 100 images per frame.

You didn't give much details about your actual problem, so I can't give better guidance.

[Droggelbecher]

Thanks!

You asked for it so here is more detail on what I'm trying to do ;-)

I want to build a fast tilemap implementation (much like bevy_simple_tilemap or bevy_ecs_tilemap), some first experiments suggest its much faster if the work is moved to GPU.

The way I am doing this is like this:

• The shader receives 2 textures: One with the tile indices and one with the tile graphics (think: texture atlas)
• In addition, some uniforms like tile size etc.. are sent over
• The shader then can render the whole tilemap on a single quad which speeds up rendering a lot

This implies that the "index" texture would regularly be modified by the end user whenever tiles change. In some initial benchmarks this still seemed siginificantly faster than the existing tilemap approaches ( I guess it saves more in quads than it costs in texture transfers?).

FYI the "modify two images in parallel" question came from the desire to have a third texture that for each tile would define a tint color.
For a user of the tilemap than it would be nice to set the tile index and tint color together for the same "pixel". As user would often modify multiple "pixels" at once I'd like to avoid the handle lookup for each one, hence I was looking for a way to have two mutable image references.

My take is for that last part the removing approach should do the trick, but any comments you have on the general approach are much appreciated.

[nicopap]

Well, you'll only have a single or not many index maps, so the approach looks sane/good. Though I spied in the file I mentioned earlier a removed: Vec<Handle<A>> which means there might be a very low C O(C*n) cost on remove/set. Probably something that can be fixed if it becomes a problem.