Cycling through texture UVs on many meshes efficiently (WGSL???)

[64-Tesseract]

I'm creating a Minecraft clone and I'm trying to figure out how to efficiently animate block textures. It doesn't need to be anything fancy, just cycling through a couple Rects in an atlas at regular intervals, and all blocks of the same type should be at the same "frame" at any time.

The way I designed the system is by arranging each block's animation frames in a column, and every 0.2 seconds, go through each mesh's UVs and shift the coordinates down as needed. It works, but the framerate drops quite noticably; the biggest problem isn't even iterating through potentially hundreds of vertices, it's just fetching the Mesh asset!
I haven't gotten to figuring out which chunks are visible and which are occluded yet, which I'm sure would help a lot, but I'd still like to figure out a more reliable, cleaner method of animating block textures than... this.

pub fn animate_chunks(
    time: Res<Time>,
    mut timers: ResMut<WorldTimers>,
    mut meshes: ResMut<Assets<Mesh>>,
    mut chunk_graphics: Query<&mut ChunkGraphic>,
) {
    if timers.render_timer.tick(time.delta()).just_finished() {
        for mut graphic in chunk_graphics.iter_mut() {

            // This line right here seems to be causing most of the frame drops
            // Commenting it (and by necessity the line below) makes the game run smoothly
            let mesh = meshes.get_mut(&mut graphic.mesh_handle).unwrap();

            // Actually iterating the vertices and moving the UVs is less intensive than getting the mesh
            // Commenting just this makes very little difference
            graphic.mesh_builder.animate_faces(mesh, time.elapsed_seconds());
        }
    }
}

I've thought of cycling through the global atlas asset for each frame (generate multiple atlases for each frame, then cycle though the handles), but different blocks could have different numbers of frames or animation speeds, so that wouldn't work too well. I can't have separate texture images and cycle those, either, since I'm adding all the blocks to a single mesh, and as far as I'm aware there's no way to set multiple albedo/normal textures for a single mesh.
If there isn't a way to animate textures with Bevy better than changing all the meshes' UVs, or some native built-in way I'm completely missing, I think the only way to really optimize this approach would be to cache the mutable reference to the mesh UV vector - but I'm sure rustc wouldn't take too kindly to that. I think this is a bit of a dead end.

However, I've got a suspicion that I could use WGSL shaders to access a custom attribute on each animated vertex and shift their UVs around with the GPU... that would solve all my problems, but I know next to nothing about the topic and the documentation's not great, so I'm pretty lost either way.

I'll be busy this week, so I might not have time to reply quickly, but thanks in advance!

[superdump]

Could you show more of what the animation code looks like? Then we can help you design a GPU solution. It shouldn’t be difficult, just need to understand the constraints and flexibilities of what you need.

[64-Tesseract]

I've probably way overcomplicated the whole thing, but I'll do my best to explain it...

First Bevy loads a bunch of .ron files that represent all the block models - it's a bit of a mess so I won't get into the specifics, but for each block model, it compiles a list of BlockModels:

Code

#[derive(Clone, Serialize, Deserialize)]
pub struct BlockModelFace {
    pub uv: FaceUv,
    pub vertices: [Vec3; 4],
    pub normal: Vec3,
}


// Whether to use static UV texture, or animate it as time passes
#[derive(Clone, Serialize, Deserialize)]
pub enum FaceUv {
    Static(Rect),
    Anim(FaceUvAnim),
}

impl FaceUv {
    pub fn get_uv(&self, time: f32) -> Rect {
        match &self {
            Self::Static(rect) => *rect,
            Self::Anim(anim) => anim.get_uv(time),
        }
    }
}


// Animation data is kept as its own struct so that when UVs are animated, enums don't have to be checked
#[derive(Clone, Serialize, Deserialize, PartialEq)]
pub struct FaceUvAnim {
    pub uv: Rect,
    pub frame_count: usize,
    pub frame_height: f32,
    pub anim_cycle: f32,
}

impl FaceUvAnim {
    pub fn get_uv(&self, time: f32) -> Rect {
        let anim_percent = (time % self.anim_cycle) / self.anim_cycle;
        let anim_frame = (anim_percent * self.frame_count as f32).floor();
        let anim_y = anim_frame * self.frame_height;

        let add_y = Vec2 { x: 0.0, y: anim_y };
        Rect::from_corners(self.uv.min + add_y, self.uv.max + add_y)
    }
}

These are organized into separate lists for each occlusion side, just so I don't have to check if each face is visible individually:

#[derive(Serialize, Deserialize)]
pub struct BlockModel {
    pub colour: String,
    // [0..=5] represent 6 directions, [6] is for faces that are always* visible
    pub faces: [Vec<BlockModelFace>; 7],
}

Next comes the actual rendering part. A MeshBuilder transforms faces to the cubes' positions and puts them into a list until they need to be applied to the mesh, and once they're applied it only stores the indices of faces that need to be animated.
(Unrelated, but I need to rework this part because every time a block is changed it'll need to re-calculate at worst 32^3 blocks, though storing index ranges for each block that need to be updated for every change might be worse in terms of performance... any ideas?)

Lots of code

#[derive(Component, Default)]
pub struct MeshBuilder {
    faces: Vec<block::BlockModelFace>,
    anim_uvs: HashMap<usize, block::FaceUvAnim>
}

impl MeshBuilder {
    // For each face in a block model, add its unoccluded faces
    pub fn add_model(&mut self, block_model: &block::BlockModel, position: &Vec3, occlusions: &[bool; 6]) {
        // Keep track of if all faces are occluded without iterating before-hand
        let mut all_occluded = true;

        for direction in 0..=6 {
            if direction < 6 {
                // If face occluded, ignore it, otherwise note that a face is unoccluded
                if occlusions[direction] {
                    continue;
                }
                all_occluded = false;

            } else {
                // If all previous faces occluded, don't render unoccluded cubes either
                if all_occluded {
                    continue;
                }
            }

            // Side unoccluded, add faces
            let faces = &block_model.faces[direction];

            for face in faces {
                self.faces.push(block::BlockModelFace {
                    uv: face.uv.clone(),
                    vertices: core::array::from_fn(|v| face.vertices[v] + *position),
                    normal: face.normal,
                });
            }
        }
    }


    // Expand each face into float slices for the mesh
    pub fn apply_mesh(&mut self, mesh: &mut Mesh) {
        let mut vertices: Vec<[f32; 3]> = Vec::new();
        let mut normals: Vec<[f32; 3]> = Vec::new();
        let mut uvs: Vec<[f32; 2]> = Vec::new();
        let mut indices: Vec<u32> = Vec::new();

        self.anim_uvs.clear();

        for (index, face) in self.faces.iter().enumerate() {
            // Each mesh vertex index can be derived from the face index
            let offset = (index * 4) as u32;

            // If the face is animated, cache it so all the faces in the model don't need to be checked
            if let block::FaceUv::Anim(a) = &face.uv {
                self.anim_uvs.insert(index, a.clone());
            }

            vertices.extend_from_slice(&[
                face.vertices[0].into(),
                face.vertices[1].into(),
                face.vertices[2].into(),
                face.vertices[3].into(),
            ]);

            normals.extend_from_slice(&[
                face.normal.into(), face.normal.into(),
                face.normal.into(), face.normal.into(),
            ]);

            // At first just use the first frame's UV for all faces
            uvs.extend_from_slice(&expand_uv(&face.uv.get_uv(0.0)));

            indices.extend_from_slice(&[
                offset + 0, offset + 1, offset + 2,
                offset + 2, offset + 3, offset + 0,
            ]);
        }


        self.faces.clear();
        self.faces.shrink_to_fit();

        mesh.insert_attribute(
            Mesh::ATTRIBUTE_POSITION,
            vertices,
        );

        mesh.insert_attribute(
            Mesh::ATTRIBUTE_NORMAL,
            normals,
        );

        mesh.insert_attribute(
            Mesh::ATTRIBUTE_UV_0,
            uvs,
        );

        mesh.set_indices(Some(Indices::U32(indices)));
        mesh.generate_tangents();
    }


    // Used the cached animated faces to translate UVs as needed
    pub fn animate_faces(&self, mesh: &mut Mesh, time: f32) {
        let VertexAttributeValues::Float32x2(uvs) = mesh.attribute_mut(Mesh::ATTRIBUTE_UV_0).unwrap() else { return; };

        for (index, face) in self.anim_uvs.iter() {
            let offset = index * 4;
            uvs[offset..offset + 4].copy_from_slice(&expand_uv(&face.get_uv(time)));
        }
    }
}

So the MeshBuilder has access to the animation properties of each face/vertex, and it also has access to the Mesh - it could add a custom vertex attribute for the GPU instead of animating UVs itself.

Probably not as important, but just to put things into perspective, I'll explain how my horribly unoptimized chunk system works:

Bit more code

Each chunk is made up of 2 components, since once I eventually implement multiplayer, the server won't need to store any graphical mesh. First up is the actual block storage, as structs in a hashmap so that they can be accessed by their coordinates, and secondly a separate mesh rendering entity in an actual Bevy ECS bundle:

pub struct Chunk {
    pub blocks: Vec<block::Block>,
    graphic: Entity,
    position: coords::ChunkCoords,
}

impl Chunk {
    pub fn new_empty(
        commands: &mut Commands,
        asset_reg: &Res<registry::AssetRegistry>,
        meshes: &mut ResMut<Assets<Mesh>>,
        position: coords::ChunkCoords,
    ) -> Chunk {
        let air = asset_reg.get_block_type("air".to_string()).unwrap();
        let mut graphic = commands.spawn_empty();
        ChunkGraphic::new_entity(&mut graphic, meshes, &asset_reg, position.clone());

        Chunk {
            blocks: (0..CHUNK_SIZE * CHUNK_SIZE * CHUNK_SIZE).map(|_| block::Block::new(&air)).collect(),
            graphic: graphic.id(),
            position: position,
        }
    }

    /* Insert functions related to getting block coords by index and vice-versa */

    // Add blocks to the graphical component's mesh builder
    pub fn update_mesh(
        &self,
        commands: &mut Commands,
        meshes: &mut ResMut<Assets<Mesh>>,
        block_types: &Res<Assets<block::BlockType>>,
        adjacent_chunks: &[Option<&Chunk>; 6],
        chunk_graphics: &mut Query<&mut ChunkGraphic>,
    ) {
        if let Ok(mut graphic) = chunk_graphics.get_mut(self.graphic) {
            for (i, block) in self.blocks.iter().enumerate() {
                let local_coords = Self::index_to_xyz(i);
                let mut occlusions: [bool; 6] = [false; 6];

                // For each direction, figure out if there's a block there
                for d in 0..6 {
                    // Get the chunk the block in that direction is in, possibly this chunk
                    let adjacent_coords = coords::LocalBlockCoords::from(coords::dir_to_coords(d)) + local_coords;
                    let (adjacent_chunk, block_coords) = {
                        if 0 <= adjacent_coords.0 && adjacent_coords.0 < CHUNK_SIZE as i64 &&
                           0 <= adjacent_coords.1 && adjacent_coords.1 < CHUNK_SIZE as i64 &&
                           0 <= adjacent_coords.2 && adjacent_coords.2 < CHUNK_SIZE as i64
                        {
                            (Some(self), adjacent_coords)
                        } else {
                            (
                                adjacent_chunks[d],
                                coords::LocalBlockCoords::from((
                                    adjacent_coords.0.rem_euclid(CHUNK_SIZE as i64),
                                    adjacent_coords.1.rem_euclid(CHUNK_SIZE as i64),
                                    adjacent_coords.2.rem_euclid(CHUNK_SIZE as i64)
                                ))
                            )
                        }
                    };

                    // Get the block (if any) in that chunk (if any) and return if it's solid
                    if let Some(chunk) = adjacent_chunk {
                        if let Some(block) = chunk.get_block_at_xyz(block_coords) {
                            let block_type = block_types.get(&block.block_type).unwrap();
                            occlusions[d] = block_type.solidity == block::Solidity::Solid;

                        } else {
                            occlusions[d] = false;
                        }
                    } else {
                        occlusions[d] = true;
                    }
                };

                // Get the block model from its type (should probably cache them...)
                let block_type = block_types.get(&block.block_type).unwrap();
                graphic.mesh_builder.add_model(&block_type.model, &Self::index_to_vec3(i), &occlusions);
            }

            // Marks rendering component as needing a re-render
            commands.entity(self.graphic).insert(ChunkGraphicNeedsRender);
        }
    }
}


// Marker for queries instead of iterating all `ChunkGraphic`s and checking a `bool`
#[derive(Component)]
pub struct ChunkGraphicNeedsRender;

// Graphical part of chunk, rendered independently of other chunk processes
#[derive(Component)]
pub struct ChunkGraphic {
    pub mesh_handle: Handle<Mesh>,
    pub mesh_builder: meshgen::MeshBuilder,
    position: coords::ChunkCoords,
}

impl ChunkGraphic {
    // Adds itself to an existing entity to avoid passing `Commands` as an argument
    pub fn new_entity(
        entity: &mut EntityCommands,
        meshes: &mut ResMut<Assets<Mesh>>,
        asset_reg: &Res<registry::AssetRegistry>,
        position: coords::ChunkCoords,
    ) {
        let graphic = Self {
            mesh_handle: {
                let mut mesh = Mesh::new(PrimitiveTopology::TriangleList);
                mesh.insert_attribute(
                    Mesh::ATTRIBUTE_POSITION,
                    Vec::<[f32; 3]>::new(),
                );
                mesh.insert_attribute(
                    Mesh::ATTRIBUTE_NORMAL,
                    Vec::<[f32; 3]>::new(),
                );
                mesh.insert_attribute(
                    Mesh::ATTRIBUTE_UV_0,
                    Vec::<[f32; 2]>::new(),
                );
                meshes.add(mesh)
            },
            mesh_builder: meshgen::MeshBuilder::default(),
            position: position,
        };

        entity.insert((
            PbrBundle {
                transform: Transform::from_translation(position.to_vec3()),
                material: asset_reg.texture_material.clone_weak(),
                mesh: graphic.mesh_handle.clone(),
                ..default()
            },
            graphic,
        ));
    }
}

And finally there's the rendering and animating systems:

// Re-generate mesh if needed
pub fn render_chunks(
    mut commands: Commands,
    mut meshes: ResMut<Assets<Mesh>>,
    mut chunk_graphics: Query<(Entity, &mut ChunkGraphic), With<ChunkGraphicNeedsRender>>,
) {
    for (entity, mut graphic) in chunk_graphics.iter_mut() {
        let mesh = meshes.get_mut(&mut graphic.mesh_handle).unwrap();

        graphic.mesh_builder.apply_mesh(mesh);
        commands.entity(entity).remove::<ChunkGraphicNeedsRender>();
    }
}


// Shift faces' UVs as needed to animate the textures
pub fn animate_chunks(
    time: Res<Time>,
    mut timers: ResMut<super::WorldTimers>,
    mut meshes: ResMut<Assets<Mesh>>,
    mut chunk_graphics: Query<&mut ChunkGraphic>,
) {
    if timers.render_timer.tick(time.delta()).just_finished() {
        for mut graphic in chunk_graphics.iter_mut() {

            // This line right here seems to be causing most of the frame drops
            // Commenting it (and by necessity the line below) makes the game run smoothly
            let mesh = meshes.get_mut(&mut graphic.mesh_handle).unwrap();

            // Actually iterating the vertices and moving the UVs is less intensive than getting the mesh
            // Commenting just this makes very little difference
            graphic.mesh_builder.animate_faces(mesh, time.elapsed_seconds());
        }
    }
}

So yeah, it's a bit of a mess. I'm hoping if I can offload at least a tiny chunk of the code to the GPU it'll speed things up, but I'll need to work on everything else that can't run on the GPU afterwards.

[mikeam565]

Any update on this? I am also finding that if you have a mesh with many vertices you want to modify frame by frame, it's the meshes.get_mut() function that slows everything to a halt.