Adding PBR lighting to the shader_instancing example

[DavidHospital]

Hi all,
I'm pretty new to bevy and graphics programming in general, so please bear with me if I get some of the terminology wrong.

I'm interested in using GPU instancing to render fields of grass (upwards of a million blades of grass at a time), and I have managed to implement some simple rendering of exactly that by building off of the shader_instancing example. What I would like to do is customize the instancing.wgsl shader to apply basic PBR lighting to the grass blade meshes, like you would get for free using something like StandardMaterial. I've managed to change the output of the vertex shader to be bevy_pbr::forward_io::VertexOutput, so I can use pbr_input_from_standard_material(in, is_front) in my fragment shader like so:

@fragment
fn fragment(
	in: VertexOutput,
        @builtin(front_facing) is_front: bool,
) -> @location(0) vec4<f32> {
#ifdef VERTEX_COLORS
	let pbr_input = pbr_input_from_standard_material(in, is_front);
        return apply_pbr_lighting(pbr_input);
#else
	return vec4<f32>(0.3, 0.8, 0.1, 1.0);
#endif
}

The first issue I ran into was the following:

2024-01-18T17:35:33.113992Z ERROR bevy_render::render_resource::pipeline_cache: failed to process shader:
error: failed to build a valid final module: Entry point fragment at Fragment is invalid
  ┌─ /home/frosty/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bevy_pbr-0.12.1/src/render/pbr_bindings.wgsl:5:23
  │
5 │ @group(1) @binding(0) var<uniform> material: bevy_pbr::pbr_types::StandardMaterial;
  │                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ naga::GlobalVariable [23]
  │
  = Bindings for [23] conflict with other resource

I was able to figure out (correct me if I'm wrong) that this was due to the grass blade mesh using the same bind group (1) as the what the shader thinks the standard material should be bound to. I was able to solve this by removing the MESH_BINDGROUP_1 shader def in SpecializedMeshPipeline::specialize() function in my CustomPipeline:

descriptor
            .vertex
            .shader_defs
            .push("MESH_BINDGROUP_1".into());

I also changed SetMeshBindGroup<1> to SetMeshBindGroup<2> in:

type DrawCustom = (
    SetItemPipeline,
    SetMeshViewBindGroup<0>,
    SetMeshBindGroup<2>,
    DrawMeshInstanced,
);

But now I am receiving the following error, and I've been struggling to understand/fix this for some time now:

2024-01-18T17:40:23.839249Z ERROR log: Handling wgpu errors as fatal by default
thread 'Compute Task Pool (4)' panicked at /home/frosty/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-0.17.1/src/backend/direct.rs:3056:5:
wgpu error: Validation Error

Caused by:
    In Device::create_render_pipeline
      note: label = `opaque_mesh_pipeline`
    Error matching ShaderStages(VERTEX) shader requirements against the pipeline
    Shader global ResourceBinding { group: 2, binding: 0 } is not available in the layout pipeline layout
    Binding is missing from the pipeline layout


note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Encountered a panic in system `bevy_render::render_resource::pipeline_cache::PipelineCache::process_pipeline_queue_system`!
thread 'Compute Task Pool (1)' panicked at /home/frosty/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bevy_render-0.12.1/src/pipelined_rendering.rs:145:45:
called `Result::unwrap()` on an `Err` value: RecvError

Any help/advice would be greatly appreciated! Maybe I am just missing something obvious here. Like I said, I am a bit fuzzy on my knowledge of the rendering pipeline, and honestly I've been having a hard time finding good resources online to learn. Thanks in advance 😃

[JMS55]

Don't look at bevy main. Bind group numbers moved around recently. Make sure you look at bevy 0.12's code https://github.com/bevyengine/bevy/blob/release-0.12.1/examples/shader/shader_instancing.rs

[DavidHospital]

Ah, I was looking at that branch actually. I linked the wrong one, sorry.

[DavidHospital]

I have figured it out. For anyone else who has a similar issue: the problem is wgpu cannot create "skip" a bindgroup. In my case, bindgroup 0 is used for the vertex data, and bindgroup 2 is used for the mesh uniform data. wgpu doesn't like this as bindgroup 1 is unused. So to make it work, I am now using bindgroup 1 for a PBR material uniform (which I was already planning on doing since I want PBR lighting on my mesh instances).

[EmiOnGit]

Hey!
If you publish your code I would gladly take a look :)
I am the author of warblers grass which is a bevy crate for rendering grass.
Sadly it doesn't support shadows since I haven't found a performant way to calculate them.
However, I am using instancing and many other things you might be interested in.

[dror-g]

You sir, need to add support for Bevy 0.12 🤣

/joking. Thanks for a great crate full of inspiration!

[EmiOnGit]

You're right, I am working on a port to 0.12 but it seems that some parts of the crate are tricky to accomplish with the new rendering pipeline.

Also I'm wondering how much my approach actually improves performance compared to the automatic batching implemented in 0.12. If you happen to further work on your grass pipeline, I would be glad if you could share some benchmarks in comparison to the crate 😄

[dror-g]

Yes many changes in 0.12.. I tried to update Warblers Grass first but it did seem like a lot of work (and the fact half your code is way out of my league 🤣 )

[DavidHospital]

I've been quite busy lately so I've only had time to reply now, but I appreciate your comment! I've actually been using your crate as inspiration/reference while working on my own. The rendering pipeline is still a bit fuzzy for me, so your code has been a great help. If you're interested in checking out my solution, I've just published my grass rendering work: https://github.com/DavidHospital/FrostyGrass.

[dror-g]

Interesting!
I'll just note that I think instancing is done automatically now?
I'm able to render millions of grass blades by manually generating each blade from verts, then bunch them into a single mesh with let mut mesh = Mesh::new(PrimitiveTopology::TriangleList); and iterating each blade into this mesh.

I can render 1.5 million blades easy on laptop at +300fps, and even on Android low end device at ~20fps.
I now generate smaller square grass meshes, 4096 blades each, and clone them multiple times, so I can get culling (otherwise all million blades are always rendered in giant mesh that never leaves view).

I'll try applying fragment shader at some point too like you do.

I create a custom material:

#[derive(Asset, AsBindGroup, TypeUuid, TypePath, Debug, Clone)]
#[uuid = "f690fdae-d598-42ab-8225-97e2a3f05666"]
pub struct GrassMaterial {
    #[uniform(0)]
    color: Color,
}

impl Material for GrassMaterial {
    fn fragment_shader() -> ShaderRef {
        "shaders/grass_material.wgsl".into()
    }

    fn vertex_shader() -> ShaderRef {
        "shaders/grass_material.wgsl".into()
    }

   fn specialize(
        _pipeline: &bevy::pbr::MaterialPipeline<Self>,
        descriptor: &mut bevy::render::render_resource::RenderPipelineDescriptor,
        _layout: &bevy::render::mesh::MeshVertexBufferLayout,
        _key: bevy::pbr::MaterialPipelineKey<Self>,
    ) -> Result<(), bevy::render::render_resource::SpecializedMeshPipelineError> { 
        if let Some(label) = &mut descriptor.label {
            *label = format!("Grass_{}", *label).into();
        }
        Ok(())

    }

And spawn it with:

            commands.spawn(MaterialMeshBundle {
                mesh: grass_mesh_handle.clone(),
                material: material_handle.clone(),
                transform: Transform::from_xyz(-100, 0., -100),
                ..default()
            })
            .insert(Grass {} );

My shader is thus more or less:

#import bevy_pbr::{
    mesh_view_bindings::globals,
    mesh_bindings::mesh,
    mesh_functions as mesh_functions,
    view_transformations::position_world_to_clip,
    forward_io::{Vertex,VertexOutput},
}
#import bevy_render::instance_index::get_instance_index
#import bevy_pbr::{mesh_vertex_output, MeshVertexOutput}

struct CustomMaterial {
    color: vec4<f32>,
};

@group(1) @binding(0)
var<uniform> material: CustomMaterial;

@vertex
fn vertex(vertex: Vertex) -> VertexOutput {

    var model = mesh_functions::get_model_matrix(vertex.instance_index);
    var out: VertexOutput;
    out.world_normal = mesh_functions::mesh_normal_local_to_world(vertex.normal, get_instance_index(vertex.instance_index));


    var position: vec3<f32>;
    position = vec3(vertex.position.x + y, vertex.position.y, vertex.position.z + y);
    out.world_position = mesh_functions::mesh_position_local_to_world(model, vec4<f32>(position, 1.0));

    // out.position is clip_position
    out.position = position_world_to_clip(out.world_position.xyz);

    return out;
}

@fragment
fn fragment(
    mesh: VertexOutput
) -> @location(0) vec4<f32> {
    var out = material.color * vec4(6.0, 0.4, 0.0, 1.0) * mesh.world_position.y; 
    return out;
}

[EmiOnGit]

Yes, batching is done automatically in bevy 0.12 (see https://bevyengine.org/news/bevy-0-12/#automatic-batching-and-instancing-of-draw-commands).
This should be sufficiently performant for most use-cases :)

[dror-g]

But come to think of it - I'm not sure my approach below actually benefits from instancing / batching, as blades are stored in a single mesh?
Perhaps I should split them to even smaller meshes... 100 blades per mesh or less?

[DavidHospital]

Interesting, I wasn't aware of this automatic batching. This would greatly simplify the rendering pipeline

[dror-g]

Another grass solution that slipped under my radar -

https://github.com/jadedbay/bevy_procedural_grass

Looks neat!