Add MeshPass

[momoluna444]

Hello everyone,

This is my first pull request. I've been experimenting with this for a while, and now it's time to share it.

Objective

To simplify the effort required to create and add a custom mesh pipeline.

This pull request currently only simplifies the work needed for the specialization and queue stages. I haven't yet found a suitable way to abstract the other stages. I think that's enough to close #21127.

As a reward, this enables a form of "multi-material" support by allowing custom MeshPasses to be added to a material.

Solution

I've abstracted MaterialPlugins into the more general MeshPassPlugin<P>, which customizes the behavior of a pass via the MeshPass generic parameter.

A MeshPass can be defined like this:

pub struct MainPass;

impl MeshPass for MainPass {
    type ViewKeySource = Self;
    type Specializer = MaterialPipelineSpecializer;
    type PhaseItems = (Opaque3d, AlphaMask3d, Transmissive3d, Transparent3d);
    type RenderCommand = DrawMaterial;
}

Where Specializer is expected to implement PipelineSpecializer, and PhaseItem is expected to implement PhaseItemExt.

Then a material can utilize MeshPass as follows:

impl Material for CustomMaterial {
    fn shaders() -> PassShaders {
        let mut pass_shaders = PassShaders::default();
        pass_shaders.insert(MainPass::id(), ShaderSet{...});
        ...
        pass_shaders
    }
}

Beyond this, some additional work is still required for usage. Please refer to main_pass.rs and prepass/mod.rs for details.

Implementation Challenges

To achieve this, I explored multiple approaches:

• Abstracting the specialize and queue systems to be per-phase item: This meant using systems like specialize<Pass, PhaseItem> and queue<Pass, PhaseItem>. Each system would only handle one phase item, and we would add multiple systems based on the tuple MeshPass::PhaseItems.

  • Pros: The implementation was simple, and systems for each phase item could run in parallel.
  • Cons: Each phase's system iterates over visible_entities containing entities from all phases, and I couldn't find a simple or efficient way to filter them down.

• Switching to a per-pass system approach: To convert the MeshPass::PhaseItems tuple into parameters for the specialize and queue systems and to directly call the trait methods through the tuple within the systems, I ended up writing a lot of complicated and verbose macros. This eventually led to an issue with HRTB (Higher-Rank Trait Bounds), which propagated from the internal systems to the user's implementation, making the whole approach completely unusable.

Current Approach:

We continue to use the per-pass system approach but hard-code four Option<Res<RenderPhasesN<P>>> parameters in both the specialize and queue systems. Any unused parameters will be filled by the resource ViewBinnedRenderPhases<DummyPhaseN> which will never be initialized. This approach is also simple to implement, and the resulting boilerplate code is considered acceptable.

Design Choices

Entity Level Filtering (Unimplemented)

Previously, when iterating over visible_entity in specialize_prepass_material_meshes, we had this:

if !material.properties.prepass_enabled && !material.properties.shadows_enabled {
    // If the material was previously specialized for prepass, remove it
    view_specialized_material_pipeline_cache.remove(visible_entity);
    continue;
}

Now that we support adding custom MeshPasses, and users can even disable the MainPass, this filtering needs to be extended into a general implementation. Also, since users can add two materials with completely different passes, when iterating over visible_entity, it would be best to filter for the relevant passes.

Both issues fundamentally boil down to the same problem: We need a way within the systems to determine if an entity is valid for the current pass.

Potential Solutions:

1. Add a field to MaterialProperties: For example, passes_enabled: SmallVec<[PassId]>. We would also need to add a corresponding method in the Material trait.
2. Use Material purely for defining material data: Then, use an additional PassMarker component to indicate whether the corresponding pass is enabled. This can be combined with VisibilityClass.

I personally prefer the second approach and will give it a try to see what challenges might arise.

View Level Filtering

Currently, structures implementing the MeshPass trait are required to implement ExtractComponent. This allows them to be added to cameras to indicate that "this camera should be used to render this pass."

This is somewhat similar to our existing DepthPrepass and NormalPrepass, except that it controls whether the entire pass is enabled for a camera instead of a feature.

Internally, we can directly filter views using fn queue<P: MeshPass>(views: Query<(&ExtractedView), With<P>>, ...).

For compatibility, we automatically add built-in passes to all cameras via Required Components.

Placement of RenderCommand

I currently place the definition of RenderCommand within the MeshPass, but if needed, it could certainly be moved to PhaseItemExt.

The One-to-One Constraint Between MeshPass and ShaderSet

A ShaderSet contains a vertex shader and a fragment shader.

In the previous Prepass implementation, we had two forward PhaseItems and two deferred PhaseItems. During specialization, we actually need to use two different sets of vertex shaders and fragment shaders for these two cases, which conflicts with the current design.

My solution is: since we have already abstracted the MeshPass, we can implement Prepass and Deferred as two independent MeshPass. This means we have two specialize systems (and queue systems as well), with each system handling two PhaseItems.

In addition to the points mentioned above, there are also API naming and design. If you have any suggestions, please let me know.

Testing

In this commit, MainPass, Prepass, and DeferredPass have been switched to the new MeshPass implementation and have been tested in the shader_prepass, motion_blur, and deferred_rendering examples.

[github-actions]

Welcome, new contributor!

Please make sure you've read our contributing guide and we look forward to reviewing your pull request shortly ✨

[tychedelia]

Okay, I haven't had a chance to look deeply at the code but a few thoughts:

1. I'm very intrigued! Abstracting specialization and queue is a big win in itself even if it doesn't solve everything.
2. This isn't meant as a critique especially without first reviewing the code, but I've been wanting to move the renderer away from type level constructs and towards being more ECS driven, basically to lean on dynamic dispatch where possible. This code is particularly hot and it may be the case that traits are a clear win.
3. I'd love an example to help review the code to better understand what this looks like for users. This can be totally silly just showing the APIs.
4. No need to do this right now but we'll obv need to perf test. Let me know if you need help here. We're particularly interested in perf on realistic scenes like https://github.com/DGriffin91/bevy_caldera_scene.

I'm particularly encouraged by more experimentation here as this is an area of the renderer that really needs to be more modular.

[momoluna444]

I totally understand you.

While adding the example, I discovered a problem:
If a single Material contains multiple MeshPasses that share the same PhaseItem, the entity will fail to render. I suddenly realized that trying to add the same entity to the BinnedRenderPhase multiple times with different pipelines is actually invalid behavior, and BinnedRenderPhase cannot distinguish this situation from the case where an entity’s pipeline changes over time.

Adding MeshPasses that share a PhaseItem across different entities’ materials is fine, but this use case is very limited. If a user wants to add multiple MeshPasses to the same material, they must use completely different PhaseItems. Whether they implement them from scratch or use a newtype, they ultimately need to provide a corresponding render graph node.

Update:
After thinking about this problem, the solution is to detect and prevent repeated use of PhaseItem, then provide #[derive(Binned)] and #[derive(Sorted)] to make it easy for users to quickly create newtypes. I am still not entirely sure about the render graph node part and need to do some experimentation.

[github-actions]

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

[tychedelia]

If a single Material contains multiple MeshPasses that share the same PhaseItem, the entity will fail to render. I suddenly realized that trying to add the same entity to the BinnedRenderPhase multiple times with different pipelines is actually invalid behavior, and BinnedRenderPhase cannot distinguish this situation from the case where an entity’s pipeline changes over time.

Separately from your work here, this has been an issue before when users try to use a custom VisibilityClass, which @IceSentry ran into with our custom mesh pipeline example. Basically, the binned phase assumes that entities are unique, when really they should be some kind of composite key that includes their VisibilityClass. It sounds like you may be running into something here and are conceptually creating another item in this composite key. When I investigated fixing this, it seemed to not be a huge fix but required some careful thinking about how we handle render commands looking up their required data.

[momoluna444]

When I investigated fixing this, it seemed to not be a huge fix but required some careful thinking about how we handle render commands looking up their required data.

I agree with you. I’ll start by using a macro to solve my issue and create a working example to validate the overall design.

I also have another question: I saw in the VisibilityClass documentation that “Generally, an object will belong to only one visibility class, but in rare cases it may belong to multiple.” In my case, we definitely need to add multiple XXXPassMarker components to a large number of entities with Materials in order to indicate which passes are enabled, and I’m concerned that this might have a serious performance impact.

Personally, I feel that using tag components fits the ECS philosophy better than having something like fn enabled_passes -> Vec<PassId> inside the Material. Do you have any thoughts on this?