Implement minimal reflection probes (fixed macOS, iOS, and Android). (#11366)

This pull request re-submits #10057, which was backed out for breaking
macOS, iOS, and Android. I've tested this version on macOS and Android
and on the iOS simulator.

# Objective

This pull request implements *reflection probes*, which generalize
environment maps to allow for multiple environment maps in the same
scene, each of which has an axis-aligned bounding box. This is a
standard feature of physically-based renderers and was inspired by [the
corresponding feature in Blender's Eevee renderer].

## Solution

This is a minimal implementation of reflection probes that allows
artists to define cuboid bounding regions associated with environment
maps. For every view, on every frame, a system builds up a list of the
nearest 4 reflection probes that are within the view's frustum and
supplies that list to the shader. The PBR fragment shader searches
through the list, finds the first containing reflection probe, and uses
it for indirect lighting, falling back to the view's environment map if
none is found. Both forward and deferred renderers are fully supported.

A reflection probe is an entity with a pair of components, *LightProbe*
and *EnvironmentMapLight* (as well as the standard *SpatialBundle*, to
position it in the world). The *LightProbe* component (along with the
*Transform*) defines the bounding region, while the
*EnvironmentMapLight* component specifies the associated diffuse and
specular cubemaps.

A frequent question is "why two components instead of just one?" The
advantages of this setup are:

1. It's readily extensible to other types of light probes, in particular
*irradiance volumes* (also known as ambient cubes or voxel global
illumination), which use the same approach of bounding cuboids. With a
single component that applies to both reflection probes and irradiance
volumes, we can share the logic that implements falloff and blending
between multiple light probes between both of those features.

2. It reduces duplication between the existing *EnvironmentMapLight* and
these new reflection probes. Systems can treat environment maps attached
to cameras the same way they treat environment maps applied to
reflection probes if they wish.

Internally, we gather up all environment maps in the scene and place
them in a cubemap array. At present, this means that all environment
maps must have the same size, mipmap count, and texture format. A
warning is emitted if this restriction is violated. We could potentially
relax this in the future as part of the automatic mipmap generation
work, which could easily do texture format conversion as part of its
preprocessing.

An easy way to generate reflection probe cubemaps is to bake them in
Blender and use the `export-blender-gi` tool that's part of the
[`bevy-baked-gi`] project. This tool takes a `.blend` file containing
baked cubemaps as input and exports cubemap images, pre-filtered with an
embedded fork of the [glTF IBL Sampler], alongside a corresponding
`.scn.ron` file that the scene spawner can use to recreate the
reflection probes.

Note that this is intentionally a minimal implementation, to aid
reviewability. Known issues are:

* Reflection probes are basically unsupported on WebGL 2, because WebGL
2 has no cubemap arrays. (Strictly speaking, you can have precisely one
reflection probe in the scene if you have no other cubemaps anywhere,
but this isn't very useful.)

* Reflection probes have no falloff, so reflections will abruptly change
when objects move from one bounding region to another.

* As mentioned before, all cubemaps in the world of a given type
(diffuse or specular) must have the same size, format, and mipmap count.

Future work includes:

* Blending between multiple reflection probes.

* A falloff/fade-out region so that reflected objects disappear
gradually instead of vanishing all at once.

* Irradiance volumes for voxel-based global illumination. This should
reuse much of the reflection probe logic, as they're both GI techniques
based on cuboid bounding regions.

* Support for WebGL 2, by breaking batches when reflection probes are
used.

These issues notwithstanding, I think it's best to land this with
roughly the current set of functionality, because this patch is useful
as is and adding everything above would make the pull request
significantly larger and harder to review.

---

## Changelog

### Added

* A new *LightProbe* component is available that specifies a bounding
region that an *EnvironmentMapLight* applies to. The combination of a
*LightProbe* and an *EnvironmentMapLight* offers *reflection probe*
functionality similar to that available in other engines.

[the corresponding feature in Blender's Eevee renderer]:
https://docs.blender.org/manual/en/latest/render/eevee/light_probes/reflection_cubemaps.html

[`bevy-baked-gi`]: https://github.com/pcwalton/bevy-baked-gi

[glTF IBL Sampler]: https://github.com/KhronosGroup/glTF-IBL-Sampler

Author: pcwalton

Cargo.toml

@@ -2508,6 +2508,17 @@ name = "fallback_image"
 path = "examples/shader/fallback_image.rs"
 doc-scrape-examples = true
 
+[[example]]
+name = "reflection_probes"
+path = "examples/3d/reflection_probes.rs"
+doc-scrape-examples = true
+
+[package.metadata.example.reflection_probes]
+name = "Reflection Probes"
+description = "Demonstrates reflection probes"
+category = "3D Rendering"
+wasm = false
+
 [package.metadata.example.fallback_image]
 hidden = true
 

assets/environment_maps/cubes_reflection_probe_specular_rgb9e5_zstd.ktx2

@@ -7,8 +7,8 @@
     @group(0) @binding(3) var dt_lut_texture: texture_3d<f32>;
     @group(0) @binding(4) var dt_lut_sampler: sampler;
 #else
-    @group(0) @binding(15) var dt_lut_texture: texture_3d<f32>;
-    @group(0) @binding(16) var dt_lut_sampler: sampler;
+    @group(0) @binding(16) var dt_lut_texture: texture_3d<f32>;
+    @group(0) @binding(17) var dt_lut_sampler: sampler;
 #endif
 
 fn sample_current_lut(p: vec3<f32>) -> vec3<f32> {

assets/models/cubes/Cubes.glb

@@ -58,7 +58,10 @@ symphonia-vorbis = ["bevy_audio/symphonia-vorbis"]
 symphonia-wav = ["bevy_audio/symphonia-wav"]
 
 # Shader formats
-shader_format_glsl = ["bevy_render/shader_format_glsl"]
+shader_format_glsl = [
+  "bevy_render/shader_format_glsl",
+  "bevy_pbr?/shader_format_glsl",
+]
 shader_format_spirv = ["bevy_render/shader_format_spirv"]
 
 serialize = [

crates/bevy_core_pipeline/src/tonemapping/tonemapping_shared.wgsl

@@ -10,6 +10,7 @@ keywords = ["bevy"]
 
 [features]
 webgl = []
+shader_format_glsl = ["naga_oil/glsl"]
 pbr_transmission_textures = []
 
 [dependencies]
@@ -34,8 +35,17 @@ fixedbitset = "0.4"
 # direct dependency required for derive macro
 bytemuck = { version = "1", features = ["derive"] }
 radsort = "0.1"
-naga_oil = "0.11"
+smallvec = "1.6"
 thread_local = "1.0"
 
+[target.'cfg(target_arch = "wasm32")'.dependencies]
+naga_oil = { version = "0.11" }
+
+[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
+# Omit the `glsl` feature in non-WebAssembly by default.
+naga_oil = { version = "0.11", default-features = false, features = [
+  "test_shader",
+] }
+
 [lints]
 workspace = true

crates/bevy_internal/Cargo.toml

@@ -1,4 +1,7 @@
-use crate::{MeshPipeline, MeshViewBindGroup, ScreenSpaceAmbientOcclusionSettings};
+use crate::{
+    environment_map::RenderViewEnvironmentMaps, MeshPipeline, MeshViewBindGroup,
+    ScreenSpaceAmbientOcclusionSettings, ViewLightProbesUniformOffset,
+};
 use bevy_app::prelude::*;
 use bevy_asset::{load_internal_asset, Handle};
 use bevy_core_pipeline::{
@@ -14,25 +17,17 @@ use bevy_render::{
     extract_component::{
         ComponentUniforms, ExtractComponent, ExtractComponentPlugin, UniformComponentPlugin,
     },
-    render_asset::RenderAssets,
-    render_graph::{NodeRunError, RenderGraphContext, ViewNode, ViewNodeRunner},
-    render_resource::{
-        binding_types::uniform_buffer, Operations, PipelineCache, RenderPassDescriptor,
-    },
+    render_graph::{NodeRunError, RenderGraphApp, RenderGraphContext, ViewNode, ViewNodeRunner},
+    render_resource::binding_types::uniform_buffer,
+    render_resource::*,
     renderer::{RenderContext, RenderDevice},
-    texture::Image,
-    view::{ViewTarget, ViewUniformOffset},
-    Render, RenderSet,
-};
-
-use bevy_render::{
-    render_graph::RenderGraphApp, render_resource::*, texture::BevyDefault, view::ExtractedView,
-    RenderApp,
+    texture::BevyDefault,
+    view::{ExtractedView, ViewTarget, ViewUniformOffset},
+    Render, RenderApp, RenderSet,
 };
 
 use crate::{
-    EnvironmentMapLight, MeshPipelineKey, ShadowFilteringMethod, ViewFogUniformOffset,
-    ViewLightsUniformOffset,
+    MeshPipelineKey, ShadowFilteringMethod, ViewFogUniformOffset, ViewLightsUniformOffset,
 };
 
 pub struct DeferredPbrLightingPlugin;
@@ -151,6 +146,7 @@ impl ViewNode for DeferredOpaquePass3dPbrLightingNode {
         &'static ViewUniformOffset,
         &'static ViewLightsUniformOffset,
         &'static ViewFogUniformOffset,
+        &'static ViewLightProbesUniformOffset,
         &'static MeshViewBindGroup,
         &'static ViewTarget,
         &'static DeferredLightingIdDepthTexture,
@@ -165,6 +161,7 @@ impl ViewNode for DeferredOpaquePass3dPbrLightingNode {
             view_uniform_offset,
             view_lights_offset,
             view_fog_offset,
+            view_light_probes_offset,
             mesh_view_bind_group,
             target,
             deferred_lighting_id_depth_texture,
@@ -218,6 +215,7 @@ impl ViewNode for DeferredOpaquePass3dPbrLightingNode {
                 view_uniform_offset.offset,
                 view_lights_offset.offset,
                 view_fog_offset.offset,
+                **view_light_probes_offset,
             ],
         );
         render_pass.set_bind_group(1, &bind_group_1, &[]);
@@ -403,28 +401,27 @@ pub fn prepare_deferred_lighting_pipelines(
             &ExtractedView,
             Option<&Tonemapping>,
             Option<&DebandDither>,
-            Option<&EnvironmentMapLight>,
             Option<&ShadowFilteringMethod>,
             Has<ScreenSpaceAmbientOcclusionSettings>,
             (
                 Has<NormalPrepass>,
                 Has<DepthPrepass>,
                 Has<MotionVectorPrepass>,
             ),
+            Has<RenderViewEnvironmentMaps>,
         ),
         With<DeferredPrepass>,
     >,
-    images: Res<RenderAssets<Image>>,
 ) {
     for (
         entity,
         view,
         tonemapping,
         dither,
-        environment_map,
         shadow_filter_method,
         ssao,
         (normal_prepass, depth_prepass, motion_vector_prepass),
+        has_environment_maps,
     ) in &views
     {
         let mut view_key = MeshPipelineKey::from_hdr(view.hdr);
@@ -471,11 +468,10 @@ pub fn prepare_deferred_lighting_pipelines(
             view_key |= MeshPipelineKey::SCREEN_SPACE_AMBIENT_OCCLUSION;
         }
 
-        let environment_map_loaded = match environment_map {
-            Some(environment_map) => environment_map.is_loaded(&images),
-            None => false,
-        };
-        if environment_map_loaded {
+        // We don't need to check to see whether the environment map is loaded
+        // because [`gather_light_probes`] already checked that for us before
+        // adding the [`RenderViewEnvironmentMaps`] component.
+        if has_environment_maps {
             view_key |= MeshPipelineKey::ENVIRONMENT_MAP;
         }
 

crates/bevy_pbr/Cargo.toml

@@ -3,10 +3,10 @@ pub mod wireframe;
 mod alpha;
 mod bundle;
 pub mod deferred;
-mod environment_map;
 mod extended_material;
 mod fog;
 mod light;
+mod light_probe;
 mod lightmap;
 mod material;
 mod parallax;
@@ -17,10 +17,10 @@ mod ssao;
 
 pub use alpha::*;
 pub use bundle::*;
-pub use environment_map::EnvironmentMapLight;
 pub use extended_material::*;
 pub use fog::*;
 pub use light::*;
+pub use light_probe::*;
 pub use lightmap::*;
 pub use material::*;
 pub use parallax::*;
@@ -37,9 +37,12 @@ pub mod prelude {
             DirectionalLightBundle, MaterialMeshBundle, PbrBundle, PointLightBundle,
             SpotLightBundle,
         },
-        environment_map::EnvironmentMapLight,
         fog::{FogFalloff, FogSettings},
         light::{AmbientLight, DirectionalLight, PointLight, SpotLight},
+        light_probe::{
+            environment_map::{EnvironmentMapLight, ReflectionProbeBundle},
+            LightProbe,
+        },
         material::{Material, MaterialPlugin},
         parallax::ParallaxMappingMethod,
         pbr_material::StandardMaterial,
@@ -71,7 +74,6 @@ use bevy_render::{
     ExtractSchedule, Render, RenderApp, RenderSet,
 };
 use bevy_transform::TransformSystem;
-use environment_map::EnvironmentMapPlugin;
 
 use crate::deferred::DeferredPbrLightingPlugin;
 
@@ -255,12 +257,12 @@ impl Plugin for PbrPlugin {
                     ..Default::default()
                 },
                 ScreenSpaceAmbientOcclusionPlugin,
-                EnvironmentMapPlugin,
                 ExtractResourcePlugin::<AmbientLight>::default(),
                 FogPlugin,
                 ExtractResourcePlugin::<DefaultOpaqueRendererMethod>::default(),
                 ExtractComponentPlugin::<ShadowFilteringMethod>::default(),
                 LightmapPlugin,
+                LightProbePlugin,
             ))
             .configure_sets(
                 PostUpdate,