Commit e7b64b6
Add support for arbitrary/third party glTF Extension processing via GltfExtensionHandler (#22106)
# Objective
Currently Bevy doesn't support arbitrary glTF extensions. The ones it
does support are hardcoded.
We should support glTF extensions, as this is a primary mechanism for
sharing behavior via data exported from applications like Blender.
I personally have found usecases in exporting component data, lightmap
textures/information, and processing other kinds of data
(AnimationGraph, 3d meshes into 2d, etc).
## Solution
This PR introduces a new `GltfExtensionHandler` trait that users can
implement and add to the glTF loader processing via inserting into a
Resource.
There are two example processors currently added, with a third that I'd
like to add after this PR.
- `examples/gltf/gltf_extension_animation_graph.rs` duplicates the
functionality of `animation_mesh`, constructing AnimationGraphs via
extension processing and applying them to be played on the relevant
nodes.
- `examples/gltf/gltf_extension_mesh_2d.rs` duplicates the functionality
of the `custom_gltf_vertex_attribute` example, showing how the extension
processing could be used to convert 3d meshes to 2d meshes alongside
custom materials.
Both of these examples re-use existing assets and thus don't *actually
use* extension data, but show how one could access the relevant data to
say, only convert specifically labelled Mesh3ds to 2d, or process many
animations into multiple graphs based on extension-data based labelling
introduced in Blender.
A third example I want to introduce after this PR is the same core
functionality Skein requires: an example that uses reflected component
data stored in glTF extensions and inserts that data onto the relevant
entities, resulting in scenes that are "ready to go".
## Comparison to Extras
In comparison to extensions: data placed in glTF extras is well
supported through the `GltfExtras` category of components.
Extras only support adding an additional `extras` field to any object.
Data stored in extras is application-specific. It should be usable by
Bevy developers to implement their own, application-specific, data
transfer. This is supported by applications like Blender through the
application of Custom Properties.
Once data is used by more than one application, it belongs in a glTF
extension.
## What is a glTF Extension?
Extensions are named with a prefix like `KHR` or `EXT`. Bevy has already
reserved the `BEVY` namespace for this, which is listed in the official
[prefix
list](https://github.com/KhronosGroup/glTF/blob/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions/Prefixes.md).
For a glTF file, an extension must be listed in `extensionsUsed` and
optionally `extensionsRequired`.
```
{
"extensionsRequired": [
"KHR_texture_transform"
],
"extensionsUsed": [
"KHR_texture_transform"
]
}
```
Extension data is allowed in any place extras are also allowed, but also
allow much more flexibility.
Extensions are also allowed to define global data, add additional binary
chunks, and more.
For meshes, extensions can add additional attribute names, accessor
types, and/or component types
`KHR_lights_punctual` is a contained and understandable example of an
extension:
https://github.com/KhronosGroup/glTF/blob/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions/2.0/Khronos/KHR_lights_punctual/README.md
. This one happens to be already hardcoded into Bevy's handling, so it
doesn't benefit from arbitrary extension processing, but there are
additional
[ratified](https://github.com/KhronosGroup/glTF/tree/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions#ratified-khronos-extensions)
and
[in-progress](https://github.com/KhronosGroup/glTF/tree/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions#in-progress-khronos-and-multi-vendor-extensions-and-projects)
extensions, as well as
[vendor](https://github.com/KhronosGroup/glTF/tree/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions#vendor-extensions)
and other arbitrary extensions that would benefit from userland support.
## Implementation
This initial implementation is reasonably minimal: enabling extension
processing for objects/etc as they're loaded which may also define
extension data, including the scene world. This may leave out useful
functionality; as detailed in the next section: "What's not
implemented".
Extension handlers are defined by implementing a trait which can
optionally define hooks and data.
Extension handler data is cloned to start with a fresh slate for each
glTF load, which limits scope to "one glTF load".
So while state can be maintained across hooks during a single load,
users who want to combine or handle multiple glTF assets should do so in
the main app, not in an extension handler.
Following this, because the extensions are stored as `dyn GltfExtension`
*and* we want to clone them to isolate state to a single load,
`dyn_clone` must be included as a workaround to enable this cloning.
An extension handler has to be added to the list of handler by accessing
a `Resource` and pushing an instantiated handler into it.
This Resource keeps the list of extension handlers so that a new glTF
loader can bootstrap them.
The design of the hooks is such that:
- If no extensions handlers are registered, none are called for
processing
- If an extension handler is defined, it receives all "events"
- handlers are defined by a trait, and default implementations are
called if an override is not specified.
- default implementations are no-ops
It is important that extensions receive all events because certain
information is not embedded in extension data.
For example, processing animation data into an animation graph could
require both processing animations with extension data, tracking the
animation roots through hooks like `on_node`, *and* applying those
graphs in the `on_scene_completed` hook.
- Extension data is passed to hooks as `Option<&serde_json::Value>`
which is only passing references around as the data has already been
converted to `Value` by the `gltf` crate.
- `LoadContext` is required for creating any new additional assets, like
`AnimationGraph`s.
- *scene* World access is provided in hooks like `on_scene_completed`,
which allows calculating data over the course of a glTF load and
applying it to a Scene.
### What's not implemented
This PR chooses to *not* implement some features that it could. Instead
the approach in this PR is to offer up the data that Bevy has already
processed to extensions to do more with that data.
- Overriding `load_image`/`process_loaded_texture`
- This could allow projects like bevy_web_codecs, [which currently forks
the entire gltf
loader](https://github.com/jf908/bevy_web_codecs/tree/373bbf29be6555c7603fd6867a01159ab0f20fed/bevy_web_codecs_gltf).
Associated [issue](#21185).
However I believe this needs some design work dedicated to what exactly
happens here to support that use case.
- This PR doesn't include any refactoring of the glTF loader, which I
feel is important for a first merge.
- ~~There is some benefit to passing in the relevant `gltf::*` object to
every hook. For example, I believe this is the only way to access
extension data for `KHR_lights_punctual`, and
[`KHR_materials_variants`](https://docs.rs/gltf/1.4.1/gltf/struct.Document.html#method.variants)
or other extensions with "built-in" support. I haven't done this in all
places.~~ (edit: after external implementation I decided this was a good
idea and added it to more places)
## Testing
```
cargo run --example gltf_extension_animation_graph
cargo run --example gltf_extension_mesh_2d
```
---
## Showcase
Both examples running:
https://github.com/user-attachments/assets/f9e7c3c9-cdad-4d33-ace7-7c2ca5469d5e
https://github.com/user-attachments/assets/baa9bc92-ca3b-46ad-a3f0-2f74bbc29b68
<details>
<summary>An example that showcases converting Mesh3d to Mesh2d</summary>
```rust
#[derive(Default, Clone)]
struct GltfExtensionProcessorToMesh2d;
impl GltfExtensionProcessor for GltfExtensionProcessorToMesh2d {
fn extension_ids(&self) -> &'static [&'static str] {
&[""]
}
fn dyn_clone(&self) -> Box<dyn GltfExtensionHandler> {
Box::new((*self).clone())
}
fn on_spawn_mesh_and_material(
&mut self,
load_context: &mut LoadContext<'_>,
_gltf_node: &gltf::Node,
entity: &mut EntityWorldMut,
) {
if let Some(mesh3d) = entity.get::<Mesh3d>()
&& let Some(_) = entity.get::<MeshMaterial3d<StandardMaterial>>()
{
let material_handle =
load_context.add_loaded_labeled_asset("AColorMaterial", (CustomMaterial {}).into());
let mesh_handle = mesh3d.0.clone();
entity
.remove::<(Mesh3d, MeshMaterial3d<StandardMaterial>)>()
.insert((Mesh2d(mesh_handle), MeshMaterial2d(material_handle.clone())));
}
}
}
```
</details>
---------
Co-authored-by: Kristoffer Søholm <k.soeholm@gmail.com>1 parent d042f8d commit e7b64b6
File tree
9 files changed
+821
-9
lines changed- assets/models/barycentric
- crates/bevy_gltf
- src
- loader
- extensions
- examples
- gltf
9 files changed
+821
-9
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
732 | 732 | | |
733 | 733 | | |
734 | 734 | | |
| 735 | + | |
735 | 736 | | |
736 | 737 | | |
737 | 738 | | |
| |||
4201 | 4202 | | |
4202 | 4203 | | |
4203 | 4204 | | |
| 4205 | + | |
| 4206 | + | |
| 4207 | + | |
| 4208 | + | |
| 4209 | + | |
| 4210 | + | |
| 4211 | + | |
| 4212 | + | |
| 4213 | + | |
| 4214 | + | |
| 4215 | + | |
| 4216 | + | |
| 4217 | + | |
| 4218 | + | |
| 4219 | + | |
| 4220 | + | |
| 4221 | + | |
| 4222 | + | |
| 4223 | + | |
| 4224 | + | |
| 4225 | + | |
| 4226 | + | |
4204 | 4227 | | |
4205 | 4228 | | |
4206 | 4229 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
2 | 16 | | |
3 | 17 | | |
4 | 18 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
| 58 | + | |
58 | 59 | | |
59 | 60 | | |
60 | 61 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| 158 | + | |
| 159 | + | |
158 | 160 | | |
159 | 161 | | |
160 | 162 | | |
| |||
249 | 251 | | |
250 | 252 | | |
251 | 253 | | |
252 | | - | |
| 254 | + | |
| 255 | + | |
253 | 256 | | |
254 | 257 | | |
255 | 258 | | |
| |||
267 | 270 | | |
268 | 271 | | |
269 | 272 | | |
| 273 | + | |
| 274 | + | |
270 | 275 | | |
271 | 276 | | |
272 | 277 | | |
273 | 278 | | |
274 | 279 | | |
| 280 | + | |
275 | 281 | | |
276 | 282 | | |
277 | 283 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
7 | 23 | | |
8 | 24 | | |
9 | 25 | | |
10 | 26 | | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
0 commit comments