Wait, the Handle is the Asset?

[ElliottjPierce]

With assets as entities on the horizon, I've been thinking more about the asset system and how we could redesign it with assets as entities to improve performance and user experience in a number of ways.

What is an asset? I mean really, what is the definition of an asset in bevy? The best I can come up with is an asset is "some data referenced in many places but only stored in one." Not all assets are loadable; not all are savable, and not all of them are ever on the file system. It's just some data stored in one place (soon on an entity) and referenced in many places (currently with a handle instead of a pointer).

Existing Problems

Currently, assets live in a standard storage in an Assets resource and are referenced by asking that resource for the asset with a particular Handle. There's a few issues with that. For one, assets can't be used in async code without first cloning the asset out of Assets storage. That's especially painful for saving an asset. Another issue is that they have to be cloned between worlds. That puts one copy on the render world and one on the main world for things like meshes, images, etc. RenderAssetUsage is a clunky way around that, but ideally, we have a general solution to this. Third, for immutable assets (like most images) it's really painful (and slow) to need to look up an asset in the Assets resource. The image isn't changing, so why jump through those maps?

#15813 (putting every asset in arcs) is one solution to this. But while it's better than present, it still has many drawbacks. For one, not all assets benefit from being arced. Take player save data for example. That is common data stored in one placed and referenced from many. (It's an asset.) Ideally, we want to modify it as the player progresses and occasionally clone it for a save process. With Arcs We can't modify it easily, and we'd end up needing to clone it anyway for mutation. In other words, Arc helps with data you read many times and rarely write to (ex: Meshes, Images, etc.), but Arc is really painful for assets that are rarely sent between threads, world, etc but are updated often (ex: save data, procedural meshes, player preferences, etc.).

A New Solution

What if the handle was the asset?

Consider this layout:

pub struct Handle<A> {
    storage: Option<Entity>,
    keep_alive: Arc<()>,
    asset: Arc<AssetRef<A>>,
}

struct AssetRef<A> {
    current: AssetHolder<A>,
    updated: OnceLock<Arc<AssetRef<A>>>
}

enum AssetHolder<A> {
    Missing,
    Present(A),
    Static(&'static A),
}

There's a lot going on here, so let me explain as best I can.

The Handle stores the Entity that holds it's actual storage (more on that in a bit). If the Handle::storage is None, the asset has no storage entity yet. This is to replace the current Handle::weak_from_128; instead of being weak, the storage can just be None. Handle::keep_alive is just a per-asset arc to track if the asset is used or not. The handle also stores the Handle::asset itself, which is an Arc<AssetRef>.

AssetRef stores the current value of the asset in AssetRef::current, an AssetHolder. This is either Missing (ex: the asset is loading), Present (ex: the asset is loaded) or Static (ex: to facilitate Handle::weak_from_128 stuff). AssetRef also stores an updated which is a OnceLock<Arc<AssetRef<A>>>. This tracks, the next version of the asset. For example, if the asset is modified, a new Arc<AssetRef<A>> is created, and the OnceLock on the old one is set.

To get an asset from &Handle, just grab the current, AssetRef's AssetHolder, deref it and return the &'handle A. If it is Missing, return None (just like Assets::get). To try to update an asset from &mut Handle, check the AssetRef::updated's OnceLock and replace the Handle::asset with the new one if it exists.

I'm just scratching the surface here. The big deal is that Handle never have to guess when their asset has changed, they can see it (and update it) immediately. This is lossless, inexpensive, granular change detection. Fast for rarely changed assets and usable for frequently changed.

Now the storage:

#[derive(Component)]
struct AssetStorage<A> {
    asset: Option<A>,
    handle: Handle<A>,
    mode: AssetStorageMode,
    path: Option<AssetPath>,
    // any other asset data/metadata the asset server may need.
}

enum AssetStorageMode {
    DontArc,
    UpdatedOften,
    UpdatedRarely,
}

AssetStorage is the component that lives on the Handle::storage Entity. It holds a root Handle. This is how it can update AssetRef::updated's OnceLock. It also allows users to get handles from a Query<AssetStorage<MyAsset>>. It also stores an Option<AssetPath> in case it was loaded from a file. This is pretty standard, but what about AssetStorage::asset and AssetStorageMode?

AssetStorage::asset is the most up to date asset version. Any changes to the asset will flow through this value, ex: Query<&mut AssetStorage<MyAsset>>. Systems can then run on Query<&mut AssetStorage<MyAsset>, Changed<AssetStorage<MyAsset>>> (or something) to post those updates to other handles. This is another way to do asset change detection! If a system want's the most up-to-date version, it can just pull from Query<&AssetStorage<MyAsset>> (just like the current Assets).

AssetStorageMode is the most contentious part of this. The simplest one is UpdatedRarely. That means this AssetStorage::asset is None by default. Then, when something tries to change the asset, it clones from AssetStorage::handle's asset, putting the cloned value into AssetStorage::asset, the allowing that to be modified. This change is later picked up by a different system to post the change to the Handle, leaving AssetStorage::asset None again. UpdatedOften is similar, except that AssetStorage::asset is always Some, and when it is updated, it is cloned into the Handle. Finally, DontArc keeps the asset only in AssetStorage::asset as Some. It never clones into the Handle, instead keeping it AssetHolder::Missing until the mode is changed. (We can change the enum to help with invariants; I'm trying to be brief.)

UpdatedRarely is great for static items like Image, and it only stores one copy of the asset; UpdatedOften is great for assets that change frequently like a procedural Mesh. DontArc is great for assets that change so much or are referenced so rarely that arcing isn't worth it (ex: save data).

Each AssetStorageMode has a purpose, and a user can change it as needed. Users also don't need to wait for a system to post any changes; we can provide a post method, etc. There's lots of room for expansion here; we could add other handle types for different modes that could skip some checks, etc. But I'm trying to be brief.

UX

Example: A Handle::weak_from_128 shader. Include the shader in the binary and create a static LazyLock<Handle<Shader>> which creates the Handle by giving it a Handle::storage of None and a AssetHolder::Present of the parsed Shader. Now clone and pass around the Handle as you like. If you want the Shader to be discoverable, you can spawn a AssetStorage entity for it. If you want to respond to mutations from it being discoverable, you'll need to either clone and update the static handle before using it, or make it a RwLock<Option<Handle<Shader>>> or something.

Example: An image loaded from a file. Ask the asset server to load the asset, giving it a AssetStorageMode. (We can provide a default here too.) The asset server reserves an entity remotely, creates a Handle with that entity and AssetHolder::missing, and starts loading the asset. The caller can check on the loading process by seeing if its Handle's AssetRef::updated has changed yet.

Example: Adding an asset manually. (We'd probably make a command for this, but this is what it would do.) Reserve an Entity. Create a Handle with that entity and AssetHolder::present. Spawn that Entity with a AssetStorage configured according to the desired AssetStorageMode.

Example: Changing a mesh. Make a Query<&mut AssetStorage<Mesh>> and get the entity assocated with the desired Handle::storage. If that is None or the entity doesn't exist, the Handle is not meant to be changed (ex: for static assets in the binary). On that &mut AssetStorage<Mesh>, request an &mut Mesh and change it. A change detection system will post the updated mesh to all Handles.

Example: Cleaning unused assets. Make a Query<(Entity, &mut AssetStorage<Mesh>)>. If any AssetStorage's``Handle::keep_aliveis the only strong arc, the entity can be despawned, dropping theHandle` and all copies of the asset.

There's a ton of flexibility here but in the interest of time, I'll leave the examples here. The performance and UX of this is much better than needing to go though Assets and mapping ids all the time. The built-in Arcs make sharing between threads, worlds, and async trivial. Change detection is fast, easy, and hard to accidentally miss. It works well with assets as entities, etc.

That said, I'm no asset expert, so there could be things I'm missing. Your thoughts are welcome.

[andriyDev]

Ok after giving this a more careful read I see some of the merits. I still don't like it, but it is more understandable. Just to summarize my understanding: to mutate an asset, you now go to AssetStorage::handle, clone the value stored there into AssetStorage::asset, mutate that value, then in the future, another system comes along and moves AssetStorage::asset into a new Arc<AssetRef> which is then inserted into AssetRef::updated for the previous handle. Or we add a post method that users need to explicitly call to commit the mutated asset (which then will require a whole other clone if you want to mutate again).

There's a couple things that I don't like here. The first is just the complexity here. The indirection goes crazy. Mutates are actually clones that don't apply immediately, and you have to wait for a frame for it to be applied.

The second is that every write (for AssetStorageMode != DontArc) unconditionally clones. Today, Assets don't require clone-ability. This is something I think is worth considering requiring but ideally our system would still allow non-clone assets. Without specialization, I don't think we can disable arcing for non-Clone assets. Even putting that aside, ideally we shouldn't need to clone unless we are aliasing - if there is no one referencing an Image right now (through an Arc), why should we need to clone it? Asset Arc-ing doesn't have this issue (though of course it has other considerations).

One thing I wasn't clear on is if you have a handle, does accessing the asset through it access the currently held Arc? Or does it chain up to the most recent instance? If it's the latter then A) that can be a lot of indirection if you don't have a mutable reference to the handle to collapse the chain, B) a mutation can change the handle in the middle of the read resulting in different data between each deref. Alternatively, if it's the former, then what are we even winning here? Now users need to regularly get a mutable reference to their handle and "upgrade" it to the newest version? That also doesn't seem ideal, from a UX experienece or a memory-usage perspective (if you forget a handle, you may accidentally be holding many different versions of an asset in memory).

Please let me know if I've misunderstood something here!

[ElliottjPierce]

I think we're mostly on the same page now, and you make some good points.

First, about the indirection and linked list chain situation, we could use RwLock instead of OnceLock to make this flat. I kinda like that. That would also fix the large memory problem if a handle isn't being updated. It would also mean the handle deref would always go to the current version of the asset, not the most up-to-date.

For the cloning everywhere, I don't like that either, but it almost never happens. When's the last time you couldn't do something without modifying an image every frame? Even so, Arc'd assets has the same problem. The only time it avoids the clone is if nobody else has an arc reference. That works fine for main world assets, but it becomes a book keeping pain as soon as it's shipped anywhere. Plus, if the asset is being saved via an arc clone (or any other async task), you still need to clone on write. For the use case of a consistently changing asset, you'd use AssetStorageMode::DontArc.

About the discrepancy between a handle and the true asset value, the handle would be designed for lazy use (by far the most common). If an image changes, who cares about a one frame delay? If a system truly needs the most up-to-date asset, it would need to Query<&AssetStorage<MyAsset>>. That could still have a delay (as could the current system) depending on system ordering.

It is definitely compelling though that Arc'd assets could avoid the clone "out of the box" for local assets. Maybe we could make Assets<A> hold a custom type that wraps Arc<A>. Then maybe have a RemoteHandle<A>? I do like arc'd assets (a lot) but the book keeping is what bothers me. People have to constantly either detect changes or check for updates. If we could bake that book keeping into a RemoteHandle that automatically detects changes and provides a direct upgrade path, I'd be satisfied.

Does that kinda make sense? I think you've persuaded me away from an Arc in the Handle because of the cloning issues. But I don't like putting the change detection book keeping on users with arc'd assets. I'd much rather create a pretty interface that handles itself automagically.

[ElliottjPierce]

Oh, btw this proposal also ends weak_from_128, but IDK how big a deal that is with Construct coming.

[PixelDust22]

It sounds like you're enforcing a copy-on-write scheme for everyone. What if I have a really big piece of asset that I really don't want to clone?

When's the last time you couldn't do something without modifying an image every frame?

Yeah small images are probably fine. But what if I have a 2GB 3D volumetric texture and I want to update 1 pixel of that texture?