Using an asset handle inside an async task?

[viridia]

I have an async task which is used for terrain generation. The task generates new terrain patches as the player moves around the world, allowing for worlds of unbounded size.

The terrain generator depends on an asset that contains data used during the generation process. This asset only needs to be loaded once at startup, and remains resident in memory for the life of the program.

I'm having trouble figuring out how to code this. If I store the asset handle in a resource, the borrow checker complains when I try to use the handle inside the closure for the task.

Also, I would like the task closure to be able to wait until the asset is loaded, but there seems to be no way to await for an asset. An alternative is to delay spawning the task until the asset is loaded, but that makes the logic more complex and doesn't generalize to other use cases very well. (In the future I might need more assets inside of tasks, and having to delay spawning the task until all those assets are loaded makes the code very complicated. It's simpler to just await for each loadable resource.)

Can anyone provide me with a code example showing how to do this?

[RCoder01]

I may be misunderstanding your question, but handles and the asset server are inexpensively cloneable:

#[derive(Debug, Resource)]
struct MyHandle(Handle<Foo>);

fn terrain_gen(my_handle: Res<MyHandle>, server: Res<AssetServer>) {
    let handle = my_handle.0.clone();
    let server = server.as_ref().clone();
    ComputeTaskPool::get().spawn(async move {
        pre_calculation().await; // do some calculation before
        // from now on, asset is needed
        loop { // idk if this blocks the compute thread
            match server.get_load_state(&handle) {
                LoadState::Loaded => { break; },
                _ => {}, // unloaded, loading, or failed
            }
        }
        async_foo(handle).await; // use handle somehow
    }).detach();
}

[RCoder01]

assets.get() only gives you a reference to the asset, to get an owned asset without cloning, you have to remove it from asset storage with assets.remove()

[viridia]

OK, that's progress. Here's what I am getting now:

if let Some(asset) = assets.remove(&handle.handle) {
  commands.insert_resource(TerrainShapesResource(Arc::new(Mutex::new(asset))));
}

cannot borrow data in dereference of `bevy::prelude::Res<'_, bevy::prelude::Assets<TerrainShapesAsset>>` as mutable
trait `DerefMut` is required to modify through a dereference, but it is not implemented for `bevy::prelude::Res<'_, bevy::prelude::Assets<TerrainShapesAsset>>`

[viridia]

This seems like it would be so much simpler if I could call assets.get() inside the task closure. I understand that the closure can't have references to the world - but is the asset server part of the world? I would think that an asset server would have a very long lifetime. The closure will have a strong handle, which will prevent the asset from being unloaded while the closure is running.

Also, what happens when TerrainShapes changes (due to terrain sculpting in the editor)? At this point, I'd want to re-run the task with the updated asset. But if the asset is stored in a resource, it means I have to write additional code to keep the resource in sync, so that the task can pick up the changes.

My concern here is that "TerrainShapes" isn't the only resource that I have which follows this pattern. My game (which I'm porting from three.js) has many different kinds of assets that are handled similarly - for example, there's a table of Biome types which is also a global asset and is also used in terrain mesh generation. Having to create two resources and two systems for every asset type seems like a lot of boilerplate - at least, when you compare it to the JavaScript version.

Apologies for all the questions, I'm fairly new to both Bevy and Rust.

[RCoder01]

I'm also pretty new to bevy, and I'm certain there's a better way to do all this (I have to do something similar in my project), but I don't know what it is.

The reason I don't think giving the task closure any sort of reference to the assets is possible is that TaskPool::spawn requires a "future: impl Future<Output = T> + Send + 'static", and since the asset resource itself will live only as long as the world, I don't think any kind of 'static reference to it is (safely) possible.

Also the reason this is much easier in JS is because JS is garbage collected and single threaded so there's no worry about leaking memory and less concurrent access memory concerns (maybe none at all, but I'm not a JS expert).

[viridia]

OK so I think I figured it out. The problem is that "Arc" can't derive Deserialize. This has to do with the fact that in the code example for custom resources, the type of the asset, and the raw type being loaded from a file are the same type. So that type derives all of the traits needed for both deserialization and for being an asset.

The problem comes when you need to do some kind of data conversion in the asset loader - in this case, wrapping an Arc and Mutex around the returned value. So you need two types (which I realized early on), but you also need to insure that those two types aren't trying to do too much by deriving from traits you don't need. So the "source" type needs the Deserialize trait, while the "asset" type needs the TypeUuid and TypePath traits. If you accidentally cut & paste such that your asset type is deriving deserialize, it won't work because Arc can't support that trait - but it doesn't need it either.

The other thing I realized is "don't poll the asset loading status, use asset events". That is, if you have a system which is checking whether the asset has finished loading, use an event reader, because this will not only tell you when the asset is loaded, it will also tell you when it has changed - which allows you to update the consumers of that asset and mark them for rebuild.