Is there any way to use references in background tasks that take a lot of time to execute?

[davids91]

I am currently looking at the examples, videos and trying to understand how AsyncComputeTaskPool works.

I am getting the error '1 must outlive lifetime 'static when I try to use anything I previously used in my system ( query, commands, etc.. )

According to this video, anything to be used in an AsyncComputeTaskPool needs to be copied..

https://www.youtube.com/watch?v=QTUEyAZmdv4

Is that the only way to have computations in the background? If so I feel like this wastes an enormous waste of runtime..

I have a global object based on which I want to generate a mesh, which is a really expensive operation, but what is more expensive is copying the whole thing every time a new mesh needs to be created. I find that practice wasteful.. is there any other way to run a computation in the background or is copying the only way?

[nicopap]

It depends on two factors:

1. What part of that "global object" is used in the computation for the mesh? Is it the whole object? A subset of it?
2. Is the global object read only or mutable? Can the subset of the object be mutated while you run the mesh generation?

If the answer to (2) is "yes". Then the rust type system actually saved you from hitting nasty bugs! You should consider an alternative design. It's because you can't predict what happens first when two different threads try to access the same data, and it is likely that you would get two different results when running the software two different times, for the same code… This is called a race condition.

So assuming you don't mutate the global object under the feet of the generation process, you have a few options:

• Only clone the relevant data of the global object to the generation process. Divide your global object in smaller part so that it's easier to do
• Wrap the parts of the global objects that is meant to be used by mesh generation in Arcs. This way, only the reference to it will be cloned into the generation process when starting it up.

Some people might recommend using a lock such as RwLock or Mutext, but it's a bad solution in this case. Since it doesn't solve the race condition, and if you are not careful enough, some processes might stall on the lock so much that you are back to single-threaded performances.

If you use locks, consider using many locks with fine-grained access, otherwise you risk just forcing your whole game to be single threaded.

[davids91]

Hi and thank you for the swift answer! While it is easy to imagine that only part of the global object is to be used, It is really hard to actually separate it. Although it is possible, at first I'd like to investigate about the other possibilities, just as you wrote, e.g. Arc.

Mutation is to be expected on the global object, but not right now, so so far using Arc seems to be the best bet.
However I fail to see why RwLock would be a bad solution, if implemented well.. can you please elaborate on that?

[nicopap]

So the reason a RwLock could be bad is that: your mesh generation code will lock access to the global object. This means: any system that writes the global object (as part of the normal "foreground" game logic) will have to wait until generation ends (or rather, until you free the lock). This means you don't gain anything from the async task, since only one process can run at the same time. It ends up being the same processing time as running them one after the other.

Arc requires the underlying object to be immutable as long as several references to the same data exist. Since you said there is no mutation on the global object going on concurrently with the mesh generation, it means you could just wrap the global object into an Arc and call it a day.

Maybe you could give more details. What's up with that global object? Could you be more specifics? What exactly is preventing you from putting related data together?

[davids91]

As mentioned, Arc is a great first step, so thanks again for that! :)

I think I understand where you are getting at with the mesh generation logic blocking other processes, and there I see that the only way to handle this in a safe way is to copy the data.

The data object I am working with a Sparse Voxel Octree, in which a position can mean a multitude of things, and the data struct itself does not yet have a clone method and there are various questions I need to clarify before I can safely assume an effective cloning mechanism.