Add more control over resources loading

[arthuro555]

Description

When GDevelop loads up, it takes much time preloading everything. Sometimes though, you may want to have a game that loads fast initially and then have some loading screens to load the missing stuff.

Solution suggested

Instead of forcing preloading resources to the user, we could take a similar approach to what was done in #2006 :

1. Add a property on the resources to choose if it should be preloaded or not
2. Add actions to preload and unload resources
3. Modify the loaded resources getters to load the resource if it hasn't been loaded yet or has been unloaded

That way, the game never breaks, but it gives a way for users to control what is loaded or not to have a better memory management, and to make their own "loading screens" by initially loading only the loading screen assets so that the gdevelop splash screen can be hidden as it is too fast to see anyways, and using a GDevelop scene using the actions to load what they need and display the on screen feedback they want in the meantime.

[4ian]

In short: yes! :)

That's definitely something I want to have at some point: be able to disable the initial loading of some/almost all resources, then do the loading whenever it's needed (and potentially also have a way to unload, or at least specify that something can be unloaded).
See this card: https://trello.com/c/TjYFGhQV/302-improve-loading-by-only-loading-resources-of-the-first-scene-at-startup-and-load-the-rest-when-needed

Modify the loaded resources getters to load the resource if it hasn't been loaded yet or has been unloaded

This may create some weird behavior in the sense that for example, if you set a sprite image to be not loaded at the beginning, then it should be loaded whenever an instance of an object that use it is created in the runtime scene.
The problem is that loading a resource is not something that is immediate: it will have at least a few frames during which the image is not loaded. What should we do at this moment:

• Block the game while the resource is loaded? (if you don't want to block the game, then take care of preloading it in a scene before using the action)
• Don't block the game, so that the object won't show anything until the image is loaded at which point it will "pop" on the screen. This is what we do for the scene editor... but it's because the scene editor, we don't care about this effect and value the simplicity first. Not sure for a game it's a good idea because this means for example that while the image is loading, the object width and height will be 0... can be a problem for collisions and even for events/logic of the game.

unload resources

If you want to investigate a bit, it would be interesting to see what Pixi.js offers for this. Could be a good idea to have a way to monitor the usage in terms of memory/GPU memory too. That's something we could maybe expose using some Electron apis (at least just for developers) (I was also thinking of having a dialog in the editor that would also show the usage in terms of memory, to see how well the editor handles big games)

[arthuro555]

I am also thinking about some methods to make loading easier:

1. We could introduce resource groups. Resource groups can be used to group resources, and the user can simply tell load resources "resource group" and tell to unload them. For pixi this is very simple to do, we can simply use a separate loader for every group and call loader.destroy(), if I understood correctly this will destroy all webgl context of the textures, and remove them from the textures cache. All that would be left to do is removing them from the image manager so that they can be garbage collected. There would be a default resource group "all resources" and one per scene that contains resources used by scenes so that users can make their own loading screen by disabling the default one and loading all resources in their scene or loading the scene resources before switching to any scene. The groups that contain multiple resources would be internally split in two groups, so that only resources of one type are grouped together to make loading of a resources group easier.
2. Add a way to load a scene in the background (make the loading of scene async and load it's used resources asynchronously), with callback variables so that user can make a simple loading scene (load the scene in the background, get percentage done via callback variables, and once the callback variables says it is done switch to the newly loaded scene).

Both of those methods would have a callback variable with the amount of things to load, the amount of things loaded and if it finished loading.

The problem is that loading a resource is not something that is immediate: it will have at least a few frames during which the image is not loaded. What should we do at this moment:

Imo we should do both. For that, we can either use a project property or an action "block until all resources are loaded" or a condition "all resources loaded". The default should be not blocking though, as it is up to the user to implement a loading screen to hide the loading process I think.

the object width and height will be 0... can be a problem for collisions and even for events/logic of the game.

We could pregenerate a transparent image of the size of the image to load to put there in the meantime maybe?

[4ian]

We could introduce resource groups

Speaking of this, there is already a notion of resource groups/resource folders in the codebase, which is not exposed in the editor. Though it requires a lot of painful manual work to organize your resources. Might be better to focus on loading resources for a scene - or autogenerate groups for a scene :)

get percentage done via callback variables, and once the callback variables says it is done switch to the newly loaded scene)

We might not need "callback" variables, just an expression giving the percentage of the progress of the currently loaded resources, and a condition checking if the resources have just finished loading.

We could pregenerate a transparent image of the size of the image to load to put there in the meantime maybe?

I would avoid this because of confusion that this can make ("why is my object not showing", "actually it is showing", "but the animation is messed up because it started not at the right time because it was loading").

Maybe to get things rolling on this, we could start with an experiment:

• See how calling destroy on the Pixi loader impacts a running game. We could imagine using one loader per scene, so that when we switch scene we destroy the old one and load the "resource group" of the next scene with a new loader (see the discussion here: https://www.html5gamedevs.com/topic/42153-pixiloader-how-to-clear-resources-up/).
• See how calling destroy on a texture individually works.

I think overall the biggest challenge is to do the work of identifying which resources are used in the scenes. Fortunately, it should be doable with gd::Project::ExposeResources method (we should probably make one for each scene), though it will need a bit of C++ and testing to make it correct. We can then persist in the exported/preview game file the list of resources of each scene.

Then, we it should be fairly easy when loading a new scene/unloading a scene/changing to a new scene to compute the difference (the set of new resources minus the set of old resources), and load the new resources/unload the ones that are now useless :)

If you want to experiment but don't feel like doing the ExposeResources part, just ignore this and I'll do this later myself.
It would be good to check how the Pixi texture cache reacts when we call destroy - to verify things are unloaded. Also all of this discussion is about images. For audio, we would need probably your refactoring PR to go ahead first. For JSON, it's easy to just drop the reference if asked to unload.