Rooms "scripting" by Webassembly

[DerKarlos]

A Mozilla Hubs room now is static. I would like to bring re/activity and “AI” into the system.
The user should be able to “script” cyclically and reactive actions,
not in an extra Hubs cloud but for all in the the default hubs server.
Enabling JavaScript code is much do dangerous.
It could be done by an extra scripting editor, in world or external, and an interpreter or compiler, as I know it from Second Life.

This is just my idea, how to it using web-assembly, to make it fast, save and “easy" to handle:

A) The user may write the scripts in any language, supported by WASM. Well may be not the languages with large included interpreters. The size of the WASM module should be limited.

B) The room editor offers to upload a WASM module. (Later we may have modules for single objects.)
The server should check this module if it is valid WASM, does not use too much memory, only uses the expected exports and imports, defined by an API.
The server even could start the module some milliseconds and check the CPU load.

C) When a hubs user enters a room, the Hubs client will loaded, compiled and initialize the WASM module.

D) Cyclically an export “render(dtMsec)” is called to give life to the module. It will call API imports to get stats or “manipulate” objects in the Hubs room.
This may be one or all position- and rotation axis. Colors or what ever. The API may get extended by versions.
An object will be selected by its unigue name/path string. Or by an object ID, defined during initialization of the WASM module.

Example:
let clockFingerID = export.makeId(“Clock1/Minutes”);
import.set(clockFingerID, rotationZ_enum, angle);

Collisions or touches of users could be “send” to WASM too. We should start to play around with features in the API.
In later versions, even new objects in the room may be dynamically created, from simple cubes to geometries and meshes, and materials.

Let’s talk about it.

-karlos-

[robinkwilson]

This would really be awesome. You can follow our extensibility discussion on this issue: #1976
Or on our community discord #extensibility channel.

I totally agree that would be awesome. I think you could accomplish some of what you'd like to accomplish with playing with a custom client.

Definitely an on-going discussion, would love to see you on discord.

[DerKarlos]

Yep, I am following extensibility on Discord. Triggers and Actions are a good start.
Later we may use them as in- and outputs of a WASM module to get my proposed solution.
And yes, I could code it myselve in thr Hubs Cloude. I woud, if my other projects are done, wich may be never 🤪