Adding new interactive features to OpenGLRenderer

[MertAltug]

Hi,
I was wondering if there is a way to move camera in Z direction during rendering.
Currently, WSAD or arrows are used but it would be great if we could move camera along normal to WSAD plane or Z in global coordinates as well.
In general is there a way to add new interactive features to render scene? For example, when mouse left-click is done adding some particles or applying some forces/displacements.
Ideally, these new interactive features should automatically be deduced from an action space of an RL environment. Moreover, rendering should take into account the observation space of that environment too.
So, a user can record some trajectories for imitation learning in simulation.
Best regards

[momo-van]

@MertAltug

• Global-Z camera motion: I believe this is not available today as a built-in viewer/navigation control. Camera pose can be set programmatically though through through APIs like set_camera(...)
• Custom interactive viewer features: the hooks appear to exist already, so users should be able to implement more complex custom logic on top of the current viewer. The renderer/viewer seems to expose extensibility points such as UI callbacks, picking, and interactive force application (register_ui_callback(...), viewer.apply_forces(...), picking_enabled) which could support features like click-to-apply-force, click-to-add-particles, custom displacement tools, or other task-specific interactions.
• Automatic RL action/observation-driven interaction and rendering: @rubengrandia for viz.

@preist-nvidia for viz