Draw debug text fixed to camera frame or screen space?

[LemonPi]

Currently I don't believe there's an option to draw debug text that moves with the camera (and specified in the camera frame). This would be useful for video logging of trial statistics, like which random seed the run is using, run name, and other info.

Debugging via text in the scene is created by p.addUserDebugText, which takes cartesian world frame coordinates. It should already be possible to draw relative to the camera frame via the transform you get from p.getDebugVisualizerCamera, but then you'd have to move all the rendered items manually whenever camera moves. It may not be convenient to track all the texts that are rendered in one place.

Could this be a feature for future versions? (an alternative function to p.addUserDebugText or an argument to change the specified input to be with respect to the camera frame rather than world frame). Also, could you give an example of how to draw to the camera frame (fixed) using the current functionality (given text and 3D coordinates with respect to the camera frame)? I'm not sure how to use the view matrix or projection matrix. My guess is that the view matrix is from the world frame to the camera frame?

[LemonPi]

My current solution is to cache the inverse camera transform whenever it's moved:

p.resetDebugVisualizerCamera(...)
# wait for reset
for _ in range(1000):
    p.stepSimulation()
info = p.getDebugVisualizerCamera()
view_matrix = np.array(info[2]).reshape(4, 4).T
self._inv_camera_tsf = np.linalg.inv(view_matrix)

And then for drawing to screen,

pos_in = np.r_[camera_frame_pos, 1]
world_frame_pos = self._inv_camera_tsf @ pos_in
p.addUserDebugText(text, world_frame_pos[:3], ...)

Let me know if there's a better option.