[Question] How to import a 3DGS model into a RLTask in Isaac Lab

[carlulxy]

Question

Hello everyone, I'm currently developing a reinforcement learning task in Isaac Lab and encountered a confusing issue that I hope to get some help with.
I need to import a 3D Gaussian model into the RL simulation environment, but I can't find the corresponding official API for 3D Gaussian in Isaac Lab. For reference, we have dedicated APIs for different assets: Articulation for articulated multi-joint interactive objects, Camera for camera sensors, but there is no clear API for 3D Gaussian assets.
I have tried two different methods to import the 3D Gaussian model into the environment, both of which work individually:

1. Import the 3D Gaussian model via the RigidObject API;
2. Attach the 3D Gaussian model as a child component/part of the Robot asset and load it with the robot together.

With either of the above two methods, I can successfully see the 3D Gaussian model rendered properly in the simulation environment when running with zero_agent.py.
However, a strange problem occurs: once I add a Camera component to the same task configuration, the entire simulation fails to launch completely. What's more confusing is that there are NO error messages — the simulation just hangs and cannot start.
Additional verification results:

• The simulation launches and runs normally if only the Camera component is loaded (without the 3D Gaussian model);
• The simulation also works fine if only the 3D Gaussian model is loaded (without the Camera component).

The problem only arises when the 3D Gaussian model and the Camera component exist in the environment at the same time.
My core questions are:

1. What is the correct official API/method to import a 3D Gaussian model into an Isaac Lab RL task environment?
2. Why does the combination of 3D Gaussian and Camera cause the simulation to fail to launch with no errors, and how can this issue be fixed?

Build Info

Describe the versions that you are currently using:

• Isaac Lab Version: [2.3.0]
• Isaac Sim Version: [5.0]

[RandomOakForest]

Thank you for posting this. Could you elaborate on what your task is about and what you are trying to achieve? I'll move this issue to our Discussions section for follow up. If you think you have found a bug, please post a bug report and provide details and files to reproduce the issue. Thank you.

[carlulxy]

I just want to use 3DGS-rendered backgrounds in Isaac Lab's RLtask so that the images captured by my camera appear more realistic. However, whenever both 3DGS and camera scenes appear simultaneously in the simulation, it freezes. Specifically, it freezes at env = gym.make(args_cli.task, cfg=env_cfg) in zero_agent.py. More precisely, it freezes at self.sim.reset() in direct_rl_env.py without any error.