[Question] RayCaster mesh_prim_paths: How to scan multiple objects with single path in multi-environment setup?

[ducmngx]

Issue Description:

I'm trying to configure a RayCaster (lidar) to scan room walls in a multi-environment setup, but I'm having trouble with mesh_prim_paths configuration.

My Setup:

• Multi-environment training with /World/envs/env_0/, /World/envs/env_1/, etc.
• Room with 5 walls created as separate RigidObjectCfg objects:
  • /World/envs/env_0/wall_north
  • /World/envs/env_0/wall_south
  • /World/envs/env_0/wall_east
  • /World/envs/env_0/wall_west
  • /World/envs/env_0/wall_mid

Current RayCaster Config:

ray_caster = RayCasterCfg(
    prim_path="/World/envs/env_.*/Robot/base_link/lidar_cage",
    mesh_prim_paths=???  # This is my problem
    pattern_cfg=patterns.LidarPatternCfg(channels=1, vertical_fov_range=[0., 0.])
)

Questions:

1. Single path requirement: RayCaster seems to only accept one mesh_prim_paths string, not a list. How can I scan all 5 walls with a single path?

2. Multi-environment support: Does mesh_prim_paths support regex patterns like "/World/envs/env_.*/wall_*" or do I need explicit paths like "/World/envs/env_0/wall_north"?

3. Parent container approach: If I target a parent path like "/World/envs/env_0", will it automatically discover all child mesh objects, or do I need each wall to be under a single mesh container?

Errors I've encountered:

• ValueError: Prim path '/World/envs/env_.*' is not valid when using regex
• RuntimeError: Invalid mesh prim path when using CuboidCfg (I understand this needs to be actual mesh)
• ValueError: Number of markers cannot be zero in debug visualization

What's the recommended approach for scanning multiple room walls in a multi-environment RL setup?

Thanks for any guidance! If there is other way to use LiDAR, I am happy to hear about it too. I wonder how we can just bring the from isaacsim.sensors.rtx.LidarRtx into IsaacLab

[RandomOakForest]

Thanks for posting this. We are working on improving the RayCaster, likely out in Isaac Lab 2.3. For now you may try combining the prims into one.

[ducmngx]

Thank you for clarifying. I'm trying to build a mesh for the room instead of adding 4, 5 walls at the moment. However, can you help me with this question?

Does mesh_prim_paths support regex patterns like "/World/envs/env_./wall_" or do I need explicit paths like "/World/envs/env_0/wall_north"?

Thank you a lot

[tohsin]

Were you able to make the raycaster work with the CUBOID_CFG?

[RandomOakForest]

Thank you for following up. You may want to try the latest versions of the tools. Follow #3021 to update. I'll move this post to our Discussions for follow up.

[Jdvakil]

@ducmngx I am trying something similar out with multiple meshes in a scene. Were you able to figure out a solution for this?