[Bug Report] multi_mesh_ray_caster

[Gaoxinyigithub]

If you are submitting a bug report, please fill in the following details and use the tag [bug].

Describe the bug

I have a global scene, and it contains a lot of meshes. Also i create a robot in each env.so that，i have one scene with multi robots in it.Then，i want to use multi_mesh_ray_caster to get the ray point. when move the meshes,it should give the move point.but there is a error

Steps to reproduce

# ray

ray = MultiMeshRayCasterCfg(
    prim_path="{ENV_REGEX_NS}/Robot/base",
    update_period=1/30,
    offset=RayCasterCfg.OffsetCfg(pos=(0.0, 0.0, 0.5), rot=(0.0, 0.0, 0.0, 0.0)),
    max_distance=3.0,
    pattern_cfg=patterns.LidarPatternCfg(
        channels=40, vertical_fov_range=[-90, 90], horizontal_fov_range=[-180, 180], horizontal_res=8
    ),
    debug_vis=True,
    ray_alignment='base',
    reference_meshes=True,
    mesh_prim_paths=[MultiMeshRayCasterCfg.RaycastTargetCfg(
        prim_expr="/World/obs",
        is_shared=True,              
        merge_prim_meshes=False,
        track_mesh_transforms=True,  
    )],
)

and if i modify prim_expr="/World/obs" to prim_expr="/World/obs/.*" , the api do not seem it as a global scene, the mesh num of each env is wrong

the example prim of the mesh '/world/obs/Rock2_315'[rigidbody],and i move mesh with

tree_point = scene['Rock2_315'].data.root_state_w
print(tree_point)
tree_point[:, 0:3] = tree_point[:, 0:3] + torch.tensor([[random.random() * 10, random.random() * 10, 0]],
device=device)
scene['Rock2_315'].write_root_state_to_sim(tree_point)
sim.step()
scene.reset()

• Commit: [e.g. 8f3b9ca]
• Isaac Sim Version: 5.0
• OS: 22.04
• GPU: RTX 5090
• CUDA: 12.8
• GPU Driver:580.95.05

[RandomOakForest]

Thank you for posting this. MultiMeshRayCaster will only return updated hit points for moving meshes if those meshes are explicitly configured to have their transforms tracked; otherwise the ray-caster treats them as static and you see “stale” hit points or errors when they move.¹² Could you confirm the following are in place? Thanks.

Key requirement

For each mesh group you want to raycast against, the config must:

• Use a MultiMeshRayCaster (not the basic RayCaster).²¹
• Provide a prim expression that actually matches your meshes (e.g. "/World/envs/env_.*/MyMesh" for per‑env meshes).
• Enable transform tracking for those meshes: track_mesh_transforms=True in the per‑target config.

The new MultiMesh implementation was added specifically to support dynamic meshes, and tracking is wired through mesh_views that are only created when track_mesh_transforms is set.²

What is typically missing

When users have a global scene with many meshes and multiple robots per env, the usual missing pieces are:

• Using RayCasterCfg instead of the MultiMesh variant exposed in isaaclab.sensors (so the sensor can only handle static meshes).³²
• Supplying mesh_prim_paths=[... ] without enabling tracking on those targets, so mesh_views never update and the ray points do not move with the meshes.
• A prim expression that matches only the first env (e.g. /World/envs/env_0/...) so meshes in other envs are ignored, which can also trigger “no meshes found” or similar runtime errors.

Footnotes

1. https://isaac-sim.github.io/IsaacLab/main/source/api/lab/isaaclab.sensors.html ↩ ↩²

2. https://isaac-sim.github.io/IsaacLab/main/source/refs/changelog.html ↩ ↩² ↩³ ↩⁴

3. https://isaac-sim.github.io/IsaacLab/main/source/overview/core-concepts/sensors/ray_caster.html ↩