[Bug Report] TiledCamera.data.quat_w_ros returns incorrect quaternion for camera with ROS convention offset

[ta-klauz]

Describe the bug

When using a TiledCameraCfg with convention="ros" and a rotation offset, the data.quat_w_ros attribute returns an incorrect world-frame quaternion that does not match the actual camera orientation. The parent link's quaternion (ee_frame.data.target_quat_w) is correct, but the camera's computed quaternion appears to be incorrectly transformed.

Steps to reproduce

1. Create a TiledCamera mounted on a robot end-effector with ROS convention and rotation offset:

  ee_camera: TiledCameraCfg = TiledCameraCfg(
      prim_path="{ENV_REGEX_NS}/Robot/ee_link/Camera",
      offset=TiledCameraCfg.OffsetCfg(
          pos=(0.0, 0.0, 0.0),
          rot=(0, 0.707, 0, 0.707), 
          convention="ros",
      ),
      data_types=["rgb", "distance_to_image_plane"],
      spawn=sim_utils.PinholeCameraCfg(...),
      width=1936,
      height=1216,
  )

2. At runtime, compare the sensor-reported quaternion with the correctly computed quaternion:

  # Get sensor-reported quaternion (INCORRECT)
  quat_sensor = ee_camera.data.quat_w_ros[env_idx]  # [w, x, y, z]

  # Compute correct quaternion by composing parent orientation (using frame transformer) with camera offset
  quat_parent_world = ee_frame.data.target_quat_w[env_idx, 0]  # [w, x, y, z]
  quat_cam_offset = torch.tensor(ee_camera.cfg.offset.rot, device=device)  # [w, x, y, z]

  # Quaternion multiplication: q_result = q1 * q2
  def quat_multiply(q1, q2):
      w1, x1, y1, z1 = q1[0], q1[1], q1[2], q1[3]
      w2, x2, y2, z2 = q2[0], q2[1], q2[2], q2[3]
      w = w1*w2 - x1*x2 - y1*y2 - z1*z2
      x = w1*x2 + x1*w2 + y1*z2 - z1*y2
      y = w1*y2 - x1*z2 + y1*w2 + z1*x2
      z = w1*z2 + x1*y2 - y1*x2 + z1*w2
      return torch.tensor([w, x, y, z], device=q1.device, dtype=q1.dtype)

  quat_correct = quat_multiply(quat_parent_world, quat_cam_offset)

  # These quaternions don't match
  print(f"Sensor-reported:    {quat_sensor}")
  print(f"Correctly computed: {quat_correct}")

3. Convert both to rotation matrices to verify the sensor quaternion produces incorrect camera frame axes that don't match the expected camera orientation.

System Info

• Commit: 995070d
• Isaac Lab Version: 2.2.1
• OS: Ubuntu
• GPU: RTX 3070
• CUDA: 13.0
• GPU Driver: 580.95.05

Additional context

Workaround: Manually compose the parent link's world quaternion with the camera offset quaternion as shown in the reproduction steps above. This produces the correct camera orientation.

The parent frame transformer (ee_frame.data.target_quat_w) correctly reports its orientation, suggesting the issue could be in the camera's quaternion computation logic.

Checklist

[x] I have checked that there is no similar issue in the repo (required)
[x] I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

• TiledCamera.data.quat_w_ros returns the correct world-frame quaternion computed as parent_quat ⊗ offset_quat

[Mayankm96]

Is the ee_link a physics-enabled rigid object?

[ta-klauz]

Hi Mayank, yes, it appears to be a physics-enabled rigid object

[ta-klauz]

@Mayankm96 any thoughts on this issue? Thanks!

[Mayankm96]

You probably need to set this setting to True: https://isaac-sim.github.io/IsaacLab/main/source/api/lab/isaaclab.sensors.html#isaaclab.sensors.TiledCameraCfg.update_latest_camera_pose