Behavior different when simulating robot using Isaac Sim GUI and when launching through Isaac Lab. Also, IdealPDActuatorCfg throws an error

[dhruvkm2402]

Describe the bug

Hi,
I've setup an RL task for a mobile robot with Isaac Lab.
What I've noticed is that when I test the motion through Isaac Sim GUI, behavior looks reasonable. However, when launching through Isaac lab with the exact same settings, behavior is drastically different. I am implementing velocity control so I am setting stiffness to 0.0 and damping to 20.0.
Additionally If I use IdealPDActuatorCfg intstead of ImplicitActuatorCfg, robot does not get spawned and throws Nan if I try to access joint velocity values

Steps to reproduce

Physics scene settings in Isaac Sim:
Solver: PGS
dt: 200
physics material: static_friction - 2.0
dynamic_friction - 1.0
restitution - 0.0

System Info

• Isaac Sim Version: [4.2.0]
• OS: [Ubuntu 22.04]
• GPU: [RTX A6000]
• CUDA: [12.2]
• GPU Driver: [535.183.01]

Additional context

Add any other context about the problem here.

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

Add the criteria for which this task is considered done. If not known at issue creation time, you can add this once the issue is assigned.

• Criteria 1
• Criteria 2

[RandomOakForest]

Thank you for posting this. Could you provide more details on how are you are using Isaac Lab, and a script to reproduce the behavior you mention? Regarding the NaN output, it is likely due to the fact that the robot didn't get spawned in the first place. Could you provide your error log so we can investigate? Thanks.

[dhruvkm2402]

Hello @RandomOakForest , Sure.
So, the way I am using Isaac Lab is that I have a skid-steered robot and through camera data I need to apply wheel joint velocities which will control the robot motion like a typical autonomous setup. Entire setup is done using the DRL framework.
As of now I am using ImplicitActuatorCfg class and setting:
Stiffness: 0
Damping: 100
This is done to do velocity control.
What I noticed is that when I set left wheel velocities as 12 rad/s and right wheel velocities at 4 rad/s or vice versa, the angular velocity should be about 2 rad/s.
When I use the above mentioned solver and friction settings in Isaac Sim GUI, it works reasonably because it immediately sets the joint velocities. However, when I use Isaac Lab, the wheel joint velocities for each wheel are different which I believe is the cause for the behavior to be different.
Here is the baseline code I use:

from future import annotations

import torch
import gymnasium as gym
import math
import numpy as np
import torch
from collections.abc import Sequence

import omni.isaac.lab.sim as sim_utils
from omni.isaac.lab.assets import Articulation, ArticulationCfg
from omni.isaac.lab.envs import DirectRLEnv, DirectRLEnvCfg
from omni.isaac.lab.scene import InteractiveSceneCfg
from omni.isaac.lab.sensors import Camera, ContactSensor, ContactSensorCfg, CameraCfg
from omni.isaac.lab.sim import SimulationCfg, PhysxCfg
from omni.isaac.lab.terrains import TerrainImporterCfg
from omni.isaac.lab.utils import configclass
from omni.isaac.lab.sim.spawners.from_files import GroundPlaneCfg, spawn_ground_plane
from omni.isaac.lab.utils.math import euler_xyz_from_quat
from omni.isaac.lab.sim.spawners.materials import RigidBodyMaterialCfg
from PIL import Image

Pre-defined configs

from omni.isaac.lab_assets.husky import HUSKY_CFG # isort: skip # Change this

@configclass
class HuskyEnvCfg(DirectRLEnvCfg):
# simulation
sim: SimulationCfg = SimulationCfg(
dt=1 / 200,
use_gpu_pipeline=True,
use_fabric=True,
enable_scene_query_support=False,
disable_contact_processing=False,
gravity=(0.0, 0.0, -9.81),

physics_material=RigidBodyMaterialCfg(
    static_friction=2.0,
    dynamic_friction=1.0,
    restitution=0.0
),

physx=PhysxCfg(
    solver_type=0,
    use_gpu=True,
    max_position_iteration_count=4,
    max_velocity_iteration_count=0,
    bounce_threshold_velocity=0.2,
    friction_offset_threshold=0.04,
    friction_correlation_distance=0.025,
    enable_stabilization=True,
    gpu_max_rigid_contact_count=2**23,
    gpu_max_rigid_patch_count=5 * 2**15,
    gpu_found_lost_pairs_capacity=2**21,
    gpu_found_lost_aggregate_pairs_capacity=2**25,
    gpu_total_aggregate_pairs_capacity=2**21,
    gpu_heap_capacity=2**26,
    gpu_temp_buffer_capacity=2**24,
    gpu_max_num_partitions=8,
    gpu_max_soft_body_contacts=2**20,
    gpu_max_particle_contacts=2**20,
)

)

# robot 
robot: ArticulationCfg = HUSKY_CFG.replace(prim_path="/World/envs/env_.*/Robot")
camera: CameraCfg =  CameraCfg(
    prim_path="/World/envs/env_.*/Robot/Camera_Link/Camera",
    update_period=0.1,
    spawn=None,
    width=100,
    height=100,
    colorize_semantic_segmentation=False,
    colorize_instance_id_segmentation=False,
    colorize_instance_segmentation=False,
    
)
frontleft_wheel = "front_left_wheel"
frontright_wheel = "front_right_wheel"

rearleft_wheel = "rear_left_wheel"
rearright_wheel = "rear_right_wheel"



#scene
scene: InteractiveSceneCfg = InteractiveSceneCfg(num_envs=4096, env_spacing=40.0, replicate_physics=True)

#env

num_channels = 1
decimation = 10
episode_length_s = 20.0
action_scale = 1.0  # [N]
observation_space=X
action_space=2
state_space=0

class HuskyEnv(DirectRLEnv):
cfg: HuskyEnvCfg

def __init__(self, cfg: HuskyEnvCfg, render_mode: str | None = None, **kwargs):
    super().__init__(cfg, render_mode, **kwargs)

    self._frontleft_dof_idx, _ = self.husky.find_joints(self.cfg.frontleft_wheel)   #DOF index of rear wheel left                           
    self._frontright_dof_idx, _ = self.husky.find_joints(self.cfg.frontright_wheel)

    self._rearleft_dof_idx, _ = self.husky.find_joints(self.cfg.rearleft_wheel)   #DOF index of rear wheel left                           
    self._rearright_dof_idx, _ = self.husky.find_joints(self.cfg.rearright_wheel)
   
    
    self.action_scale = self.cfg.action_scale

    self.joint_pos = self.husky.data.joint_pos
    self.joint_vel = self.husky.data.joint_vel
    

def close(self):
    """Cleanup for the environment."""
    super().close()

def _setup_scene(self):
    self.robot = Articulation(self.cfg.robot)
    # add ground plane
    #spawn_ground_plane(prim_path="/World/ground", cfg=GroundPlaneCfg())
    # clone, filter, and replicate
    self.scene.clone_environments(copy_from_source=False)
    self.scene.filter_collisions(global_prim_paths=[])
    # add articultion to scene
    self.scene.articulations["husky"] = self.husky
    # add lights
    self.camera = Camera(self.cfg.camera)
    self.scene.sensors["camera"] = self.camera
    
    
    #light_cfg = sim_utils.DomeLightCfg(intensity=2000.0, color=(0.75, 0.75, 0.75))
    #light_cfg.func("/World/Light", light_cfg)

def _pre_physics_step(self, actions: torch.Tensor) -> None:
    actions[:,0] = 0.0
    actions[:,1] = 1.0
    self.actions[:,0] = 12.0*actions[:,0].clone()        # Action of base linear vel
    self.actions[:,1] = 12.0*actions[:,1].clone()  # Action for base angular vel
    self.actions[:,0] = 12.0        # Action of base linear vel
    self.actions[:,1] = 4.0  # Action for base angular vel

def _apply_action(self) -> None:

    self.husky.set_joint_velocity_target(self.actions[:,0].unsqueeze(-1), joint_ids=self._frontright_dof_idx)
    self.husky.set_joint_velocity_target(self.actions[:,0].unsqueeze(-1), joint_ids=self._rearright_dof_idx)

    self.husky.set_joint_velocity_target(self.actions[:,1].unsqueeze(-1), joint_ids=self._frontleft_dof_idx)
    self.husky.set_joint_velocity_target(self.actions[:,1].unsqueeze(-1), joint_ids=self._rearleft_dof_idx)

If you require additional information, I'd be happy to send it separately.

[dhruvkm2402]

Hi @RandomOakForest,
Any update on this front?

[RandomOakForest]

Thanks for following up. What is the error message you get? Could you share the log or stdout with the error? Thanks

[dhruvkm2402]

Hi @RandomOakForest,
Here is the error log

tensor([nan], device='cuda:0')
2025-01-08 21:07:56 [20,261ms] [Warning] [omni.usd] Warning: in Orthonormalize at line 495 of /builds/omniverse/usd-ci/USD/pxr/base/gf/matrix4d.cpp -- OrthogonalizeBasis did not converge, matrix may not be orthonormal

What I noticed is, If I keep damping to 10.0, It does not show an error but the moment I make it 20.0, there is this error for IdealPDActuatorCfg. I briefly noticed that with high damping, robot kind of flies off and is not spawned again.

Additionally, when I set the target joint velocity of left wheels to 12.0 rad/s and right wheels to 4 rad/s, they dont reach those velocities. Here are the values of joint velocities (This happens even with ImplicitActuatorCfg):

tensor([[ 4.9236, 12.1095, 3.7108, 6.4523]], device='cuda:0')

tensor([[ 6.4652, 6.5897, 3.5293, 7.8635]], device='cuda:0')

tensor([[ 6.4083, 11.4830, 6.3876, 10.1888]], device='cuda:0')
whereas in the Isaac Sim GUI these velocities are immediately set to 12.0 and 4.0. Which I believe is the cause for a different behavior when launching the task through Isaac Lab.
Please feel free let me know if you need more information.

[dhruvkm2402]

Hi @RandomOakForest,
Just a quick follow up again. Were you able to get an idea about the issue from the information provided?

[dhruvkm2402]

Hi @RandomOakForest ,
Sorry but a quick follow up again. Any suggestions on the isssue? We want to train it and get some results for a publication and the parallelization is a great tool so we want to continue doing it in Isaac Lab. I've tried different actutator configs but nothing works.

[RandomOakForest]

Thanks for following up. There is a related conversation with suggestions that may help. Having said that, we are able to repro the error message you get in some examples we are working on. Will keep you posted.

[StrainFlow]

How are you getting those joint velocities? The tensors you're showing look like quaternions, not velocities. Are you using ArticulationData.Joint_vel?

[dhruvkm2402]

Hi @StrainFlow ,
No, they are not quaternions,
I am printing the values using the following
self.joint_vel = self.husky.data.joint_vel as you mentioned.
There are 5 joints in total, I am just printing the output of four wheels.

[dhruvkm2402]

Hi @StrainFlow and @RandomOakForest,
While the error of NaN values is something you are working on and thank you for the follow up.
I wanted to ask, w.r.t to setting joint velocities, is there a reason why these joint velocities are not being set instantly as in the case of running on Isaac Sim itself?

[dhruvkm2402]

Hi Everyone,
Wanted to quickly follow up on this. Can anybody suggest the right way to use velocity control? I believe the problem is that the joints don't reach the velocity when I set it using set_joint_velocity_target.

[RandomOakForest]

Following up on this. Regarding the original issue on the different actuators, it seems the way they are implemented they differ in how they handle the step in simulation. However, if you have small enough dt, both should yield similar results. I will move this post to a discussion for the team to continue to follow up. Thanks for your interest in Isaac Lab.