[Question] Explanation of Sim2Real for universal robot as explained in blog

[HariKrishnan06082k]

Question

Hi, Isaac Lab team, we are working with the hardware of ur5e with robotiq 2f85 gripper and I'm trying to make the reach policy for UR robot happen in real robot. There are 2 ways to approach this as described earlier use ros2 control or develop a controller which you can closely mimic for robot as well as your simulated asset. I came across this article which was interesting and I believe this would be of great use if you could provide more information on the implementation aspect of it.

1. https://developer.nvidia.com/blog/bridging-the-sim-to-real-gap-for-industrial-robotic-assembly-applications-using-nvidia-isaac-lab/
2. https://github.com/isaac-sim/IsaacLab/discussions/2343

I wrote a impedence controller using ur_rtde which directly applies torque_command to real robot which works like a simple torque_pd_control in joint space. I tuned it for real robot by sending a sinusoid to each joint and getting the optimal stiffness and damping gains which gave me decent trajectory following. However I wasn't able to make the robot work with the hierarchical dual control where you run higher level impedence loop on teach pendant at 500hz and send only position setpoints from external pc like using socket.

In summary I tried two approaches,

1. Single stage PD control only using rtde directly working at 60hz and communicating with real robot no hardware sync issues and has decent tracking
2. Two stage where the teach pendant impedence loop listens to a socket at 60hz for desired joint position updates via socket string and executes the main control compensating for dynamics at 500hz. This approach made the controller fight itself and its struggling, partly could be because i only send joint position updates without any velocity or acceleration profile, and its data hungry too. So I was wondering how did this blog address this issue and properly ensuring this two stage approach was working the way its desired to do. ( In this loop i did numerical desired velocity and acceleration calculation { new-prev/dt }

My goal is to transfer policy with good constraints set in simulation including modeling of delay using delayed PD actuator model and make the task happen on real robot. Since the intention of the blog was to explain people how to do it, I was wondering if it would be possible to help me out with this issue.

TODO:
Make the actuator model available in Isaaclab follow same control law and model delay so I could do sim 2 real, and also ensure impedence control works as the way isaaclab would output updates.

import time
import numpy as np
from rtde_control import RTDEControlInterface as RTDEControl
from rtde_receive import RTDEReceiveInterface as RTDEReceive

def main():
    # RTDE setup
    rtde_frequency = 60.0
    rtde_c = RTDEControl("192.168.1.102", rtde_frequency, 
                         RTDEControl.FLAG_VERBOSE | RTDEControl.FLAG_UPLOAD_SCRIPT)
    rtde_r = RTDEReceive("192.168.1.102", rtde_frequency)

    # --- Joint PD torque control with ABSOLUTE positions ---
    
    # UR5e max torques
    max_torque = np.array([150.0, 150.0, 150.0, 28.0, 28.0, 28.0])

    # Example gains (tune these for your application)
    Kp = max_torque / np.array([0.25, 0.25, 0.5, 1, 1, 1])
    Kd = max_torque / (np.pi * 0.5)
    
    print("Connected to robot!")
    print("Kp (stiffness):", Kp)
    print("Kd (damping):", Kd)
    print("Max torques:", max_torque)

    # Get initial robot position
    current_q_startup = np.array(rtde_r.getActualQ())
    print("\nInitial joint angles (rad):", current_q_startup)
    print("Initial joint angles (deg):", np.rad2deg(current_q_startup))

    # Initialize
    q_err = np.zeros(6)
    q_err_prev = np.zeros(6)
    dt = 1.0 / rtde_frequency

    # PD Controller loop
    timecounter = 0.0
    print(f"\nStarting PD controller at {rtde_frequency} Hz")
    print("Press Ctrl+C to stop\n")

    try:
        while True:
            t_start = rtde_c.initPeriod()
            
            # ===== DEFINE ABSOLUTE TARGET POSITION =====
            amplitude = np.deg2rad(15)
            frequency = 0.2
            q_desired = current_q_startup.copy()
            q_desired[0] = current_q_startup[0] + amplitude * np.sin(2 * np.pi * frequency * timecounter)
            
            # ===== END TARGET DEFINITION =====
            
            current_q = np.array(rtde_r.getActualQ())
            
            q_err_prev = q_err
            q_err = q_desired - current_q
            q_err_d = (q_err - q_err_prev) / dt
            
            torque_stiffness = Kp * q_err
            torque_damping = Kd * q_err_d
            
            torque_damping_clamped = np.clip(torque_damping, 
                                             a_min=0.2 * -max_torque, 
                                             a_max=0.2 * max_torque)
            
            torque_target = torque_stiffness + torque_damping_clamped
            torque_target = np.clip(torque_target, a_min=-max_torque, a_max=max_torque)
            
            rtde_c.directTorque(torque_target.tolist())
            
            if int(timecounter * 10) % 10 == 0:
                print(f"\n=== t: {timecounter:.1f}s ===")
                print(f"Target (deg):  {np.rad2deg(q_desired)}")
                print(f"Current (deg): {np.rad2deg(current_q)}")
                print(f"Error (deg):   {np.rad2deg(q_err)}")
                print(f"Stiffness torque: {torque_stiffness}")
                print(f"Damping torque:   {torque_damping_clamped}")
                print(f"Total torque:     {torque_target}")
            
            rtde_c.waitPeriod(t_start)
            timecounter += dt

    except KeyboardInterrupt:
        print("\n\nStopping controller...")
    finally:
        zero_torque = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
        rtde_c.directTorque(zero_torque)
        print("Controller stopped - torques set to zero")
        rtde_c.stopScript()

[RandomOakForest]

Thank you for posting this. This is a great question for our Discussions section. I'll move the post there for follow up. Here is a summary that may be of help in your case.

The NVIDIA blog “Bridging the Sim-to-Real Gap for Industrial Robotic Assembly Applications Using Isaac Lab” demonstrates a concrete and working sim-to-real workflow for Universal Robots (UR10e) using torque-based impedance control. The key insight relevant to your dual-control loop issue is that NVIDIA’s implementation eliminates control conflicts by consolidating the impedance loop fully into the robot-side torque interface, while the external PC sends delta joint targets (not full position/velocity control) within this inner high-frequency loop.¹²³

How NVIDIA’s Two-Stage Control Works

The architecture described in the NVIDIA blog uses a layered design:

• High-level policy: Runs on an external PC (Isaac Lab or Isaac ROS node). The policy network predicts delta joint positions based on sensory input (e.g., vision and joint states).
• Low-level loop: Runs entirely on the robot controller (via URScript or UR’s torque interface) at 500 Hz, computing torques using an impedance model. This inner loop consumes the delta joint targets and computes:

$$ \tau = K_p (\theta_d - \theta) + K_d (\dot{\theta}_d - \dot{\theta}) $$

where $\theta_d$ is locally updated from target deltas received over the network.

• Communication rate: The external PC transmits only at 60 Hz, similar to your setup. However, the 500 Hz impedance layer has its own trajectory generator internally, smoothing incoming target deltas and estimating local velocity/acceleration.

Why Your Current Dual-Loop Setup Struggled

Your second setup resembles NVIDIA’s structure but differs in two critical ways:

1. The inner UR impedance loop receives absolute position targets without smooth interpolation or velocity feedforward. Consequently, the high-frequency UR loop interprets these as sharp trajectory steps, causing oscillations and “controller fighting.”
2. Velocity and acceleration estimates ($\dot{q} = (q_t - q_{t-1})/dt$) are susceptible to quantization and delay errors over the network, which destabilizes an outer 60 Hz position stream feeding a 500 Hz torque loop.

Implementation Fix Strategies

To approach NVIDIA’s stable two-tier controller:

• Switch to target deltas: Send incremental deltas (Δq) rather than full joint positions. This matches Isaac Lab’s impedance formulation used in IndustReal.
• Add trajectory smoothing in the URScript impedance loop: Implement a local interpolator, e.g., first-order filter:

$$ q_{target}(t) = (1 - \alpha) q_{target}(t-1) + \alpha (q_{desired}) $$

with $ \alpha $ tuned for smoothness (~0.01–0.05 for 500 Hz loop).

• Include feedforward terms: Send both Δq and coarse velocity estimates so the local loop can internally reconstruct a compliant velocity target.
• Model delay and impedance in Isaac Lab: Use the delayed PD actuator in Isaac Lab’s XML config to match end-to-end communication delay (2–3 cycle delay typical for Ethernet-based RTDE).

Practical Integration

• NVIDIA’s blog implementation used UR’s new Direct Torque Command interface instead of the traditional RTDE socket control. If you remain on RTDE, mimic this approach by computing torques externally in sync with the inner 125 Hz RTDE update rate (rather than 60 Hz user loop).
• In Isaac Lab, specify actuator_model_cfg=“DelayedPDActuatorCfg” and tune the parameters delay_steps and stiffness/damping to match your real impedance gains.
• Reference example: IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality (Narang et al.) describes the same impedance architecture used in the blog.¹

Recommended Path for You

1. Continue with the single-stage torque PD control for experimental validation.
2. For hierarchical control, make the robot-side loop impedance-aware and interpolate incoming 60 Hz deltas to internal 500 Hz update rate.
3. If possible, migrate to UR torque interface (early-access SDK) for impedance consistency with Isaac Lab’s training environment.
4. When training in Isaac Lab, use a delayed PD actuator model with tuned communication delay to approximate RTDE latency.

By aligning your impedance structure to NVIDIA’s internal interpolation and delta-driven design, your real UR5e + 2F85 setup can achieve smoother and stable sim-to-real behavior matching the Isaac Lab workflow.³⁴¹

Footnotes

1. https://developer.nvidia.com/blog/bridging-the-sim-to-real-gap-for-industrial-robotic-assembly-applications-using-nvidia-isaac-lab/ ↩ ↩² ↩³

2. https://forums.developer.nvidia.com/t/bridging-the-sim-to-real-gap-for-industrial-robotic-assembly-applications-using-nvidia-isaac-lab/333818 ↩

3. https://research.nvidia.com/labs/srl/post/bridging-sim-to-real-gap-industrial-robotic-assembly-applications-nvidia-isaac-lab/ ↩ ↩²

4. https://forums.developer.nvidia.com/t/robotiq-2f85-and-ur5e-assembly/314415 ↩

[HariKrishnan06082k]

Hi thanks for your detailed response this cleared some of the doubts I had and the steps I will have to take to debug this issue.

1. In RTDE, now there's integration to directly set torque which is what I'm doing in my python loop as well. But I do understand the strict timing requirements of python is a problem to sync with internal loop run as urscript in teach pendant because of GIL. First question would be to like how do you make this communication framework from external PC to urscript stable and respect the strict timing constraints. As far as i know there's three ways registers, socket string (which I'm doing currently) and I know in the recent polyscope X you can do it in ROS2 based way.

2. Im using 5.23 (If possible, migrate to UR torque interface (early-access SDK)) for impedance consistency with Isaac Lab’s training environment which is directly setting torque, and whenever I implement a logic to send set points let it be delta pos or absolute pos , I'm unable to meet the strict timing requirements like script is 0.02 ms definite and my python or cpp node has to run at like ~0.16 (1/8th) or something, how do ensure the pipeline for dual hierarchy is stable. (The RTDE registers exposed via API are like I think non atomic reads so that doesn't really work for me so with socket when I'm send all delta as single string separated by commas I'm facing this communication based delay.

3. With regard to your second suggestion of sending coarse velocity estimates, I can think of this in 2 ways one where you let the RL policy predict both delta joint position and also delta coarse velocity, or you let it predict delta joint pos and do a finite difference to estimate the delta coarse velocity term to give to your 500 hz loop. Which one would you ideally do?

4. For interpolation would just exponential smoothing work or should I do like a cubic polynomial based one which is dependent on hermite basis function.

On top of all this, I'm mainly concerned about the exact sync update and stable framework on this two stage approach both communication wise and also system design wise that would be like the part which would help me most. Thanks and really appreciate your input in advance and I believe this would help other people as well struggling with Sim 2 real bridge.

Isaac Lab Dual-Rate Communication Framework Gist

• Communication: RTDE registers (non-atomic) vs Socket strings (~0.6ms delay) vs ROS2 - which maintains stable 60Hz→500Hz dual-rate performance?

• Timing sync: How do you bridge 16.67ms policy updates to 2ms URScript loops without breaking real-time constraints? Zero-order hold + interpolation details?

• Velocity estimation: Do RL policies predict both delta positions AND velocities (12D) or just positions with finite difference computation (6D)?

• Interpolation method: Exponential smoothing vs Cubic Hermite - which is sufficient for stable impedance control within URScript timing budget?

• Validation: How do you measure 500Hz loop timing compliance and validate sim-to-real transfer success on real UR hardware?