Skip to content

Commit 98eb8a5

Browse files
haixuanTaohaixuanTao
authored
Minor fix for mujoco sim (dora-rs#1023)
@ShashwatPatil I have made some overall improvement. Could you check the commits and see if its ok for you to merge?
2 parents a7f4140 + dc6f247 commit 98eb8a5

File tree

26 files changed

+2070
-1533
lines changed

26 files changed

+2070
-1533
lines changed

‎examples/mujoco-sim/README.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# MuJoCo Sim Tutorial
2+
3+
This comprehensive tutorial demonstrates how to build a robot control system using Dora with the `dora-mujoco` simulation node and control logic.
4+
5+
## Tutorial Structure
6+
7+
### [01. Basic Simulation](basic_simulation/)
8+
Load a robot in simulation using the `dora-mujoco` node.
9+
- Learn the fundamentals of MuJoCo simulation in Dora
10+
- Understand the simulation node architecture
11+
- See how robot descriptions are loaded automatically
12+
13+
### [02. Target Pose Control](target_pose_control/)
14+
Add control logic with pose commands as target.
15+
- Implement Cartesian space control.
16+
- Create generic controller node that is able to control any robotic arm by using `dora-pytorch-kinematics`
17+
18+
### [03. Gamepad Control](gamepad_control/)
19+
Connect a gamepad for real-time interactive control.
20+
- Integrate with dora's `gamepad` node
21+
- Demonstrate real-time teleoperation

‎examples/mujoco-sim/basic_simulation/README.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# 01. Basic Simulation
2+
3+
This example demonstrates the simplest possible setup: loading and running a robot simulation using the `dora-mujoco` node.
4+
5+
- Understand how the `dora-mujoco` node works
6+
- See how robot models are loaded from `robot-descriptions`
7+
- Learn the basic dataflow for physics simulation
8+
9+
10+
The simulation runs at 500Hz and outputs:
11+
- Joint positions for all robot joints
12+
- Joint velocities
13+
- Sensor data (if available)
14+
- Current simulation time
15+
16+
### Running the Example
17+
18+
```bash
19+
cd basic_simulation
20+
dora build basic.yml
21+
dora run basic.yml
22+
```
23+
24+
You should see:
25+
1. MuJoCo viewer window opens with GO2 robot
26+
2. Robot is effected by gravity (enabled by default)
27+
28+
### What's Happening
29+
30+
1. **Model Loading**: The `dora-mujoco` node loads the RoboDog (go2) model using `load_robot_description("go2_mj_description")`
31+
2. **Physics Loop**: Timer triggers simulation steps at 500Hz (This is default step time for Mujoco)
32+
3. **Data Output**: Joint states are published
33+
4. **Visualization**: MuJoCo viewer shows real-time simulation
34+
35+
### Configuration Details
36+
37+
The `basic.yml` configures:
38+
- Model name: `"go2"` you change this to other robots name
39+
- Update rate: 2ms (500Hz)
40+
- Outputs: Joint positions, velocities, and sensor data

‎examples/mujoco-sim/basic_simulation/basic.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
nodes:
2+
- id: mujoco_sim
3+
build: pip install -e ../../../node-hub/dora-mujoco
4+
path: dora-mujoco
5+
inputs:
6+
tick: dora/timer/millis/2 # 500 Hz simulation
7+
outputs:
8+
- joint_positions
9+
- joint_velocities
10+
- sensor_data
11+
env:
12+
MODEL_NAME: "go2_mj_description" # Load GO2

‎examples/mujoco-sim/gamepad_control/README.md‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# 03. Gamepad Control
2+
3+
This example demonstrates real-time interactive control by connecting a gamepad to the controller. It builds upon the target pose control example by adding gamepad input processing for teleoperation of the robot arm.
4+
5+
## Controller Types
6+
7+
### 1. Basic Gamepad Control (`gamepad_control_basic.yml`)
8+
- **Direct IK approach**: Uses simple IK solver for immediate position updates
9+
- The movement fells jumpy
10+
11+
### 2. Advanced Gamepad Control (`gamepad_control_advanced.yml`)
12+
- **Differential IK approach**: Smooth velocity-based control with Jacobian
13+
- **Smooth**: Continuous motion interpolation
14+
- **Use case**: Precise manipulation, smooth trajectories
15+
16+
## Gamepad Controls
17+
18+
- **D-pad Vertical**: Move along X-axis (forward/backward)
19+
- **D-pad Horizontal**: Move along Y-axis (left/right)
20+
- **Right Stick Y**: Move along Z-axis (up/down)
21+
- **LB/RB**: Decrease/Increase movement speed (0.1-1.0x scale)
22+
- **START**: Reset to home position [0.4, 0.0, 0.3]
23+
24+
## Running the Examples
25+
26+
1. **Connect a gamepad** (Xbox/PlayStation controller via USB or Bluetooth)
27+
28+
### Basic Gamepad Control
29+
```bash
30+
cd gamepad_control
31+
dora build gamepad_control_basic.yml
32+
dora run gamepad_control_basic.yml
33+
```
34+
35+
### Advanced Gamepad Control
36+
```bash
37+
cd gamepad_control
38+
dora build gamepad_control_advanced.yml
39+
dora run gamepad_control_advanced.yml
40+
```
41+
42+
You should see:
43+
1. Robot responds to gamepad input in real-time
44+
2. **Basic**: Immediate position jumps based on gamepad input
45+
3. **Advanced**: Smooth incremental movement with velocity control
46+
4. Speed scaling with bumper buttons
47+
5. Reset functionality with START button
48+
49+
50+
### **Gamepad Node** (`gamepad`)
51+
Built-in Dora node that interfaces with system gamepad drivers using Pygame.
52+
53+
```yaml
54+
- id: gamepad
55+
build: pip install -e ../../../node-hub/gamepad
56+
path: gamepad
57+
outputs:
58+
- cmd_vel # 6DOF velocity commands
59+
- raw_control # Raw button/stick states
60+
inputs:
61+
tick: dora/timer/millis/10 # 100Hz polling
62+
```
63+
64+
- **`cmd_vel`**: 6DOF velocity array `[linear_x, linear_y, linear_z, angular_x, angular_y, angular_z]`
65+
- Generated from D-pad and analog stick positions
66+
- Continuous updates while controls are held
67+
68+
- **`raw_control`**: JSON format gamepad state
69+
70+
71+
### **Gamepad Controller Scripts**
72+
73+
**Basic Controller (`gamepad_controller_ik.py`)**:
74+
```
75+
Gamepad Input → Target Position Update → IK Request → Joint Commands
76+
```
77+
- Updates target position incrementally based on gamepad
78+
- Immediate position jumps
79+
80+
**Advanced Controller (`gamepad_controller_differential_ik.py`)**:
81+
```
82+
Gamepad Input → Target Position → Pose Error → Velocity → Joint Commands
83+
```
84+
- Continuous pose error calculation and velocity control
85+
- Smooth interpolation between current and target poses
86+
- Real-time Jacobian-based control
87+
88+
## Gamepad Mapping
89+
90+
The controller expects standard gamepad layout: (The mapping may change based on controller)
91+
92+
| Control | Function |
93+
|---------|----------|
94+
| D-pad Up/Down | X-axis movement |
95+
| D-pad Left/Right | Y-axis movement |
96+
| Right Stick Y | Z-axis movement |
97+
| Left Bumper | Decrease speed |
98+
| Right Bumper | Increase speed |
99+
| START button | Reset position |
100+
101+
## Key Features
102+
103+
**Real-time Teleoperation:**
104+
- Incremental position updates based on continuous input
105+
- Immediate feedback through robot motion
106+
107+
**Speed Control:**
108+
- Dynamic speed scaling (0.1x to 1.0x)
109+
- Allows both coarse and fine manipulation
110+
111+
**Features:**
112+
- Home position reset capability
113+
- Bounded movement through incremental updates
114+
- Working on collision avoidance
115+
116+
## Troubleshooting
117+
118+
**"Gamepad not responding"**
119+
```bash
120+
# Check if gamepad is connected
121+
ls /dev/input/js*
122+
# Test gamepad input
123+
jstest /dev/input/js0
124+
# Grant permissions
125+
sudo chmod 666 /dev/input/js0
126+
```
127+
128+
**"Robot doesn't move with D-pad"**
129+
- Check `cmd_vel` output: should show non-zero values when D-pad pressed
130+
- Verify correct gamepad mapping for your controller model
131+
132+
**"Movement too fast/slow"**
133+
- Use LB/RB buttons to adjust speed scale
134+
- Modify `delta = cmd_vel[:3] * 0.03` scaling factor in code if required

‎examples/mujoco-sim/gamepad_control/gamepad_control_advanced.yml‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
nodes:
2+
- id: gamepad
3+
build: pip install -e ../../../node-hub/gamepad
4+
path: gamepad
5+
outputs:
6+
- cmd_vel
7+
- raw_control
8+
inputs:
9+
tick: dora/timer/millis/10
10+
11+
- id: mujoco_sim
12+
build: pip install -e ../../../node-hub/dora-mujoco
13+
path: dora-mujoco
14+
inputs:
15+
tick: dora/timer/millis/2 # 500 Hz simulation
16+
control_input: gamepad_controller/joint_commands
17+
outputs:
18+
- joint_positions
19+
- joint_velocities
20+
- actuator_controls
21+
- sensor_data
22+
env:
23+
MODEL_NAME: "iiwa14_mj_description"
24+
25+
- id: gamepad_controller
26+
path: nodes/gamepad_controller_differential_ik.py
27+
inputs:
28+
joint_positions: mujoco_sim/joint_positions
29+
joint_velocities: mujoco_sim/joint_velocities
30+
raw_control: gamepad/raw_control
31+
cmd_vel: gamepad/cmd_vel
32+
fk_result: pytorch_kinematics/fk_request
33+
jacobian_result: pytorch_kinematics/jacobian_request
34+
outputs:
35+
- joint_commands
36+
- fk_request
37+
- jacobian_request
38+
39+
- id: pytorch_kinematics
40+
build: pip install -e ../../../node-hub/dora-pytorch-kinematics
41+
path: dora-pytorch-kinematics
42+
inputs:
43+
fk_request: gamepad_controller/fk_request
44+
jacobian_request: gamepad_controller/jacobian_request
45+
outputs:
46+
- fk_request
47+
- jacobian_request
48+
env:
49+
MODEL_NAME: "iiwa14_description"
50+
END_EFFECTOR_LINK: "iiwa_link_7"
51+
TRANSFORM: "0. 0. 0. 1. 0. 0. 0."

‎examples/mujoco-sim/gamepad_control/gamepad_control_basic.yml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
nodes:
2+
- id: gamepad
3+
build: pip install -e ../../../node-hub/gamepad
4+
path: gamepad
5+
outputs:
6+
- cmd_vel
7+
- raw_control
8+
inputs:
9+
tick: dora/timer/millis/10
10+
11+
- id: mujoco_sim
12+
build: pip install -e ../../../node-hub/dora-mujoco
13+
path: dora-mujoco
14+
inputs:
15+
tick: dora/timer/millis/2 # 500 Hz simulation
16+
control_input: pytorch_kinematics/cmd_vel
17+
outputs:
18+
- joint_positions
19+
- joint_velocities
20+
- actuator_controls
21+
- sensor_data
22+
env:
23+
MODEL_NAME: "iiwa14_mj_description"
24+
25+
- id: pytorch_kinematics
26+
build: pip install -e ../../../node-hub/dora-pytorch-kinematics
27+
path: dora-pytorch-kinematics
28+
inputs:
29+
cmd_vel: gamepad/cmd_vel
30+
pose: mujoco_sim/joint_positions
31+
outputs:
32+
- cmd_vel
33+
- pose
34+
env:
35+
MODEL_NAME: "iiwa14_description"
36+
END_EFFECTOR_LINK: "iiwa_link_7"
37+
TRANSFORM: "0. 0. 0. 1. 0. 0. 0."

0 commit comments

Comments
 (0)