Connecting an Xbox Controller for Teleoperation

[morganchendev]

Hello,

I've been trying to follow the Teleoperation and Imitation Learning tutorial and have had success with going through the steps with my keyboard as my input device but would like to use an Xbox controller instead. However, I'm experiencing difficulties in getting the arm to respond to the inputs from the Xbox controller (no response at all).

I have a Wired Controller For Xbox Series X|S. I'm running Isaac Lab 2.2.0 inside Docker on a computer running Ubuntu 24.04. I know to run with the "--teleop-device" parameter set to "gamepad", have tested to make sure that the controller and its inputs are recognized inside the container environment (ls /dev/input/js*), and have tried changing the index of the gamepad = app_window.get_gamepad(0) line in the se3_gamepad.py file.

Has anyone been able to connect and use the gamepad device as input for this tutorial? Thank you.

[IrvingF7]

I was able to use a PS5 controller but only after I installed Isaac Lab+Sim in my native environment. I couldn't get it to work under docker as well. I know there's gotta be a way as long as I configure the usb device correctly, but there seems to be a lot of moving parts so I gave up on that and switched to local install, and it works immediately

I do think if you want to spend more time on the docker setup, an easier way is to directly test with omni's gamepad api to see if omni is even seeing this device