UR5e Automated Data Collection

1. Environment Setup

1.1 Create and Activate Conda Environment

source ~/miniconda3/bin/activate
conda create -n ur_data python=3.10
conda activate ur_data

1.2 Install lerobot

# Install specific version 0.3.4
git clone https://github.com/huggingface/lerobot.git
cd lerobot
git checkout da5d2f3e9187fa4690e6667fe8b294cae49016d6
pip install -e .

1.3 Clone and Install UR5e Teleoperation Control Source

mkdir ur_data_collection && cd ur_data_collection
git clone https://github.com/scy-v/lerobot_ur5e_auto.git
cd lerobot_ur5e_auto
pip install -r requirements.txt # install all packages in editable mode

1.4 Compile ROS Tool Source Code

• Already compiled. If you wish to recompile, execute:

# Enter /robot_ros_interface directory
sudo apt-get install python-numpy
catkin build -w . --cmake /usr/bin/cmake --cmake-args -DCMAKE_POLICY_VERSION_MINIMUM=3.5 -Wno-dev

1.5 View Available Commands

Run the following command to display all available scrtips:

ur5e-help
# ==================================================
#  UR5e Teleoperation Utilities - Command Reference
# ==================================================

# Core Commands:
#   ur5e-record           Record teleoperation dataset
#   ur5e-replay           Replay a recorded dataset
#   ur5e-visualize        Visualize recorded dataset
          
# Tool Commands:
#   tools-check-dataset   Check local dataset information
#   tools-check-rs        Retrieve connected RealSense camera serial numbers
#   tools-get-ur          Get UR state 
          
# Shell Tools:
#   map_gripper.sh        Map Gripper Serial Port
#   check_master_port.sh  Get the Master Arm's Persistent Serial Identifier
#   init_ros.sh           Initialize ROS environment (source setup.bash)
          
# Test Commands:
#   test-gripper-ctrl     Run gripper control command (operate the gripper)
#   test-ur-freedrive     Run UR freedrive command (enter/exit freedrive mode)
# --------------------------------------------------
#  Tip: Use 'ur5e-help' anytime to see this summary.
# ==================================================

2. Obtain and Configure Necessary Parameters

2.1 Get RealSense Camera Serial Number

Please make sure only one camera is connected at a time.

tools-check-rs

2.2 Map Gripper Serial Port

For example, map the gripper to /dev/ur5e_left_gripper and make sure only one gripper USB device is connected:

map_gripper.sh ur5e_left_gripper

Next, enter the mapped device into the gripper_port field in cfg.yaml.

3. Dataset Recording

3.1 Upload Data to Hugging Face (Optional)

1. Set the following in cfg.yaml:

push_to_hub: True

2. Obtain your Hugging Face account token and login:

huggingface-cli login --token ${HUGGINGFACE_TOKEN} 
huggingface-cli whoami  # Verify successful login

3.2 Start Recording

1. Turn on Remote Control on the UR5e teach pendant.
2. Verify that the parameters in cfg.yaml are correct.
3. To ensure standardized data collection, please read 9. Dataset Naming and Recording Details before running the following command:

bash
source ~/miniconda3/bin/activate
conda activate ur_data
source init_ros.sh
ur5e-record

4. Dataset Replay

ur5e-replay # not used

5. Dataset Visualization

ur5e-visualize # Make sure cfg.yaml is properly configured

6. Dataset Appending and Resume

If you have already recorded a dataset under the specified repo_id, set resume: True in cfg.yaml and enter the dataset name in resume_dataset to continue recording new data on the existing dataset. Then, run the following command:

ur5e-record

7. Merge Datasets

If you recorded data in multiple stages, ensure each dataset has a unique repo_id. Use the following command to merge them into a single dataset:

lerobot-edit-dataset 
    --repo_id <merged_repo_id> 
    --operation.type merge 
    --operation.repo_ids "['<repo_id_1>', '<repo_id2>']"

• For more dataset processing commands, see LeRobot.

8. Dataset Naming and Recording Details

1. Dataset Naming

Figure 3: Dataset

Figure 4: Dataset Info

1. The dataset is by default stored in the ~/.cache/huggingface/lerobot directory and contains three types of items:

   • dataset_info.txt: Automatically records local dataset information, including the following fields: record_id, name, task, date, version, user_info, and type. The user_info field can be annotated via the user_notes entry in cfg.yaml.

   • dataset_info_backup: When tools-check-dataset is used to manually update dataset_info.txt, this folder stores backups of the previous records.

   • Dataset folders: Stores the actual dataset contents.

2. Dataset names follow the format [description]_[date]_[version], where:

   • description comes from cfg.yaml as repo_id=<user_name>/<description>;
   • date is generated automatically;
   • version is incremented automatically if a dataset with the same repo_id already exists.

3. Description naming rule: task.description -> Verb_SourceObject_prep_TargetObject.
   For example:
   Pick up the green cube and put it into the trash bin -> pick_greencube_into_trashbin.

2. Recording Instructions

• After recording, a dataset_info.txt file will be automatically created in the same directory to store local dataset information. If you have manually deleted any datasets, update the dataset_info.txt using:

tools-check-dataset

9. Other Issues

sudo find ~/miniconda3/envs/ur_data/ -name 'libffi*'
sudo find /usr/lib/ -name 'libffi*'
mv ~/miniconda3/envs/ur_data/lib/libffi.so.7 ~/miniconda3/envs/ur_data/lib/libffi.so.7.bak
sudo ln -s /usr/lib/x86_64-linux-gnu/libffi.so.7.1.0 ~/miniconda3/envs/ur_data/lib/libffi.so.7