Skip to content

README.md

Latest commit

 

History

History
70 lines (48 loc) · 2.49 KB

README.md

File metadata and controls

70 lines (48 loc) · 2.49 KB

Finetuning GR00T for Point Navigation

This guide shows how to finetune GR00T for point navigation tasks using datasets generated by COMPASS.

Dataset

To generate and prepare the dataset, follow the COMPASS GR00T Post-training Guide:

  1. Train residual RL specialists
  2. Collect specialist distillation data
  3. Convert HDF5 dataset to GR00T LeRobot format using scripts/hdf5_to_lerobot_episodic.py

For a quick start, a pre-collected G1 robot dataset is available on HuggingFace.

Modality Configuration

The point navigation task uses the following input and output modalities defined in modality.json:

Inputs

Modality Key Indices Dimension Description
Video ego_view - H×W×3 Ego-centric RGB camera image
State speed [0, 1) 1 Robot forward speed
State route [1, 41) 40 Route segments in robot frame (10 segments × 4 values: x_start, y_start, x_end, y_end)
State goal_heading [41, 43) 2 Goal heading direction (cos θ, sin θ)
Language task_description - - Robot Navigation Task

Outputs

Modality Key Indices Dimension Description
Action vel_cmd [0, 3) 3 Velocity command (vx, vy, ωz)

Finetuning

Run the finetuning script:

uv run bash examples/PointNav/finetune_point_nav.sh

Remember to update the script with your dataset and output paths:

  • --dataset-path: Path to the converted LeRobot format dataset
  • --output-dir: Directory to save checkpoints

Evaluation

Step 1: Launch the GR00T inference server:

uv run --extra=gpu python gr00t/eval/run_gr00t_server.py \
    --model-path <path/to/checkpoint> \
    --embodiment-tag NEW_EMBODIMENT \
    --device cuda:0 \
    --host 0.0.0.0 \
    --port 8888

Step 2: Run the COMPASS evaluation following the evaluation instructions.

Results

Task success rate on 640 randomized test cases:

Model In-Distribution Out-of-Distribution
GR00T N1.6 86.3% 76.5%
GR00T N1.5 86.1% 77.6%
COMPASS (baseline) 84.7% 45.6%