Skip to content

Conversation

@shay-ff
Copy link

@shay-ff shay-ff commented Oct 22, 2025

🐛 Fix: Training crash when using --policy.n_obs_steps=1

Summary

Fixes a bug where training crashed when --policy.n_obs_steps=1.
Previously, the dataset returned tensors with the wrong number of channels, causing a RuntimeError in the encoder.

Changes

  • Converts single-frame images from [H, W, C][C, H, W]
  • Converts multi-frame images from [N, H, W, C][N*C, H, W]
  • Applies image_transforms per frame
  • Works for any positive n_obs_steps

🧪 How it was tested

  • Verified that training starts successfully with both:
    • n_obs_steps=1
    • n_obs_steps>1
  • Checked that output image tensors have correct dimensions:
    [n_obs_steps*3, H, W]
  • Confirmed that no other functionality is affected.

🔍 How to checkout & try

# Install dependencies if needed
pip install -r requirements.txt

# Train with n_obs_steps=1 (should not crash)
lerobot-train --policy.n_obs_steps=1

# Train with n_obs_steps=2 (should also work)
lerobot-train --policy.n_obs_steps=2

Fixes #2277

Copilot AI review requested due to automatic review settings October 22, 2025 12:04
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Fixes a bug where training crashed when using n_obs_steps=1 due to incorrect tensor shape handling for image observations. The code now properly converts image tensors from HWC to CHW format and handles both single-frame and multi-frame cases.

  • Adds tensor shape conversion logic to handle both [H, W, C] and [N, H, W, C] formats
  • Applies image transforms per-frame for multi-frame observations
  • Reshapes multi-frame tensors to [N*C, H, W] format expected by the encoder

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@shay-ff
Copy link
Author

shay-ff commented Oct 22, 2025

Hi @jadechoghari this PR addresses issue #2277 and applies all suggested fixes.
Could you please review when you have a moment? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Diffusion Policy don't support n_obs_steps=1?

1 participant