KeyError: 'goal_pixel_values'

[YangJC112]

Bug Report

Problem Description

When running the train_generalist_calvin.py script, I encountered a KeyError during the data loading process. The error indicates that the key 'goal_pixel_values' is missing from the instance dictionary.

Error Message

Traceback (most recent call last):
File "/inspire/hdd/project/agentend2end/zhanghongquan-240108120118/RoboDual/RoboDual/vla-scripts/train_generalist_calvin.py", line 272, in

Investigation

I checked the keys of the instance dictionary and printed them. The output is as follows:
First instance keys: ['pixel_values', 'pixel_values_dp', 'prev_pixel_values_dp', 'depth_image', 'gripper_image', 'depth_gripper', 'rgb_tactile', 'input_ids', 'labels', 'dataset_name', 'raw_action', 'lang', 'proprio', 'hist_action', 'valid_cond_mask']

It seems that the key 'goal_pixel_values' is missing from the instance dictionary, which is causing the KeyError.

Steps to Reproduce

1. Run the train_generalist_calvin.py script.
2. The error occurs during the data loading process.

Expected Behavior

The script should be able to load the data without encountering a KeyError.

Possible Solution

I suspect that the data loading process might not be correctly configured to include the 'goal_pixel_values' key. Could you please provide some guidance on how to fix this issue or what might be causing it?

Thank you for your help!