Skip to content

Commit 4d08bbe

Browse files
authored
Keep asset video by default for rerun visualization (#135)
1 parent a7881b5 commit 4d08bbe

File tree

3 files changed

+306
-62
lines changed

3 files changed

+306
-62
lines changed

docs/source/tutorials/visualization.rst

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,40 @@ The following example shows how to visualize the first episode (index 0) of the
99
1010
opentau-dataset-viz --repo-id lerobot/droid_100 --episode-index 0
1111
12+
When the dataset is already available in a local directory, ``--repo-id`` is optional and you can point directly to it with ``--root``:
13+
14+
.. code-block:: bash
15+
16+
opentau-dataset-viz --root ~/.cache/huggingface/lerobot/lerobot/droid_100 --episode-index 0
17+
1218
This command will open a `rerun <https://rerun.io>`_ window displaying the selected episode, allowing you to explore the episode interactively.
1319

20+
Camera Logging Mode
21+
-------------------
22+
23+
For datasets that include camera streams stored as MP4 files, ``opentau-dataset-viz`` supports different camera logging modes:
24+
25+
- ``frames``: logs decoded frames with ``rr.Image``.
26+
- ``asset_video``: logs source MP4 files with ``rr.AssetVideo`` and synchronizes playback using ``rr.VideoFrameReference``.
27+
- ``auto`` (default): same behavior as ``asset_video`` with graceful fallback to ``frames`` when MP4 assets or required Rerun APIs are unavailable.
28+
29+
If your episodes contain one or more videos and the generated ``.rrd`` files are large, prefer ``asset_video`` (or ``auto``) to take advantage of video compression.
30+
31+
.. code-block:: bash
32+
33+
# Prefer MP4 assets (smaller .rrd when videos are available)
34+
opentau-dataset-viz --repo-id lerobot/droid_100 --episode-index 0 --camera-log-mode asset_video
35+
36+
# Always log decoded frames (existing behavior)
37+
opentau-dataset-viz --repo-id lerobot/droid_100 --episode-index 0 --camera-log-mode frames
38+
39+
To compare storage size, save both outputs and compare the resulting files:
40+
41+
.. code-block:: bash
42+
43+
opentau-dataset-viz --repo-id lerobot/droid_100 --episode-index 0 --save 1 --output-dir ./rrd_frames --camera-log-mode frames
44+
opentau-dataset-viz --repo-id lerobot/droid_100 --episode-index 0 --save 1 --output-dir ./rrd_asset --camera-log-mode asset_video
45+
1446
OpenTau also supports visualizing a dataset with URDF models. To do this, you need to first install ``opentau`` with optional URDF support:
1547

1648
.. code-block:: bash

src/opentau/datasets/lerobot_dataset.py

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1694,11 +1694,21 @@ def save_episode(self, episode_data: dict | None = None) -> None:
16941694
self.tolerance_s,
16951695
)
16961696

1697-
video_files = list(self.root.rglob("*.mp4"))
1698-
assert len(video_files) == self.num_episodes * len(self.meta.video_keys)
1699-
1700-
parquet_files = list(self.root.rglob("*.parquet"))
1701-
assert len(parquet_files) == self.num_episodes
1697+
expected_episodes = self.meta.total_episodes
1698+
missing_videos: list[str] = []
1699+
for ep_idx in range(expected_episodes):
1700+
for vid_key in self.meta.video_keys:
1701+
video_path = self.root / self.meta.get_video_file_path(ep_idx, vid_key)
1702+
if not video_path.is_file():
1703+
missing_videos.append(str(video_path))
1704+
assert not missing_videos, "Missing expected encoded videos:\n" + "\n".join(missing_videos)
1705+
1706+
missing_parquet: list[str] = []
1707+
for ep_idx in range(expected_episodes):
1708+
parquet_path = self.root / self.meta.get_data_file_path(ep_idx)
1709+
if not parquet_path.is_file():
1710+
missing_parquet.append(str(parquet_path))
1711+
assert not missing_parquet, "Missing expected parquet episode files:\n" + "\n".join(missing_parquet)
17021712

17031713
# delete images
17041714
img_dir = self.root / "images"

0 commit comments

Comments
 (0)