Skip to content

Commit 54fbf4e

Browse files
oxkitsunegrtlr
oxkitsune
and
grtlr
authored
Add MCAP docs page (#11011)
### Related * Part of #9949 * Closes https://linear.app/rerun/issue/RR-2268/add-additional-sections-to-mcap-documentation. * Closes https://linear.app/rerun/issue/RR-2259/add-mcap-section-to-our-documentation. ### What This a documentation page for the MCAP support. --------- Co-authored-by: Jochen Görtler <[email protected]> Co-authored-by: Jochen Görtler <[email protected]>
1 parent 3e9060f commit 54fbf4e

File tree

7 files changed

+313
-2
lines changed

7 files changed

+313
-2
lines changed

‎docs/content/getting-started/configure-the-viewer.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Configure the viewer
3-
order: 600
3+
order: 700
44
---
55

66
By default, the Rerun Viewer uses heuristics to automatically determine an appropriate

‎docs/content/getting-started/mcap.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
---
2+
title: Working with MCAP
3+
order: 500
4+
---
5+
6+
The Rerun Viewer has built-in support for opening [MCAP](https://mcap.dev/) files, an open container format for storing timestamped messages.
7+
8+
⚠️ **This is an early version of MCAP support** that will continue to evolve and expand over time. We are actively seeking feedback from the community to guide development priorities. Reinterpretation of custom messages and enhanced query capabilities are planned for following releases.
9+
10+
## Quick start
11+
12+
### Loading MCAP files
13+
14+
The simplest way to get started is to load an MCAP file directly:
15+
16+
```bash
17+
# View an MCAP file in the Rerun Viewer
18+
rerun your_data.mcap
19+
```
20+
21+
You can also drag and drop MCAP files into the Rerun Viewer or load them using the SDK:
22+
23+
snippet: howto/load_mcap
24+
25+
### Basic conversion
26+
27+
Convert MCAP files to Rerun's native format for faster loading:
28+
29+
```bash
30+
# Convert MCAP to RRD format for faster loading
31+
rerun mcap convert input.mcap -o output.rrd
32+
33+
# View the converted file
34+
rerun output.rrd
35+
```
36+
37+
## Data model
38+
39+
Rerun's data model is based on an [entity component system (ECS)](../concepts/entity-component.md) that is a bit different to the message-based model of [MCAP](https://mcap.dev).
40+
To map MCAP messages to Rerun entities we make the following assumptions:
41+
42+
* MCAP topics corresponds to Rerun entities.
43+
* Messages from the same topic within an MCAP chunk will be placed into a corresponding [Rerun chunk](../concepts/chunks.md).
44+
* The contents of an MCAP message will be extracted to Rerun components and grouped under a corresponding Rerun archetype.
45+
* `log_time` and `publish_time` of an MCAP message will be carried over to Rerun as two distinct [timelines](../concepts/timelines.md).
46+
47+
### Layered architecture
48+
49+
Rerun uses a _layered architecture_ to process MCAP files at different levels of abstraction. This design allows the same MCAP file to be ingested in multiple ways simultaneously, from raw bytes to semantically meaningful visualizations.
50+
51+
Each layer extracts different types of information from the MCAP source and each of the following layers will create distinct Rerun archetypes:
52+
53+
- **`raw`**: Logs the unprocessed message bytes as Rerun blobs without any interpretation
54+
- **`schema`**: Extracts metadata about channels, topics, and schemas
55+
- **`stats`**: Extracts file-level metrics like message counts, time ranges, and channel statistics
56+
- **`protobuf`**: Automatically decodes protobuf-encoded messages using reflection
57+
- **`ros2msg`**: Provides semantic conversion of common ROS2 message types into Rerun's visualization components
58+
- **`recording_info`**: Extracts recording metadata such as message counts, start time, and session information
59+
60+
By default, Rerun analyzes an MCAP file to determine which layers are active to provide the most comprehensive view of your data, while avoiding duplication.
61+
You can also choose to activate only specific layers that are relevant to your use case.
62+
63+
The following shows how to select specific layers:
64+
65+
```sh
66+
# Use only specific layers
67+
rerun mcap convert input.mcap -l protobuf -l stats -o output.rrd
68+
69+
# Use multiple layers for different perspectives
70+
rerun mcap convert input.mcap -l ros2msg -l raw -l recording_info -o output.rrd
71+
```
72+
73+
For a detailed explanation of how each layer works and when to use them, see [Layers Explained](../reference/mcap/layers-explained.md).
74+
75+
## Supported message formats
76+
77+
Rerun provides automatic visualization for common ROS2 message types. Protobuf messages are automatically decoded into Arrow structs, but for now will only show up in the selection panel and in the dataframe view. The contents of these MCAP files can also be queried using the Dataframe API.
78+
79+
Unsupported message types (such as ROS1 messages) remain available as raw bytes in Arrow format.
80+
81+
The following is a screenshot of the selection panel and shows a Protobuf-encoded MCAP message. The top-level fields of the Protobuf message are imported as components in the corresponding point cloud archetype. The raw MCAP schema and message information show up as separate archetypes as well.
82+
83+
<picture style="zoom: 0.5">
84+
<img src="https://static.rerun.io/mcap_raw_arrow/17b7723690c46901d14e6c1d264298ce0ca8c3ae/full.png" alt="Screenshot of MCAP messages converted to raw Arrow data in the selection panel">
85+
<source media="(max-width: 480px)" srcset="https://static.rerun.io/mcap_raw_arrow/17b7723690c46901d14e6c1d264298ce0ca8c3ae/480w.png">
86+
<source media="(max-width: 768px)" srcset="https://static.rerun.io/mcap_raw_arrow/17b7723690c46901d14e6c1d264298ce0ca8c3ae/768w.png">
87+
<source media="(max-width: 1024px)" srcset="https://static.rerun.io/mcap_raw_arrow/17b7723690c46901d14e6c1d264298ce0ca8c3ae/1024w.png">
88+
</picture>
89+
90+
For more details about all supported message types, see [Message Formats](../reference/mcap/message-formats.md).
91+
92+
## Advanced usage
93+
94+
For advanced command-line options and automation workflows, see the [CLI Reference](../reference/mcap/cli-reference.md) for complete documentation of all available commands and flags.

‎docs/content/getting-started/navigating-the-viewer.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Navigating the viewer
3-
order: 500
3+
order: 600
44
---
55

66
This guide will familiarize you with the basics of using the Rerun Viewer with an example dataset. By the end you should be comfortable with the following topics:

‎docs/content/reference/mcap.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
title: MCAP files
3+
order: 850
4+
redirect: reference/mcap/message-formats
5+
---

‎docs/content/reference/mcap/cli-reference.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
---
2+
title: CLI Reference for MCAP
3+
order: 500
4+
---
5+
6+
This reference guide covers all command-line options and workflows for working with MCAP files in Rerun.
7+
8+
## Basic commands
9+
10+
### Direct viewing
11+
12+
Open MCAP files directly in the Rerun Viewer:
13+
14+
```bash
15+
# View a single MCAP file
16+
rerun data.mcap
17+
18+
# View multiple specific files
19+
rerun file1.mcap file2.mcap file3.mcap
20+
21+
# Use glob patterns to load all MCAP files in a directory
22+
rerun recordings/*.mcap
23+
24+
# Recursively load all MCAP files from a directory
25+
rerun mcap_data/
26+
```
27+
28+
### File conversion
29+
30+
Convert MCAP files to Rerun's native RRD format:
31+
32+
```bash
33+
# Convert MCAP to RRD format for faster loading
34+
rerun mcap convert input.mcap -o output.rrd
35+
36+
# Convert with custom output location
37+
rerun mcap convert data.mcap -o /path/to/output.rrd
38+
```
39+
40+
## Layer selection
41+
42+
### Using specific layers
43+
44+
Control which processing layers are applied during conversion:
45+
46+
```bash
47+
# Use only protobuf decoding and file statistics
48+
rerun mcap convert input.mcap -l protobuf -l stats -o output.rrd
49+
50+
# Use only ROS2 semantic interpretation for robotics data
51+
rerun mcap convert input.mcap -l ros2msg -o output.rrd
52+
53+
# Combine multiple layers for comprehensive data access
54+
rerun mcap convert input.mcap -l ros2msg -l raw -l recording_info -o output.rrd
55+
```
56+
57+
### Available layer options
58+
59+
- **`raw`**: Preserve original message bytes
60+
- **`schema`**: Extract metadata and schema information
61+
- **`stats`**: Compute file and channel statistics
62+
- **`protobuf`**: Decode protobuf messages using into generic Arrow data without Rerun visualization components
63+
- **`ros2msg`**: Semantic interpretation of ROS2 messages
64+
- **`recording_info`**: Extract recording session metadata
65+
66+
### Default behavior
67+
68+
When no `-l` flags are specified, all available layers are used:
69+
70+
```bash
71+
# These commands are equivalent (default uses all layers):
72+
73+
rerun mcap convert input.mcap -o output.rrd
74+
75+
rerun mcap convert input.mcap \
76+
-l raw \
77+
-l schema \
78+
-l stats \
79+
-l protobuf \
80+
-l ros2msg \
81+
-l recording_info \
82+
-o output.rrd
83+
```

‎docs/content/reference/mcap/layers-explained.md‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
---
2+
title: MCAP Layers Explained
3+
order: 300
4+
---
5+
6+
MCAP processing in Rerun uses a layered architecture where each layer represents a different way to interpret and extract data from the same MCAP source.
7+
By default, when opening a file Rerun analyzes an MCAP file to determine which layers are active to provide the most comprehensive view of your data, while avoiding duplication.
8+
You can specify which layers to use during conversion, allowing you to extract exactly the information you need for your analysis.
9+
10+
## Understanding layers with an example
11+
12+
When multiple layers are enabled, they each process the same messages independently, creating different component types on identical entity paths. This can result in data duplication—for instance, enabling both `raw` and `protobuf` layers stores the same message as both structured field data and raw binary blobs.
13+
14+
Consider an MCAP file from a ROS2 robot containing sensor data on the topic `/robot/camera/image_raw` with ROS2 `sensor_msgs/msg/Image` messages:
15+
16+
- With only the `ros2msg` layer: Creates an [Image](../../reference/types/archetypes/image.md) archetype for direct visualization in Rerun's viewer
17+
- With only the `raw` layer: Creates an [McapMessage](../../reference/types/archetypes/mcap_message.md) containing the original CDR-encoded message bytes
18+
- With both layers enabled: All representations coexist on the same entity path `/robot/camera/image_raw`
19+
20+
## Schema and statistics layers
21+
22+
The `schema` layer extracts structural information about the MCAP file's organization, creating metadata entities that describe channel definitions, topic names with their message types, and schema definitions. This layer is particularly useful for understanding unfamiliar MCAP files or getting an overview of available topics and channels before deeper processing.
23+
24+
The `stats` layer computes file-level metrics and statistics, creating entities with message counts per channel, temporal ranges, file size information, and data rate analysis. This gives you insight into the scale and characteristics of your dataset for quality assessment and planning storage requirements.
25+
26+
## Message interpretation layers
27+
28+
### ROS2 semantic interpretation
29+
30+
The `ros2msg` layer provides semantic interpretation and visualization of standard ROS2 message types, creating meaningful Rerun visualization components from data. Unlike the `protobuf` layer, this layer understands the semantics of ROS2 messages and creates appropriate visualizations: images become [Image](../../reference/types/archetypes/image.md), point clouds become [Points3D](../../reference/types/archetypes/points3d.md), IMU messages become [SeriesLines](../../reference/types/archetypes/series_lines.md) with the data plotted over time, and so on.
31+
32+
This layer supports standard ROS2 packages including `sensor_msgs`, `geometry_msgs`, `std_msgs`, and `builtin_interfaces`. This layer provides visualization of sensor data like cameras and LiDAR with minimal setup required.
33+
34+
See [Message Formats](message-formats.md) for the complete list of supported message types.
35+
36+
### Protobuf decoding
37+
38+
The `protobuf` layer automatically decodes protobuf-encoded messages using reflection, creating structured component data based on the protobuf schema. Message fields become Rerun components that you can query and analyze.
39+
40+
However, this layer provides structured access without semantic visualization meaning. While the data becomes queryable, it won't automatically appear as meaningful visualizations like images or point clouds, it gives you the data structure, not the visual interpretation.
41+
42+
## The raw layer
43+
44+
The `raw` layer preserves the original message bytes without any interpretation, creating blob entities containing the unprocessed message data. Each message appears as a binary blob that can be accessed programmatically for custom analysis tools.
45+
46+
## Recording info
47+
48+
The `recording_info` layer extracts metadata about the recording session and capture context, creating metadata entities with information about recording timestamps, source system details, and capture software versions.
49+
50+
## Layer selection and performance
51+
52+
### Selecting layers
53+
54+
By default, Rerun processes MCAP files with all layers active. You can control which layers are used when [converting MCAP files via the CLI](cli-reference.md) using the `-l` flag:
55+
56+
```bash
57+
# Use only specific layers
58+
rerun mcap convert input.mcap -l protobuf -l stats -o output.rrd
59+
60+
# Use multiple layers for different perspectives
61+
rerun mcap convert input.mcap -l ros2msg -l raw -l recording_info -o output.rrd
62+
```
63+
64+
## Accessing layer data
65+
66+
Each layer creates different types of components on entity paths (derived from MCAP channel topics) that can be accessed through Rerun's SDK:
67+
68+
- Data from the `protobuf` layer appears as structured components that can be queried by field name
69+
- Data from the `ros2msg` layer appears as native Rerun visualization components ([Image](../../reference/types/archetypes/image.md), [Points3D](../../reference/types/archetypes/points3d.md), etc.)
70+
- Data from the `raw` layer appears as blob components containing the original message bytes
71+
- Metadata from `schema`, `stats`, and `recording_info` layers appears as dedicated metadata entities
72+
73+
For more information on querying data and working with archetypes, see the [Data Queries documentation](../../howto/get-data-out.md).
74+
75+
Each of these layers contributes their own [chunks](../../concepts/chunks.md) to the Rerun-native data.
76+
Below is a table showing the mapping between MCAP data and Rerun components:
77+
78+
| MCAP Data | Rerun component | Description |
79+
| ---------------- | ------------------------------- | ----------------------------------------------------------------------------- |
80+
| Schema name | `mcap.Schema:name` | Message type name from schema definition |
81+
| Schema data | `mcap.Schema:data` | Raw schema definition (protobuf, ROS2 msg, etc.) |
82+
| Schema encoding | `mcap.Schema:encoding` | Schema format type |
83+
| | | |
84+
| Channel topic | `mcap.Channel:topic` | Topic name from MCAP channel |
85+
| Channel ID | `mcap.Channel:id` | Numeric channel identifier |
86+
| Message encoding | `mcap.Channel:message_encoding` | Encoding format (e.g., `protobuf`, `cdr`) |
87+
| | | |
88+
| Statistics | `mcap.Statistics` | File-level metrics like message counts and time ranges |
89+
| Raw message data | `mcap.Message:data` | Unprocessed message bytes stored as binary blobs, handled by the `raw` layer. |

‎docs/content/reference/mcap/message-formats.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
title: Supported Message Formats
3+
order: 200
4+
---
5+
6+
Rerun provides automatic visualization for common message types in MCAP files through different processing layers.
7+
8+
## ROS2 message types
9+
10+
The `ros2msg` layer provides automatic visualization for common ROS2 message types:
11+
12+
- **`sensor_msgs`**
13+
- **`std_msgs`**
14+
- **`geometry_msgs`**
15+
- **`builtin_interfaces`**
16+
17+
We are continually adding support for more standard ROS2 message types. For the complete list of currently supported messages, see the [ROS2 message parsers in our codebase](../../../../crates/utils/re_mcap/src/layers/ros2.rs).
18+
19+
### Limitations
20+
21+
The following are known limitations and link to the corresponding GitHub issues.
22+
23+
<!-- TODO(#11174) -->
24+
- [Cannot express transforms defined via `tf` messages](https://github.com/rerun-io/rerun/issues/11174)
25+
26+
## ROS1 message types
27+
28+
ROS1 messages are not currently supported for semantic interpretation through any layer.
29+
The `raw` and `schema` layers are able to preserve the original bytes and structure of the messages.
30+
31+
## Protobuf messages
32+
33+
The `protobuf` layer automatically decodes protobuf-encoded messages using schema reflection. Fields become queryable components, but no automatic visualizations are created.
34+
35+
## Adding support for new types
36+
37+
To request support for additional message types:
38+
39+
- [File a GitHub issue](https://github.com/rerun-io/rerun/issues) requesting the specific message type
40+
- Join the Rerun community on [Discord](https://discord.gg/PXtCgFBSmH) to discuss and provide feedback on message support priorities

0 commit comments

Comments
 (0)