Skip to content

Add reflection based support for deserializing ROS2 MCAP #11367

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 21 commits into
base: main
Choose a base branch
from
+2,167 −421
Open

Add reflection based support for deserializing ROS2 MCAP #11367

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
8cc7c01
IDL
oxkitsune Sep 19, 2025
e39792e
implement parser layer
oxkitsune Sep 22, 2025
17bc3f0
debug
oxkitsune Sep 23, 2025
ccc8f10
fix parsing
oxkitsune Sep 24, 2025
6c9e4f4
cleanup
oxkitsune Sep 24, 2025
289d824
refactor entire reflection module
oxkitsune Sep 25, 2025
1f5bb98
add more test cases
oxkitsune Sep 25, 2025
082a909
Deserialization + correct archetype
oxkitsune Sep 29, 2025
9a4ef3f
fmt
oxkitsune Sep 29, 2025
337521f
remove re_error dependency
oxkitsune Sep 29, 2025
65cab24
Fix empty fields
oxkitsune Sep 29, 2025
329415b
re_ros_msg
oxkitsune Oct 3, 2025
4ef2fc6
Remove unused dependencies
oxkitsune Oct 3, 2025
e7fc5b4
update snapshot
oxkitsune Oct 6, 2025
de4791a
remove fallback
oxkitsune Oct 6, 2025
a9034bb
Remove arrow data parsing from semantic extractors
oxkitsune Oct 6, 2025
0f46c16
rust-analyzer refactor artifact
oxkitsune Oct 6, 2025
40456d4
Update snapshots/snippets
oxkitsune Oct 6, 2025
eb76270
escape <name>
oxkitsune Oct 6, 2025
be990ab
escape <name> pt. 2
oxkitsune Oct 6, 2025
0ff050b
remove sneaky file
oxkitsune Oct 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ARCHITECTURE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,7 @@ Update instructions:
| re_mcap | Convert MCAP into Rerun-compatible data. |
| re_memory | Run-time memory tracking and profiling. |
| re_perf_telemetry | In and out of process performance profiling utilities for Rerun & Redap |
| re_ros_msg | Parsing and deserializing ROS messages |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we still need to update the Architecture figma (see component above table).

| re_smart_channel | A channel that keeps track of latency and queue length. |
| re_span | An integer range that always has a non-negative length |
| re_string_interner | Yet another string interning library |
Expand Down
10 changes: 10 additions & 0 deletions Cargo.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8988,6 +8988,7 @@ dependencies = [
"re_chunk",
"re_log",
"re_log_types",
"re_ros_msg",
"re_tracing",
"re_types",
"serde",
Expand Down Expand Up @@ -9300,6 +9301,15 @@ dependencies = [
"winit",
]

[[package]]
name = "re_ros_msg"
version = "0.26.0-alpha.1+dev"
dependencies = [
"anyhow",
"serde",
"thiserror 1.0.69",
]

[[package]]
name = "re_sdk"
version = "0.26.0-alpha.1+dev"
Expand Down
1 change: 1 addition & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ re_log = { path = "crates/utils/re_log", version = "=0.26.0-alpha.1", default-fe
re_mcap = { path = "crates/utils/re_mcap", version = "=0.26.0-alpha.1", default-features = false }
re_memory = { path = "crates/utils/re_memory", version = "=0.26.0-alpha.1", default-features = false }
re_perf_telemetry = { path = "crates/utils/re_perf_telemetry", version = "=0.26.0-alpha.1", default-features = false }
re_ros_msg = { path = "crates/utils/re_ros_msg", version = "=0.26.0-alpha.1", default-features = false }
re_smart_channel = { path = "crates/utils/re_smart_channel", version = "=0.26.0-alpha.1", default-features = false }
re_span = { path = "crates/utils/re_span", version = "=0.26.0-alpha.1", default-features = false }
re_string_interner = { path = "crates/utils/re_string_interner", version = "=0.26.0-alpha.1", default-features = false }
Expand Down
4 changes: 2 additions & 2 deletions crates/store/re_data_loader/tests/snapshots/test_mcap_loader__tests__ros2.snap
View file
Open in desktop
Git LFS file not shown
1 change: 1 addition & 0 deletions crates/utils/re_mcap/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ workspace = true
re_chunk.workspace = true
re_log.workspace = true
re_log_types.workspace = true
re_ros_msg.workspace = true
re_tracing.workspace = true
re_types = { workspace = true, features = ["ecolor", "glam", "image", "video"] }

Expand Down
5 changes: 4 additions & 1 deletion crates/utils/re_mcap/src/layers/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ mod protobuf;
mod raw;
mod recording_info;
mod ros2;
mod ros2_reflection;
mod schema;
mod stats;

Expand All @@ -10,7 +11,8 @@ use std::collections::{BTreeMap, BTreeSet};

pub use self::{
protobuf::McapProtobufLayer, raw::McapRawLayer, recording_info::McapRecordingInfoLayer,
ros2::McapRos2Layer, schema::McapSchemaLayer, stats::McapStatisticLayer,
ros2::McapRos2Layer, ros2_reflection::McapRos2ReflectionLayer, schema::McapSchemaLayer,
stats::McapStatisticLayer,
};

use crate::{
Expand Down Expand Up @@ -324,6 +326,7 @@ impl LayerRegistry {
.register_file_layer::<McapStatisticLayer>()
// message layers (priority order):
.register_message_layer::<McapRos2Layer>()
.register_message_layer::<McapRos2ReflectionLayer>()
.register_message_layer::<McapProtobufLayer>();

if raw_fallback_enabled {
Expand Down
1 change: 1 addition & 0 deletions crates/utils/re_mcap/src/layers/raw.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ impl MessageParser for RawMcapMessageParser {
msg: &::mcap::Message<'_>,
) -> anyhow::Result<()> {
re_tracing::profile_function!();

self.data.values().values().append_slice(&msg.data);
self.data.values().append(true);
self.data.append(true);
Expand Down
7 changes: 4 additions & 3 deletions crates/utils/re_mcap/src/layers/ros2.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
use std::collections::BTreeMap;

use super::MessageLayer;
use crate::{
parsers::MessageParser,
parsers::ros2msg::{
use crate::parsers::{
MessageParser,
ros2msg::{
Ros2MessageParser,
rcl_interfaces::LogMessageParser,
sensor_msgs::{
Expand Down Expand Up @@ -114,6 +114,7 @@ impl MessageLayer for McapRos2Layer {
"Message schema {:?} is currently not supported",
schema.name
);

None
}
}
Expand Down
Loading