Compressed Image Display #1288

mjforan · 2024-10-24T15:38:10Z

Addresses #738

I added RosTopicMultiProperty, which extends RosTopicProperty with a list of message types instead of one single type. This allows the ImageDisplay to show raw and compressed image topics together in its selector list. ImageDisplay was altered to use an image_transport subscriber so it can seamlessly display any installed image type. There is also a handful of frivolous changes to comply with the style linters.

Tested with raw and compressed images. Assuming the other types work as well because the conversion happens in the image_transport library.

Uncertainties:

RosTopicMultiProperty: Is the name ok? Is there a simpler way to achieve my goal without adding a one-off property?
plugin_description.xml: I added the other image_transport topic types here so Image Display appears in the "Add/By topic" menu. Adding the display will fail if the associated image_transport plugin is not installed. Should ros-$ROS_DISTRO-image-transport-plugins be added to package.xml?

mjforan · 2024-11-07T16:04:32Z

Added a property to manually override image transport type. Rviz determines the transport by checking the topic name against image_transport conventions, e.g. camera_name/raw, camera_name/compressed, etc. If this is not the case, the transport must be set manually.
I also fixed the crash caused by adding a topic with an unavailable transport. Instead it will provide an error on the status property saying the transport is not installed.

ahcorde

This is wierd

I had a default rviz.config file with an image display and Rviz2 crashes, do you mind to take a look ? we don't want to break user.

rviz_common/include/rviz_common/properties/ros_topic_multi_property.hpp

mjforan · 2024-11-08T14:37:49Z

@ahcorde

This is wierd

Sorry about that, I have pushed a fix.

I had a default rviz.config file with an image display and Rviz2 crashes, do you mind to take a look ? we don't want to break user.

Please provide details so I can reproduce your issue.

mjforan · 2024-11-08T14:49:34Z

I would like to list the available image transports in a status message. However, setStatusStd only seems to work when it is called from subscribe, not onInitialize or updateTopic. Any idea why?

The status message doesn't really matter except the transport->message type map is hard-coded, so we will want a warning if a new transport type is used which does not exist in the map.

mjforan · 2024-11-12T16:18:49Z

I noticed the Jazzy version of image_transport finally allows passing in custom QoS so I updated the API here to match. This change is not compatible with previous versions of ROS.

mjforan · 2024-11-13T13:32:40Z

Can ImageDisplay be combined with the parent class ImageTransportDisplay? That would allow CameraDisplay to also show compressed image topics.

…re subscribing

ahcorde

I run a image publisher and there is an issue. I'm able to subscribe to raw and compressed but I'm not able to subscribe to zstd.

rviz_default_plugins/CMakeLists.txt

rviz_common/src/rviz_common/properties/ros_topic_multi_property.cpp

rviz_default_plugins/include/rviz_default_plugins/displays/image/image_display.hpp

rviz_default_plugins/test/rviz_default_plugins/publishers/compressed_image_publisher.hpp

ahcorde · 2025-03-24T16:21:31Z

rviz_default_plugins/test/rviz_default_plugins/publishers/compressed_image_publisher.hpp

+  {
+    publisher = this->create_publisher<sensor_msgs::msg::CompressedImage>(topic_name, 10);
+    timer = this->create_wall_timer(100ms,
+        std::bind(&CompressedImagePublisher::timer_callback, this));


include <utility>

For what? Do you mean <functional> for std::bind?

rviz_default_plugins/test/rviz_default_plugins/publishers/compressed_image_publisher.hpp

mjforan · 2025-03-24T19:14:55Z

I run a image publisher and there is an issue. I'm able to subscribe to raw and compressed but I'm not able to subscribe to zstd.

All the plugins are handled by image_transport. The only transport-specific part of my code is the map saying zstd transport uses sensor_msgs/msg/CompressedImage messages. Did you use the transport override property to specify zstd?

mjforan · 2025-03-27T12:12:25Z

@ahcorde

ahcorde

You included many changes which are not related with this PR, I mean the headers inclusion. Do you mind to split the PR in two ?

rviz_default_plugins/package.xml

ahcorde · 2025-03-28T12:38:12Z

rviz_default_plugins/package.xml

  <test_depend>rviz_rendering_tests</test_depend>
  <test_depend>rviz_visual_testing_framework</test_depend>
+  <test_depend>image_transport_plugins</test_depend>
+  <test_depend>vision_opencv</test_depend>


Same idea here, vision_opencv is not a core package, we can't use this one here

OpenCV is used in compressed_image_publisher.hpp, to encode a jpeg image.

mjforan · 2025-04-02T11:59:04Z

@ahcorde

mjforan · 2025-05-15T14:36:23Z

@ahcorde

Signed-off-by: Matthew Foran <[email protected]>

mjforan · 2025-05-19T12:47:27Z

@cottsay

mjforan · 2025-05-19T12:49:54Z

@audrow @sloretz @clalancette @MichaelOrlov

…isplay

ahcorde · 2025-05-19T15:08:30Z

Pulls: #1288
Gist: https://gist.githubusercontent.com/ahcorde/753b92fff53f4ae45626059ba1fca518/raw/8ba713dd713ffadefa3801686787117bf9f33380/ros2.repos
BUILD args: --packages-above-and-dependencies rviz_common rviz_default_plugins
TEST args: --packages-above rviz_common rviz_default_plugins
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/16039

Linux
Linux-aarch64
Linux-rhel
Windows

mjforan · 2025-05-19T16:41:10Z

The "unstable" builds are caused by GitHub rate limiting. I think they changed something recently because I have been seeing it frequently in my own CI processes.

ahcorde · 2025-05-19T17:13:25Z

The "unstable" builds are caused by GitHub rate limiting. I think they changed something recently because I have been seeing it frequently in my own CI processes.

This is is related with RViz in particular with rviz_default_plugins.image_display_test, there is a segfault

mjforan · 2025-05-19T18:22:02Z

Oh, I see now. That must be the result of some changes outside of this PR because it was not failing in March. I will look into it.

ijnek · 2025-08-15T21:03:08Z

We would like to get this PR over the line. Is there anything we can do to help here?

mjforan · 2025-08-15T21:10:38Z

@ijnek sorry, I am in the middle of a cross-country move so I haven't been able to look at this for a while. As of May a new segfault was the only blocker for this PR. I can't provide much support right now but I'll be on it again in a month or so.

mjforan · 2025-08-24T02:27:01Z

@ahcorde The test build reports errors apparently stemming from point cloud display updates in 7248938. Everything builds successfully on my system so maybe there is a problem with the automated build setup.

mjforan · 2025-08-31T16:47:29Z

I putting this PR on hold until the issues regarding abstract node interfaces are worked out.

mjforan requested a review from ahcorde as a code owner October 24, 2024 15:38

cottsay assigned ahcorde Nov 1, 2024

ahcorde requested changes Nov 8, 2024

View reviewed changes

rviz_common/include/rviz_common/properties/ros_topic_multi_property.hpp Show resolved Hide resolved

mjforan mentioned this pull request Nov 12, 2024

Include QString #1298

Merged

mjforan marked this pull request as draft November 13, 2024 13:47

mjforan and others added 17 commits November 20, 2024 17:55

first draft and devcontainer

188eb92

devcontainer working, compressed image working

d2e3289

topic-multi property working (segfaults on close)

3f3296d

remove image transport plugins as dependency

f8bb97e

segfault was not my fault

5928cd9

minor cleanup and comments

fcdf022

code style

ef95ae8

add other image types

0becd68

remove docker stuff

64d0ef4

update README

b658026

remove unnecessary cast

5d7eebc

property to override transport, make sure transport is installed befo…

752fa43

…re subscribing

fix build server compilation error

b1638dc

fix duplicate transport override items

7063d9f

use custom QoS

60a69d2

combine image_transport_display and image_display

0bc298d

add test for compressed image display

40711b7

mjforan force-pushed the compressed-image-display branch from 48133ae to 40711b7 Compare November 20, 2024 23:42

remove duplicate image texture

287e57a

mjforan marked this pull request as ready for review January 11, 2025 18:24

ahcorde requested changes Mar 24, 2025

View reviewed changes

address code review

76559ad

ahcorde requested changes Mar 27, 2025

View reviewed changes

revert header changes

35cb8de

mjforan mentioned this pull request Mar 27, 2025

Fix includes #1353

Merged

ahcorde requested changes Mar 28, 2025

View reviewed changes

remove extra dependency

6c70ede

Merge branch 'rolling' into compressed-image-display

dc6d160

Signed-off-by: Matthew Foran <[email protected]>

Merge remote-tracking branch 'origin/rolling' into compressed-image-d…

c02e295

…isplay

ahcorde approved these changes May 19, 2025

View reviewed changes

mjforan added 2 commits August 22, 2025 14:49

Merge branch 'rolling' into compressed-image-display

3aa1332

Update deprecated function calls

a37c3c4

lreiher mentioned this pull request Aug 26, 2025

Add support for ffmpeg_image_transport to Image/Camera displays and point_cloud_transport/cloudini to PointCloud2 display #1568

Merged

Compressed Image Display #1288

Are you sure you want to change the base?

Compressed Image Display #1288

Uh oh!

Conversation

mjforan commented Oct 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mjforan commented Nov 7, 2024

Uh oh!

ahcorde left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mjforan commented Nov 8, 2024

Uh oh!

mjforan commented Nov 8, 2024

Uh oh!

mjforan commented Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mjforan commented Nov 13, 2024

Uh oh!

ahcorde left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ahcorde Mar 24, 2025

Choose a reason for hiding this comment

Uh oh!

mjforan Mar 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mjforan commented Mar 24, 2025

Uh oh!

mjforan commented Mar 27, 2025

Uh oh!

ahcorde left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ahcorde Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

mjforan Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

mjforan commented Apr 2, 2025

Uh oh!

mjforan commented May 15, 2025

Uh oh!

mjforan commented May 19, 2025

Uh oh!

mjforan commented May 19, 2025

Uh oh!

ahcorde commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mjforan commented May 19, 2025

Uh oh!

ahcorde commented May 19, 2025

Uh oh!

mjforan commented May 19, 2025

Uh oh!

ijnek commented Aug 15, 2025

Uh oh!

mjforan commented Aug 15, 2025

Uh oh!

mjforan commented Aug 24, 2025

Uh oh!

mjforan commented Aug 31, 2025

Uh oh!

Uh oh!

mjforan commented Oct 24, 2024 •

edited

Loading

mjforan commented Nov 12, 2024 •

edited

Loading

ahcorde commented May 19, 2025 •

edited

Loading