Support for Different Cameras and Random Answer Generation

[doruksonmez]

Hi @dusty-nv and @khannah-s, thank you for this great integration. I just wanted to test this repo on a Jetson AGX Orin 64GB with a stereo camera and found out that it doesn't support different camera types/profiles out of the box because of the cam2image package. So I made the following addition inside the launch/camera_input_example.launch.py file:

    # cam2image defaults to /dev/video0 for the camera input and 320x240(WxH) for the camera profile.
    # See v4l2-ctl --list-formats-ext command output for a full list of supported camera profiles for your camera device.
    cam2image_node = Node(
            package='image_tools',
            executable='cam2image',
            parameters=[{
                'device_id': 4, # default is 0
                'width': 640, # default is 320
                'height': 480, # default is 240
                'frequency': 30.0 # default is 30.0
            }],
            remappings=[('image', 'input_image')],
    )

However, now that I can run the application and see the generated results by echoing the /output topic, I see it just generates random outputs for random questions.

Here is the main terminal output (Publishing image #... logs from the cam2image is very annoying and couldn't find a way to suppress them):

And this is the output of ros2 topic echo /output:

I also see Rviz output implementation which would display the stream from VideoOutput class doesn't exist inside the scripts, so I wanted to give OpenCV's putText() function a shot and publish it with self.cv_br.cv2_to_imgmsg(out_frame, 'rgb8'). Even though I can get the stream on Rviz screen, it just delivers ~1-2 fps. So, is there an easy way around to get the VideoOutput stream and publish it directly?

Thanks in advance!

[kyle-redyeti]

I am having the exact same problem running on my Orin-NX ! Did you ever find a solution to this issue? Did you track down how to remove the random question/answer output? In your testing did it ever give you a response to the 'Describe the image.'?