ROS2 Fastdds Upgrade - Compatibility between Humble and Jazzy

[uff1199]

@DerBaertige and me observed the following and want to propose a solution:

Is your feature request related to a problem? Please describe.

Incompatibility between ROS2 Humble and Jazzy versions due to fastdds

Currently LibCarla is linked against fastdds v2.11.2, ROS Humble uses v2.6.1 and both versions are compatible. ROS Jazzy on the other hand uses v.2.14.5 and I noticed heavy blocking behavior (up to seconds) in fastdds recvfrom calls when Nodes want to subscribe to topics published within CARLA.

See image below.

Additionally, the whole simulation stalls because also the publish function calls executed in the game thread are blocking as well.

We also think that upgrading the fastdds version is necessary, as Humble will reach next year EOL.

Describe the solution you'd like

We upgraded the fastdds version to v.2.14.5 and communication with ROS2 Jazzy was not stalling anymore. Furthermore backward compatibility with ROS2 Humble was maintained. We also observed, that currently no toolchain and / or documentation exists to upgrade the cpp + header files with fastddsgen-Tool necessary for the fastdds publishers, based on existing ROS2 message definitions (.msg). As there are significant changes between cpp + header files for fastdds v.2.11 and v.2.14.

Also observed in a different context here: #8381 (comment)

Simply selecting a newer fastdds during the Setup step will not solve as the cpp + header files are outdated. Because the selected versions depend on it but they are not generated during build process.

If wished we would like to contribute our findings in upgrading fastdds to the CARLA project. It will not interfere with existing works on reworking the ROS2 handling see #9451.

Additional context

[berndgassmann]

Hi,

I can confirm that the v2.11 makes some issues -- especially with Jazzy/Iron.
I believe, most of the files you would need to update the ue4-dev branch (without the #9451) to v2.14 you might find here in that commit where I was switching to v2.14.4 on my branch a year ago (berndgassmann@369bd06)

That time I also updated the README on how to generate the files if required (at least the way I did it that time).
berndgassmann@7223c06

In #9451 I had left out the change of the FastDDS version, but if I can update that PR also easily; at least the v2.14.5 is straight forward.

A few weeks ago I even tried to update to FastDDS v3.2 being able to even run both variants besides each other depending on the configuration (or Ubuntu Version). But there, I was gotten stuck with some not working topics when anonymous arrays come into play (Motor-Ai@a25146b).
Since some of the idl-file updates required are a bit cumbersome, I added also a clean_idl_files.bash in that commit. Still some manual adaption have to be done where name clashes are happening.

That's the good and the bad of not using ROS2 middleware directly. Tricky to get it running (even CARLA is not visible as node in the ROS2 node graph), but then works for multiple ROS2 versions.