First version of unified Ros2ForUnity - building into Unity package. Needs Windows support re-added

Signed-off-by: Adam Dabrowski <[email protected]>

Author: adamdbrw

.gitignore

@@ -0,0 +1,7 @@
+install
+log
+build
+.idea
+src/ros2cs
+src/Ros2ForUnity/Plugins
+!src/Ros2ForUnity/Plugins/.gitkeep

README.md

@@ -0,0 +1,166 @@
+Ros2 For Unity
+=============
+
+ROS2 For Unity is a high-performance communication solution to connect Unity3D and ros2 ecosystem in a ros2 "native" way. Communication is not bridged as in several other solutions, but instead uses ros2 (rcl layer and below) middleware stack, which means your can have ros2 nodes in your simulation.
+Advantages of this module include:
+- High performance, as in supporting higher throughputs and having considerably lower latencies than bridging solutions.
+- Your simulation entities are real ros2 nodes / publishers / subscribers and will behave correctly with e.g. command line tooling such as `ros2 topic`. They will respect QoS settings and can use ros2 native time.
+- The module supplies abstractions and tools to use in your Unity project, including transformations, sensor interface, a clock, spinning loop wrapped in a Monobehavior, handling initialization and shutdown.
+- Suppots all standard ros2 messages
+- Custom messages are generated automatically with build, using standard ros2 way. It is straightforward to generate and use them without having to define `.cs` equivalents by hand.
+- The module is wrapped as Unity asset. 
+
+### Platforms
+
+Supported OSes: 
+- Ubuntu 20.04  (bash)
+- Windows 10 (powershell)
+
+Supported ROS2 distributions:
+- Foxy
+- Galactic
+
+The asset can be prepared in two flavors:
+- standalone (no ROS2 installation required on target machine, e.g. your Unity3D simulation server). All required dependencies are installed and can be used e.g. as a complete set of Unity3D plugins.
+- overlay (assuming existing (supported) ROS2 installation on target machine). Only asset libraries and generated messages are installed.
+
+#### Platform considerations
+
+Running the Editor or App using this Asset on Ubuntu requires a start script that populates the LD_LIBRARY_PATH. This is due to dynamic plugin loading mechanism used in the ros2cs library.
+On Windows, no script is necessary and both Editor and App can be ran the usual way (e.g. with a click or cmd line).
+
+### Releases
+
+You can download pre-built releases of the Asset that support both platforms and specific ros2 and Unity3D versions. (TODO - add links).
+
+### Prerequisites
+
+#### Ubuntu
+
+*  ROS2 installed on the system, along with `test-msgs` and `fastrtps` packages
+*  vcstool package - [see here](https://github.com/dirk-thomas/vcstool)
+*  .NET core 3.1 sdk - [see here](https://www.microsoft.com/net/learn/get-started)
+
+The following script can be used to install the aforementioned prerequisites on Ubuntu 20.04:
+
+```bash
+# Install tests-msgs for your ROS2 distribution
+apt install -y ros-${ROS_DISTRO}-test-msgs ros-${ROS_DISTRO}-fastrtps ros-${ROS_DISTRO}-rmw-fastrtps-cpp
+
+# Install vcstool package
+curl -s https://packagecloud.io/install/repositories/dirk-thomas/vcstool/script.deb.sh | sudo bash
+sudo apt-get update
+sudo apt-get install -y python3-vcstool
+
+# Install .NET core
+wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
+sudo dpkg -i packages-microsoft-prod.deb
+rm packages-microsoft-prod.deb
+sudo apt-get update; \
+  sudo apt-get install -y apt-transport-https && \
+  sudo apt-get update && \
+  sudo apt-get install -y dotnet-sdk-3.1
+```
+
+#### Windows
+
+*  ROS2 installed on the system
+*  vcstool package - [see here](https://github.com/dirk-thomas/vcstool)
+*  .NET 5.0 sdk - [see here](https://dotnet.microsoft.com/download/dotnet/5.0)
+*  xUnit testing framework (for tests only) - [see here](https://xunit.net/)
+
+
+### Note on project composition
+
+The project will pull `ros2cs` into the workspace, which also functions independently as it is a more general project aimed at any `C# / .Net` environment. It has its own README and scripting, but for building the Unity Asset, please use instructions and scripting in this document instead, unless you also wish to run tests or examples for `ros2cs`.
+
+### Building
+
+#### Windows considerations
+
+> For **Windows**, [path length is limited to 260 characters](https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation). Clone your repo to `C:\dev` or a similar shallow path to avoid this issue during build.
+
+> For **Windows**, a Visual Studio preconfigured powershell terminal must be used. Standard powershell prompt might not be configured properly to be used with MSVC compiler and Windows SDKs.  You should have Visual Studio already installed (ROS2 dependency) and you can find shortcut for `Developer PowerShell for VS` here: `C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2019\Visual Studio Tools`. 
+
+> A powershell terminal with administrator privileges is required for **Windows** and **ros2 galactic**. This is because python packages installation requires a privilage for creating symlinks. More about this issue: [github issue](https://github.com/ament/ament_cmake/issues/350).
+
+> There is a bug with hardcoded include exports in some **ros2 galactic** packages on **Windows**. Easiest workaround is to create a `C:\ci\ws\install\include` directory in your system. More about this bug and proposed workarounds: [github issue](https://github.com/ros2/rclcpp/issues/1688#issuecomment-858467147).
+
+#### Build instructions
+
+*  Clone this project.
+*  If you wish to include custom messages in your build, make sure to put them into `ros2_for_unity_custom_messages.repos` file. You can change this file in your fork or change `custom_messages.repos` in the ros2cs repository fork, it will work either way as the scripts will pull both sources. 
+   As an alternative, you can also add your custom messages package directly by copying it to `src/ros2cs/custom_messages` folder after the next step. Any message package in the build tree will be subjected to `.cs` file generator during the build.
+*  You need to source your ROS2 installation (e.g. `source /opt/ros/foxy/setup.bash` on Ubuntu or `C:\dev\ros2_foxy\local_setup.ps1` on Windows) before you proceed, for each new open terminal. On Ubuntu, you can also include this command in your `~/.bashrc` file.
+*  Run 'pull_repositories.sh'. This will pull `ros2cs` as well as your custom messages.
+*  Run `build.sh` (Ubuntu) or `build.ps1` (Windows) script.
+   * You can build tests by adding `--with-tests` argument to `build` command.
+   * It invokes `colcon_build` with `--merge-install` argument to simplify libraries installation.
+   * It deploys built plugins into the Asset directory. Note that only plugins built for the current platform will be deployed (there is no cross-compilation).
+   * It prepares Unity Asset that is ready to import into your Unity project.
+
+#### Standalone version
+
+By default, build process generates standalone libraries.
+You can disable this feature by setting CMake option `STANDALONE_BUILD` to `OFF` (e.g. through editing `build.sh`).
+
+## Running with Unity
+
+## Standalone mode
+
+If the Asset is built with `STANDALONE_BUILD` option set to `1` (the default), then nothing extra needs to be done. Otherwise, you have to source your ros distribution before launching either Unity3D Editor or Application.
+
+## Running the Editor
+
+Open your Unity3D project and import Ros2ForUnity Asset which is built by this project. 
+
+#### Additional considerations for Linux
+
+TODO - we are looking to replace this solution with something simpler. This section needs changes!
+
+After including the Asset, close the Editor. You need to use `start.py` from `linux-utils` directory. It is necessary to run your Unity3D Editor or built App.
+This can be done directly or through wrapping your Unity3D project into a ros2 python package (using files from `linux-utils`) and running `colcon build`, sourcing your workspace and calling `ros2 run ros2-for-unity start editor`.
+
+> This is due to the necessity for Unity3D to find all required assemblies. `dlopen` is used under the hood to dynamically
+   load necessary libraries (including custom messages and rmw implementation).
+   Note that unlike Windows `LoadLibrary`, `dlopen` uses `LD_LIBRARY_PATH` value from the start of the process execution and later modifications do not
+   affect the search paths. Thus, it is necessary to change `LD_LIBRARY_PATH` before the process starts.
+
+> Note that as an alternative, you can set the path manually in your system for the Editor, but consider that built App will have a different absolute path to plugins and can be moved and copied around, so this approach is not recommended.
+
+### Building Unity3D application
+
+When there are no errors in the Editor, you can proceed with an application build.
+You can do this standard way through `Build->Build Settings...`.
+
+## Running application 
+
+### Windows
+
+You can run your application normally by clicking it or executing from command line.
+
+### Linux
+
+On Linux, follow the [Editor instructions](#running-the-editor), but use the start script (directly or through a package wrapper: `ros2 run ros2-for-unity start app --name:=<Your app name>`) to run your application. 
+This is necessary to ensure plugin visibility by setting `LD_LIBRARY_PATH`. 
+
+## Full example (Windows)
+
+Example for setting up `ros2cs` standalone with `Unity` editor on Windows (powershell with git). Let's assume working directory is `C:\dev` and we are using `ROS2 foxy`:
+
+1. Install ros2 distribution, either Foxy or Galactic. We assume standard directory, e.g. `C:\dev\ros2_foxy`.
+   You can find instructions [here (ros2 binary)](https://docs.ros.org/en/foxy/Installation/Windows-Install-Binary.html) and [here (development tools)](https://docs.ros.org/en/foxy/Installation/Windows-Development-Setup.html). 
+2. Make sure you are running `Developer PowerShell for Visual Studio` (see **Building** section in `ros2cs` README.md).
+3. Source ROS2
+```
+C:\dev\ros2_foxy\local_setup.ps1
+```
+4. Follow Build Instructions
+5. Launch your Unity3D Project. The following command assumes that you have Unity3D installed in `C:\Program Files\Unity\Hub\Editor\2021.1.7f1\Editor\Unity.exe`:
+```
+& "C:\Program Files\Unity\Hub\Editor\2021.1.7f1\Editor\Unity.exe" -projectPath C:\dev\<UnityProjectPath>
+```
+6. Import `Ros2ForUnity` Asset into your project.
+10. You should be able to use `Ros2 For Unity` with the module now. You can test if everything works through `Ros2 For Unity` test publisher.
+> Note that after you build the Asset, you can use it on a machine that has no ros2 installation.
+> You can simply copy over the `Ros2ForUnity` subdirectory to update your Asset. 

build.sh

@@ -0,0 +1,24 @@
+#!/bin/bash
+
+if [ -z "${ROS_DISTRO}" ]; then
+    echo "Source your ros2 distro first (Foxy and Galactic are supported)"
+    exit 1
+fi
+
+TESTS=0
+MSG="Build started."
+if [ "$1" = "--with-tests" ]; then
+    TESTS=1
+    MSG="$MSG (with tests)"
+elif [ "$1" = "-h" ] || [ "$1" = "--help" ]; then
+    echo "Usage: "
+    echo "build.sh [--with-tests]"
+    echo ""
+    echo "Options:"
+    echo "--with-tests - build with tests."
+    exit 1
+fi
+
+echo $MSG
+#TODO - call ros2cs ./build.sh instead, but with this workspace directory (parametrize the script)
+colcon build --merge-install --event-handlers console_direct+ --cmake-args -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=$TESTS && ./deploy_unity_plugins.sh src/Ros2ForUnity/Plugins/

create_unity_asset.ps1

@@ -0,0 +1,59 @@
+
+<#
+.SYNOPSIS
+    Creates a 'unitypackage' from an input asset.
+.DESCRIPTION
+    This script screates a temporary Unity project in "%USERPROFILE%\AppData\Local\Temp" directory, copy input asset and makes an unity package out of it.
+.PARAMETER unity_path
+    Unity editor executable path
+.PARAMETER input_asset
+    input asset to pack into unity package
+.PARAMETER package_name
+    Unity package name
+.PARAMETER output_dir
+    output file directory
+#>
+Param (
+    [Parameter(Mandatory=$true)][string]$unity_path,
+    [Parameter(Mandatory=$true)][string]$input_asset,
+    [Parameter(Mandatory=$true)][string]$package_name,
+    [Parameter(Mandatory=$true)][string]$output_dir
+)
+
+$scriptPath = split-path -parent $MyInvocation.MyCommand.Definition
+$temp_dir = $Env:TEMP
+
+& "$unity_path" -version | Tee-Object -Variable unity_version | Out-Null
+
+if ($unity_version -match '^[0-9]{4}\.[0-9]*\.[0-9]*[f]?[0-9]*$') {
+    Write-Host "Unity editor confirmed..."
+} else {
+    Write-Host "Can't confirm Unity editor. Exiting."
+    exit 1
+}
+
+$tmp_project_path = Join-Path -Path "$temp_dir" -ChildPath "\ros2cs_unity_project\$unity_version"
+
+# Create temp project
+if(Test-Path -Path "$tmp_project_path") {
+    Write-Host "Found existing temporary project for Unity $unity_version."
+    Remove-Item -Path "$tmp_project_path\Assets\*" -Force -Recurse -ErrorAction Ignore
+} else {
+    Write-Host "Creating Unity temporary project for Unity $unity_version..."
+    & "$unity_path" -createProject "$tmp_project_path" -batchmode -quit | Out-Null
+}
+
+# Copy asset
+Write-Host "Copying asset '$input_asset' to export..."
+Copy-Item -Path "$input_asset" -Destination "$tmp_project_path\Assets\$package_name" -Recurse
+
+# Creating asset
+Write-Host "Saving unitypackage '$output_dir\$package_name.unitypackage'..."
+& "$unity_path" -projectPath "$tmp_project_path" -exportPackage "$package_name" "$output_dir\$package_name.unitypackage" -batchmode -quit | Out-Null
+
+# Cleaning up
+Write-Host "Cleaning up temporary project..."
+Remove-Item -Path "$tmp_project_path\Assets\*" -Force -Recurse -ErrorAction Ignore
+
+Write-Host "Done!" -ForegroundColor Green
+

create_unity_asset.sh

@@ -0,0 +1,101 @@
+#!/bin/bash
+
+SCRIPT=$(readlink -f $0)
+SCRIPTPATH=`dirname $SCRIPT`
+
+display_usage() {
+  echo "This script creates a temporary Unity project in '/tmp' directory, copy input asset and makes an unity package out of it."
+  echo ""
+  echo "Usage:" 
+  echo "create_unity_asset.sh -u <UNITY_PATH> -i <INPUT_ASSET> -p <PACKAGE_NAME> -o <OUTPUT_DIR>"
+  echo ""
+  echo "UNITY_PATH - Unity editor executable path"
+  echo "INPUT_ASSET - input asset to pack into unity package"
+  echo "PACKAGE_NAME - unity package name"
+  echo "OUTPUT_DIR - output file directory"
+}
+
+UNITY_PATH=""
+INPUT_ASSET="src/Ros2ForUnity"
+PACKAGE_NAME="Ros2ForUnity"
+OUTPUT_DIR="$SCRIPTPATH/install/unity_package"
+
+while [[ $# -gt 0 ]]; do
+  key="$1"
+
+  case $key in
+    -u|--unity-path)
+      UNITY_PATH="$2"
+      shift # past argument
+      shift # past value
+      ;;
+    -p|--package_name)
+      PACKAGE_NAME="$2"
+      shift # past argument
+      shift # past value
+      ;;
+    -i|--input-directory)
+      INPUT_ASSET="$2"
+      shift # past argument
+      shift # past value
+      ;;
+    -o|--output-directory)
+      OUTPUT_DIR="$2"
+      shift # past argument
+      shift # past value
+      ;;
+    -h|--help)
+      display_usage
+      exit 0
+      shift # past argument
+      shift # past value
+      ;;
+    *)    # unknown option
+      shift # past argument
+      ;;
+  esac
+done
+
+if [ -z "$UNITY_PATH" ] || [ -z "$PACKAGE_NAME" ] || [ -z "$INPUT_ASSET" ] || [ -z "$OUTPUT_DIR" ]; then
+    echo -e "\nMissing arguments!"
+    echo ""
+    display_usage
+    exit 1
+fi
+
+UNITY_VERSION=`$UNITY_PATH -version`
+
+# Test if unity editor is valid
+if [[ $UNITY_VERSION =~ ^[0-9]{4}\.[0-9]*\.[0-9]*[f]?[0-9]*$ ]]; then
+    echo "Unity editor confirmed..."
+else
+    echo "Can't confirm Unity editor. Exiting."
+    exit 1
+fi
+
+TMP_PROJECT_PATH=/tmp/ros2cs_unity_project/$UNITY_VERSION
+# Create temp project
+if [ -d "$TMP_PROJECT_PATH" ]; then
+    echo "Found existing temporary project for Unity $UNITY_VERSION."
+    rm -rf $TMP_PROJECT_PATH/Assets/*
+else
+  rm -rf $TMP_PROJECT_PATH
+  echo "Creating Unity temporary project for Unity $UNITY_VERSION..."
+  $UNITY_PATH -createProject $TMP_PROJECT_PATH -batchmode -quit
+fi
+
+# Copy asset
+echo "Copying asset to export..."
+cp -r "$INPUT_ASSET" "$TMP_PROJECT_PATH/Assets/$PACKAGE_NAME"
+
+# Creating asset
+echo "Saving unitypackage '$OUTPUT_DIR/$PACKAGE_NAME.unitypackage'..."
+mkdir -p $OUTPUT_DIR
+$UNITY_PATH -projectPath "$TMP_PROJECT_PATH" -exportPackage "Assets/$PACKAGE_NAME" "$OUTPUT_DIR/$PACKAGE_NAME.unitypackage" -batchmode -quit
+
+# Cleaning up
+echo "Cleaning up temporary project..."
+rm -rf $TMP_PROJECT_PATH/Assets/*
+
+echo "Done!"
+

deploy_unity_plugins.ps1

@@ -0,0 +1,32 @@
+$scriptPath = split-path -parent $MyInvocation.MyCommand.Definition
+$pluginDir=$args[0]
+
+function Print-Help {
+"
+Usage: 
+deploy_unity_plugins.ps1 <PLUGINS_DIR>
+
+PLUGINS_DIR - Assets/ROS2/Plugins/ directory of Unity project.
+"
+}
+
+if (([string]::IsNullOrEmpty($pluginDir)) -Or $args[0] -eq "--help" -Or $args[0] -eq "-h")
+{
+    Print-Help
+    exit
+}
+
+if (Test-Path -Path $pluginDir) {
+    Write-Host "Copying plugins to to: '$pluginDir' ..."
+    (Copy-Item -verbose -Path $scriptPath\install\lib\dotnet\* -Destination ${pluginDir} 4>&1).Message
+    Write-Host "Plugins copied to: '$pluginDir'" -ForegroundColor Green
+    if(-not (Test-Path -Path $pluginDir\Windows\x86_64\)) {
+        mkdir ${pluginDir}\Windows\x86_64\
+    }
+    Write-Host "Copying libraries to: '$pluginDir\Windows\x86_64\' ..."
+    (Copy-Item -verbose -Path $scriptPath\install\bin\*.dll -Destination ${pluginDir}\Windows\x86_64\ 4>&1).Message
+    (Copy-Item -verbose -Path $scriptPath\install\standalone\*.dll -Destination ${pluginDir}\Windows\x86_64\ 4>&1).Message
+    Write-Host "Libraries copied to '${pluginDir}\Windows\x86_64\'" -ForegroundColor Green
+} else {
+    Write-Host "Plugins directory: '$pluginDir' doesn't exist. Please create it first manually." -ForegroundColor Red
+}