-
Notifications
You must be signed in to change notification settings - Fork 7
Use pixi for windows workflow #374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 70 commits
Commits
Show all changes
78 commits
Select commit
Hold shift + click to select a range
24f0c18
Install pixi
christophfroehlich 7def4fd
Split installation
christophfroehlich 1c8e4fd
Use pixi action
christophfroehlich 00739ce
No cache
christophfroehlich c86db85
Install python later
christophfroehlich def6647
Use ROS' pixi.toml
christophfroehlich f613f3f
Cleanup
christophfroehlich e4ad042
Test pixi+ros
christophfroehlich 8e6fbd7
Call -> &
christophfroehlich fe265e9
use pixi and manual colcon steps
christophfroehlich d6d164c
Test pixi
christophfroehlich 81c9969
don't use bash
christophfroehlich fd9174e
Fix pwsh
christophfroehlich 6e81405
Don't switch to C:
christophfroehlich 4e83598
Fix vcs command
christophfroehlich 6ff9b25
Use C:\dev also for target ws
christophfroehlich c14ce32
Use github workflow folder instead
christophfroehlich 2b0061a
Why is the first pixi shell always not working?
christophfroehlich 9bcd56f
Revert "don't use bash"
christophfroehlich c971dc5
Use pixi shell-hook instead
christophfroehlich f12a43e
/ instead of \
christophfroehlich fc77924
Debug pixi shell-hook
christophfroehlich 0076de3
Debug pixi shell-hook
christophfroehlich fe3740f
Use custom shell wrapper
christophfroehlich 636f2b0
Use set-package-list-pixi
christophfroehlich f703fc0
Add set-package-list-pixi
christophfroehlich 775a938
Use consistent forward slash
christophfroehlich 7e579de
Debug action
christophfroehlich 4f1eea3
Debug action
christophfroehlich e2f78fb
Remove debug action
christophfroehlich e7981c8
Install pixi globally
christophfroehlich 0c89478
Use ros distro
christophfroehlich eb1e2f0
Revert "Install pixi globally"
christophfroehlich ab510f4
Also for Install dependencies
christophfroehlich e4a9e6d
Add echo
christophfroehlich 4b9bb29
Cache pixi
christophfroehlich 39fd176
Use pixi run in current shell
christophfroehlich 4aebdc2
Checkout first
christophfroehlich ad9a837
Don't cache
christophfroehlich 001f202
Try windows-2022
christophfroehlich f4bebbc
Revert "Use pixi run in current shell"
christophfroehlich 80ebdc6
Properly concatenate colcon command
christophfroehlich a2e587a
Use bash instead of pwsh
christophfroehlich 76ad1f0
Fix slash
christophfroehlich 2a3396d
Fix path of ros rolling
christophfroehlich bdc4d4a
Revert "Use bash instead of pwsh"
christophfroehlich 10cf25e
Add debug steps
christophfroehlich 5d05c83
Don't run tests
christophfroehlich bf41cc1
Add additional parameters and use C:\dev for pixi + ROS
christophfroehlich 2bc8936
Fix action with pixi
christophfroehlich 2eceb24
Fix typo
christophfroehlich c66d0c8
Remove ninja again
christophfroehlich f8eaef3
Debug manifest path
christophfroehlich 4421a04
Use \
christophfroehlich 91169f1
hardcode package for now
christophfroehlich c5a81a3
Hardcode pacakge in install deps step
christophfroehlich 5774a0e
Fix install deps step
christophfroehlich b3b0f41
Fix check_local_repos
christophfroehlich af76e72
Fix build step
christophfroehlich c1c3204
get colcon version
christophfroehlich b4d5f98
Fix install dependencies
christophfroehlich 3650a43
Debug output
christophfroehlich 9888004
Debug pixi
christophfroehlich 3fbc970
use shell-hook instead
christophfroehlich 74278e6
use shell-hook instead
christophfroehlich 7a042c1
fix path
christophfroehlich e46a80b
Use action for getting package names
christophfroehlich 6a2eef8
Suppress output
christophfroehlich 63188d4
Fix batch script
christophfroehlich 0cae4b5
Cleanup and add some comments
christophfroehlich af612fd
Fix paranthesis and pre-commit
christophfroehlich e339c35
Add event handler to colcon build
christophfroehlich dda66bb
Try additional input for Ninja packages
christophfroehlich 4c4eaf7
Build up-to
christophfroehlich 67f45d5
Add Ninja itself
christophfroehlich 1340575
Remove trailing ws
christophfroehlich c70800b
Merge branch 'master' into windows/pixi
christophfroehlich 112cbb8
Use master branch of CI repo
christophfroehlich File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| name: "Get package list" | ||
| # pixi has to be installed and available in the PATH | ||
| description: "Get a list of packages in the given path" | ||
| inputs: | ||
| path: | ||
| description: "Path to the repository after checkout action, e.g. src/repo-name" | ||
| required: false | ||
| default: "" | ||
| manifest-path: | ||
| description: "Path to the pixi.toml file or workspace directory" | ||
| required: true | ||
| outputs: | ||
| package_list: | ||
| description: "A white-space separated list of packages" | ||
| value: ${{ steps.colcon.outputs.package_list }} | ||
| package_path_list: | ||
| description: "A white-space separated list of package paths" | ||
| value: ${{ steps.colcon.outputs.package_path_list }} | ||
| repo_name: | ||
| description: "The name of the repo, last part of github.repository" | ||
| value: ${{ steps.split_repo.outputs.repo_name }} | ||
|
|
||
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - id: colcon | ||
| # if a path is given, list the packages in the given path and its subdirectories from the path | ||
| # if no path is given, list all packages in the workspace | ||
| run: | | ||
| call pixi shell-hook -s cmd --manifest-path ${{ inputs.manifest-path }} > pixi_env.bat | ||
| call pixi_env.bat | ||
| setlocal enabledelayedexpansion | ||
| set "package_list=" | ||
| set "package_path_list=" | ||
| if not "${{ inputs.path }}"=="" ( | ||
| for /f "delims=" %%a in ('colcon list --paths "${{ inputs.path }}" --names-only') do ( | ||
| set "package_list=!package_list! %%a" | ||
| ) | ||
| for /f "delims=" %%a in ('colcon list --paths "${{ inputs.path }}\*" --names-only') do ( | ||
| set "package_list=!package_list! %%a" | ||
| ) | ||
| for /f "delims=" %%a in ('colcon list --paths "${{ inputs.path }}" --paths-only') do ( | ||
| set "package_path_list=!package_path_list! %%a" | ||
| ) | ||
| for /f "delims=" %%a in ('colcon list --paths "${{ inputs.path }}\*" --paths-only') do ( | ||
| set "package_path_list=!package_path_list! %%a" | ||
| ) | ||
| ) else ( | ||
| for /f "delims=" %%a in ('colcon list --names-only') do ( | ||
| set "package_list=!package_list! %%a" | ||
| ) | ||
| for /f "delims=" %%a in ('colcon list --paths-only') do ( | ||
| set "package_path_list=!package_path_list! %%a" | ||
| ) | ||
| ) | ||
| echo package_list=!package_list!>> %GITHUB_OUTPUT% | ||
| echo package_path_list=!package_path_list!>> %GITHUB_OUTPUT% | ||
| endlocal | ||
| shell: cmd | ||
| - id: split_repo | ||
| run: | | ||
| echo "repo_name=$(echo ${{ github.repository }} | cut -d '/' -f 2)" >> $GITHUB_OUTPUT | ||
| shell: bash | ||
| - run: | | ||
| echo "repo ${{ steps.split_repo.outputs.repo_name }} contains the packages: ${{ steps.colcon.outputs.package_list }}" | ||
| shell: bash |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,72 +5,50 @@ on: | |
| workflow_call: | ||
| inputs: | ||
| ros_distro: | ||
| description: 'ROS 2 distribution name. One of | ||
| https://github.com/ros-tooling/setup-ros/blob/master/src/setup-ros-windows.ts' | ||
| description: "ROS 2 distribution name, e.g. rolling" | ||
| required: true | ||
| type: string | ||
| ref_for_scheduled_build: | ||
| description: 'Reference on which the repo should be checkout for scheduled build. Usually is this name of a branch or a tag.' | ||
| default: '' | ||
| description: "Reference on which the repo should be checkout for scheduled build. Usually is this name of a branch or a tag." | ||
| default: "" | ||
| required: false | ||
| type: string | ||
| os_name: | ||
| description: 'On which OS to run the build job' | ||
| description: "On which OS to run the build job" | ||
| required: false | ||
| default: 'windows-2019' | ||
| default: "windows-2022" | ||
| type: string | ||
| container: | ||
| description: '(optional) Docker container to run the job in, e.g. ubuntu:noble' | ||
| description: "Docker container to run the job in, e.g. ubuntu:noble" | ||
| required: false | ||
| default: '' | ||
| default: "" | ||
| type: string | ||
| pixi_dependencies: | ||
| description: "Whitespace separated list of additional pixi dependencies" | ||
| required: false | ||
| default: "" | ||
| type: string | ||
| windows_dependencies: | ||
| description: 'Path to a repos file with additional windows dependencies' | ||
| description: "Path to a repos file with additional windows dependencies" | ||
| required: false | ||
| default: '' | ||
| default: "" | ||
| type: string | ||
| install_boost: | ||
| description: 'Install boost for the build' | ||
| skip_packages: | ||
| description: "Packages to skip from build" | ||
| default: "" | ||
| required: false | ||
| default: false | ||
| type: boolean | ||
| type: string | ||
|
|
||
| jobs: | ||
| reusable_ros_tooling_source_build: | ||
| name: ${{ inputs.ros_distro }} ${{ inputs.os_name }} | ||
| runs-on: ${{ inputs.os_name }} | ||
| container: ${{ inputs.container }} | ||
| env: | ||
| pixi_path: C:\dev | ||
| # this will be src/{repo-owner}/{repo-name} | ||
| repo_path: src/${{ github.repository }} | ||
| steps: | ||
| - uses: actions/setup-python@v5 | ||
| # setup-ros has hardcoded python 3.8, as it is the default version acc. to REP-2000 for jazzy | ||
| # let's use the same version here for the later action-ros-ci step | ||
| with: | ||
| python-version: '3.8' | ||
|
|
||
| - uses: ros-tooling/[email protected] | ||
| with: | ||
| required-ros-distributions: ${{ inputs.ros_distro }} | ||
| use-ros2-testing: true | ||
|
|
||
| - name: Install boost | ||
| uses: MarkusJx/[email protected] | ||
| if: ${{ inputs.install_boost }} | ||
| id: install-boost | ||
| with: | ||
| # REQUIRED: Specify the required boost version | ||
| # A list of supported versions can be found here: | ||
| # https://github.com/MarkusJx/prebuilt-boost/blob/main/versions-manifest.json | ||
| boost_version: 1.86.0 | ||
| # OPTIONAL: Specify a platform version | ||
| platform_version: 2019 | ||
| # OPTIONAL: Specify a toolset | ||
| toolset: msvc | ||
| # Whether the boost libraries will be supplied through static or shared libraries | ||
| link: shared | ||
|
|
||
| - name: Checkout default ref when build is not scheduled | ||
| if: ${{ github.event_name != 'schedule' }} | ||
| uses: actions/checkout@v4 | ||
|
|
@@ -83,65 +61,74 @@ jobs: | |
| ref: ${{ inputs.ref_for_scheduled_build }} | ||
| path: ${{ env.repo_path }} | ||
|
|
||
| - id: package_list_action | ||
| uses: ros-controls/ros2_control_ci/.github/actions/set-package-list@master | ||
| - name: Bootstrap pixi | ||
| # https://docs.ros.org/en/rolling/Installation/Windows-Install-Binary.html | ||
| run: | | ||
| mkdir -p ${{ env.pixi_path }} | ||
| Set-Location -Path ${{ env.pixi_path }} | ||
| irm https://raw.githubusercontent.com/ros2/ros2/refs/heads/rolling/pixi.toml -OutFile pixi.toml | ||
|
|
||
| - name: Install pixi | ||
| uses: prefix-dev/[email protected] | ||
| with: | ||
| path: ${{ env.repo_path }} | ||
| manifest-path: ${{ env.pixi_path }}\pixi.toml | ||
|
|
||
| - name: Check for local repos file | ||
| id: check_local_repos | ||
| - name: Install additional pixi dependencies | ||
| working-directory: ${{ env.pixi_path }} | ||
| run: | | ||
| if (Test-Path ${{ env.repo_path }}\${{ steps.package_list_action.outputs.repo_name }}.${{ inputs.ros_distro }}.repos) { | ||
| Write-Output "Local repos file found" | ||
| "repo_file=${{ env.repo_path }}\${{ steps.package_list_action.outputs.repo_name }}.${{ inputs.ros_distro }}.repos" | Out-File -FilePath $Env:GITHUB_OUTPUT -Append | ||
| } else { | ||
| Write-Output "No local repos file found" | ||
| "repo_file=" | Out-File -FilePath $Env:GITHUB_OUTPUT -Append | ||
| } | ||
| pixi add ${{ inputs.pixi_dependencies }} | ||
|
|
||
| - name: Set windows dependencies variable | ||
| id: set_windows_dependencies | ||
| - name: Install ROS | ||
| working-directory: ${{ env.pixi_path }} | ||
| # Download and extract ROS 2 package | ||
| # https://docs.ros.org/en/rolling/Installation/Windows-Install-Binary.html | ||
| run: | | ||
| if (![string]::IsNullOrWhiteSpace("${{ inputs.windows_dependencies }}") -and (Test-Path "${{ env.repo_path }}\${{ inputs.windows_dependencies }}")) { | ||
| Write-Output "Windows repos file found" | ||
| "repo_file=${{ env.repo_path }}\${{ inputs.windows_dependencies }}" | Out-File -FilePath $Env:GITHUB_OUTPUT -Append | ||
| } else { | ||
| Write-Output "No windows dependencies provided or file not found" | ||
| "repo_file=" | Out-File -FilePath $Env:GITHUB_OUTPUT -Append | ||
| } | ||
| $url = "https://ci.ros2.org/view/packaging/job/packaging_windows/lastSuccessfulBuild/artifact/ws/ros2-package-windows-AMD64.zip" | ||
| $output = "ros2-package-windows-AMD64.zip" | ||
| Invoke-WebRequest -Uri $url -OutFile $output | ||
| Expand-Archive -Path $output -DestinationPath ros2_${{ inputs.ros_distro }} | ||
|
|
||
| - uses: ros-tooling/[email protected] | ||
| # tests are disabled https://github.com/ros-tooling/action-ros-ci/pull/712#issuecomment-969495087 | ||
| env: | ||
| BOOST_ROOT: ${{ steps.install-boost.outputs.BOOST_ROOT }} | ||
| - name: Get package list | ||
| id: package_list_action | ||
| uses: ros-controls/ros2_control_ci/.github/actions/set-package-list-pixi@windows/pixi | ||
| with: | ||
| target-ros2-distro: ${{ inputs.ros_distro }} | ||
| package-name: ${{ steps.package_list_action.outputs.package_list }} | ||
| vcs-repo-file-url: | | ||
| ${{ steps.check_local_repos.outputs.repo_file }} | ||
| ${{ steps.set_windows_dependencies.outputs.repo_file }} | ||
| path: ${{ env.repo_path }} | ||
| manifest-path: ${{ env.pixi_path }} | ||
|
|
||
| colcon-mixin-repository: https://raw.githubusercontent.com/colcon/colcon-mixin-repository/master/index.yaml | ||
| colcon-defaults: | | ||
| { | ||
| "test": { | ||
| "executor": "sequential" | ||
| } | ||
| - name: Install dependencies | ||
| # check for repos files, and pass them to vcstool | ||
| run: | | ||
| Invoke-Expression ((& pixi shell-hook -s powershell --manifest-path ${{ env.pixi_path }}) -join "`n") | ||
| $repo_file = "${{ env.repo_path }}\${{ steps.package_list_action.outputs.repo_name }}.${{ inputs.ros_distro }}.repos" | ||
| if (Test-Path "$repo_file") { | ||
| Write-Output "Local repos file found" | ||
| vcs import --input $repo_file src | ||
| } | ||
| if (![string]::IsNullOrWhiteSpace("${{ inputs.windows_dependencies }}") { | ||
| $repo_file_win = "${{ env.repo_path }}\${{ inputs.windows_dependencies }}" | ||
| if (Test-Path "$repo_file_win") { | ||
| Write-Output "Windows repos file found" | ||
| vcs import --input $repo_file_win src | ||
| } | ||
| id: action-ros | ||
| } | ||
|
|
||
| - name: Download issue template for target failure # Has to be a local file | ||
| if: ${{ always() && steps.action-ros.outcome == 'failure' && github.event_name == 'schedule' }} | ||
| run: | ||
| wget https://raw.githubusercontent.com/ros-controls/ros2_control_ci/master/.github/issue_template_failed_ci.md -O ${{ env.repo_path }}/.github/issue_template_failed_ci.md | ||
| - uses: JasonEtco/create-an-issue@v2 | ||
| # action-ros-ci does not report more details on test failures afaik | ||
| if: ${{ always() && steps.action-ros.outcome == 'failure' && github.event_name == 'schedule'}} | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| ACTION_NAME: ${{ inputs.ros_distro }}/source | ||
| REF: ${{ inputs.ref_for_scheduled_build }} | ||
| URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} | ||
| with: | ||
| update_existing: true | ||
| filename: ${{ env.repo_path }}/.github/issue_template_failed_ci.md | ||
| - name: Build workspace | ||
| # TODO: use Ninja generator for RSL, but doesn't work for msg packages | ||
| # https://github.com/search?q=repo%3APickNikRobotics%2FRSL%20ninja&type=code | ||
| # https://github.com/colcon/colcon-ros/issues/84#issuecomment-1862881299 | ||
| shell: cmd | ||
| run: | | ||
| call pixi shell-hook -s cmd --manifest-path ${{ env.pixi_path }} > pixi_env.bat | ||
| call pixi_env.bat >nul 2>&1 | ||
|
|
||
| call ${{ env.pixi_path }}\ros2_${{ inputs.ros_distro }}\ros2-windows\setup.bat | ||
|
|
||
| set up_to_arg= | ||
| if not "${{ steps.package_list_action.outputs.package_list }}"=="" ( | ||
| set up_to_arg=--packages-up-to ${{ steps.package_list_action.outputs.package_list }} | ||
| ) | ||
| set skip_arg= | ||
| if not "${{ inputs.skip_packages }}"=="" ( | ||
| set skip_arg=--packages-skip ${{ inputs.skip_packages }} | ||
| ) | ||
| colcon build %up_to_arg% %skip_arg% | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.