workflows/release-binaries: Enable builds on Linux/AArch64 #120786
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -18,6 +18,7 @@ on: | |||||
| type: choice | ||||||
| options: | ||||||
| - ubuntu-22.04 | ||||||
| - depot-ubuntu-22.04-arm-small | ||||||
| - windows-2022 | ||||||
| - macos-13 | ||||||
| - macos-14 | ||||||
|
|
@@ -56,10 +57,13 @@ jobs: | |||||
| ref: ${{ steps.vars.outputs.ref }} | ||||||
| upload: ${{ steps.vars.outputs.upload }} | ||||||
| target-cmake-flags: ${{ steps.vars.outputs.target-cmake-flags }} | ||||||
| ccache: ${{ steps.vars.outputs.ccache }} | ||||||
| build-flang: ${{ steps.vars.outputs.build-flang }} | ||||||
| enable-pgo: ${{ steps.vars.outputs.enable-pgo }} | ||||||
| release-binary-basename: ${{ steps.vars.outputs.release-binary-basename }} | ||||||
| release-binary-filename: ${{ steps.vars.outputs.release-binary-filename }} | ||||||
| runs-on: ${{ steps.vars.outputs.runs-on }} | ||||||
| multi-stage: ${{ steps.vars.outputs.multi-stage }} | ||||||
|
|
||||||
| steps: | ||||||
| # It's good practice to use setup-python, but this is also required on macos-14 | ||||||
|
|
@@ -120,8 +124,21 @@ jobs: | |||||
|
|
||||||
| # Detect necessary CMake flags | ||||||
| target="${{ runner.os }}-${{ runner.arch }}" | ||||||
|
|
||||||
| # The hendrikmuhs/ccache-action action does not support installing sccache | ||||||
| # on arm64 Linux. | ||||||
| if [ "$target" = "Linux-ARM64" ]; then | ||||||
| echo ccache=ccache >> $GITHUB_OUTPUT | ||||||
| else | ||||||
| echo ccache=sccache >> $GITHUB_OUTPUT | ||||||
| fi | ||||||
|
|
||||||
| if [ "${{ runner.os }}" = "Linux" ]; then | ||||||
|
There was a problem hiding this comment.
Suggested change
We don't need workflow interpolation here, and it's better to avoid it if we don't need it: https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions#using-an-intermediate-environment-variable In general I'd prefer to remove all workflow interpolation (even in instances where it is safe) to make auditing easier, but that would require plenty of changes across all workflows. There was a problem hiding this comment. Here is a patch to clean up the file: #120860 I'll update this patch with your suggestions too. |
||||||
| echo "enable-pgo=true" >> $GITHUB_OUTPUT | ||||||
| else | ||||||
| echo "enable-pgo=false" >> $GITHUB_OUTPUT | ||||||
| target_cmake_flags="-DLLVM_RELEASE_ENABLE_PGO=OFF" | ||||||
| fi | ||||||
| # The macOS builds try to cross compile some libraries so we need to | ||||||
| # add extra CMake args to disable them. | ||||||
| # See https://github.com/llvm/llvm-project/issues/99767 | ||||||
|
|
@@ -144,12 +161,29 @@ jobs: | |||||
|
|
||||||
| echo "target-cmake-flags=$target_cmake_flags" >> $GITHUB_OUTPUT | ||||||
| echo "build-flang=$build_flang" >> $GITHUB_OUTPUT | ||||||
| case "${{ inputs.runs-on }}" in | ||||||
| ubuntu-22.04) | ||||||
| runs_on="depot-${{ inputs.runs-on }}-16" | ||||||
| multi_stage="false" | ||||||
| ;; | ||||||
| depot-ubuntu-22.04-arm-small) | ||||||
| runs_on="depot-ubuntu-22.04-arm-16" | ||||||
| multi_stage="false" | ||||||
| ;; | ||||||
| *) | ||||||
| runs_on="${{ inputs.runs-on }}" | ||||||
| multi_stage="true" | ||||||
| ;; | ||||||
| esac | ||||||
| echo "runs-on=$runs_on" >> $GITHUB_OUTPUT | ||||||
| echo "multi-stage=$multi_stage" >> $GITHUB_OUTPUT | ||||||
|
|
||||||
| build-stage1: | ||||||
| name: "Build Stage 1" | ||||||
| needs: prepare | ||||||
| if: >- | ||||||
| github.repository == 'llvm/llvm-project' | ||||||
|
There was a problem hiding this comment.
Suggested change
Might make it easier to reuse workflow snippets in other org repos, but this is a very mild preference. There was a problem hiding this comment. I've put this change in a separate PR: #123797 |
||||||
| runs-on: ${{ needs.prepare.outputs.runs-on }} | ||||||
| steps: | ||||||
|
|
||||||
| - name: Checkout Actions | ||||||
|
|
@@ -192,10 +226,10 @@ jobs: | |||||
| with: | ||||||
| # Default to 2G to workaround: https://github.com/hendrikmuhs/ccache-action/issues/174 | ||||||
| max-size: 2G | ||||||
| key: ${{ needs.prepare.outputs.ccache }}-${{ runner.os }}-${{ runner.arch }}-release | ||||||
| variant: ${{ needs.prepare.outputs.ccache }} | ||||||
|
|
||||||
| - name: Configure Stage 1 Clang | ||||||
| id: build | ||||||
| shell: bash | ||||||
| run: | | ||||||
|
|
@@ -206,12 +240,37 @@ jobs: | |||||
| -C clang/cmake/caches/Release.cmake \ | ||||||
| -DBOOTSTRAP_LLVM_PARALLEL_LINK_JOBS=1 \ | ||||||
| -DBOOTSTRAP_CPACK_PACKAGE_FILE_NAME="${{ needs.prepare.outputs.release-binary-basename }}" \ | ||||||
| -DCMAKE_C_COMPILER_LAUNCHER=${{ needs.prepare.outputs.ccache }} \ | ||||||
| -DCMAKE_CXX_COMPILER_LAUNCHER=${{ needs.prepare.outputs.ccache }} | ||||||
| - name: Build Stage 1 Clang | ||||||
| shell: bash | ||||||
| run: | | ||||||
| if [ "${{ needs.prepare.outputs.multi-stage}}" = "false" ]; then | ||||||
| ninja -v -C ${{ steps.setup-stage.outputs.build-prefix }}/build stage2-package | ||||||
| mv ${{ steps.setup-stage.outputs.build-prefix }}/build/tools/clang/stage2-bins/${{ needs.prepare.outputs.release-binary-filename }} . | ||||||
| else | ||||||
| ninja -v -C ${{ steps.setup-stage.outputs.build-prefix }}/build | ||||||
| # There is a race condition on the MacOS builders and this command is here | ||||||
| # to help debug that when it happens. | ||||||
| ls -ltr ${{ steps.setup-stage.outputs.build-prefix }}/build | ||||||
| fi | ||||||
|
|
||||||
| - uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0 | ||||||
| if: needs.prepare.outputs.multi-stage == 'false' | ||||||
| with: | ||||||
| name: ${{ runner.os }}-${{ runner.arch }}-release-binary | ||||||
| # Due to path differences on Windows when running in bash vs running on node, | ||||||
| # we need to search for files in the current workspace. | ||||||
| path: | | ||||||
| ${{ needs.prepare.outputs.release-binary-filename }} | ||||||
|
|
||||||
| # Clean up some build files to reduce size of artifact. | ||||||
| - name: Clean Up Build Directory | ||||||
| if: needs.prepare.outputs.multi-stage == 'false' | ||||||
| shell: bash | ||||||
| run: | | ||||||
| find ${{ steps.setup-stage.outputs.build-prefix }}/build -iname ${{ needs.prepare.outputs.release-binary-filename }} -delete | ||||||
| rm -Rf ${{ steps.setup-stage.outputs.build-prefix }}/build/tools/clang/stage2-bins/_CPack_Packages | ||||||
|
|
||||||
| - name: Save Stage | ||||||
| uses: ./workflows-main/.github/workflows/release-binaries-save-stage | ||||||
|
|
@@ -223,8 +282,10 @@ jobs: | |||||
| needs: | ||||||
| - prepare | ||||||
| - build-stage1 | ||||||
| if: >- | ||||||
| github.repository == 'llvm/llvm-project' && | ||||||
| needs.prepare.outputs.multi-stage == 'true' | ||||||
| runs-on: ${{ needs.prepare.outputs.runs-on }} | ||||||
| steps: | ||||||
| - name: Checkout Actions | ||||||
| uses: actions/checkout@v4 | ||||||
|
|
@@ -242,7 +303,9 @@ jobs: | |||||
|
|
||||||
| - name: Build Stage 2 | ||||||
| # Re-enable once PGO builds are supported. | ||||||
| if: >- | ||||||
| needs.prepare.outputs.enable-pgo == 'true' && | ||||||
| needs.prepare.outputs.multi-stage == 'true' | ||||||
| shell: bash | ||||||
| run: | | ||||||
| ninja -C ${{ steps.setup-stage.outputs.build-prefix}}/build stage2-instrumented | ||||||
|
|
@@ -257,8 +320,10 @@ jobs: | |||||
| needs: | ||||||
| - prepare | ||||||
| - build-stage2 | ||||||
| if: >- | ||||||
| github.repository == 'llvm/llvm-project' && | ||||||
| needs.prepare.outputs.multi-stage == 'true' | ||||||
| runs-on: ${{ needs.prepare.outputs.runs-on }} | ||||||
| steps: | ||||||
| - name: Checkout Actions | ||||||
| uses: actions/checkout@v4 | ||||||
|
|
@@ -307,7 +372,9 @@ jobs: | |||||
| needs: | ||||||
| - prepare | ||||||
| - build-stage3-clang | ||||||
| if: >- | ||||||
| needs.prepare.outputs.multi-stage == 'true' | ||||||
| runs-on: ${{ needs.prepare.outputs.runs-on }} | ||||||
| steps: | ||||||
| - name: Checkout Actions | ||||||
| uses: actions/checkout@v4 | ||||||
|
|
@@ -357,7 +424,7 @@ jobs: | |||||
| needs: | ||||||
| - prepare | ||||||
| - build-stage3-flang | ||||||
| runs-on: ${{ needs.prepare.outputs.runs-on }} | ||||||
| steps: | ||||||
| - name: Checkout Actions | ||||||
| uses: actions/checkout@v4 | ||||||
|
|
@@ -409,6 +476,7 @@ jobs: | |||||
| needs: | ||||||
| - prepare | ||||||
| - build-stage3-all | ||||||
| - build-stage1 | ||||||
| if: >- | ||||||
| always() && | ||||||
| github.event_name != 'pull_request' && | ||||||
|
|
@@ -469,6 +537,7 @@ jobs: | |||||
| - prepare | ||||||
| - build-stage3-all | ||||||
| if: >- | ||||||
| always() && | ||||||
| github.repository == 'llvm/llvm-project' | ||||||
| runs-on: ${{ inputs.runs-on }} | ||||||
| steps: | ||||||
|
|
@@ -484,7 +553,17 @@ jobs: | |||||
| id: setup-stage | ||||||
| uses: ./workflows/.github/workflows/release-binaries-setup-stage | ||||||
| with: | ||||||
| previous-artifact: ${{ (needs.prepare.outputs.multi-stage == 'false' && 'build-stage1') || 'build-stage3-all' }} | ||||||
|
|
||||||
| # Need sccache installed, because some stage1 objects are being built for the tests. | ||||||
| # FIXME: This probably shouldn't be happening. | ||||||
| - name: Setup sccache | ||||||
| uses: hendrikmuhs/ccache-action@ca3acd2731eef11f1572ccb126356c2f9298d35e # v1.2.9 | ||||||
| with: | ||||||
| # Default to 2G to workaround: https://github.com/hendrikmuhs/ccache-action/issues/174 | ||||||
| max-size: 2G | ||||||
| key: ${{ needs.prepare.outputs.ccache }}-${{ runner.os }}-${{ runner.arch }}-release | ||||||
| variant: ${{ needs.prepare.outputs.ccache }} | ||||||
|
|
||||||
| - name: Run Tests | ||||||
| shell: bash | ||||||
|
|
||||||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like this should work still, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is working now, so I dropped this part of the change.