Skip to content

Rework/simplify GH Actions jobs #4364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 25, 2024
Merged

Rework/simplify GH Actions jobs #4364

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ecfccd0
Don't upload test reports as artifacts
yrodiere Oct 24, 2024
152a922
Rework/simplify GH Actions jobs
yrodiere Oct 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .github/workflows/ci-report.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: GH Actions CI reporting

on:
workflow_run:
workflows: [ "GH Actions CI" ]
types: [ completed ]

defaults:
run:
shell: bash

env:
MAVEN_ARGS: "-e -B --settings .github/mvn-settings.xml --fail-at-end"

jobs:
publish-build-scans:
name: Publish Develocity build scans
if: github.repository == 'hibernate/hibernate-search' && github.event.workflow_run.conclusion != 'cancelled'
runs-on: ubuntu-latest
steps:
# Checkout target branch which has trusted code
- name: Check out target branch
uses: actions/checkout@v4
with:
persist-credentials: false
ref: ${{ github.ref }}
- name: Set up Java ${{ matrix.os.java.version }}
uses: actions/setup-java@v4
with:
java-version: ${{ matrix.os.java.version }}
distribution: temurin
# https://github.com/actions/cache/blob/main/examples.md#java---maven
- name: Cache local Maven repository
uses: actions/cache@v4
with:
path: ~/.m2/repository
# use a different key than workflows running untrusted code
key: trusted-${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
trusted-${{ runner.os }}-maven-
- name: Set up Maven
run: ./mvnw -v
- name: Download GitHub Actions artifacts for the Develocity build scans
id: downloadBuildScan
uses: actions/download-artifact@v4
with:
pattern: build-scan-data-*
github-token: ${{ github.token }}
repository: ${{ github.repository }}
run-id: ${{ github.event.workflow_run.id }}
path: /tmp/downloaded-build-scan-data/
# Don't fail the build if there are no matching artifacts
continue-on-error: true
- name: Publish Develocity build scans for previous builds
if: ${{ steps.downloadBuildScan.outcome != 'failure'}}
run: |
shopt -s nullglob # Don't run the loop below if there are no artifacts
status=0
for build_scan_data_directory in /tmp/downloaded-build-scan-data/*
do
rm -rf ~/.m2/.develocity/build-scan-data
mkdir -p ~/.m2/.develocity/build-scan-data
tar -xzf "$build_scan_data_directory/build-scan-data.tgz" -C ~/.m2/.develocity/build-scan-data \
&& ./mvnw $MAVEN_ARGS develocity:build-scan-publish-previous || status=1
done
exit $status
env:
DEVELOCITY_ACCESS_KEY: ${{ secrets.DEVELOCITY_ACCESS_KEY_PR }}
93 changes: 32 additions & 61 deletions .github/workflows/build.yml → .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
# See https://docs.github.com/en/actions
# for more information about GitHub actions.

name: GitHub Actions Build
name: GH Actions CI

on:
push:
Expand All @@ -24,9 +24,7 @@ on:
- '!wip/**/dependency-update/**'
tags:
- '**'
# WARNING: Using pull_request_target to access secrets, but we check out the merge commit.
# See checkout action for details.
pull_request_target:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
branches:
# Pattern order matters: the last matching inclusion/exclusion wins
Expand All @@ -44,7 +42,7 @@ on:

concurrency:
group: "workflow = ${{ github.workflow }}, ref = ${{ github.event.ref }}, pr = ${{ github.event.pull_request.id }}"
cancel-in-progress: ${{ github.event_name == 'pull_request_target' || github.repository != 'hibernate/hibernate-search' }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.repository != 'hibernate/hibernate-search' }}

defaults:
run:
Expand Down Expand Up @@ -90,21 +88,9 @@ jobs:
- name: Support longpaths on Windows
if: "startsWith(matrix.os.runs-on, 'windows')"
run: git config --global core.longpaths true
- name: Check out commit already pushed to branch
if: "! github.event.pull_request.number"
uses: actions/checkout@v4
- name: Check out PR head
uses: actions/checkout@v4
if: github.event.pull_request.number
- uses: actions/checkout@v4
with:
# WARNING: This is potentially dangerous since we're checking out unreviewed code,
# and since we're using the pull_request_target event we can use secrets.
# Thus, we must be extra careful to never expose secrets to steps that execute this code,
# and to strictly limit our of secrets to those that only pose minor security threads.
# This means in particular we won't expose Develocity credentials to the main maven executions,
# but instead will execute maven a third time just to push build scans to Develocity;
# see below.
ref: "refs/pull/${{ github.event.pull_request.number }}/head"
persist-credentials: false
# Fetch the whole history to make sure that gitflow incremental builder
# can find the base commit.
fetch-depth: 0
Expand All @@ -118,9 +104,10 @@ jobs:
uses: actions/cache@v4
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
# use a different key than workflows running in trusted mode
key: ${{ github.event_name == 'push' && 'trusted' || 'untrusted' }}-${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
${{ github.event_name == 'push' && 'trusted' || 'untrusted' }}-${{ runner.os }}-maven-
- name: Set up Maven
run: ./mvnw -v
- name: Docker cleanup
Expand All @@ -131,55 +118,39 @@ jobs:
./mvnw $MAVEN_ARGS ${{ matrix.os.maven.args }} clean install \
-Pjqassistant -Pdist -Pci-build -DskipITs
env:
# WARNING: exposes secrets, so must only be passed to a step that doesn't run unapproved code.
DEVELOCITY_ACCESS_KEY: "${{ github.event_name == 'push' && secrets.GRADLE_ENTERPRISE_ACCESS_KEY || '' }}"
- name: Publish Develocity build scan for previous build (pull request)
# Don't fail a build if publishing fails
continue-on-error: true
if: "${{ !cancelled() && github.event_name == 'pull_request_target' && github.repository == 'hibernate/hibernate-search' }}"
run: |
./mvnw $MAVEN_ARGS develocity:build-scan-publish-previous
env:
# WARNING: exposes secrets, so must only be passed to a step that doesn't run unapproved code.
DEVELOCITY_ACCESS_KEY: ${{ secrets.GRADLE_ENTERPRISE_ACCESS_KEY_PR }}
DEVELOCITY_ACCESS_KEY: "${{ secrets.DEVELOCITY_ACCESS_KEY || '' }}"
# For jobs running on 'pull_request', tar and upload build scan data.
# The actual publishing must be done in a separate job (see ci-report.yml).
# We don't write to the remote cache as that would be unsafe.
- name: Tar build scan content pushed to subsequent jobs
if: "${{ github.event_name == 'pull_request' && !cancelled() }}"
run: tar -czf build-scan-data.tgz -C ~/.m2/.develocity/build-scan-data .
- name: Upload GitHub Actions artifact for the Develocity build scan
uses: actions/upload-artifact@v4
if: "${{ github.event_name == 'pull_request' && !cancelled() }}"
with:
name: build-scan-data-initial-${{ matrix.os.name }}
path: build-scan-data.tgz

- name: Run integration tests in the default environment
run: |
./mvnw $MAVEN_ARGS ${{ matrix.os.maven.args }} clean verify \
-Pskip-checks \
${{ github.event.pull_request.base.ref && format('-Dincremental -Dgib.referenceBranch=refs/remotes/origin/{0}', github.event.pull_request.base.ref) || '' }}
env:
# WARNING: exposes secrets, so must only be passed to a step that doesn't run unapproved code.
DEVELOCITY_ACCESS_KEY: "${{ github.event_name == 'push' && secrets.GRADLE_ENTERPRISE_ACCESS_KEY || '' }}"
- name: Publish Develocity build scan for previous build (pull request)
# Don't fail a build if publishing fails
continue-on-error: true
if: "${{ !cancelled() && github.event_name == 'pull_request_target' && github.repository == 'hibernate/hibernate-search' }}"
run: |
./mvnw $MAVEN_ARGS develocity:build-scan-publish-previous
env:
# WARNING: exposes secrets, so must only be passed to a step that doesn't run unapproved code.
DEVELOCITY_ACCESS_KEY: ${{ secrets.GRADLE_ENTERPRISE_ACCESS_KEY_PR }}
DEVELOCITY_ACCESS_KEY: "${{ secrets.DEVELOCITY_ACCESS_KEY || '' }}"
# Same as above, but for the build scan of the latest Maven run.
- name: Tar build scan content pushed to subsequent jobs
if: "${{ github.event_name == 'pull_request' && !cancelled() }}"
run: tar -czf build-scan-data.tgz -C ~/.m2/.develocity/build-scan-data .
- name: Upload GitHub Actions artifact for the Develocity build scan
uses: actions/upload-artifact@v4
if: "${{ github.event_name == 'pull_request' && !cancelled() }}"
with:
name: build-scan-data-integrationtest-${{ matrix.os.name }}
path: build-scan-data.tgz

- name: Docker cleanup
run: ./ci/docker-cleanup.sh
- name: Omit produced artifacts from build cache
run: rm -r ~/.m2/repository/org/hibernate/search
# Workaround for https://github.com/actions/upload-artifact/issues/240
- name: List build reports to upload (if build failed)
if: ${{ failure() || cancelled() }}
# The weird syntax is because we're setting a multiline environment variable
# See https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#example-of-a-multiline-string
run: |
{
echo 'buildReportPaths<<EOF'
find . -path '**/*-reports'
echo EOF
} >> "$GITHUB_ENV"
- name: Upload build reports (if build failed)
uses: actions/upload-artifact@v4
if: ${{ failure() || cancelled() }}
with:
name: ${{ format('build-reports-{0}', matrix.os.name ) }}
path: ${{ env.buildReportPaths }}
retention-days: 7
Loading