Skip to content

Update gradle caching #13208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 5, 2025
Merged

Update gradle caching #13208

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 1 addition & 11 deletions .github/workflows/build-common.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,6 @@ jobs:
uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0
with:
cache-read-only: ${{ inputs.cache-read-only }}
# gradle enterprise is used for the build cache
gradle-home-cache-excludes: caches/build-cache-1
Comment on lines -47 to -48
Copy link
Member Author

@trask trask Feb 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing this might help CI performance now that we aren't using gradle enterprise anymore (it should use github actions cache)


- name: Spotless
run: ./gradlew spotlessCheck ${{ inputs.no-build-cache && '--no-build-cache' || '' }}
Expand All @@ -68,8 +66,6 @@ jobs:
uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0
with:
cache-read-only: ${{ inputs.cache-read-only }}
# gradle enterprise is used for the build cache
gradle-home-cache-excludes: caches/build-cache-1

- name: Generate license report
run: ./gradlew generateLicenseReport ${{ inputs.no-build-cache && '--no-build-cache' || '' }}
Expand Down Expand Up @@ -151,8 +147,6 @@ jobs:
uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0
with:
cache-read-only: ${{ inputs.cache-read-only }}
# gradle enterprise is used for the build cache
gradle-home-cache-excludes: caches/build-cache-1

- name: Build
# javadoc task fails sporadically fetching https://docs.oracle.com/javase/8/docs/api/
Expand Down Expand Up @@ -253,8 +247,6 @@ jobs:
with:
# only push cache for one matrix option since github action cache space is limited
cache-read-only: ${{ inputs.cache-read-only || matrix.test-java-version != 11 || matrix.vm != 'hotspot' }}
# gradle enterprise is used for the build cache
gradle-home-cache-excludes: caches/build-cache-1

- name: List tests
# "check" is needed to activate all tests for listing purposes
Expand Down Expand Up @@ -379,12 +371,10 @@ jobs:
with:
# only push cache for one matrix option per OS since github action cache space is limited
cache-read-only: ${{ inputs.cache-read-only || matrix.smoke-test-suite != 'tomcat' }}
# gradle enterprise is used for the build cache
gradle-home-cache-excludes: caches/build-cache-1

- name: Build
# running suite "none" compiles everything needed by smoke tests without executing any tests
run: ./gradlew :smoke-tests:test -PsmokeTestSuite=none --no-daemon ${{ inputs.no-build-cache && ' --no-build-cache' || '' }}
run: ./gradlew :smoke-tests:test -PsmokeTestSuite=none ${{ inputs.no-build-cache && ' --no-build-cache' || '' }}
Copy link
Member Author

@trask trask Feb 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--no-daemon was only used in two places, so removed for consistency

Copy link
Contributor

@laurit laurit Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the --no-daemon was used here to reduce memory usage.

Copy link
Member Author

@trask trask Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I run with --no-daemon locally it logs

To honour the JVM settings for this build a single-use Daemon process will be forked.

and reading the CodeQL docs they say they need --no-daemon or ./gradlew --stop before running (in order to pick up the initial forking/child process I guess?), which makes me think that --no-daemon doesn't really do much here except prevent it from reusing the existing worker started in "Set up Gradle cache", but I really don't know...

Copy link
Contributor

@laurit laurit Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't check the commit history, but I suspect the issue was that the tests sometimes got an OOM. So it was split in 2 parts where in the first part we compile the tests with --no-daemon (-PsmokeTestSuite=none doesn't run any tests) so whatever memory gradle used for building gets freed. Second step just runs the tests and hopefully has gradle consuming less memory. It is possible that the OOM isn't an issue any more. If that is the case we could remove the Build step from that workflow and just keep Test.

Copy link
Member Author

@trask trask Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh! I missed the second step below, that makes sense, I'll add this back


- name: Test
run: ./gradlew :smoke-tests:test -PsmokeTestSuite=${{ matrix.smoke-test-suite }}${{ inputs.no-build-cache && ' --no-build-cache' || '' }}
Expand Down
3 changes: 0 additions & 3 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,6 @@ jobs:

- name: Setup Gradle
uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0
with:
# gradle enterprise is used for the build cache
gradle-home-cache-excludes: caches/build-cache-1

- name: Build and publish artifact snapshots
env:
Expand Down
16 changes: 11 additions & 5 deletions .github/workflows/codeql.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,21 @@ jobs:
- name: Free disk space
run: .github/scripts/gha-free-disk-space.sh

- name: Set up Java 17
- name: Set up JDK for running Gradle
uses: actions/setup-java@3a4f6e1af504cf6a31855fa899c6aa5355ba6c12 # v4.7.0
with:
distribution: temurin
java-version-file: .java-version

- name: Increase gradle daemon heap size
run: |
sed -i "s/org.gradle.jvmargs=/org.gradle.jvmargs=-Xmx3g /" gradle.properties

- name: Setup Gradle
uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0
with:
cache-read-only: ${{ github.event_name == 'pull_request' }}

- name: Initialize CodeQL
uses: github/codeql-action/init@dd746615b3b9d728a6a37ca2045b68ca76d4841a # v3.28.8
with:
Expand All @@ -51,12 +60,9 @@ jobs:
# see https://github.com/github/codeql-action/issues/1555#issuecomment-1452228433
tools: latest

- name: Setup Gradle
uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0

- name: Build
# skipping build cache is needed so that all modules will be analyzed
run: ./gradlew assemble -x javadoc --no-build-cache --no-daemon
run: ./gradlew assemble -x javadoc
Copy link
Member Author

@trask trask Feb 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing --no-build-cache should help with CodeQL performance now that it's a PR check


- name: Perform CodeQL analysis
uses: github/codeql-action/analyze@dd746615b3b9d728a6a37ca2045b68ca76d4841a # v3.28.8
2 changes: 0 additions & 2 deletions .github/workflows/pr-smoke-test-early-jdk8-images.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,6 @@ jobs:
uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0
with:
cache-read-only: true
# gradle enterprise is used for the build cache
gradle-home-cache-excludes: caches/build-cache-1

- name: Build Docker image
run: ./gradlew :smoke-tests:images:early-jdk8:imageBuild
2 changes: 0 additions & 2 deletions .github/workflows/pr-smoke-test-fake-backend-images.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,6 @@ jobs:
uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0
with:
cache-read-only: true
# gradle enterprise is used for the build cache
gradle-home-cache-excludes: caches/build-cache-1

- name: Build Docker image
run: ./gradlew :smoke-tests:images:fake-backend:jibDockerBuild -Djib.httpTimeout=120000 -Djib.console=plain
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/reusable-test-indy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,6 @@ jobs:
uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0
with:
cache-read-only: ${{ inputs.cache-read-only }}
# gradle enterprise is used for the build cache
gradle-home-cache-excludes: caches/build-cache-1

- name: List tests
run: >
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/reusable-test-latest-deps.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,6 @@ jobs:
uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0
with:
cache-read-only: ${{ inputs.cache-read-only }}
# gradle enterprise is used for the build cache
gradle-home-cache-excludes: caches/build-cache-1

- name: List tests
run: >
Expand Down
Loading