Skip to content

Commit 059ae87

Browse files
yrodieregsmet
authored and
beikov
committed
Improve build cache strategy
Ripped off from Quarkus. Here's how it will work: 1. We create a cache entry on push only. Pull requests only restore it. 2. We create a new cache entry every day, prefixed with something like 2024-10-25. 3. When restoring the cache, we try the entry for the day first (2024-10-25) and default to the one for the month (2024-10-*). Critically, this means we will build each day's cache based on the previous day's cache. 4. Atlas infra uses its own, separate cache entries. Co-Authored-By: Guillaume Smet <[email protected]> (cherry picked from commit 5eaaff2)
1 parent 61bf908 commit 059ae87

File tree

2 files changed

+91
-28
lines changed

2 files changed

+91
-28
lines changed

‎.github/workflows/ci-report.yml

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -26,23 +26,30 @@ jobs:
2626
with:
2727
distribution: 'temurin'
2828
java-version: '11'
29-
- name: Get year/month for cache key
30-
id: get-date
31-
run: echo "yearmonth=$(/bin/date -u "+%Y-%m")" >> $GITHUB_OUTPUT
32-
shell: bash
33-
# Note we only restore the caches, we never populate them
34-
- name: Restore Maven/Gradle local caches
29+
30+
- name: Generate cache key
31+
id: cache-key
32+
run: |
33+
CURRENT_BRANCH="${{ github.repository != 'hibernate/hibernate-orm' && 'fork' || github.base_ref || github.ref_name }}"
34+
CURRENT_MONTH=$(/bin/date -u "+%Y-%m")
35+
CURRENT_DAY=$(/bin/date -u "+%d")
36+
ROOT_CACHE_KEY="buildtool-cache"
37+
echo "buildtool-monthly-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}" >> $GITHUB_OUTPUT
38+
echo "buildtool-monthly-branch-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}" >> $GITHUB_OUTPUT
39+
echo "buildtool-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}-${CURRENT_DAY}" >> $GITHUB_OUTPUT
40+
- name: Restore Maven/Gradle Local Caches
3541
uses: actions/cache/restore@v4
36-
id: cache-maven-gradle
3742
with:
3843
path: |
3944
~/.m2/repository/
4045
~/.m2/wrapper/
4146
~/.gradle/caches/
4247
~/.gradle/wrapper/
43-
# refresh cache every month to avoid unlimited growth
44-
# use a different key than workflows running untrusted code
45-
key: trusted-maven-gradle-caches-${{ steps.get-date.outputs.yearmonth }}
48+
key: ${{ steps.cache-key.outputs.buildtool-cache-key }}
49+
restore-keys: |
50+
${{ steps.cache-key.outputs.buildtool-monthly-branch-cache-key }}-
51+
${{ steps.cache-key.outputs.buildtool-monthly-cache-key }}-
52+
4653
- name: Download GitHub Actions artifacts for the Develocity build scans
4754
id: downloadBuildScan
4855
uses: actions/download-artifact@v4

‎.github/workflows/ci.yml

Lines changed: 74 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -61,22 +61,50 @@ jobs:
6161
with:
6262
distribution: 'temurin'
6363
java-version: '11'
64-
- name: Get year/month for cache key
65-
id: get-date
66-
run: echo "yearmonth=$(/bin/date -u "+%Y-%m")" >> $GITHUB_OUTPUT
67-
shell: bash
68-
- name: Cache Maven/Gradle local caches
64+
65+
- name: Generate cache key
66+
id: cache-key
67+
run: |
68+
CURRENT_BRANCH="${{ github.repository != 'hibernate/hibernate-orm' && 'fork' || github.base_ref || github.ref_name }}"
69+
CURRENT_MONTH=$(/bin/date -u "+%Y-%m")
70+
CURRENT_DAY=$(/bin/date -u "+%d")
71+
ROOT_CACHE_KEY="buildtool-cache"
72+
echo "buildtool-monthly-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}" >> $GITHUB_OUTPUT
73+
echo "buildtool-monthly-branch-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}" >> $GITHUB_OUTPUT
74+
echo "buildtool-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}-${CURRENT_DAY}" >> $GITHUB_OUTPUT
75+
- name: Cache Maven/Gradle Local Caches
76+
id: cache-maven
6977
uses: actions/cache@v4
70-
id: cache-maven-gradle
78+
# if it's not a pull request, we restore and save the cache
79+
if: github.event_name != 'pull_request'
7180
with:
7281
path: |
7382
~/.m2/repository/
7483
~/.m2/wrapper/
7584
~/.gradle/caches/
7685
~/.gradle/wrapper/
77-
# refresh cache every month to avoid unlimited growth
78-
# use a different key depending on whether we run in trusted or untrusted mode
79-
key: ${{ github.event_name == 'push' && 'trusted' || 'untrusted' }}-maven-gradle-caches-${{ steps.get-date.outputs.yearmonth }}
86+
# A new cache will be stored daily. After that first store of the day, cache save actions will fail because the cache is immutable but it's not a problem.
87+
# The whole cache is dropped monthly to prevent unlimited growth.
88+
# The cache is per branch but in case we don't find a branch for a given branch, we will get a cache from another branch.
89+
key: ${{ steps.cache-key.outputs.buildtool-cache-key }}
90+
restore-keys: |
91+
${{ steps.cache-key.outputs.buildtool-monthly-branch-cache-key }}-
92+
${{ steps.cache-key.outputs.buildtool-monthly-cache-key }}-
93+
- name: Restore Maven/Gradle Local Caches
94+
uses: actions/cache/restore@v4
95+
# if it a pull request, we restore the cache but we don't save it
96+
if: github.event_name == 'pull_request'
97+
with:
98+
path: |
99+
~/.m2/repository/
100+
~/.m2/wrapper/
101+
~/.gradle/caches/
102+
~/.gradle/wrapper/
103+
key: ${{ steps.cache-key.outputs.buildtool-cache-key }}
104+
restore-keys: |
105+
${{ steps.cache-key.outputs.buildtool-monthly-branch-cache-key }}-
106+
${{ steps.cache-key.outputs.buildtool-monthly-cache-key }}-
107+
80108
- name: Run build script
81109
run: ./ci/build-github.sh
82110
shell: bash
@@ -142,22 +170,50 @@ jobs:
142170
with:
143171
distribution: 'graalvm'
144172
java-version: '21'
145-
- name: Get year/month for cache key
146-
id: get-date
147-
run: echo "yearmonth=$(/bin/date -u "+%Y-%m")" >> $GITHUB_OUTPUT
148-
shell: bash
149-
- name: Cache Maven/Gradle local caches
173+
174+
- name: Generate cache key
175+
id: cache-key
176+
run: |
177+
CURRENT_BRANCH="${{ github.repository != 'hibernate/hibernate-orm' && 'fork' || github.base_ref || github.ref_name }}"
178+
CURRENT_MONTH=$(/bin/date -u "+%Y-%m")
179+
CURRENT_DAY=$(/bin/date -u "+%d")
180+
ROOT_CACHE_KEY="buildtool-cache-atlas"
181+
echo "buildtool-monthly-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}" >> $GITHUB_OUTPUT
182+
echo "buildtool-monthly-branch-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}" >> $GITHUB_OUTPUT
183+
echo "buildtool-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}-${CURRENT_DAY}" >> $GITHUB_OUTPUT
184+
- name: Cache Maven/Gradle Local Caches
185+
id: cache-maven
150186
uses: actions/cache@v4
151-
id: cache-maven-gradle
187+
# if it's not a pull request, we restore and save the cache
188+
if: github.event_name != 'pull_request'
152189
with:
153190
path: |
154191
~/.m2/repository/
155192
~/.m2/wrapper/
156193
~/.gradle/caches/
157194
~/.gradle/wrapper/
158-
# refresh cache every month to avoid unlimited growth
159-
# use a different key than jobs running in trusted mode
160-
key: untrusted-maven-gradle-caches-${{ steps.get-date.outputs.yearmonth }}
195+
# A new cache will be stored daily. After that first store of the day, cache save actions will fail because the cache is immutable but it's not a problem.
196+
# The whole cache is dropped monthly to prevent unlimited growth.
197+
# The cache is per branch but in case we don't find a branch for a given branch, we will get a cache from another branch.
198+
key: ${{ steps.cache-key.outputs.buildtool-cache-key }}
199+
restore-keys: |
200+
${{ steps.cache-key.outputs.buildtool-monthly-branch-cache-key }}-
201+
${{ steps.cache-key.outputs.buildtool-monthly-cache-key }}-
202+
- name: Restore Maven/Gradle Local Caches
203+
uses: actions/cache/restore@v4
204+
# if it a pull request, we restore the cache but we don't save it
205+
if: github.event_name == 'pull_request'
206+
with:
207+
path: |
208+
~/.m2/repository/
209+
~/.m2/wrapper/
210+
~/.gradle/caches/
211+
~/.gradle/wrapper/
212+
key: ${{ steps.cache-key.outputs.buildtool-cache-key }}
213+
restore-keys: |
214+
${{ steps.cache-key.outputs.buildtool-monthly-branch-cache-key }}-
215+
${{ steps.cache-key.outputs.buildtool-monthly-cache-key }}-
216+
161217
- name: Run build script
162218
env:
163219
RDBMS: ${{ matrix.rdbms }}

0 commit comments

Comments
 (0)