Skip to content

Commit 5eaaff2

Browse files
yrodieregsmet
yrodiere
and
gsmet
committed
Improve build cache strategy
Ripped off from Quarkus. Here's how it will work: 1. We create a cache entry on push only. Pull requests only restore it. 2. We create a new cache entry every day, prefixed with something like 2024-10-25. 3. When restoring the cache, we try the entry for the day first (2024-10-25) and default to the one for the month (2024-10-*). Critically, this means we will build each day's cache based on the previous day's cache. 4. Atlas infra uses its own, separate cache entries. Co-Authored-By: Guillaume Smet <[email protected]>
1 parent 4e8eb91 commit 5eaaff2

File tree

2 files changed

+91
-28
lines changed

2 files changed

+91
-28
lines changed

‎.github/workflows/ci-report.yml‎

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -26,23 +26,30 @@ jobs:
2626
with:
2727
distribution: 'temurin'
2828
java-version: '17'
29-
- name: Get year/month for cache key
30-
id: get-date
31-
run: echo "yearmonth=$(/bin/date -u "+%Y-%m")" >> $GITHUB_OUTPUT
32-
shell: bash
33-
# Note we only restore the caches, we never populate them
34-
- name: Restore Maven/Gradle local caches
29+
30+
- name: Generate cache key
31+
id: cache-key
32+
run: |
33+
CURRENT_BRANCH="${{ github.repository != 'hibernate/hibernate-orm' && 'fork' || github.base_ref || github.ref_name }}"
34+
CURRENT_MONTH=$(/bin/date -u "+%Y-%m")
35+
CURRENT_DAY=$(/bin/date -u "+%d")
36+
ROOT_CACHE_KEY="buildtool-cache"
37+
echo "buildtool-monthly-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}" >> $GITHUB_OUTPUT
38+
echo "buildtool-monthly-branch-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}" >> $GITHUB_OUTPUT
39+
echo "buildtool-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}-${CURRENT_DAY}" >> $GITHUB_OUTPUT
40+
- name: Restore Maven/Gradle Local Caches
3541
uses: actions/cache/restore@v4
36-
id: cache-maven-gradle
3742
with:
3843
path: |
3944
~/.m2/repository/
4045
~/.m2/wrapper/
4146
~/.gradle/caches/
4247
~/.gradle/wrapper/
43-
# refresh cache every month to avoid unlimited growth
44-
# use a different key than workflows running untrusted code
45-
key: trusted-maven-gradle-caches-${{ steps.get-date.outputs.yearmonth }}
48+
key: ${{ steps.cache-key.outputs.buildtool-cache-key }}
49+
restore-keys: |
50+
${{ steps.cache-key.outputs.buildtool-monthly-branch-cache-key }}-
51+
${{ steps.cache-key.outputs.buildtool-monthly-cache-key }}-
52+
4653
- name: Download GitHub Actions artifacts for the Develocity build scans
4754
id: downloadBuildScan
4855
uses: actions/download-artifact@v4

‎.github/workflows/ci.yml‎

Lines changed: 74 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -60,22 +60,50 @@ jobs:
6060
with:
6161
distribution: 'temurin'
6262
java-version: '17'
63-
- name: Get year/month for cache key
64-
id: get-date
65-
run: echo "yearmonth=$(/bin/date -u "+%Y-%m")" >> $GITHUB_OUTPUT
66-
shell: bash
67-
- name: Cache Maven/Gradle local caches
63+
64+
- name: Generate cache key
65+
id: cache-key
66+
run: |
67+
CURRENT_BRANCH="${{ github.repository != 'hibernate/hibernate-orm' && 'fork' || github.base_ref || github.ref_name }}"
68+
CURRENT_MONTH=$(/bin/date -u "+%Y-%m")
69+
CURRENT_DAY=$(/bin/date -u "+%d")
70+
ROOT_CACHE_KEY="buildtool-cache"
71+
echo "buildtool-monthly-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}" >> $GITHUB_OUTPUT
72+
echo "buildtool-monthly-branch-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}" >> $GITHUB_OUTPUT
73+
echo "buildtool-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}-${CURRENT_DAY}" >> $GITHUB_OUTPUT
74+
- name: Cache Maven/Gradle Local Caches
75+
id: cache-maven
6876
uses: actions/cache@v4
69-
id: cache-maven-gradle
77+
# if it's not a pull request, we restore and save the cache
78+
if: github.event_name != 'pull_request'
7079
with:
7180
path: |
7281
~/.m2/repository/
7382
~/.m2/wrapper/
7483
~/.gradle/caches/
7584
~/.gradle/wrapper/
76-
# refresh cache every month to avoid unlimited growth
77-
# use a different key depending on whether we run in trusted or untrusted mode
78-
key: ${{ github.event_name == 'push' && 'trusted' || 'untrusted' }}-maven-gradle-caches-${{ steps.get-date.outputs.yearmonth }}
85+
# A new cache will be stored daily. After that first store of the day, cache save actions will fail because the cache is immutable but it's not a problem.
86+
# The whole cache is dropped monthly to prevent unlimited growth.
87+
# The cache is per branch but in case we don't find a branch for a given branch, we will get a cache from another branch.
88+
key: ${{ steps.cache-key.outputs.buildtool-cache-key }}
89+
restore-keys: |
90+
${{ steps.cache-key.outputs.buildtool-monthly-branch-cache-key }}-
91+
${{ steps.cache-key.outputs.buildtool-monthly-cache-key }}-
92+
- name: Restore Maven/Gradle Local Caches
93+
uses: actions/cache/restore@v4
94+
# if it a pull request, we restore the cache but we don't save it
95+
if: github.event_name == 'pull_request'
96+
with:
97+
path: |
98+
~/.m2/repository/
99+
~/.m2/wrapper/
100+
~/.gradle/caches/
101+
~/.gradle/wrapper/
102+
key: ${{ steps.cache-key.outputs.buildtool-cache-key }}
103+
restore-keys: |
104+
${{ steps.cache-key.outputs.buildtool-monthly-branch-cache-key }}-
105+
${{ steps.cache-key.outputs.buildtool-monthly-cache-key }}-
106+
79107
- name: Run build script
80108
run: ./ci/build-github.sh
81109
shell: bash
@@ -141,22 +169,50 @@ jobs:
141169
with:
142170
distribution: 'graalvm'
143171
java-version: '21'
144-
- name: Get year/month for cache key
145-
id: get-date
146-
run: echo "yearmonth=$(/bin/date -u "+%Y-%m")" >> $GITHUB_OUTPUT
147-
shell: bash
148-
- name: Cache Maven/Gradle local caches
172+
173+
- name: Generate cache key
174+
id: cache-key
175+
run: |
176+
CURRENT_BRANCH="${{ github.repository != 'hibernate/hibernate-orm' && 'fork' || github.base_ref || github.ref_name }}"
177+
CURRENT_MONTH=$(/bin/date -u "+%Y-%m")
178+
CURRENT_DAY=$(/bin/date -u "+%d")
179+
ROOT_CACHE_KEY="buildtool-cache-atlas"
180+
echo "buildtool-monthly-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}" >> $GITHUB_OUTPUT
181+
echo "buildtool-monthly-branch-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}" >> $GITHUB_OUTPUT
182+
echo "buildtool-cache-key=${ROOT_CACHE_KEY}-${CURRENT_MONTH}-${CURRENT_BRANCH}-${CURRENT_DAY}" >> $GITHUB_OUTPUT
183+
- name: Cache Maven/Gradle Local Caches
184+
id: cache-maven
149185
uses: actions/cache@v4
150-
id: cache-maven-gradle
186+
# if it's not a pull request, we restore and save the cache
187+
if: github.event_name != 'pull_request'
151188
with:
152189
path: |
153190
~/.m2/repository/
154191
~/.m2/wrapper/
155192
~/.gradle/caches/
156193
~/.gradle/wrapper/
157-
# refresh cache every month to avoid unlimited growth
158-
# use a different key than jobs running in trusted mode
159-
key: untrusted-maven-gradle-caches-${{ steps.get-date.outputs.yearmonth }}
194+
# A new cache will be stored daily. After that first store of the day, cache save actions will fail because the cache is immutable but it's not a problem.
195+
# The whole cache is dropped monthly to prevent unlimited growth.
196+
# The cache is per branch but in case we don't find a branch for a given branch, we will get a cache from another branch.
197+
key: ${{ steps.cache-key.outputs.buildtool-cache-key }}
198+
restore-keys: |
199+
${{ steps.cache-key.outputs.buildtool-monthly-branch-cache-key }}-
200+
${{ steps.cache-key.outputs.buildtool-monthly-cache-key }}-
201+
- name: Restore Maven/Gradle Local Caches
202+
uses: actions/cache/restore@v4
203+
# if it a pull request, we restore the cache but we don't save it
204+
if: github.event_name == 'pull_request'
205+
with:
206+
path: |
207+
~/.m2/repository/
208+
~/.m2/wrapper/
209+
~/.gradle/caches/
210+
~/.gradle/wrapper/
211+
key: ${{ steps.cache-key.outputs.buildtool-cache-key }}
212+
restore-keys: |
213+
${{ steps.cache-key.outputs.buildtool-monthly-branch-cache-key }}-
214+
${{ steps.cache-key.outputs.buildtool-monthly-cache-key }}-
215+
160216
- name: Run build script
161217
env:
162218
RDBMS: ${{ matrix.rdbms }}

0 commit comments

Comments
 (0)