Skip to content

Commit c67f8d3

Browse files
authored
ci: remove "max-workers" and test with our gradle defaults (#15307)
If our defaults need improvement, let's improve them to make tests run faster. We want to take advantage of the hardware. Tests use all runner cores with defaults and some macos stability hacks were added to speed up the worst-case builds.
1 parent 4166a03 commit c67f8d3

File tree

4 files changed

+20
-10
lines changed

4 files changed

+20
-10
lines changed

.github/workflows/run-checks-all.yml

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ jobs:
5858
echo "lucene.tool.ast-grep=ast-grep" >> build-options.local.properties
5959
6060
- name: Run gradle check (without tests)
61-
run: ./gradlew check -x test "-Ptask.times=true" --max-workers 2
61+
run: ./gradlew check -x test "-Ptask.times=true"
6262

6363

6464
# This runs all tests without any other validation checks.
@@ -87,8 +87,20 @@ jobs:
8787
- name: Configure tools
8888
uses: ./.github/actions/prepare-for-build
8989

90+
- name: Speedup MacOS runner
91+
if: ${{ runner.os == 'macOS' }}
92+
run: |
93+
mkdir /tmp/tmpfs
94+
sudo mount_tmpfs -o noowners -s 1g /tmp/tmpfs
95+
sudo sysctl debug.lowpri_throttle_enabled=0
96+
echo "tests.workDir=/tmp/tmpfs/lucene" >> build-options.local.properties
97+
9098
- name: Run gradle tests
91-
run: ./gradlew test "-Ptask.times=true" "-Pvalidation.errorprone=false" --max-workers 2
99+
run: ./gradlew test "-Ptask.times=true" "-Pvalidation.errorprone=false"
100+
env:
101+
# Set to the defaults to override the "CI"-based logic that would enable C2
102+
# we can't afford C2 on github runners.
103+
TEST_JVM_ARGS: "-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1"
92104

93105
- name: List automatically-initialized gradle.properties
94106
run: cat gradle.properties

build-tools/build-infra/src/main/java/org/apache/lucene/gradle/GradlePropertiesGenerator.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,10 +54,10 @@ public void run(Path source, Path destination) throws IOException {
5454
}
5555

5656
// Approximate a common-sense default for running gradle/tests with parallel
57-
// workers: half the count of available cpus but not more than 12.
57+
// workers: the count of available cpus but not more than 12.
5858
var cpus = Runtime.getRuntime().availableProcessors();
59-
var maxWorkers = (int) Math.max(1d, Math.min(cpus * 0.5d, 12));
60-
var testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 12));
59+
var maxWorkers = Math.min(cpus, 12);
60+
var testsJvms = Math.min(cpus, 12);
6161

6262
var replacements = Map.of("@MAX_WORKERS@", maxWorkers, "@TEST_JVMS@", testsJvms);
6363

build-tools/build-infra/src/main/java/org/apache/lucene/gradle/plugins/java/TestsAndRandomizationPlugin.java

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -186,9 +186,7 @@ public void apply(Project project) {
186186
.getProviders()
187187
.provider(
188188
() -> {
189-
return ((int)
190-
Math.max(
191-
1, Math.min(Runtime.getRuntime().availableProcessors() / 2.0, 4.0)));
189+
return Math.min(12, Runtime.getRuntime().availableProcessors());
192190
}));
193191

194192
// GITHUB#13986: Allow easier configuration of the Panama Vectorization provider with newer Java

gradle/template.gradle.properties

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,15 +16,15 @@
1616
###############
1717
#
1818
# Gradle build can run tasks in parallel but by default it consumes all CPU cores which
19-
# is too optimistic a default for Lucene tests. You can disable the parallelism
19+
# may be too optimistic a default for Lucene tests. You can disable the parallelism
2020
# entirely or assign it a 'low' priority with these properties:
2121
#
2222
# org.gradle.parallel=[true, false]
2323
# org.gradle.priority=[normal, low]
2424
#
2525
# The default level of parallelism is computed based on the number of cores on
2626
# your machine (on the first run of gradle build). By default these are fairly conservative
27-
# settings (half the number of cores for workers, for example):
27+
# settings (the number of cores for workers up to a limit of 12, for example):
2828
#
2929
# org.gradle.workers.max=[X]
3030
# tests.jvms=[N <= X]

0 commit comments

Comments
 (0)