Commit 32eab85
committed
build: Optimize CI with test splitting and 32-core runner
Split 5 monolithic test targets into 139 individual test binaries for
better ctest parallelism, and upgrade ubuntu-debug to 32-core runner
with tuned build settings.
Test target splits (CMakeLists.txt changes only):
- velox_exec_test -> 76 individual tests
- velox_exec_util_test -> 9 individual tests
- velox_aggregates_test -> 43 individual tests
- velox_cache_test -> 5 individual tests
- velox_serializer_test -> 5 individual tests
CI workflow changes (ubuntu-debug):
- Runner: 16-core -> 32-core (128 GB RAM)
- NUM_THREADS: 16 -> 32
- MAX_HIGH_MEM_JOBS: 4 -> 12 (128 GB / 6.8 GB per heavy TU)
- MAX_LINK_JOBS: 2 -> 12 (shared links use ~1 GB each)
- ctest: -j 8 -> -j 24 --timeout 900
- ulimit -n 65536 to prevent "Too many open files"
Adapters job: add -DVELOX_BUILD_SHARED=ON for faster linking.
Validated on 48-core test machine (throttled to 32 threads):
- Build: 26m 34s (cold, no ccache)
- Test: 17m 45s (631 tests, 625 passed)
- Total: 44m 28s vs estimated ~70 min current CI
BUILD_PLAN.md contains the full analysis and profiling data.1 parent 24ac0df commit 32eab85
File tree
6 files changed
+818
-81
lines changed- .github/workflows
- velox
- common/caching/tests
- exec/tests
- functions/prestosql/aggregates/tests
- serializers/tests
6 files changed
+818
-81
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
156 | 156 | | |
157 | 157 | | |
158 | 158 | | |
| 159 | + | |
159 | 160 | | |
160 | 161 | | |
161 | 162 | | |
| |||
222 | 223 | | |
223 | 224 | | |
224 | 225 | | |
225 | | - | |
| 226 | + | |
226 | 227 | | |
227 | 228 | | |
228 | 229 | | |
| |||
263 | 264 | | |
264 | 265 | | |
265 | 266 | | |
266 | | - | |
| 267 | + | |
267 | 268 | | |
268 | 269 | | |
269 | 270 | | |
| |||
297 | 298 | | |
298 | 299 | | |
299 | 300 | | |
300 | | - | |
| 301 | + | |
| 302 | + | |
301 | 303 | | |
302 | 304 | | |
303 | 305 | | |
| |||
0 commit comments