Skip to content

Commit 6ef45bc

Browse files
author
MarcoFalke
committed
Merge #18581: ci: Print ccache stats, add pip cache, and cleanups
73f8303 ci: Add pip cache (Hennadii Stepanov) 6d48066 ci: Print ccache statistics summary (Hennadii Stepanov) a6eea71 ci: Drop unused WINEDEBUG variable (Hennadii Stepanov) Pull request description: The Travis [pip cache](https://docs.travis-ci.com/user/caching/#pip-cache) is free and saves a dozen of seconds :) Here are some excerpts from the Travis logs with `ccache` statistics (I found useful): 2) [Arm64](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507749) ``` ccache version 3.6 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Fri Apr 10 18:21:06 2020 stats zeroed Fri Apr 10 18:03:04 2020 cache hit (direct) 156 cache hit (preprocessed) 198 cache miss 143 cache hit rate 71.23 % called for link 8 cleanups performed 9 files in cache 1255 cache size 80.6 MB max cache size 100.0 MB ``` 3) [s390x](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507750) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:15:35 2020 cache hit (direct) 115 cache hit (preprocessed) 163 cache miss 94 cache hit rate 74.73 % called for link 6 cleanups performed 0 files in cache 804 cache size 304.8 MB max cache size 5.0 GB ``` 4) [Win64](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507751) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:38:07 2020 cache hit (direct) 147 cache hit (preprocessed) 199 cache miss 138 cache hit rate 71.49 % called for link 7 cleanups performed 7 files in cache 1242 cache size 87.0 MB max cache size 100.0 MB ``` 5) [CentOS 7](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507752) ``` ccache version 3.7.7 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Fri Apr 10 17:45:59 2020 stats zeroed Fri Apr 10 17:34:27 2020 cache hit (direct) 152 cache hit (preprocessed) 200 cache miss 144 cache hit rate 70.97 % called for link 8 cleanups performed 4 files in cache 1352 cache size 86.1 MB max cache size 100.0 MB ``` 6) [bionic](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507753) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:21:10 2020 cache hit (direct) 136 cache hit (preprocessed) 35 cache miss 330 cache hit rate 34.13 % called for link 8 cleanups performed 18 files in cache 1302 cache size 83.8 MB max cache size 100.0 MB ``` 7) [xenial](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507754) ``` ccache version 3.2.4 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf cache hit (direct) 161 cache hit (preprocessed) 174 cache miss 95 called for link 7 files in cache 3411 cache size 1.1 GB max cache size 5.0 GB ``` 10) [focal](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507757) ``` ccache version 3.7.7 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Fri Apr 10 17:35:57 2020 stats zeroed Fri Apr 10 17:21:32 2020 cache hit (direct) 78 cache hit (preprocessed) 125 cache miss 107 cache hit rate 65.48 % called for link 120 cleanups performed 0 files in cache 6218 cache size 1.8 GB max cache size 5.0 GB ``` 14) [macOS 10.12](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507761) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:47:19 2020 cache hit (direct) 28 cache hit (preprocessed) 1 cache miss 469 cache hit rate 5.82 % called for link 8 cleanups performed 17 files in cache 1946 cache size 82.6 MB max cache size 100.0 MB ``` 15) [macOS 10.14 native](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507762) ``` ccache version 3.7.8 cache directory /Users/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /Users/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /usr/local/Cellar/ccache/3.7.8/etc/ccache.conf stats updated Fri Apr 10 17:38:44 2020 stats zeroed Fri Apr 10 17:22:36 2020 cache hit (direct) 213 cache hit (preprocessed) 293 cache miss 144 cache hit rate 77.85 % called for link 11 called for preprocessing 56 compile failed 30 preprocessor error 56 bad compiler arguments 14 autoconf compile/link 77 no input file 66 cleanups performed 0 files in cache 1564 cache size 284.6 MB max cache size 5.0 GB ``` Top commit has no ACKs. Tree-SHA512: f9f2c6d72680bafc7a982efc04dc37ab9bba77ffd2a4cbbc56bb0e86c5c3501d57aa2f0ebeac0113e9d8daac1ee4eac1eddf74c1172c267b37ee2d9324366c8a
2 parents 130b644 + 73f8303 commit 6ef45bc

File tree

4 files changed

+8
-4
lines changed

4 files changed

+8
-4
lines changed

.travis.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ jobs:
7171
- stage: lint
7272
name: 'lint'
7373
env:
74-
cache: false
74+
cache: pip
7575
language: python
7676
python: '3.5' # Oldest supported version according to doc/dependencies.md
7777
install:

ci/test/00_setup_env.sh

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ echo "Fallback to default values in env (if not yet set)"
2424
export MAKEJOBS=${MAKEJOBS:--j4}
2525
# A folder for the ci system to put temporary files (ccache, datadirs for tests, ...)
2626
# This folder only exists on the ci host.
27-
export BASE_SCRATCH_DIR=${BASE_SCRATCH_DIR:-$BASE_ROOT_DIR/ci/scratch/}
27+
export BASE_SCRATCH_DIR=${BASE_SCRATCH_DIR:-$BASE_ROOT_DIR/ci/scratch}
2828
# What host to compile for. See also ./depends/README.md
2929
# Tests that need cross-compilation export the appropriate HOST.
3030
# Tests that run natively guess the host
@@ -53,7 +53,6 @@ export DEPENDS_DIR=${DEPENDS_DIR:-$BASE_ROOT_DIR/depends}
5353
export BASE_OUTDIR=${BASE_OUTDIR:-$BASE_SCRATCH_DIR/out/$HOST}
5454
export PREVIOUS_RELEASES_DIR=${PREVIOUS_RELEASES_DIR:-$BASE_ROOT_DIR/releases/$HOST}
5555
export SDK_URL=${SDK_URL:-https://bitcoincore.org/depends-sources/sdks}
56-
export WINEDEBUG=${WINEDEBUG:-fixme-all}
5756
export DOCKER_PACKAGES=${DOCKER_PACKAGES:-build-essential libtool autotools-dev automake pkg-config bsdmainutils curl ca-certificates ccache python3 rsync git procps}
5857
export GOAL=${GOAL:-install}
5958
export DIR_QA_ASSETS=${DIR_QA_ASSETS:-${BASE_SCRATCH_DIR}/qa-assets}

ci/test/04_install.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ export ASAN_OPTIONS="detect_stack_use_after_return=1:check_initialization_order=
2626
export LSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/lsan"
2727
export TSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/tsan:log_path=${BASE_SCRATCH_DIR}/sanitizer-output/tsan"
2828
export UBSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1"
29-
env | grep -E '^(BITCOIN_CONFIG|BASE_|QEMU_|CCACHE_|WINEDEBUG|LC_ALL|BOOST_TEST_RANDOM|CONFIG_SHELL|(ASAN|LSAN|TSAN|UBSAN)_OPTIONS|TEST_PREVIOUS_RELEASES|PREVIOUS_RELEASES_DIR)' | tee /tmp/env
29+
env | grep -E '^(BITCOIN_CONFIG|BASE_|QEMU_|CCACHE_|LC_ALL|BOOST_TEST_RANDOM|CONFIG_SHELL|(ASAN|LSAN|TSAN|UBSAN)_OPTIONS|TEST_PREVIOUS_RELEASES|PREVIOUS_RELEASES_DIR)' | tee /tmp/env
3030
if [[ $HOST = *-mingw32 ]]; then
3131
DOCKER_ADMIN="--cap-add SYS_ADMIN"
3232
elif [[ $BITCOIN_CONFIG = *--with-sanitizers=*address* ]]; then # If ran with (ASan + LSan), Docker needs access to ptrace (https://github.com/google/sanitizers/issues/764)

ci/test/06_script_a.sh

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
export LC_ALL=C.UTF-8
88

99
BITCOIN_CONFIG_ALL="--disable-dependency-tracking --prefix=$DEPENDS_DIR/$HOST --bindir=$BASE_OUTDIR/bin --libdir=$BASE_OUTDIR/lib"
10+
DOCKER_EXEC "command -v ccache > /dev/null && ccache --zero-stats"
1011
if [ -z "$NO_DEPENDS" ]; then
1112
DOCKER_EXEC ccache --max-size=$CCACHE_SIZE
1213
fi
@@ -44,3 +45,7 @@ trap 'DOCKER_EXEC "cat ${BASE_SCRATCH_DIR}/sanitizer-output/* 2> /dev/null"' ERR
4445
BEGIN_FOLD build
4546
DOCKER_EXEC make $MAKEJOBS $GOAL || ( echo "Build failure. Verbose build follows." && DOCKER_EXEC make $GOAL V=1 ; false )
4647
END_FOLD
48+
49+
BEGIN_FOLD ccache_stats
50+
DOCKER_EXEC "command -v ccache > /dev/null && ccache --version | head -n 1 && ccache --show-stats"
51+
END_FOLD

0 commit comments

Comments
 (0)