unix: use sccache more during gcc build

I discovered this optimization as part of extracting the toolchain
bootstrap logic to a separate, not-yet-published project. Let's
"backport" it to python-build-standalone to hopefully realize a
build speedup.

In order to make this work, we had to move `sccache` to PATH. Well,
we strictly didn't need to. But I think that makes more sense.

Author: indygreg

cpython-unix/build-gcc.sh

@@ -7,12 +7,17 @@ set -ex
 
 cd /build
 
-ROOT=$(pwd)
-SCCACHE="${ROOT}/sccache"
-
 tar -C /tools -xf /build/binutils-${BINUTILS_VERSION}-linux64.tar
 export PATH=/tools/host/bin:$PATH
 
+HAVE_SCCACHE=
+
+if [ -x sccache ]; then
+  HAVE_SCCACHE=1
+  mv sccache /tools/host/bin/
+  cp sccache-wrapper.sh /tools/host/bin/
+fi
+
 tar -xf gcc-${GCC_VERSION}.tar.xz
 tar -xf gmp-${GMP_VERSION}.tar.xz
 tar -xf isl-${ISL_VERSION}.tar.bz2
@@ -26,10 +31,11 @@ ln -sf ../mpc-${MPC_VERSION} mpc
 ln -sf ../mpfr-${MPFR_VERSION} mpfr
 popd
 
-if [ -x "${SCCACHE}" ]; then
-  "${SCCACHE}" --start-server
-  export CC="${SCCACHE} /usr/bin/gcc"
-  export CXX="${SCCACHE} /usr/bin/g++"
+if [ -n "${HAVE_SCCACHE}" ]; then
+  sccache --start-server
+  export CC="sccache /usr/bin/gcc"
+  export CXX="sccache /usr/bin/g++"
+  export STAGE_CC_WRAPPER=sccache-wrapper.sh
 fi
 
 mkdir gcc-objdir
@@ -51,3 +57,5 @@ pushd gcc-objdir
 make -j `nproc`
 make -j `nproc` install DESTDIR=/build/out
 popd
+
+sccache -s

cpython-unix/build.py

@@ -294,6 +294,7 @@ def build_gcc(client, image, host_platform):
             build_env.copy_file(a)
 
         build_env.copy_file(toolchain_archive_path("binutils", host_platform))
+        build_env.copy_file(SUPPORT / "sccache-wrapper.sh")
         build_env.copy_file(SUPPORT / "build-gcc.sh")
 
         env = {

cpython-unix/sccache-wrapper.sh

@@ -0,0 +1,31 @@
+#!/usr/bin/env bash
+# This script is needed to handle a race in GCC's build system and a bug in
+# sccache.
+#
+# GCC's build system could materialize `xgcc` and invoke commands like
+# `xgcc -dumpspecs` before make materializes `cc1`. This is relevant because
+# sccache invokes `<compiler> -E` on the first invocation of a compiler to
+# determine which flavor of compiler to treat it as. And `gcc -E` requires
+# support binaries like `cc1` in order to work. Our wrapper script sniffs
+# for existence of `cc1` to mitigate this race condiion.
+#
+# Furthermore, sccache doesn't honor `-B` arguments when running
+# `<compiler> -E`. So even if a support binary like `cc1` may exist, GCC may
+# not know where to find it. Our wrapper script works around this by ensuring
+# the compiler's directory is always on PATH.
+#
+# This script/approach is arguably not sound for use outside of the value of
+# STAGE_CC_WRAPPER in GCC's build system. You have been warned.
+
+set -o errexit
+set -o pipefail
+
+dir=$(dirname $1)
+cc1=${dir}/cc1
+
+if [ -e "${cc1}"  ]; then
+  export PATH=${dir}:${PATH}
+  exec sccache "$@"
+else
+  exec "$@"
+fi