Use a shared cache when testing examples, but better (#481)

### Motivation

The examples pipeline currently takes around one hour to run. The CI
timeout is 60 minutes so we sometimes don't even finish it before the CI
job gets cancelled. We previously experimented with enabling a shared
build cache but this wasn't working as desired because the paths to the
source files were different for each of the examples we build. However,
by using the same package harness path, we should be able to get better
build cache performance (and keep the cache smaller during the run).

### Modifications

- Use a shared `--cache-path` and `--scratch-path`.
- Use a stable `--package-path` for each example (we delete it entirely
and recreate to be sure it's clean).

I experimented briefly with `ccache` which I thought might help with
some of the builds (e.g. the BoringSSL build, which comes in a few times
as a dependency) but didn't get very far.

### Result

CI pipeline should run much quicker.

It's possible that we will not catch failures where we depend on
something that we haven't declared properly, because these object files
will still be in the cache. I can't say with confidence that this is how
the build system works though; potentially it's a non-issue.

### Test Plan

CI.

Author: simonjbeaumont

scripts/test-examples.sh

@@ -27,42 +27,60 @@ TMP_DIR=$(/usr/bin/mktemp -d -p "${TMPDIR-/tmp}" "$(basename "$0").XXXXXXXXXX")
 
 PACKAGE_PATH=${PACKAGE_PATH:-${REPO_ROOT}}
 EXAMPLES_PACKAGE_PATH="${PACKAGE_PATH}/Examples"
+SHARED_EXAMPLE_HARNESS_PACKAGE_PATH="${TMP_DIR}/swift-openapi-example-harness"
+SHARED_PACKAGE_SCRATCH_PATH="${TMP_DIR}/swift-openapi-example-cache"
+SHARED_PACKAGE_CACHE_PATH="${TMP_DIR}/swift-openapi-example-scratch"
 
 for EXAMPLE_PACKAGE_PATH in $(find "${EXAMPLES_PACKAGE_PATH}" -maxdepth 2 -name Package.swift -type f -print0 | xargs -0 dirname); do
 
     EXAMPLE_PACKAGE_NAME="$(basename "${EXAMPLE_PACKAGE_PATH}")"
-    EXAMPLE_COPY_DIR="${TMP_DIR}/${EXAMPLE_PACKAGE_NAME}"
 
     if [[ "${SINGLE_EXAMPLE_PACKAGE:-${EXAMPLE_PACKAGE_NAME}}" != "${EXAMPLE_PACKAGE_NAME}" ]]; then
         log "Skipping example: ${EXAMPLE_PACKAGE_NAME}"
         continue
     fi
 
-    log "Copying example ${EXAMPLE_PACKAGE_NAME} to ${EXAMPLE_COPY_DIR}"
-    mkdir "${EXAMPLE_COPY_DIR}"
-    git archive HEAD "${EXAMPLE_PACKAGE_PATH}" --format tar | tar -C "${EXAMPLE_COPY_DIR}" -xvf- --strip-components 2
+    log "Recreating shared example harness directory: ${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}"
+    rm -rf "${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}"
+    mkdir -v "${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}"
 
-    log "Overriding dependency in ${EXAMPLE_PACKAGE_NAME} to use ${PACKAGE_PATH}"
+    log "Copying example contents from ${EXAMPLE_PACKAGE_NAME} to ${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}"
+    git archive HEAD "${EXAMPLE_PACKAGE_PATH}" --format tar | tar -C "${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}" -xvf- --strip-components 2
+
+    # GNU tar has --touch, but BSD tar does not, so we'll use touch directly.
+    log "Updating mtime of example contents..."
+    find "${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}" -print0 | xargs -0 -n1 touch -m
+
+    log "Re-overriding dependency in ${EXAMPLE_PACKAGE_NAME} to use ${PACKAGE_PATH}"
+    "${SWIFT_BIN}" package \
+        --package-path "${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}" \
+        --cache-path "${SHARED_PACKAGE_CACHE_PATH}" \
+        --skip-update \
+        --scratch-path "${SHARED_PACKAGE_SCRATCH_PATH}" \
+        unedit swift-openapi-generator || :
     "${SWIFT_BIN}" package \
-        --package-path "${EXAMPLE_COPY_DIR}" \
+        --package-path "${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}" \
+        --cache-path "${SHARED_PACKAGE_CACHE_PATH}" \
+        --skip-update \
+        --scratch-path "${SHARED_PACKAGE_SCRATCH_PATH}" \
         edit swift-openapi-generator \
         --path "${PACKAGE_PATH}"
 
     log "Building example package: ${EXAMPLE_PACKAGE_NAME}"
-    "${SWIFT_BIN}" build \
-        --package-path "${EXAMPLE_COPY_DIR}"
+    "${SWIFT_BIN}" build --build-tests \
+        --package-path "${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}" \
+        --cache-path "${SHARED_PACKAGE_CACHE_PATH}" \
+        --skip-update \
+        --scratch-path "${SHARED_PACKAGE_SCRATCH_PATH}"
     log "✅ Successfully built the example package ${EXAMPLE_PACKAGE_NAME}."
 
-    if [ -d "${EXAMPLE_COPY_DIR}/Tests" ]; then
+    if [ -d "${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}/Tests" ]; then
         log "Running tests for example package: ${EXAMPLE_PACKAGE_NAME}"
         "${SWIFT_BIN}" test \
-            --package-path "${EXAMPLE_COPY_DIR}"
+            --package-path "${SHARED_EXAMPLE_HARNESS_PACKAGE_PATH}" \
+            --cache-path "${SHARED_PACKAGE_CACHE_PATH}" \
+            --skip-update \
+            --scratch-path "${SHARED_PACKAGE_SCRATCH_PATH}"
         log "✅ Passed the tests for the example package ${EXAMPLE_PACKAGE_NAME}."
     fi
-
-    log "Deleting example ${EXAMPLE_PACKAGE_NAME} at ${EXAMPLE_COPY_DIR}"
-    rm -rf "${EXAMPLE_COPY_DIR}"
 done
-
-log "Deleting temporary directory"
-rm -rf "${TMP_DIR}"