Merge #6421: ci: merge bitcoin#30451, add macOS builds to GitHub Actions, refactor cache keys

106500c ci: add x86_64 macOS cross-compilation build to GitHub Actions (Kittywhiskers Van Gogh)
56d8a8e ci: add SDK (extracted) sources to separate SDK cache (Kittywhiskers Van Gogh)
9841155 ci: cleanup restore keys (Kittywhiskers Van Gogh)
2f28f3e ci: replace `runner.os` with adding CI `Dockerfile` to `hashFiles` (Kittywhiskers Van Gogh)
1d8868b merge bitcoin#30451: remove Darwin ENV unsetting (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Depends on #6400

  * ~~`WINEPREFIX` needs to be overridden because of permissions issues with GitHub Actions ([build](https://github.com/kwvg/dash/actions/runs/11952384181/job/33318706080#step:8:3297))~~ Superseded by [dash#6563](#6563)

  * [bitcoin#30451](bitcoin#30451) will enable `ccache` to work with macOS cross-compilation builds (i.e. they will no longer be registered as uncacheable).

  * After merging in [dash#6564](#6564), existing caches resulted in across-the-board build failures ([build](https://github.com/dashpay/dash/actions/runs/13290871849/job/37113644102#step:7:2768), [build](https://github.com/dashpay/dash/actions/runs/13290791962/job/37113418275#step:7:2940), [build](https://github.com/dashpay/dash/actions/runs/13289932901/job/37109498600#step:7:2177)) due to differing compiler versions (GCC 13 build on GCC 11 based cache) and differing glibc versions (2.39 build vs 2.35 cache) due to the change from `jammy` (on which the caches were built) to `noble` (the new base image for CI).

    To prevent this from occurring in the future, we will add CI's [`Dockerfile`](https://github.com/dashpay/dash/blob/265d9b5f9f5af07ff9da3d6fcc318b783d18caea/contrib/containers/ci/Dockerfile) to the `hashFiles()` used to generate the root of the cache key to ensure that the cache is discarded if it is modified.

    * `runner.os` doesn't help us as it tells us what platform the _runner_ is on and is independent from what the base image in the container is using, we only care about the latter.

  * The SDKs were cached separately because
    * The sources cache is shared between variants and the fastest runner (usually a Linux runner) will set the cache and as the download and extraction of the macOS SDK is only done when expected to target macOS, the cache associated with the shared key will not have the requisite SDKs.

      Then when that cache key is used to pull the sources cache by the macOS build, it will succeed due to a cache hit and then fail building because the compiler cannot locate the SDK.
    * The build cache is expected to have a higher churn rate as it is tied to the platform, which will trigger more than necessary downloads of the SDK.
      * If placed here, the cache would need to be re-evaluated if the version of Xcode has been bumped, which involves modifying `depends/hosts/darwin.mk`. Meaning, that file needs to be included when computing `hashFiles()`. But this also would result in unnecessary cache misses for all the platforms that _aren't_ macOS (because the structure of the build key is the same among all variants).

  ## Breaking Changes

  None expected.

  ## Checklist

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    utACK 106500c
  PastaPastaPasta:
    utACK 106500c; gonna do CI on my repo, and push an empty commit and ensure caches are working as expected

Tree-SHA512: b51608d8791a0d87b6d43388d4173e31123fd62fc3ae346907102504a454133e7b862bf3be784a3610ebd2b816b2aaab8c1d4a2f33a4400cc6cf7bce380e9adf

Author: PastaPastaPasta

.github/workflows/build-depends.yml

@@ -51,23 +51,37 @@ jobs:
             depends/sources
           key: depends-sources-${{ hashFiles('depends/packages/*') }}
           restore-keys: |
-            depends-sources-${{ hashFiles('depends/packages/*') }}
             depends-sources-
 
+      - name: Cache SDKs
+        uses: actions/cache@v4
+        if: inputs.build-target == 'mac'
+        with:
+          path: |
+            depends/SDKs
+          key: depends-sdks-${{ hashFiles('depends/hosts/darwin.mk') }}
+          restore-keys: |
+            depends-sdks-
+
       - name: Restore cached depends
         uses: actions/cache/restore@v4
         id: restore
         with:
           path: |
             depends/built
             depends/${{ steps.setup.outputs.HOST }}
-          key: ${{ runner.os }}-depends-${{ inputs.build-target }}-${{ steps.setup.outputs.DEP_HASH }}-${{ hashFiles('depends/packages/*') }}
+          key: depends-${{ hashFiles('contrib/containers/ci/Dockerfile') }}-${{ inputs.build-target }}-${{ steps.setup.outputs.DEP_HASH }}-${{ hashFiles('depends/packages/*') }}
           restore-keys: |
-            ${{ runner.os }}-depends-${{ inputs.build-target }}-${{ hashFiles('depends/packages/*') }}
-            ${{ runner.os }}-depends-${{ inputs.build-target }}
+            depends-${{ hashFiles('contrib/containers/ci/Dockerfile') }}-${{ inputs.build-target }}-${{ steps.setup.outputs.DEP_HASH }}-
+            depends-${{ hashFiles('contrib/containers/ci/Dockerfile') }}-${{ inputs.build-target }}-
 
       - name: Build depends
-        run: env ${{ steps.setup.outputs.DEP_OPTS }} HOST=${{ steps.setup.outputs.HOST }} make -j$(nproc) -C depends
+        run: |
+          export HOST="${{ steps.setup.outputs.HOST  }}"
+          if [ "${HOST}" = "x86_64-apple-darwin" ]; then
+            ./contrib/containers/guix/scripts/setup-sdk
+          fi
+          env ${{ steps.setup.outputs.DEP_OPTS }} make -j$(nproc) -C depends
 
       - name: Save depends cache
         uses: actions/cache/save@v4

.github/workflows/build-src.yml

@@ -44,6 +44,15 @@ jobs:
           echo "PR_BASE_SHA=${{ github.event.pull_request.base.sha || '' }}" >> $GITHUB_OUTPUT
         shell: bash
 
+      - name: Restore SDKs cache
+        uses: actions/cache/restore@v4
+        if: inputs.build-target == 'mac'
+        with:
+          path: |
+            depends/SDKs
+          key: depends-sdks-${{ hashFiles('depends/hosts/darwin.mk') }}
+          fail-on-cache-miss: true
+
       - name: Restore depends cache
         uses: actions/cache/restore@v4
         with:
@@ -58,11 +67,9 @@ jobs:
         with:
           path: |
             /cache
-          key: ${{ runner.os }}-${{ inputs.build-target }}-${{ github.sha }}
+          key: ccache-${{ hashFiles('contrib/containers/ci/Dockerfile', 'depends/packages/*') }}-${{ inputs.build-target }}-${{ github.sha }}
           restore-keys: |
-            ${{ runner.os }}-${{ inputs.build-target }}-${{ github.sha }}
-            ${{ runner.os }}-${{ inputs.build-target }}-${{ steps.setup.outputs.HOST }}
-            ${{ runner.os }}-${{ inputs.build-target }}
+            ccache-${{ hashFiles('contrib/containers/ci/Dockerfile', 'depends/packages/*') }}-${{ inputs.build-target }}-
 
       - name: Build source and run unit tests
         run: |

.github/workflows/build.yml

@@ -49,6 +49,14 @@ jobs:
       build-target: linux64_nowallet
       container-path: ${{ needs.container.outputs.path }}
 
+  depends-mac:
+    name: x86_64-apple-darwin
+    uses: ./.github/workflows/build-depends.yml
+    needs: [container]
+    with:
+      build-target: mac
+      container-path: ${{ needs.container.outputs.path }}
+
   depends-win64:
     name: x86_64-w64-mingw32
     uses: ./.github/workflows/build-depends.yml
@@ -129,6 +137,15 @@ jobs:
       container-path: ${{ needs.container.outputs.path }}
       depends-key: ${{ needs.depends-linux64.outputs.key }}
 
+  src-mac:
+    name: mac-build
+    uses: ./.github/workflows/build-src.yml
+    needs: [container, depends-mac]
+    with:
+      build-target: mac
+      container-path: ${{ needs.container.outputs.path }}
+      depends-key: ${{ needs.depends-mac.outputs.key }}
+
   src-win64:
     name: win64-build
     uses: ./.github/workflows/build-src.yml

contrib/guix/libexec/build.sh

@@ -73,11 +73,9 @@ unset OBJCPLUS_INCLUDE_PATH
 
 export C_INCLUDE_PATH="${NATIVE_GCC}/include"
 export CPLUS_INCLUDE_PATH="${NATIVE_GCC}/include/c++:${NATIVE_GCC}/include"
-export OBJC_INCLUDE_PATH="${NATIVE_GCC}/include"
-export OBJCPLUS_INCLUDE_PATH="${NATIVE_GCC}/include/c++:${NATIVE_GCC}/include"
 
 case "$HOST" in
-    *darwin*) export LIBRARY_PATH="${NATIVE_GCC}/lib" ;;
+    *darwin*) export LIBRARY_PATH="${NATIVE_GCC}/lib" ;; # Required for qt/qmake
     *mingw*) export LIBRARY_PATH="${NATIVE_GCC}/lib" ;;
     *)
         NATIVE_GCC_STATIC="$(store_path gcc-toolchain static)"
@@ -182,6 +180,14 @@ make -C depends --jobs="$JOBS" HOST="$HOST" \
                                    x86_64_linux_NM=x86_64-linux-gnu-gcc-nm \
                                    x86_64_linux_STRIP=x86_64-linux-gnu-strip
 
+case "$HOST" in
+    *darwin*)
+        # Unset now that Qt is built
+        unset C_INCLUDE_PATH
+        unset CPLUS_INCLUDE_PATH
+        unset LIBRARY_PATH
+        ;;
+esac
 
 ###########################
 # Source Tarball Building #

depends/hosts/darwin.mk

@@ -50,17 +50,11 @@ darwin_STRIP=$(shell $(SHELL) $(.SHELLFLAGS) "command -v llvm-strip")
 #         Disable adhoc codesigning (for now) when using LLVM tooling, to avoid
 #         non-determinism issues with the Identifier field.
 
-darwin_CC=env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH \
-              -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH \
-              -u LIBRARY_PATH \
-              $(clang_prog) --target=$(host) \
+darwin_CC=$(clang_prog) --target=$(host) \
               -isysroot$(OSX_SDK) -nostdlibinc \
               -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks
 
-darwin_CXX=env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH \
-               -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH \
-               -u LIBRARY_PATH \
-               $(clangxx_prog) --target=$(host) \
+darwin_CXX=$(clangxx_prog) --target=$(host) \
                -isysroot$(OSX_SDK) -nostdlibinc \
                -iwithsysroot/usr/include/c++/v1 \
                -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks

depends/packages/qt.mk

@@ -24,6 +24,7 @@ $(package)_patches += memory_resource.patch
 $(package)_patches += clang_18_libpng.patch
 $(package)_patches += utc_from_string_no_optimize.patch
 $(package)_patches += windows_lto.patch
+$(package)_patches += darwin_no_libm.patch
 $(package)_patches += zlib-timebits64.patch
 
 $(package)_qttranslations_file_name=qttranslations-$($(package)_suffix)
@@ -256,6 +257,7 @@ define $(package)_preprocess_cmds
   patch -p1 -i $($(package)_patch_dir)/utc_from_string_no_optimize.patch && \
   patch -p1 -i $($(package)_patch_dir)/guix_cross_lib_path.patch && \
   patch -p1 -i $($(package)_patch_dir)/windows_lto.patch && \
+  patch -p1 -i $($(package)_patch_dir)/darwin_no_libm.patch && \
   patch -p1 -i $($(package)_patch_dir)/zlib-timebits64.patch && \
   mkdir -p qtbase/mkspecs/macx-clang-linux &&\
   cp -f qtbase/mkspecs/macx-clang/qplatformdefs.h qtbase/mkspecs/macx-clang-linux/ &&\

depends/patches/qt/darwin_no_libm.patch

@@ -0,0 +1,17 @@
+build: remove explicit -lm link from qttools
+    
+This causes issues with at least the macOS cross build, and shouldn't
+actually be required anywhere else. GCC with libstdc++ will already get libm.
+
+--- a/qtbase/src/corelib/tools/tools.pri
++++ b/qtbase/src/corelib/tools/tools.pri
+@@ -111,9 +111,6 @@ qtConfig(easingcurve) {
+         tools/qtimeline.cpp
+ }
+ 
+-# Note: libm should be present by default becaue this is C++
+-unix:!macx-icc:!vxworks:!haiku:!integrity:!wasm: LIBS_PRIVATE += -lm
+-
+ TR_EXCLUDE += ../3rdparty/*
+ 
+ # MIPS DSP