Nix: Only write caches in main branch

For a long time we are having issues with Github Actions cache evictions when
multiple PRs change the nix setup. What happens in that case it that each of
these PRs will create it's own cache entries exceeding 5 GB for the 3 platforms
we require. As the total cache size limit is 10 GB, Github often evicted the
older cases which often is the cache from the main branch.
This will consequently slow down _all_ PRs as they have to build all
dependencies from scratch. The symptom of this is commonly the autogeneration
CI jobs taking 2 hours or more as they have to compile the cross toolchains
from scratch.

This commit fixes that for good. Caches are only written in the main branch,
and, hence, no cache evictions can occur do to PRs.
PRs changing the the nix setup will no longer be faster on second execution,
however, they still benefit from the cache from the main branch as that has
most of the packages they need anyway in most cases.
For example, a PR changing CBMC, will only have to rebuild CBMC, but can use
the cached z3 and HOL-Light.

Signed-off-by: Matthias J. Kannwischer <[email protected]>

Author: mkannwischer

.github/actions/setup-nix/action.yml

@@ -104,9 +104,8 @@ runs:
       if: ${{ env.NIX_CACHE_ENABLED != 1 && inputs.cache == 'true' }}
       continue-on-error: true
       with:
-        primary-key: ${{ steps.nix-post-check.outputs.cache_prefix }}-${{ hashFiles('**/*.nix') }}
-        restore-prefixes-first-match: ${{ steps.nix-post-check.outputs.cache_prefix }}
-        gc-max-store-size-linux: 536870912
+        primary-key: ${{ steps.nix-post-check.outputs.cache_prefix }}
+        gc-max-store-size-linux: 3221225472
         purge: ${{ inputs.save_cache == 'true' }}
         save: ${{ inputs.save_cache == 'true' }}
         purge-prefixes: cache-${{ steps.nix-post-check.outputs.cache_prefix }}

.github/workflows/nix.yml

@@ -65,7 +65,7 @@ jobs:
         with:
           cache: true
           verbose: true
-          save_cache: true
+          save_cache: ${{ github.ref == 'refs/heads/main' }}
           devShell: ci
           gh_token: ${{ secrets.GITHUB_TOKEN }}
           script: |