Improve performance on big projects

[wjt]

On the Threadbare repo this workflow routinely takes nearly 2 minutes now:

https://github.com/endlessm/threadbare/actions/workflows/github-pages.yml

I think this is because the game is now pretty large: 29MB of assets per branch, resulting github-pages artifact is nearly 400 MB.

[wjt]

In #31 I've added timestamps.

In local development one of the slowest parts of the cycle is repeatedly downloading the same build artifacts. I have a WIP branch that uses https://requests-cache.readthedocs.io/ to cache HTTP responses which with a warm cache reduces the time to build https://github.com/wjt/github-pages-branches-test locally to under 2 seconds. (Without a cache it takes 14 seconds.)

But it's surprisingly fiddly to wire up to the GitHub Actions cache mechanism because the cache key needs to be derived from the state of the whole repo: if the cache already exists with a specific name it won't be re-uploaded. I think we can do this by generating a cache key in the programme based on the assets that are in the resulting site.

[wjt]

#32 introduced the cache for local development.

I tried the approach of keying the cache based on the assets but that also doesn't work. The actions YAML looks like this:

     - name: Save HTTP request cache
       uses: actions/cache/save@v4
       with:
         path: http_cache.sqlite
         # The cache key should actually be the state of the repository. This is
         # hard to calculate, so instead we hash the generated site.
         key: http_cache-${{ hashFiles('_site/**') }}

plus a corresponding change to the script to build the site into _site in the working directory rather than a fresh temporary directory because hashFiles can only operate on files in the current working directory.

But then in the case where that cache key already exists, you get a warning when you try to save it. I don't think persisting the cache is worth the spurious warnings.

As I wrote this I realised that there is actually a way to deal with this in many (but not all) cases. actions/cache/restore gives you the primary key for the cache entry that was restored in its output. https://github.com/actions/cache/tree/main/restore#outputs

So in principle we could calculate the cache key that we would save, and then skip the save step if it's the same key that was just restored. I can think of cases where this would fail:

• Publish the project (cache key A)
• Add a new branch (cache key B)
• Delete that branch. Now you'd expect the cache to be A, but B would have been restored because it's more recent, so you'd once again try to push the "new" cache as A and get a warning.

I think I'll pause further work here. Probably the best way to speed this up is to do less, i.e. make the game smaller.