Skip to content

Conversation

gpshead
Copy link
Member

@gpshead gpshead commented Nov 28, 2024

https://github.com/mozilla/sccache from Mozilla is a modern ccache replacement that uses cloud storage such as GHA directly.

If successful, this adds ccache to places we didn't have it before (macOS), and uses GHA storage directly instead of the old ccache based flow of storing a bundle into the GHA cache after the run.

Q: is that an accurate description of how the old ccache flow actually worked?

Windows? it can! That'll take a little more plumbing into its build flow but it should just work.

TODO items

  • look at the dumped cache stats across GHA CI runs on commits below, why are many of them so low when they should've just rebuilt the same thing? what inputs does sccache use as a key? is there a unique path or env var mixing things up? they in theory designed this in part for CI usage so...
  • figure out the debugging story for sccache in CI to understand why it is or isn't caching some stuff.
  • decide which places this is worth enabling vs the older ccache stuff.
  • Q: how do we monitor GHA Cache storage and rate limits for the project?

@gpshead gpshead added skip news skip issue infra CI, GitHub Actions, buildbots, Dependabot, etc. labels Nov 28, 2024
@gpshead gpshead changed the title [testing] Explore using sccache instead of ccache in CI 🦃 Use sccache in CI with GHA storage, mac & wasi included Nov 28, 2024
@eli-schwartz
Copy link
Contributor

If successful, this adds ccache to places we didn't have it before (macOS), and uses GHA storage directly instead of the old ccache based flow of storing a bundle into the GHA cache after the run.

ccache itself should work fine on macOS already (and Windows!) so configuring that directly should be an option too.

look at the dumped cache stats across GHA CI runs on commits below, why are many of them so low when they should've just rebuilt the same thing? what inputs does sccache use as a key? is there a unique path or env var mixing things up? they in theory designed this in part for CI usage so...

I know that sccache does cache on the full path despite that this is fundamentally a pretty bad idea (and renders it totally useless for scenarios such as speeding up Gentoo compilations) so that may very well be the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infra CI, GitHub Actions, buildbots, Dependabot, etc. skip issue skip news

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants