Incremental / partial caching

[yacinehmito]

Hello! 👋

I'd to start a conversation about incremental / partial caching and whether it's something the project has considered.

The problem

Moon uses task-level caching that's all-or-nothing: either the input hash matches exactly (cache hit) or you rebuild from scratch (cache miss). This works well for small packages, but becomes painful when working with large packages.

Example: I have a package with 500 test files. If I change one file, I have to re-run all 500 tests because the task hash changed. The current workaround is using VCS integration to identify affected tests (moon run --affected). It works OK in CI, but locally this requires being disciplined with one's git workflow. I'd rather have something more robust that works regardless of how git is used.

Many tools support their own incremental caching at the file level:

• TypeScript: tsBuildInfo files track which files changed and only recompile those
• ESLint: --cache flag stores lint results per file in .eslintcache
• Vitest: plugins like vite-plugin-vitest-cache (https://github.com/raegen/vite-plugin-vitest-cache) cache test results per file, skipping tests whose dependencies haven't changed

The problem is that moon's task-level caching can't leverage these. On a cache miss, these incremental artifacts aren't restored, so tools start from scratch.

What I'd like

The ability to cache and restore task outputs at a finer granularity, so that changing one file in a large package doesn't invalidate the entire task cache. Tool-level incremental caches (like tsBuildInfo, .eslintcache, or test result caches) can be preserved and reused across runs

Questions

1. Has this been discussed before? I couldn't find existing issues or discussions on the topic.
2. Is this something that's been considered for the roadmap?
3. Are there architectural constraints that would make this difficult?

I understand this would be a significant change to how caching works. Curious to hear your thoughts/

[milesj]

@yacinehmito What does your setup look like?

We don't do anything with files like tsbuildinfo/.eslintcache/etc, so I'm curious how moon is involved here. Those should persist between runs and different hashes.

The only time files are removed, is if they are defined as outputs and there's a cache hit, as we remove the previous state: https://github.com/moonrepo/moon/blob/master/crates/task-runner/src/output_hydrater.rs#L116 You can simply not have them as outputs.