Support Remote Cache for golangci-lint

[ryancurrah]

Welcome

• Yes, I've searched similar issues on GitHub and didn't find any.
• I agree to follow this project's Code of Conduct

Your feature request related to a problem? Please describe

Yes, this feature request is related to the problem of long linting times, especially in large codebases or CI environments where there are multiple runners/agents.

Currently, golangci-lint lacks a mechanism to cache results remotely, which could help in reducing the overall time taken for linting when running it repeatedly over the same or similar code. This lack of caching increases build times and resource usage, especially when dealing with distributed and large build environments.

Describe the solution you'd like

I propose introducing a feature in golangci-lint to support remote caching by abstracting the cache mechanism. Similar to how Go has implemented the GOCACHEPROG environment variable, which allows a custom cache implementation via a child process that communicates using JSON over stdin/stdout, golangci-lint could introduce a GOLANGCICACHEPROG environment variable. This would allow users to specify a binary responsible for handling cache operations, thereby enabling remote caching similar to https://go-review.googlesource.com/c/go/+/486715.

Describe alternatives you've considered

Persisting cache on individual agents. In our environment, multiple agents are used, and they may not always have an up-to-date cache, leading to inconsistencies and longer build times.

Additional context

Willing to implement this myself if it is deemed an acceptable proposal. Remote caching has proven for us to reduce build times in Go projects. By using a similar approach in golangci-lint, users could experience similar reductions in linting times, especially in CI/CD environments or large projects when linting a lot of code. The ability to cache results remotely would lead to more efficient use of resources and faster feedback loops during builds.

Supporter

• I am a sponsor through GitHub or OpenCollective

[ryancurrah]

Note this feature relies on golang/go#64876

[ldez]

Just to clarify why I put no-decision: I'm working on a re-sync of the internal code related to cache.

It requires a lot of time to ensure it is non-breaking and provides a reviewable PR.

My priority is maintainability, so the new feature (if not added by the sync) will be evaluated after the sync because this a really really really long and complex job to just re-sync the code.

The changes in the original code have never been documented, I already added a "summary" but this is not enough to work quickly on this.
I don't want (because I'm not able to do it) to follow 2 things on the same complex topic at the same time.

[ryancurrah]

Did not realize you were refactoring or reimplementing the caching behaviour. Will wait for that and then we can see how to extend it for remote cache.

[ldez]

@ryancurrah I updated the internals inside #5098.

As a side effect, the programmatic cache is supported but I didn't test it.

If you plan to create a cache implementation, I'm interested in playing with it or contributing.