HTTP(S) Proxy support

[caquino]

Feature Request

Description

Support for an HTTP/HTTPS caching proxy for VMs.

Context

Some of the big pains of maintaining self-hosted runners infrastructure are downloading external tolling using actions like actions/setup-java, actions/setup-go, and all other language actions.

This not only consumes time but is also prone to connectivity issues, which can cause failures on pipelines unrelated to the code being built/tested.

The standard solution for that is to use the tool cache on the base images, but managing base images and keeping cached tooling up to date increases your workload.

Implementing a transparent caching proxy for both HTTP and HTTPS can help with this, just as the hosted registry helps with docker images.

The complex part is handling the HTTPS caching; without it, the cache's effectiveness is almost zero, considering all dependencies are downloaded from HTTPS endpoints, and for that, a local CA is necessary.

I'm documenting this to check if this is a feature that interests the maintainers of fireactions, and if positive, I will see if I can find some time to work on a PR.

Possible Implementation

When the feature is enabled on the host machine during setup, a CA is created. When the fireactions binary starts on the runner, it should talk to the host machine fireactions server to download the CA root certificates and add them to the local system ca-certificates.

The certificates could have been installed directly on the base image, but that increases the operational cost and also issues when the certificates expire.

Having the certificates generated by the fireactions server allows for short-lived certificates and always has valid certificates installed on the VM.

Another necessary part is that, similarly to the registry local caching, a proxy service needs to be installed on the host machine. There are many options for caching proxies, but it looks like the old squid is still the most stable and feature-rich solution for this.

This also allows for more resilient workflows, as the local cache can return stale data if the external service is offline.

Then, the runner configuration needs to contain both the http_proxy and https_proxy configurations pointing to the local cache endpoint.

With this in place, local tooling connects to the proxy using the internal CA, and the proxy connects externally with the proper external certificate, allowing caching to be stored locally on the host machine.