How to deal with disappear-y packages

[leoluk]

Fedora mirrors tend to remove outdated packages relatively quickly. Is there an established workflow for caching those to a mirror?

[rmohr]

We have a generic tool in kubevirt: https://github.com/kubevirt/project-infra/tree/main/robots/cmd/uploader. It uploads all our dependencies to gce.

In general it is multi-layered:

• the upload tool adds the RPMs to gce
• Inside our CI we have a bazel greenhouse cache which stores them as well, to not always go to GCE

[leoluk]

This obscure downloader feature could be an option to avoid the WORKSPACE rewriting: https://blog.aspect.dev/configuring-bazels-downloader

[rmohr]

This obscure downloader feature could be an option to avoid the WORKSPACE rewriting: https://blog.aspect.dev/configuring-bazels-downloader

That is a nice option. In general I like having them in the workspace, since that reflects where they are, and allows easy checking based on the WORKSPACE that dependencies are indeed mirrored.

[manuelnaranjo]

@leoluk the download is something we use extensively at booking to avoid our CI hitting external repos that may not be as reliable as ours, it has the advantage of not having to deal with rewritting URLs into your workspace and make it easier to tell what the source of the file is. We have a jfrog transparent proxy that we get a few URLs through.

I wonder if also part of the problem you're facing is due to the fact that the current workspace approach to download RPMs is a bit hard to keep up to date