Feature request: open source script that mirrors the Julia Computing storage servers and serves them statically

[DilumAluthge]

I do think that we should have an open source script that mirrors the Julia Computing storage servers and serves them statically. That will act as a backup in case the JC storage servers go down.

Originally posted by @StefanKarpinski in JuliaRegistries/General#16777 (comment)

[johnnychen94]

https://github.com/johnnychen94/StorageMirrorServer.jl is used to build the mirror storage server in BFSU, TUNA and SJTUG.

Unlike the previous gen_static.jl script, this one doesn't build any tarball locally, it simply downloads the resources from the existing upstream.

[StefanKarpinski]

I'm not sure this is the right place for this issue since it's not really a PkgServer feature, but I'm not sure where is the right place, so this will do. The basic idea is for storage servers to (optionally) add /packages and /artifacts end-points that list all the package versions and artifacts that they know about in the same way that /registries lists the current heads of all registries that they serve (one resource per line). Then you can mirror a storage server simply by fetching those lists and fetching each item (and there's never any eed to fetch items that you already have).