Package download failure not categorized as spurious

[RalfJung]

This run had the following two regressions:

[INFO] [stderr] error: failed to download `aho-corasick v1.1.3`
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   unable to get packages from source
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   config.json not found in registry

[INFO] [stderr] error: failed to download `itoa v1.0.10`
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   unable to get packages from source
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   config.json not found in registry

Those should both be categorized as spurious.

[Skgland]

I think #788 should fix this and rust-lang/rustwide#98 should reduce false positives
Nope, this error doesn't appear to be in the prepare phase so those won't help.

Weird I would have expected that to already fail during prepare maybe there was a TOCTOU race condition.

[RalfJung]

Not sure how often this happens, but https://crater-reports.s3.amazonaws.com/pr-147185/index.html also had one of these:

[INFO] [stderr] error: failed to download `anyhow v1.0.64`
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   unable to get packages from source
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   config.json not found in registry

[Skgland]

Thinking more about this, the only registry used by crater/rustwide is crates.io and that should always have a config.json¹² (it's not even a file that would be regularly updated so TOCTOU doesn't make sense after all). Makes me think there might be a network error underneath.

Footnotes

1. https://index.crates.io/config.json ↩

2. https://github.com/rust-lang/crates.io-index/blob/master/config.json ↩

[RalfJung]

Yeah I assume this is a network error.

[Skgland]

Looking at the logs I don't think this can be a network error as the command is run withe the --frozen flag (implying --offline) so no network requests should be made (also doesn't crater run without network access anyways during the test phase?).

So looking at the code at https://github.com/rust-lang/cargo/blob/79df5f4d52c8e8e30be353be20d4572c8653fa69/src/cargo/sources/registry/http_remote.rs#L505-L519 I would expect this to be one of those two error conditions resulting in the error we see being emitted at https://github.com/rust-lang/cargo/blob/79df5f4d52c8e8e30be353be20d4572c8653fa69/src/cargo/sources/registry/http_remote.rs#L439.

Could it be that between the prepare phase an the actual test a disk space clean-up ran and removed the cache?