Capture per-crate build timings from cargo's --timings output#825
Capture per-crate build timings from cargo's --timings output#825oriongonza wants to merge 1 commit intorust-lang:masterfrom
Conversation
cb63c57 to
064db93
Compare
|
Are the majority of the changes here (in particular the comments) generated with some LLM tool? I don't want to accept those changes at least (they aren't related to this PR and I'm not a fan of generated comments, that feels broadly unhelpful and would take a good bit of review bandwidth). I haven't looked at the actual changes but I'm pretty sceptical we would want to store any extra information into crater's database. The approaches I've discussed with others broadly speaking leave the output in the files sent to S3 and analysis external; I'd prefer to keep it out of the crater codebase until we are least have evidence there's enough signal to noise ratio for it to be valuable. |
Rebased. Feel free to leave the other PR open, no hurry. I didn't think that it would be a controversial change.
|
|
Introduces a timings module with a visitor pattern for optionally intercepting cargo timing-info messages during builds. Timings are captured only when capture_timings is enabled in config, the toolchain is nightly/CI, and running in local/CLI mode (not agent mode).
064db93 to
a0e346d
Compare
Summary
Closes #812.
Depends on #824.
When
capture_timings = trueinconfig.tomland the toolchain is nightly, crater now passes--timingsto cargo and captures the per-compilation-unittiming-infoJSON messages. Each entry records the package, target, mode, duration, and rmeta time. The data is stored in abuild_timingsSQLite table keyed by (experiment, crate, toolchain).This gives crater the infrastructure to collect rough per-crate build durations alongside every run, and discussed on Zulip (#t-infra > crater get rough perf numbers). The data is not surfaced in reports yet, so future work is needed.
Motivation
Several use cases came up in the discussion:
Crater's environment is noisy (oversubscribed machines, non-deterministic build order, cache effects), so these numbers are rough, but rough timings are better than no timings :D