Skip to content

Returning early pipelined .rmeta like rustc #2496

@kornelski

Description

@kornelski

Cargo<>rustc integration supports pipelining. The compiler can write only .rmeta file and emit a JSON message indicating that this file is ready, before anything else is compiled. This allows Cargo to increase parallelism of the compilation.

sccache could take advantage of this too. A network-based cache is latency-sensitive. Currently sccache can't respond to Cargo at all until the entire cache entry is downloaded and extracted. With a bit of async magic, CacheRead could be extended to wait only as long as needed to get stdout and .rmeta, and emit these first. This would reduce latency, and further increase concurrency, hiding more of cache latency.

I'm interested in implementing this if that's a change you'd accept.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions