You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
generator: Use a streaming pipeline to unpack host Swift toolchain (#148)
Currently, unpacking the host Swift toolchain happens in several steps
connected by temporary files. Switching to a single concurrent pipeline
reduces the time taken to extract the toolchain from around 30s to under
6s on my machine.
### Changes
The `xar` manual page says that it is no longer under development:
> DEPRECATION WARNING
> xar is no longer under active development by Apple. Clients of
> xar should pursue alternative archive formats.
On Darwin, `tar` is able to unpack `xar` archives. Unlike `xar`, it
can also extract a subset of the files from the archive and stream them
to standard output. This lets us extract the Swift toolchain in
a single pipeline, without writing temporary files to disk.
The first `tar` process unpacks the .pkg file and extracts the payload,
writing it out on standard output. The second `tar` process can now
start in parallel instead of waiting for the payload to be fully unpacked
to disk. This saves a couple of seconds.
`tar` is also able to extract gzipped cpio archives. This lets
us also extract the package payload as part of the same pipeline.
Switching to `tar` also lets us use `--include` and `--exclude`
patterns to control which files are extracted from the payload,
instead of extracting the entire archive and copying out only the
files we need. On my machine, the `cat | gunzip | cpio` pipeline
takes about 27 seconds to unpack the entire archive; `tar`, without
`--include` or `--exclude` patterns, takes about 8 seconds to extract
everything.
### Result
Overall, this PR reduces the time to build an SDK from Debian packages
on an M3 MacBook Air from 1m30s to about 53s, excluding download time.
The SDKs generated by the old and new methods are identical.
0 commit comments