Skip to content

Commit 9d484c9

Browse files
committed
Apply review suggestions
1 parent 56d9c20 commit 9d484c9

File tree

6 files changed

+21
-10
lines changed

6 files changed

+21
-10
lines changed

Sources/XCRemoteCache/Commands/SwiftFrontend/SwiftFrontendOrchestrator.swift

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ class CommonSwiftFrontendOrchestrator {
8686
/// the critical section, the code realizes that remote cache cannot be used
8787
/// (in practice - a new file has been added)
8888
/// None of compilation process (so with '-c' args) can continue until the entire emit-module logic finishes
89-
/// Because it is expected to happen no that often and emit-module is usually quite fast, this makes the
89+
/// Because it is expected to happen not that often and emit-module is usually quite fast, this makes the
9090
/// implementation way simpler. If we ever want to optimize it, we should release the lock as early
9191
/// as we know, the remote cache cannot be used. Then all other compilation process (-c) can run
9292
/// in parallel with emit-module

docs/design/SwiftDriverIntegration.md

Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,46 @@
11
## Swift Driver Integration
22

3-
### Pre Swift Driver Integration
3+
### Pre Swift Driver Integration
44

55
Historically (prior to Xcode 14), Swift compilation step was invoked by Xcode as a single external process. Xcode was calling `swiftc` and passing all required parameters (like all input files, output destinations, header paths etc.), and reading its standard output to recognize the status/state of a compilation. Essentially, there were two build systems: "the big one" from Xcode and "small one" by Swift.
66

77
That design was easy to mock in the XCRemoteCache, where the `xcswiftc` wrapper was first inspecting if the cached artifact can be reused (e.g. no new input `.swift` files were added to the list of compilation files) and based on that either continuing with the local compilation (cache miss) or mocking the compilation and existing early (cache hit).
88

9-
![Pre Swift Driver Integration](./../img/pre-driver.png)
10-
9+
<p>
10+
<img src="docs/img/pre-driver.png#gh-light-mode-only">
11+
<img src="docs/img/pre-driver-dark.png#gh-dark-mode-only">
12+
</p>
1113

1214
### Swift Driver Integration Design
1315

1416
With the upgraded design (aka Swift Driver Integration), Xcode splits the work into `n` subprocesses (when `n` is ~CPU), each responsible to compile a subset of files/actions. To align with that, XCRemoteCache meeds to specify a single place to identify if the cached artifact is applicable. `swift-frontend` has been picked for that - process responsible for module emitting. By reviewing Xcode's behavior, it has been found that this process is scheduled very early in the workflow timeline (with some approximation, we could say it is scheduled as a first step) so it seems as best candidate for the "pre-work".
1517

16-
As the same executable `swift-frontend` is invoked multiple times for the same target (e.g. to emit module, multiple batches of compilation etc.), XCRemoteCaches uses a file lock-based synchronization. Each `xcswift-frontend` (the wrapper for `swift-frontend`) tries to acquire a unique lock file. The lock has a name `$LLBUILD_BUILD_ID.lock`, which is unique for each build, placed in the `Intermediate` directory. `xcswift-frontend` process reads its content to find if the "pre-work" from the emit-module has already been done - if not, it releases a lock a gives a way to other processes (presumably the "emit-module") to do the required work. As a lock file is unique per target and a build (it is actually unique per target compilation, placed in `TARGET_TEMP_DIR`), initially the file is empty.
18+
As the same executable `swift-frontend` is invoked multiple times for the same target (e.g. to emit module, multiple batches of compilation etc.), XCRemoteCaches uses a file lock-based synchronization. Each `xcswift-frontend` (the wrapper for `swift-frontend`) tries to acquire a unique lock file. The lock has a name `$LLBUILD_BUILD_ID.lock`, which is unique for each build, placed in the `Intermediate` directory. `xcswift-frontend` process reads its content to find if the "pre-work" from the emit-module has already been done - if not, it releases a lock a gives a way to other processes (presumably the "emit-module") to do the required work. As a lock file is unique per target and a build (it is actually unique per target compilation, placed in `TARGET_TEMP_DIR`), initially the file is empty.
1719

1820
Note the emit module step holds a shared lock for the time of the entire process lifetime, so only once the "pre-work" is finished, all other `xcswift-frontend` processes can continue their job (with either noop or fallbacking to the `swift-frontend` in case a cache miss). Non emit-module steps (compilation steps) acquire a lock only for a very short period - to read the content of that file, thus multiple batches of compilation can run in parallel.
1921

20-
![Pre Swift Driver Integration](./../img/driver.png)
22+
<p>
23+
<img src="docs/img/driver.png#gh-light-mode-only">
24+
<img src="docs/img/driver-dark.png#gh-dark-mode-only">
25+
</p>
2126

2227
<img src="./../img/sample-driver-timeline.png" width="600px">
2328

2429
### Sample timelines
2530

26-
#### Emit Module acquires a lock first (common)
31+
#### Emit Module acquires a lock first (common)
2732

28-
![Swift Driver Integration Sample timeline](./../img/driver-scenario1.png)
33+
<p>
34+
<img src="docs/img/driver-scenario1.png#gh-light-mode-only">
35+
<img src="docs/img/driver-scenario1-dark.png#gh-dark-mode-only">
36+
</p>
2937

30-
#### A compilation step acquires a lock first (uncommon but possible)
38+
#### A compilation step acquires a lock first (uncommon but possible)
3139

32-
![Swift Driver Integration Sample timeline](./../img/driver-scenario2.png)
40+
<p>
41+
<img src="docs/img/driver-scenario2.png#gh-light-mode-only">
42+
<img src="docs/img/driver-scenario2-dark.png#gh-dark-mode-only">
43+
</p>
3344

3445
### Other considerations/open questions
3546

docs/img/driver-dark.png

34.6 KB
Loading

docs/img/driver-scenario1-dark.png

76.3 KB
Loading

docs/img/driver-scenario2-dark.png

61.7 KB
Loading

docs/img/pre-driver-dark.png

28.9 KB
Loading

0 commit comments

Comments
 (0)