add a bit more on parallel compilation

Author: mark-i-m

src/SUMMARY.md

@@ -44,6 +44,7 @@
         - [Profiling Queries](./queries/profiling.md)
         - [Salsa](./salsa.md)
     - [Memory Management in Rustc](./memory.md)
+    - [Parallel Compilation](./parallel-rustc.md)
 
 - [Part 3: Source Code Representations](./part-3-intro.md)
     - [The Rustc Driver and Interface](./rustc-driver.md)

src/parallel-rustc.md

@@ -0,0 +1,45 @@
+# Parallel Compilation
+
+Most of the compiler is not parallel. This represents an opportunity for
+improving compiler performance. Much effort has been put into parallelizing
+`rustc`, but it is still pretty early days for this work. There is a lot of
+design and correctness work that needs to be done.
+
+One can try out the current parallel compiler work by enabling it in the
+`config.toml`.
+
+There are a few basic ideas in this effort:
+
+- There are a lot of loops in the compiler that just iterate over all items in
+  a crate. These can possibly be parallelized.
+- We can use (a custom fork of) [`rayon`] to run tasks in parallel. The custom
+  fork allows the execution of DAGs of tasks, not just trees.
+- There are currently a lot of global data structures that need to be made
+  thread-safe. A key strategy here has been converting interior-mutable
+  data-structures (e.g. `Cell`) into their thread-safe siblings (e.g. `Mutex`).
+
+[`rayon`]: https://crates.io/rayon
+
+As of this writing, much of this effort is on hold due to lack of manpower. We
+have a working prototype with promising performance gains in many cases.
+However, there are two blockers:
+
+- It's not clear what invariants need to be upheld that might not hold in the
+  face of concurrency. An auditing effort was underway, but seems to have
+  stalled at some point.
+
+- There is a lot of lock contention, which actually degrades performance as the
+  number of threads increases beyond 4.
+
+Here are some resources that can used to learn more (note that some of them are
+a bit out of date):
+
+- [This IRLO thread by Zoxc, when of the pioneers of the effort][irlo0]
+- [This list of interior mutability in the compiler by nikomatsakis][imlist]
+- [This IRLO thread by alexchricton about performance][irlo1]
+- [This tracking issue][tracking]
+
+[irlo0]: https://internals.rust-lang.org/t/parallelizing-rustc-using-rayon/6606
+[imlist]: https://github.com/nikomatsakis/rustc-parallelization/blob/master/interior-mutability-list.md
+[irlo1]: https://internals.rust-lang.org/t/help-test-parallel-rustc/11503
+[tracking]: https://github.com/rust-lang/rust/issues/48685