-
Notifications
You must be signed in to change notification settings - Fork 16
Description
The change in #1811 replaced parallel fixturization (= infusing a snippet into a fixture to convert it from human readable but non-compiling code to less human friendly but compiling code) with a serial approach to limit file system spikes.
As it turns out the loading of fixtures is highly inefficient and involves multiple FS calls per snippet. However there is a 1:N relationship between fixtures and snippets, i.e. a single fixture can be used for multiple snippets. That means the number of FS calls to fixturize snippets can be limited to the number of fixtures.
While in the worst case this can still be equal to the number of snippets, in practice it is unlikely.
We can address this inefficiency by loading all fixtures in serial (which will have FS calls), and then execute fixturization of the snippets in parallel (now without FS calls).
Additionally we can look into limiting the parallelism to a constant load.