Skip to content

Commit 9a478e3

Browse files
committed
Clarify make build -j double red herring
During the migration to 3.13 it was suggested to use `make build -j` to automatically speed up the building process of the documentation to use all available cores. This is red herring in two different ways: * `make`'s `-j` flag can be used to parallelise the processing of `make` target dependencies instead of magically apply parallelisation to the underlying commands. In our case the `make build` command has a sequential sequence of dependencies (venv -> setup -> do_build) so no actual parallelisation is achieved when passing `-j`. `sphinx-build` can be parallelised by giving *it* a `-j` flag, which we already do (`-j auto`). In other words, there's nothing to do on our side to speed things up even further. What we *do* have though is a `make` `SPHINX_JOBS` variable, which acts as the argument to `sphinx-build -j`. Using this users can reduce the number of CPUs they want to use for the build process. * Secondly, an empty `make -j` option (i.e., without a number afterwards), contrary to popular belief, runs as many recipes in parallel as possible [1], *with no regards to the number of available cores*. In most scenarios this is fine because the number of dependencies for a given target isn't that great, or because the target instructions are not that heavy, but it can generate difficult-to-understand problems on occasion. For example, if you are building a shared library that consists on 250 files of heavy C++, you'll end up spawning 250 C++ compilers at the same time, which surely will consume all the memory of your computer, causing them to be OOM-killed. Bottom-line is: never use -j without a value. Signed-off-by: Rodrigo Tobar <[email protected]>
1 parent b637d30 commit 9a478e3

File tree

1 file changed

+4
-2
lines changed

1 file changed

+4
-2
lines changed

.overrides/upgrade-python-version.rst

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,9 +48,11 @@ We are currently in branch 3.12, and we want to update the strings from 3.13.
4848

4949
.. note::
5050

51-
You can also run `make build -j` to use more than 1 core (but keep in mind
52-
this is not always faster).
51+
The underlying ``sphinx-build`` command uses all available cores by default.
52+
Use the ``SPHINX_JOBS`` ``make`` variable (defaults to ``auto``)
53+
to specify an explicit amount, e.g. ``make build SPHINX_JOBS=10``.
5354

55+
.. note::
5456
It may fail the build because there may be files
5557
that don't exist anymore in the new branch.
5658
If that's the case, just continue with the steps

0 commit comments

Comments
 (0)