"make clean html" still searches for "now outdated files" = inefficient

[Dutchtallman]

Subject: "make clean html" still searches for "now outdated files" = inefficient

Problem

When you issue the command:
make html
on any project, you will see in the output:
looking for now-outdated files...
Sometimes, some unchanged files are selected for reprocessing.

When you issue the command:
make clean html
on any project, you'll see in the output that sphinx is also " ...looking for outdated files ... ", but all files are already selected for being reprocessed due to the "clean" option and no additional files are selected.

"Looking for outdated files " can take a long time, especially for larger projects and is an unnecessary step in case of "clean" being used. It's an inefficiency.

Proposed solution

Do not "look for outdated files" in case a "clean" documentation is generated.

Environment info

• OS: Kernel: 4.15.0-43-generic x86_64 bits
• Distro: Kubuntu 18.04.1 LTS
• Python version: 2.7.15rc1
• Sphinx version: Sphinx v1.8.1

[tk0miya]

Against the message, Sphinx assigns section and figure numbers on that action. Therefore, it takes long for large document enabled numbered toctree or numfig. I know this implementation is a bit hacky. So it would be refactored.
Anyway, total processing time will not be changed.

[Dutchtallman]

Aha
Perhaps rename the action into "assign numbers to sections, figures, etc ..." ? instead of "looking for now-outdated files..."

[jakobandersen]

Note that make clean html is purely on the Makefile level, and is the same as make clean followed by make html, so Sphinx doesn't know it should do a full rebuild. I guess the -a and -E options for sphinx-build are for doing a full read and write? Though I don't know if they make a difference in execution time compared to make clean html (I expect not).

[tk0miya]

All make clean, -E and -a options have different purposes.

• make clean: Removes all built files. It means next build will start with brand new state. make clean html is same as make clean && make html.
• sphinx-build -E: Ignores last built files. It also starts with brand new state. But, it does not remove old files. So some files might be remained if document structures changed.
• sphinx-build -a: Re-writes target files. It uses last built files if available. So it sometimes skips reading phase.

I suppose the execution time of make clean html and sphinx-build -E are almost same. But sphinx-build -a might be fast because it's a kind of incremental build.

As @jakobandersen said, make works as a wrapper of sphinx-build command. So sphinx-build doesn't have a proper way to know the old files are removed or not.

[tk0miya]

Perhaps rename the action into "assign numbers to sections, figures, etc ..." ? instead of "looking for now-outdated files..."

Unfortunately, these numbering are doing as a part of "looking for outdated files". It is strongly coupled. On incremental build, sometimes numbers of sections or figures are shifted by changes of document. Sphinx uses it to look for outdated files.
To change the message, we have to separate the numbering process with looking for outdated process. I suppose it would be hard work. But the benefit for the change is only better message... -0 for the change.

[marc-hb]

For what it's worth I found that numfig slows down BOTH the incremental build and build from scratch in the zephyr project by a factor of ... 5-6 times!!

• drivers: pinctrl: new state based API proposal zephyrproject-rtos/zephyr#37572 (comment)

I've disabled it locally.