[DOC] PDF build has missing Unicode characters

[jfbu]

Bug report

Bug description:

When compiling locally the PDFs (see python/docsbuild-scripts#187 (comment)) for documentation a number of missing characters are reported.

howto-unicode.log:Missing character: There is no ♞ (U+265E) in font [FreeMono.otf]/OT:script=latn
howto-unicode.log:Missing character: There is no ♟ (U+265F) in font [FreeMono.otf]/OT:script=latn
howto-unicode.log:Missing character: There is no 😀 (U+1F600) in font [FreeMono.otf]/OT:script=lat
howto-unicode.log:Missing character: There is no 😉 (U+1F609) in font [FreeMono.otf]/OT:script=lat
library.log:Missing character: There is no ٤ (U+0664) in font [FreeMonoOblique.otf]/OT:scri
library.log:Missing character: There is no ௪ (U+0BEA) in font [FreeMonoOblique.otf]/OT:scri
library.log:Missing character: There is no ౪ (U+0C6A) in font [FreeMonoOblique.otf]/OT:scri
library.log:Missing character: There is no ໔ (U+0ED4) in font [FreeMonoOblique.otf]/OT:scri
tutorial.log:Missing character: There is no 景 (U+666F) in font [FreeMono.otf]/OT:script=latn
tutorial.log:Missing character: There is no 太 (U+592A) in font [FreeMono.otf]/OT:script=latn
tutorial.log:Missing character: There is no 郎 (U+90CE) in font [FreeMono.otf]/OT:script=latn

This means that the respective PDFs are missing the glyphs. The build was done using the French locale, but a previous build in English of the sole library.pdf reported a number of those already (those lines above from library.log).

CPython versions tested on:

3.13

Operating systems tested on:

macOS

[terryjreedy]

On Windows, all the characters appear here in Firefox. Same when I cut and paste into IDLE (though editing would not work). What happens in Safari? What happens building on Windows or Linux?

I am puzzled as the docsbuild issue was closed as fixed before you posted this. What bug do you think exists in cpython? What do you expect to be done?

[AA-Turner]

I am puzzled as the docsbuild issue was closed as fixed before you posted this.

That was for a complete failure to build PDFs, not just missing characters (which probably means we need to use additional fonts).

A

[jfbu]

There is no bug in cpython and the problem does not depend on the platform. Web browsers query the system (or the web I guess when fetching fonts from server) for fonts supporting the needed Unicode, but LaTeX has no equivalent. The closest is a new experimental LuaLaTeX (not XeLaTeX) feature where via some syntax one can provide a fixed limited list of potential fallback fonts, for example for Emoji glyphs. On XeLaTeX the only way short of replacing FreeMono with some Unicode monospace font with more extensive yet Unicode support is to declare to TeX the problematic characters as being "catcode active", then they behave like TeX macros which expand and can be replaced by anything in particular trigger a font change at their location. The issue here is to locate fonts, I can do that on my system but It is time-costly for me to investigate deeper the cpython docs build set-up and how to guarantee it will have the required additional fonts.

The LaTeX community for decades has been very unwilling to change the state of affairs because it was (still is) deemed a typographical crime to do automatic replacement of glyphs when not available in given font.

Besides it always reasons from the point of view of manually edited sources where, using for example babel or polyglossia one usually manually indicates changes of langage for example to supporte CJK characters in an otherwise English document.

(sorry for spelling but I forgot my laptop power cable and my battery is almost empty I had to rush and I can't use this until computer tomorrow)