Make docs build less platform-dependent

[Ickaser]

This should make it easier to build docs locally cross-platform, making explicit a Ghostscript_jll dependency that wasn't obvious in #795 . Depends on korsbo/Latexify.jl#344 getting merged and released (hi @gustaphe), and Latexify compat should get bumped accordingly in docs/project.toml.

Existing caveats:

• Still depends on system having the FreeSerif and FreeMono fonts installed. The default font in tectonic for Latexify.render was not sufficiently clear for my tastes. Fortunately, these fonts might be installed by default by some unix packages, and if not can be installed for free with sudo apt install fonts-freefont-ttf, or from places online it seems.
• Until Drop UnitfulLatexify as a dependency for v1 JuliaPlots/Plots.jl#5174 merges, the docs can only be built once per Julia session. (The docs build loads Plots, which loads UnitfulLatexify, which seems to overwrite some of the Latexify recipes and generate incorrect LaTeX on the next build.)

That second can possibly be assuaged (for the meantime) by adjusting docs/make.jl to check if the images already exist:

if !isfile((@__DIR__) * "/src/assets/latex-examples.png") || !isfile((@__DIR__) * "/src/assets/latex-allunits.png")
    @info "Generating latex images for documentation"
    include("generate_latex_images.jl")
end

where currently the latex images are generated either way.

[giordano]

This has to be fixed upstream: korsbo/Latexify.jl#343. Won't do much here.

[Ickaser]

Once the fix is available upstream, all this PR does is remove apt install ghostscript from the docs CI.

[sostock]

Still depends on system having the FreeSerif and FreeMono fonts installed. The default font in tectonic for Latexify.render was not sufficiently clear for my tastes. Fortunately, these fonts might be installed by default by some unix packages, and if not can be installed for free with sudo apt install fonts-freefont-ttf, or from places online it seems.

These fonts are only used in the second table. The first table uses the standard fonts. Is there a specific glyph that is not “sufficiently clear” in the standard font and that only appears in the second table? IMO, having to install a font to build the docs is not ideal, especially since it does not error when the font is missing, but silently generates the table with the header and the first column just empty.

[Ickaser]

Still depends on system having the FreeSerif and FreeMono fonts installed. The default font in tectonic for Latexify.render was not sufficiently clear for my tastes. Fortunately, these fonts might be installed by default by some unix packages, and if not can be installed for free with sudo apt install fonts-freefont-ttf, or from places online it seems.

These fonts are only used in the second table. The first table uses the standard fonts. Is there a specific glyph that is not “sufficiently clear” in the standard font and that only appears in the second table? IMO, having to install a font to build the docs is not ideal, especially since it does not error when the font is missing, but silently generates the table with the header and the first column just empty.

I couldn't remember specifically what it was, but the default monospaced font has no glyph at all for pi:

We could use Courier New instead (which is probably available by default on Windows and Mac), but I had a hard time ascertaining whether that would be available on the Ubuntu used by a GitHub runner (or how to conveniently install it).

(Courier doesn't look the greatest either:

)

[Ickaser]

It looks like, as far as font for this LaTeX render, the options are:

1. Let the pi glyph be absent in the official online documentation (by using default monospace font)
2. Use Courier New, which Win/Mac users will probably have but has to be installed for CI
3. Use e.g. LiberationMono, which is installed by default on Ubuntu (for CI) but probably not on Win/Mac
4. Use another font (e.g. FreeMono), which probably everyone will have to install

I am happy to do whichever seems best, but lean probably away from option 1.

[sostock]

We could use fontconfig to get the name of an installed monospace font (but I have not checked whether this font name can then be used to render the tables):

julia> using Fontconfig: format, Pattern

julia> format(match(Pattern("monospace")), "%{family}")
"Noto Sans Mono"

It should even be possible to check whether the font has a π glyph (U+03C0), by checking whether 3c0 is in one of the ranges in this list:

julia> format(match(Pattern("monospace")), "%{charset}")
"20-7e a0-ac ae-377 37a-37f 384-38a 38c 38e-3a1 3a3-3e1 3f0-52f 10fb 1ab0-1ac0 1ac5 1ac7-1ace 1c80-1c88 1d00-1df9 1dfb-1f15 1f18-1f1d 1f20-1f45 1f48-1f4d 1f50-1f57 1f59 1f5b 1f5d 1f5f-1f7d 1f80-1fb4 1fb6-1fc4 1fc6-1fd3 1fd6-1fdb 1fdd-1fef 1ff2-1ff4 1ff6-1ffe 2000-2064 2066-2" ⋯ 504 bytes ⋯ "2736 2758-275a 27d5-27d7 27dc 27e6-27eb 27f5-27f6 2987-2988 29a3 29b8 2a00 2a05-2a06 2c60-2c7f 2de0-2e5d a640-a69f a700-a7ca a7d0-a7d1 a7d3 a7d5-a7d9 a7f2-a7ff a92e ab30-ab6b fe00 fe20-fe2f feff ff5b ff5d fffc-fffd 10780-10785 10787-107b0 107b2-107ba 1df00-1df1e 1f67c-1f67f"

Another option would be to render the individual cells in LaTeX and put them into a markdown table in the docs (the first column and the header would not be rendered by LaTeX, thus no font problems). But mixing the non-LaTeX text and the rendered PNGs would lead to inconsistent font sizes (depending on screen DPI).

[Ickaser]

Using Fontconfig turns out to be simple and easy and handle this perfectly--match(Pattern(spacing=100, charset=0x3c0)) finds a monospaced font with the π glyph. And the upstream Latexify change is merged and released, so I think this is ready to go if it looks alright.

[giordano]

Thanks for taking the time to improve this! Some comments:

• overall this seems to be a major improvement compared to previous setup
• while we're making platform-independent improvements, wrapping

  Unitful.jl/docs/make.jl

  Lines 3 to 33 in 243efcb

  	@info "Generating latex images for documentation"
  	include("generate_latex_images.jl")
  	DocMeta.setdocmeta!(Unitful, :DocTestSetup, :(using Unitful))
  	makedocs(
  	sitename = "Unitful.jl",
  	format = Documenter.HTML(prettyurls = get(ENV, "CI", nothing) == "true"),
  	warnonly = [:missing_docs],
  	modules = [Unitful],
  	workdir = joinpath(@__DIR__, ".."),
  	pages = [
  	"Home" => "index.md"
  	"Highlighted features" => "highlights.md"
  	"Types" => "types.md"
  	"Defining new units" => "newunits.md"
  	"Conversion/promotion" => "conversion.md"
  	"Manipulating units" => "manipulations.md"
  	"How units are displayed" => "display.md"
  	"Logarithmic scales" => "logarithm.md"
  	"Temperature scales" => "temperature.md"
  	"Interoperability with `Dates`" => "dates.md"
  	"Latexifying units" => "latexify.md"
  	"Extending Unitful" => "extending.md"
  	"Troubleshooting" => "trouble.md"
  	"Pre-defined dimensions, units, and constants" => "defaultunits.md"
  	"License" => "LICENSE.md"
  	]
  	)
  	deploydocs(repo = "github.com/JuliaPhysics/Unitful.jl.git")

  with

  withenv("UNITFUL_FANCY_EXPONENTS" => "false") do
      # ...
  end

  would make doctests pass on macOS
• with the withenv above, I can successfully build the docs once in a new session, but an attempt at rebuilding the docs in the same session fails with

  julia> include("make.jl")
  [ Info: Generating latex images for documentation
  warning: accessing absolute path `/dev/null`; build may not be reproducible in other environments
  warning: accessing absolute path `/System/Library/Fonts/Supplemental/Andale Mono.ttf`; build may not be reproducible in other environments
  error: main.tex:23: Extra }, or forgotten \endgroup
  error: halted on potentially-recoverable error as specified
  ERROR: LoadError: an error occured while rendering LaTeX:
  	Extra }, or forgotten \endgroup.
  	l.23 ...b+\qtylist1;2;4}{\meter}+ & \qtylist1;2;4}
  Check the log file at /var/folders/v2/hmy3kzgj4tb3xsy8qkltxd0r0000gn/T/jl_zruriq.log for more information
  Stacktrace:
     [1] pipeline_error
       @ ./process.jl:602 [inlined]
     [2] run(::Base.CmdRedirect; wait::Bool)
       @ Base ./process.jl:517
     [3] run(::Base.CmdRedirect)
       @ Base ./process.jl:514
     [4] (::Latexify.var"#172#173"{String, Bool, String, @Kwargs{packages::Vector{String}, documentclass::String}, LaTeXString})()
       @ Latexify ~/.julia/packages/Latexify/IJYMW/src/utils.jl:51
     [5] cd(f::Latexify.var"#172#173"{String, Bool, String, @Kwargs{packages::Vector{String}, documentclass::String}, LaTeXString}, dir::String)
       @ Base.Filesystem ./file.jl:112
     [6] #170
       @ ~/.julia/packages/Latexify/IJYMW/src/utils.jl:47 [inlined]
     [7] mktempdir(fn::Latexify.var"#170#171"{Bool, String, @Kwargs{packages::Vector{String}, documentclass::String}, LaTeXString, String}, parent::String; prefix::String)
       @ Base.Filesystem ./file.jl:899
   ┌ [8] mktempdir
   │   @ ./file.jl:895 [inlined]
   ╰──── repeated 2 times
    [10] _compile(s::LaTeXString, cmd::Cmd, ext::String; debug::Bool, name::String, open::Bool, use_tectonic::Bool, kw::@Kwargs{packages::Vector{String}, documentclass::String})
       @ Latexify ~/.julia/packages/Latexify/IJYMW/src/utils.jl:46
    [11] _compile
       @ ~/.julia/packages/Latexify/IJYMW/src/utils.jl:37 [inlined]
    [12] render(s::LaTeXString, ::MIME{…}; use_tectonic::Bool, tectonic_flags::Cmd, lualatex_flags::Cmd, kw::@Kwargs{…})
       @ TectonicExt ~/.julia/packages/Latexify/IJYMW/ext/TectonicExt.jl:8
    [13] render
       @ ~/.julia/packages/Latexify/IJYMW/ext/TectonicExt.jl:7 [inlined]
    [14] (::Latexify.var"#182#183"{Bool, Symbol, String, Cmd, Cmd, @Kwargs{…}, LaTeXString, String})(aux_name::String, ::IOStream)
       @ Latexify ~/.julia/packages/Latexify/IJYMW/src/utils.jl:175
    [15] mktemp(fn::Latexify.var"#182#183"{Bool, Symbol, String, Cmd, Cmd, @Kwargs{use_tectonic::Bool, packages::Vector{…}, documentclass::String}, LaTeXString, String}, parent::String)
       @ Base.Filesystem ./file.jl:870
    [16] mktemp
       @ ./file.jl:868 [inlined]
    [17] render(s::LaTeXString, mime::MIME{…}; debug::Bool, convert::Symbol, name::String, callshow::Bool, open::Bool, dpi::Int64, ghostscript_flags::Cmd, dvipng_flags::Cmd, kw::@Kwargs{…})
       @ Latexify ~/.julia/packages/Latexify/IJYMW/src/utils.jl:161
    [18] kwcall(::@NamedTuple{use_tectonic::Bool, name::String, packages::Vector{String}, documentclass::String}, ::typeof(render), s::LaTeXString, mime::MIME{Symbol("image/png")})
       @ Latexify ~/.julia/packages/Latexify/IJYMW/src/utils.jl:148
    [19] top-level scope
       @ ~/.julia/dev/Unitful/docs/generate_latex_images.jl:35
    [20] include(mapexpr::Function, mod::Module, _path::String)
       @ Base ./Base.jl:309
    [21] IncludeInto
       @ ./Base.jl:310 [inlined]
    [22] (::var"#26#27")()
       @ Main ~/.julia/dev/Unitful/docs/make.jl:5
    [23] withenv(f::var"#26#27", keyvals::Pair{String, String})
       @ Base ./env.jl:265
    [24] top-level scope
       @ ~/.julia/dev/Unitful/docs/make.jl:3
    [25] include(mapexpr::Function, mod::Module, _path::String)
       @ Base ./Base.jl:309
    [26] top-level scope
       @ REPL[1]:1
  in expression starting at /Users/mose/.julia/dev/Unitful/docs/generate_latex_images.jl:35
  in expression starting at /Users/mose/.julia/dev/Unitful/docs/make.jl:3
  Some type information was truncated. Use `show(err)` to see complete types.
  

  Any idea of what's going on?

[Ickaser]

Thanks for taking the time to improve this! Some comments:

* overall this seems to be a major improvement compared to previous setup

* while we're making platform-independent improvements, wrapping https://github.com/JuliaPhysics/Unitful.jl/blob/243efcb0f1da3496218250997e68da5b167c87e0/docs/make.jl#L3-L33
   with
  ```julia
  withenv("UNITFUL_FANCY_EXPONENTS" => "false") do
      # ...
  end
  ```
    
  would make doctests pass on macOS

I like that, would be happy to make that change.

* with the `withenv` above, I can successfully build the docs once in a new session, but an attempt at rebuilding the docs in the same session fails

  Any idea of what's going on?

That error on second build is a consequence of Plots still loading UnitfulLatexify as a dependency, which I can fix in JuliaPlots/Plots.jl#5174 once Unitful has a new release. (Interlinking dependencies have been a bit of a bugbear here.)

[giordano]

I can confirm I can successfully build the docs multiple times if I ]add https://github.com/Ickaser/Plots.jl#drop-unitful-latexify-dep in the environment! One minor annoyance is that during the build I get windows displaying latex-allunits.png and latex-examples.png, I'm not sure what's causing those windows to appear. It's not blocking, but it's a small nuisance during the build.

If you add UNITFUL_FANCY_EXPONENTS=false as suggested above, I'm happy with this PR.

[sostock]

That error on second build is a consequence of Plots still loading UnitfulLatexify as a dependency, which I can fix in JuliaPlots/Plots.jl#5174 once Unitful has a new release. (Interlinking dependencies have been a bit of a bugbear here.)

Should we add Plots to our [compat] section (does that work when Plots is not a dependency?), so that users cannot install the new Unitful with an old Plots version?

[giordano]

(does that work when Plots is not a dependency?)

I don't think so.

[sostock]

Then I think we should retroactively mark UnitfulLatexify (all versions) as incompatible with Unitful ≥ 1.25 in the general registry.

[Ickaser]

Then I think we should retroactively mark UnitfulLatexify (all versions) as incompatible with Unitful ≥ 1.25 in the general registry.

I intend to have new version of Plots require Unitful ≥ 1.25 and drop UnitfulLatexify, and UnitfulLatexify is now marked as having an upper bound of Unitful 1.23; would that be sufficient?

[sostock]

No, because it would not prevent users from installing an old Plots, old UnitfulLatexify und new Unitful version. If they have those three packages in their Project.toml and run Pkg.update, that could happen since the resolver will make sure that compatible versions of all packages are installed. (It also could resolve to old Plots, old Unitful, new UnitfulLatexify, I don't know.)

I don't know whether that is a real problem, since some users may just have Unitful and Plots in their Project.toml (with UnitfulLatexify being installed as weakdep of Plots). But if they need Latexify functionality independent from plotting, they might have UnitfulLatexify in their Project.toml.

Another option might be to release a new UnitfulLatexify version that does nothing (or only prints a warning that it is no longer required) and is only compatible with upcoming Unitful versions. Then users who update an existing project would get the newest versions of everything. But it would not stop anyone from manually installing new Unitful and old UnitfulLatexify. If we want to prevent that, we need an upper bound on Unitful in all UnitfulLatexify versions.

[Ickaser]

I can confirm I can successfully build the docs multiple times if I ]add https://github.com/Ickaser/Plots.jl#drop-unitful-latexify-dep in the environment! One minor annoyance is that during the build I get windows displaying latex-allunits.png and latex-examples.png, I'm not sure what's causing those windows to appear. It's not blocking, but it's a small nuisance during the build.

If you add UNITFUL_FANCY_EXPONENTS=false as suggested above, I'm happy with this PR.

Fixed that!

This doesn't need to be in this PR, but to make sure that UnitfulLatexify doesn't get loaded over the top of the extension, maybe the best solution is to make a non-SemVer-breaking release of UnitfulLatexify that has no code, just prints a depwarn and loads both Unitful and Latexify. That could be compatible with new Unitful. @sostock and @gustaphe , thoughts?

[gustaphe]

Sure, sounds good. I could make a change like that, but not in the next week or so.

[sostock]

Yes, sounds good! This will be helpful for upgrading. Additionally, we could still make a PR to the general registry to make older UnitfulLatexify versions incompatible with Unitful ≥1.25. What do you think about that?

[Ickaser]

Yes, sounds good! This will be helpful for upgrading. Additionally, we could still make a PR to the general registry to make older UnitfulLatexify versions incompatible with Unitful ≥1.25. What do you think about that?

Intimidating but I took a stab with JuliaRegistries/General#138160. Very possible that I did something wrong or that I should redo it after the Unitful and UnitfulLatexify new releases are already on the registry, this is my first time working with the registry.

In that case, since we would be introducing a way to block users from getting a bad installation, perhaps the new release of UnitfulLatexify should be (correctly) declared a breaking 2.0 release rather than a non-breaking 1.8?

[sostock]

In that case, since we would be introducing a way to block users from getting a bad installation, perhaps the new release of UnitfulLatexify should be (correctly) declared a breaking 2.0 release rather than a non-breaking 1.8?

I’m not sure which would be better. In what way do you consider the release breaking? Because of the printed warning?

[Ickaser]

In that case, since we would be introducing a way to block users from getting a bad installation, perhaps the new release of UnitfulLatexify should be (correctly) declared a breaking 2.0 release rather than a non-breaking 1.8?

I’m not sure which would be better. In what way do you consider the release breaking? Because of the printed warning?

We took this as a chance to break a little of the API from UnitfulLatexify--replacing e.g. latexsquareunitlabel(l, u) with latexify(l, u; labelformat=:square). So technically it's breaking for some users who used UnitfulLatexify directly.

[sostock]

I see – then I think it should definitely be UnitfulLatexify 2.0.

[Ickaser]

I think this PR should be good to go, then. We can continue conversation about UnitfulLatexify compat at gustaphe/UnitfulLatexify.jl#22 if necessary, but once this PR and that PR are merged & released I can double check JuliaRegistries/General#138160 (pending advice from someone who knows more than I do) to make sure that all old Unitful is treated as incompatible with new UnitfulLatexify and vice versa.