Adjust test outputs in test/examples to avoid warnings; fix bug in `expandfirst` handling on Windows #2804

fingolfin · 2025-11-03T23:49:21Z

... so that we can run those tests with warnonly = false, at least in some specific Julia versions.

Resolves #2798.

fingolfin · 2025-11-04T09:02:06Z

test/examples/pages/a.jl

- [`ccall`](@ref)
+- [`deepcopy`](@ref)
 - [`while`](@ref)
- [`@time(x)`](@ref)


Explanation: ccall and @time docstrings reference other docstrings, and some of those have further ref's; that then leads to "missing reference" warnings, unless one pulls in all of those. So I replaced them with alternatives that (currently) don't reference anything in their docstrings

fingolfin · 2025-11-04T09:02:52Z

test/examples/src/index.md

 ```jldoctest
 julia> [1.0, 2.0, 3.0]
-3-element Array{Float64,1}:
+3-element Vector{Float64}:


Adjust to changed output in... Julia 1.6 at least? Which is the minimum version supported by Documenter, so...

test/examples/make.jl

fingolfin · 2025-11-04T09:07:04Z

test/examples/src/index.md

 ```jldoctest
 julia> f(2)
-ERROR: UndefVarError: f not defined
+ERROR: UndefVarError: `f` not defined in `Main`


This version of the message is shown in Julia >= 1.11; this is one of the reasons why we have warnonly = (VERSION.major == 1 && VERSION.minor <= 10) further above

fingolfin · 2025-11-04T09:09:00Z

test/examples/src/index.md


 julia> fullname(@__MODULE__)
-(:Main,)
+(:Main, :Main)


I am unsure how this was ever correct... I mean, yeah, ideally it would print (:Main,) because that's what is printed in Main. But we are in a sandbox module, and we are replacing the module name in output by Main, so I think this is kinda the best we can hope for?

C.f.:

julia> fullname(@__MODULE__) (:Main,) julia> module Foo println(fullname(@__MODULE__)) end (:Main, :Foo) Main.Foo

fingolfin · 2025-11-04T09:09:33Z

test/examples/src/index.md

 ```

-# Issue398
+# Issue #398


That's just for consistency (I am also tempted to sort these sections by issue number... or maybe each issue test should be in a file of its own?

fingolfin · 2025-11-04T09:10:10Z

test/examples/src/index.md


 julia> f("")
 ERROR: MethodError: no method matching f(::String)
+The function `f` exists, but no method is defined for this combination of argument types.


This line is new in Julia 1.11

fingolfin · 2025-11-04T09:11:25Z

test/examples/src/index.md

 julia> println(Point)
 Point

+julia> import Base: sqrt


This is necessary to make this test pass in Julia >= 1.12

fingolfin · 2025-11-04T09:20:05Z

test/examples/src/index.md

-# Bad links (Windows)
-
-* [Colons not allowed on Windows -- `some:path`](some:path)
-* [No "drive" -- `:path`](:path)
-* [Absolute Windows paths -- `X:\some\path`](X:\some\path)
-


This test caused a bunch of warnings. And it is unclear to me what it even is supposed to test. It was added in 80721fd from PR #485.

My best guess is in commit 80721fd which was part of PR #485 before squashing. It introduces a function fixlinks! which has some Windows specific code:

if is_windows() && ':' in img.url Utilities.warn("Invalid local image: colons not allowed in paths on Windows\n '$(img.url)' in $(get(navnode.page))") return end

That function is long gone, but a descendant of the code lives on in local_links!:

elseif Sys.iswindows() && ':' in path @docerror( doc, :cross_references, "invalid local link/image: colons not allowed in paths on Windows in $(Documenter.locrepr(page.source))", link = node ) return end

So perhaps this line is about getting coverage for that @docerror

But despite this, I think it is the right move to remove this here: because right now, the test results in "fake coverage": we wouldn't notice if the test for colons was broken because we ignore the output via @quietly and even if we didn't, a failure here would most likely not trigger a CI failure.

So instead, we need a new test which triggers this and then specifically watches out for the result, and checks that it shows a warning/error with the appropriate message. (And that should probably be three tests, one for each instance).

As I mentioned elsewhere, I'd like such a setup also for testing the quality of the locrepr output, i.e. to ensure it shows the "right" file paths.

See PR #2822

fingolfin · 2025-11-04T09:25:21Z

test/examples/src/issue491.md

    ```@eval
-    "expanded_"*"eval"
+    import Markdown
+    Markdown.parse("expanded_"*"eval")


Without this we get a warning because @eval should return nothing or a Markdown.MD. We don't want that here. Instead there should be a dedicated test for this warning, see issue #2805

Such a test is added in PR #2823

fingolfin · 2025-11-04T09:25:57Z

test/examples/src/man/tutorial.md

-
-```@eval
-rand(20, 20)
+Markdown.MD("Hello")


Base.banner was moved to REPL.banner in Julia 1.11. But anyway, there seems no point in having a complex test here, is there?

fingolfin · 2025-11-04T09:26:32Z

test/examples/tests.jl

                        end
-                        @test length(inv(":jl:keyword:`for`")) == 1
-                        if length(inv(":jl:keyword:`for`")) == 1
-                            item = inv[":jl:keyword:`for`"]


I had to replace for by while because in recent Julia versions, the for docstring as a @ref to outer.

fingolfin · 2025-11-04T09:30:19Z

@mortenpi I tried to explain the reasoning behind all changes in PR review comments.

But tests fail on Windows, any help would be appreciated:

===================== @quietly: output from the expression =====================
[ Info: SetupBuildDirectory: setting up build directory.
[ Info: Doctest: skipped.
[ Info: ExpandTemplates: expanding markdown templates.
┌ Warning: expandorder/AA.md in expandfirst does not exist
└ @ Documenter D:\a\Documenter.jl\Documenter.jl\src\expander_pipeline.jl:45
┌ Error: failed to run `@example` block in examples\src\expandorder\01.md:9-11
│ ```@example
│ isfile("AA.txt") || error("AA.txt missing")
│ ```
│   exception =
│    AA.txt missing
│    Stacktrace:
│      [1] error(s::String)
│        @ Base .\error.jl:44
│      [2] top-level scope
│        @ 01.md:10
│      [3] eval(m::Module, e::Any)
│        @ Core .\boot.jl:489
│      [4] #61
│        @ D:\a\Documenter.jl\Documenter.jl\src\expander_pipeline.jl:856 [inlined]
│      [5] cd(f::Documenter.var"#61#62"{Module, Expr}, dir::String)
│        @ Base.Filesystem .\file.jl:101

... so that we can run those tests with `warnonly = false`, at least in some specific Julia versions.

fingolfin · 2025-11-07T14:00:19Z

Ah of course it's right there in the error:

┌ Warning: expandorder/AA.md in expandfirst does not exist
└ @ Documenter D:\a\Documenter.jl\Documenter.jl\src\expander_pipeline.jl:45

My guess would be this is due to a different in path separators, combined with dumb string comparision on the paths. Indeed, the warning is produced here:

function expand(doc::Documenter.Document)
    priority_pages = filter(doc.user.expandfirst) do src
        if src in keys(doc.blueprint.pages)
            return true
        else
            @warn "$(src) in expandfirst does not exist"
            return false
        end
    end

I'll see if using joinpath will help.

But of course other kwargs for mkdocs such as pages use slashes in path just fine. Perhaps those arguments are processed somewhere, and the same processing should be applied to expandfirst ? Perhaps @mortenpi knows.

fingolfin · 2025-11-07T14:48:29Z

CHANGELOG.md


-* Modules for `@example` environments are now generated by `eval`'ing an expression, rather than invoking the `Module` constructor, which is not recommended. (#2683)
+* Modules for `@example` environments are now generated by `eval`'ing an expression, rather than invoking the `Module` constructor, which is not recommended. ([#2683])
+* Fix handling of `expandfirst` entries containing slashes as path separators on Windows. ([#2804])


So this uncovered a genuine bug in expandfirst handling on Windows, yay. I've thus added a changelog entry. (And fixed the link in the entry above)

fingolfin added the Skip Changelog Allows the CHANGELOG.md check to pass without edit to the file. label Nov 3, 2025

fingolfin mentioned this pull request Nov 4, 2025

Don't use the Module(...) constructor to create a module #2683

Merged

fingolfin commented Nov 4, 2025

View reviewed changes

test/examples/make.jl Outdated Show resolved Hide resolved

fingolfin commented Nov 4, 2025

View reviewed changes

This was referenced Nov 4, 2025

Report file names in which "Unexpected Julia interpolation" occurred #2793

Merged

Move Windows path warning tests (for path containing :) and actually take their results into account #2822

Open

Adjust test outputs in test/examples to avoid warnings

4af527b

... so that we can run those tests with `warnonly = false`, at least in some specific Julia versions.

fingolfin force-pushed the mh/clean-and-fix-old-tests branch 2 times, most recently from ed14185 to 4af527b Compare November 7, 2025 13:56

Resolve Windows issue (hopefully)

2a0b103

fingolfin force-pushed the mh/clean-and-fix-old-tests branch from f88baf9 to 2a0b103 Compare November 7, 2025 14:45

fingolfin added Type: Bugfix and removed Skip Changelog Allows the CHANGELOG.md check to pass without edit to the file. labels Nov 7, 2025

fingolfin commented Nov 7, 2025

View reviewed changes

fingolfin changed the title ~~Adjust test outputs in test/examples to avoid warnings~~ Adjust test outputs in test/examples to avoid warnings; fix bug in expandfirst handling on Windows Nov 7, 2025

Add to changelog

80f2540

fingolfin force-pushed the mh/clean-and-fix-old-tests branch from 1005449 to 80f2540 Compare November 7, 2025 14:49

Adjust test outputs in test/examples to avoid warnings; fix bug in expandfirst handling on Windows #2804

Are you sure you want to change the base?

Adjust test outputs in test/examples to avoid warnings; fix bug in expandfirst handling on Windows #2804

Uh oh!

Conversation

fingolfin commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fingolfin commented Nov 4, 2025

Uh oh!

fingolfin commented Nov 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Adjust test outputs in test/examples to avoid warnings; fix bug in `expandfirst` handling on Windows #2804

Adjust test outputs in test/examples to avoid warnings; fix bug in `expandfirst` handling on Windows #2804

fingolfin commented Nov 3, 2025 •

edited

Loading