Add/ensure ToC includes abstract, acknowledgements, bibliography

[dginev]

A working concept for resolving #2316 .

This PR attempts consistency with the current handlers in CrossRef and Scan post-processors, trying to ensure inlist="toc" is set on ltx:abstract, ltx:acknowledgements, ltx:bibliography. And that those elements are included and used in the TOC filling algorithms (also requiring a set xml:id).

• The consistency is ensured by subs executed on every element via the Tag() technique.

To my surprise this leads to a schema change as well, allowing Labelled.attributes.

What is less surprising is that such a change leads to a very large diff in the tests. If nothing else, that change is very uniform.

Feedback most welcome.

[dginev]

Update: I have returned \tableofcontents to the mainline behavior, as per feedback.

[brucemiller]

Wow, there's a lot of diff noise here. Of course, if abstract is going be a valid target, it's going to need an xml:id. But adding inlist="toc" to any of these elements is going to affect the normal TOC -- and it does, even though I thought you said you'd backed that off.

However, if we're going to have the navigation TOC include more or different elements than the standard LaTeX TOC, then I guess we're stuck doing one of two things: We could modify Post::CrossRef to handle navigation TOC's specially, with hand coded extra candidate elements (which is kinda yucky); or use an extra keyword, like inlist="navtoc" to indicate that an element should be included in the navigation toc (or inlist="toc,navtoc" for both). The latter seems the most Correct, but of course, along with new ID's, will also cause lots of diff noise.

[dginev]

My memory is:

• we agreed to back off the changes to \tableofcontents so that the original LaTeX behavior is undisturbed
• we agreed if we are including these new kinds of content in the ToC, we should treat them equivalently to the existing ones, i.e. give them the inlist="navtoc" attribute.
• indeed the PR contains the "kind of yucky" continuing of precedent in Scan.pm, by adding the entries as an allowlist, e.g.

  $self->registerHandler('ltx:acknowledgements' => \&ack_handler);

I don't really have an appetite for a larger refactor right now, and this code has already been deployed on our end, so this can idle until we have a path forward.

[brucemiller]

* we agreed to back off the changes to |\tableofcontents| so that the original LaTeX behavior is undisturbed

Good. Except that the PR as it stand *is* disturbing the latex behavior.

[dginev]

Which LaTeX behavior ?

[brucemiller]

Now \tableofcontents includes the bibliography, which plain latex omits. (curiously it omits abstract from \tableofcontents; though both are included in the navtoc, as you wished)

[brucemiller]

This ought to be easy, and almost everything is there to do it, except for a "clean" design hook. We should not be adding \inlist="toc" to anything (for this purpose); OTOH, adding navtoc to the XML is awkward, as it pollutes all the XML but should be a purely post-processing decision.

[dginev]

If you could name the mechanisms you think are capable of achieving this simply, I could try and verify the suggestion.

[brucemiller]

It's kinda speculative, ATM, without some experimentation. I'm thinking we should rely more on schema data than explicit lists of elements. Something like all the elements in FrontMatter.class, document.body.class,BackMatter.class that allow the attribute inlist (which is Listable.attributes) would seem to be potential candidates for TOC's (though ltx:abstract,ltx:acknowlegements currently don't get that attribute, but seemingly could). If that list made sense, it might drive CrossRef's selector, maybe only for navtoc, or optionally, or ...

[dginev]

I have tried to remove the newly added inlist="toc" attributes and use $normaltoctypes instead, specifically for --navtoc=context.

[dginev]

adding ltx:document here includes the document title. So I either need a separate $contexttoctypes for --navtoc=context, or I can exclude it here. I suspect it is used somewhere else, just not sure where...

[dginev]

In the call today we agreed this can wait for mainline support and can be approached differently, with a potentially larger refactor that also includes dynamic navtoc element selection based on the schema.