A callout label is dropped from the output rather than demoted from a heading

[yfedoseev]

Regression against v0.3.77, found by human review of the release regression bundle. Filed separately from the ordering regressions because a token is lost, not moved.

Symptom

irs_f1040es.pdf, .text, .md and .html:

v0.3.77:  taxpayer. See *Special Rules*, later.

          #### TIP

          **Exception.** You don't have to pay…

v0.3.78:  taxpayer. See *Special Rules*, later.

          **Exception.** You don't have to pay…

It is not relocated — it is gone

TIP is genuine drawn text on page 1 at (47.2, 305.4), and again at (320.2, 649.9). The reviewer opened the file to confirm it.

The first byte-divergence between the two arms falls exactly at this point, which means the token cannot have moved earlier in the output. It is either dropped outright or pushed somewhere after its own position and lost.

What the right fix is, and is not

v0.3.77 emitted it as #### TIP — a callout label wrongly promoted to a heading. Demoting it from a heading is correct. Removing the word is not. The label is content: it marks an IRS callout box, and a reader of the extracted text loses the cue that the following sentence is an aside.

Related: #1330 covers runs being fused; this is the opposite outcome from the same area of change — a short isolated run disappearing rather than being merged.

[yfedoseev]

Measured against the frozen v0.3.77 baseline. The token is not lost — it is relocated. Correcting the diagnosis here so nobody hunts for a drop that is not happening.

Occurrence counts of TIP in irs_f1040es.pdf, both arms:

surface	v0.3.77	HEAD
.text	3	3
.md	3	3
.html	3	3

What changed is where they sit. In .md:

• v0.3.77 — #### TIP at line 40, between **Note.** These percentages… and **Exception.** You don't have to pay…; the others at 71 and ~199.
• HEAD — that position is empty and the surrounding text is otherwise identical; the TIPs are at 69, 73 and 230.

So the callout label at (47.2, 305.4) has moved out of its callout and down into the masthead cluster near the other one, roughly thirty lines later. The count is preserved end to end, which is why nothing is actually deleted.

The reasoning in the issue that led to "dropped" — that the first byte-divergence falls at this point, so it cannot have moved earlier — is sound, and the second branch of it is the one that holds: it is pushed after its own position. It just is not lost once it gets there.

That makes this a reading-order defect in the masthead region rather than a content-loss defect, and it should be diagnosed alongside the other ordering regressions rather than with #1330. The requirement in the issue still stands unchanged: the label is content and must appear at its callout, not merely somewhere in the document.

Re-titling would be reasonable — something like "a callout label is relocated out of its callout into the masthead". Leaving it open.

[yfedoseev]

Fixed on release/v0.3.78 in 4d6489a.

Correction to the title first: the label is not dropped. Both arms emit three TIP runs in .text, .md and .html. It is relocated — lifted out of its callout and emitted with the other TIP just before the masthead, about fifteen lines late.

Root cause — 564548ca ("scope marked-content ids to their content stream")

Found by bisect, 8 steps, every one built and ran, no skips.

Both TIP runs are drawn inside a Form XObject, and the producer numbers one continuous MCID space across the page stream and the form. Instrumented:

glyph buckets:  Page(0)/0 ... Page(0)/39, Form(3102)/40, Page(0)/41 ... Form(3102)/73
structure tree: Page(0)/0 ... Page(0)/40 ... Page(0)/73     <- all page-scoped
unconsumed:     (Form(3102),40) = "TIP" at (47.25, 477.78)
                (Form(3102),73) = "TIP" at (320.25, 133.31)

The structure elements reference 40 and 73 as bare integers with no /Stm, which ISO 32000-1:2008 §14.7.4.3 resolves against the page. The glyphs carry Form(3102). 564548ca widened the lookup key from a bare u32 to (scope, id), so (Page(0),40) no longer matches (Form(3102),40), the structure slot emits nothing, and the unreferenced-id tail appends both labels after the entire page — which is the TIP / TIP / Instructions for Form 1040es… / January 21, 2025… cluster.

The reading-order strategy has the same shape (an unmatched span is demoted to the untagged tail), which is why markdown and HTML show it too.

The fix does not undo what 564548ca was for

That commit fixed a real collision: a page and a form may both number from 0, and keyed on the bare id the form's MCID 0 was emitted at the page's MCID 0 slot.

Here there is no collision — ids 40 and 73 are numbered by the form and by nothing else. So a cross-scope match is now allowed only for a bare id that exactly one stream numbers, checked on both sides. Where an id is ambiguous, which is precisely 564548ca's case, the fallback declines and the scoped key stands alone. tests/mcid_scope_collision_table_cells.rs passes unaltered.

Verified

• Page 0 .text is byte-identical to v0.3.77.
• Markdown: the first TIP is back at line 39, between **Note.** and **Exception.**.
• 497-document sweep against the unfixed tree: exactly one file differs — this one — and every hunk moves a callout label (TIP, CAUTION) off the page tail back between the paragraphs it interrupts, on pages 0, 1, 2 and 4. No other text moves anywhere.
• cargo test --lib 5901 passed; 18 structure/tagged/ActualText/reading-order test binaries green; clippy clean.

Test

A page whose form-drawn run is referenced by a bare id, asserting it stays in the slot that names it; plus a counter-case where page and form both number from zero and the form names its stream, asserting the run stays at its own slot. The first fails without this change, the counter-case passes either way.

Left alone deliberately

A third lookup of the same class remains scoped-only. It has no failing case and the corpus sweep is clean without touching it, so I did not change behaviour on inference. Narrowing it would need the ambiguity measured over the span population rather than that map's own keys, or 564548ca's fixture breaks again.