Update to Pandoc 3.6.3 and Typst 0.13.0 #12262

cderv · 2025-03-11T12:41:06Z

Opening this PR to run the tests as I noticed some issues locally.

LaTeX template updates are done separately in

Pandoc LaTeX template updates #12260

upstream pandoc no longer has a definitions.typst but we are keeping definitions.typ they moved horizontalrule into their default.typst base template and IMO our template.typ is already so dissimilar from their default.typst that we might as well keep this where it was

some deprecation warnings. i am less sure about the s/amount/abs change but was getting the error error: length does not contain field amount ┌─ docs/smoke-all/typst/callout.typ:38:32 │ 38 │ fields.below = fields.below.amount │ ^^^^^^ can't find any documentation of amount so i'm hoping abs was meant here

and small change in HTML styles template

- jgm/pandoc@0a2e8dd

from jgm/pandoc@1eed55f Add CRediT roles to JATS Enable annotating author roles using the Contribution Role Taxonomy (CRediT) and export this information in conformant JATS

cderv · 2025-03-11T13:07:55Z

We have a few issues in our snapshot test that I have seen in the other PR I am working one.

Some differences related to table it seems.

docs/smoke-all/2024/01/31/8507.qmd

+     <w:sectPr />
-     <w:sectPr>
-       <w:footnotePr>
-         
-                 
-         <w:numRestart w:val="eachSect" />
-         
-               
-       </w:footnotePr>
-     </w:sectPr>

tests/docs/smoke-all/crossrefs/float/gfm/gfm-float.qmd

  | Dog      | 4/5    |
  | Cat      | 4/5    |
  | Horse    | 3/5    |
  | Ant      | 5/5    |
  
+ 
  </div>
  
  See <a href="#tbl-pets_rating" class="quarto-xref">Table 1</a>.

docs/smoke-all/2023/11/16/tbl-cap-classes.qmd

Lua error

./run-tests.sh docs/smoke-all/2023/11/16/tbl-cap-classes.qmd 
> Activating virtualenv from .venv for Python tests in Quarto
> Using Python from /home/cderv/project/quarto-cli/tests/.venv/bin/python
> VIRTUAL_ENV: /home/cderv/project/quarto-cli/tests/.venv
docs/smoke-all/2023/11/16/tbl-cap-classes.qmd
running 1 test from ./smoke/smoke-all.test.ts
[smoke] > quarto render docs/smoke-all/2023/11/16/tbl-cap-classes.qmd --to html ...
------- output -------
[verify] > No Errors or Warnings
----- output end -----
[smoke] > quarto render docs/smoke-all/2023/11/16/tbl-cap-classes.qmd --to html ... FAILED (1s)

 ERRORS 

[smoke] > quarto render docs/smoke-all/2023/11/16/tbl-cap-classes.qmd --to html => ./test.ts:329:8
error: AssertionError: Failed assertion: 

--------------------------------------------------------------------------------
[smoke] > quarto render docs/smoke-all/2023/11/16/tbl-cap-classes.qmd --to html
          ./run-tests.sh smoke/smoke-all.test.ts

[verify] > No Errors or Warnings

Error or Warnings During Execution
|Error
    at renderFiles (file:///home/cderv/project/quarto-cli/src/command/render/render-files.ts:350:23)
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async render (file:///home/cderv/project/quarto-cli/src/command/render/render-shared.ts:104:18)
    at async Command.actionHandler (file:///home/cderv/project/quarto-cli/src/command/render/cmd.ts:251:26)
    at async Command.execute (https://deno.land/x/[email protected]/command/command.ts:1948:7)
    at async Command.parseCommand (https://deno.land/x/[email protected]/command/command.ts:1780:14)
    at async quarto (file:///home/cderv/project/quarto-cli/src/quarto.ts:190:5)
    at async Object.execute (file:///home/cderv/project/quarto-cli/tests/test.ts:119:7)
    at async fn (file:///home/cderv/project/quarto-cli/tests/test.ts:221:13)
    at async innerWrapped (ext:cli/40_test.js:191:5)|
AssertionError: Error or Warnings During Execution
|Error
    at renderFiles (file:///home/cderv/project/quarto-cli/src/command/render/render-files.ts:350:23)
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async render (file:///home/cderv/project/quarto-cli/src/command/render/render-shared.ts:104:18)
    at async Command.actionHandler (file:///home/cderv/project/quarto-cli/src/command/render/cmd.ts:251:26)
    at async Command.execute (https://deno.land/x/[email protected]/command/command.ts:1948:7)
    at async Command.parseCommand (https://deno.land/x/[email protected]/command/command.ts:1780:14)
    at async quarto (file:///home/cderv/project/quarto-cli/src/quarto.ts:190:5)
    at async Object.execute (file:///home/cderv/project/quarto-cli/tests/test.ts:119:7)
    at async fn (file:///home/cderv/project/quarto-cli/tests/test.ts:221:13)
    at async innerWrapped (ext:cli/40_test.js:191:5)|
    at Module.assert (https://jsr.io/@std/assert/0.224.0/assert.ts:18:11)
    at assert (https://jsr.io/@std/testing/0.224.0/asserts.ts:608:11)
    at Object.verify (file:///home/cderv/project/quarto-cli/tests/verify.ts:126:7)
    at fn (file:///home/cderv/project/quarto-cli/tests/test.ts:240:25)
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async innerWrapped (ext:cli/40_test.js:191:5)
    at async outerWrapped (ext:cli/40_test.js:134:14)

Because of check in Lua

quarto-cli/tests/docs/smoke-all/2023/11/16/checktable.lua

Lines 1 to 5 in 52beccf

    
           function Table(tbl) 
        
             if #tbl.caption.long ~= 0 then 
        
               internal_error() 
        
             end 
        
           end

I am looking into those, starting by last one.

cderv · 2025-03-11T13:14:53Z

So it seems the feature to set classes on table is no more working as it was

caption: Caption {
    clone: function: 0x7fcfb1eafbf0
    long: Blocks {
      [1] Plain {
        clone: function: 0x7fcfb1eb0f80
        content: Inlines {
          [1] Str {
            clone: function: 0x7fcfb1eb2750
            text: "{.striped"
            walk: function: 0x7fcfb1eb29d0
          }
          [2] Space {
            clone: function: 0x7fcfb1eb3600
            walk: function: 0x7fcfb1eb3870
          }
          [3] Str {
            clone: function: 0x7fcfb1eb4360
            text: ".hover}"
            walk: function: 0x7fcfb1eb45e0
          }
        }
        show: function: 0x7fcfb1eb1200
        walk: function: 0x7fcfb1eb1440
      }
    }
  }

This is why the check fails

It seems setting to nil is no more working to remove the caption

quarto-cli/src/resources/filters/quarto-pre/table-classes.lua

Lines 55 to 57 in 91c665a

    
           if #quarto.utils.as_inlines(tbl.caption.long) == 0 then 
        
             tbl.caption.long = nil 
        
           end

I did not found which change caused this, but Caption long is using Block, so I changed to empty blocks

cderv · 2025-03-11T14:14:55Z

Regarding the docx issue, it is a change in Pandoc for Docx (jgm/pandoc@6fbb50f) to solve jgm/pandoc#10577

So I'll update the snapshot

Related to this change in pandoc - jgm/pandoc@6fbb50f) to solve jgm/pandoc#10577

cderv · 2025-03-11T14:56:16Z

And I finally found the last change. Pandoc is no more collapsing initial newline in Rawblock
See jgm/pandoc@d9281d1 to solve jgm/pandoc#10477

Previous pandoc

> qv 1.6.42 pandoc --to commonmark
something

```{=markdown}

</div>
```
^Z
something

</div>

Updated pandoc

> quarto pandoc --to commonmark
something

```{=markdown}

</div>
```
^Z
something


</div>

We see one more newline in newest pandoc.

This comes from our Table processing with add some <div> around table using Rawblock

quarto-cli/src/resources/filters/customnodes/floatreftarget.lua

Lines 1053 to 1054 in 52beccf

    
           local open_block = pandoc.RawBlock("markdown", "<div id=\"" .. float.identifier .. "\">\n") 
        
           local close_block = pandoc.RawBlock("markdown", "\n</div>")

and we use newline at the start.

It seems we do it also at

quarto-cli/src/resources/extensions/quarto/docusaurus/docusaurus.lua

Line 142 in 52beccf

admonition:insert(pandoc.RawBlock("markdown", "\n:::" .. node.type))

So I'll fix that too.

They add no effect prior to Pandoc 3.6.3 are they were collapse. From Pandoc 3.6.3 they are not anymore Related Pandoc change: jgm/pandoc@d9281d1 to solve jgm/pandoc#10477

follow up of 0d0fc8d

This reverts commit 73244fb.

…>` when caption location is bottom.

cderv · 2025-03-11T17:49:19Z

This PR passes tests now. So update to Pandoc 3.6.3 seems ok. We could even do the update without the latex template change, as in Quarto, we are providing all the templates, and it seems the new version of Pandoc didn't break our previous template. To be discussed.

cderv · 2025-03-13T15:19:56Z

src/resources/filters/quarto-pre/table-classes.lua

      if #quarto.utils.as_inlines(tbl.caption.long) == 0 then
-        tbl.caption.long = nil
+        tbl.caption.long = pandoc.Blocks({})


@tarleb this is surprising to us. Do you know why setting to nil was not replacing the value ?

tbl.caption.long was not modified

This is related to pandoc 3.4 to 3.6.3 update

I think it's kind of the same as when filter functions return nil. But I'll need to check.

Is this new for element content ?

Only the update triggered it.

From jgm/pandoc@9be0963 Introducing new `babeloptions` and moving option in `doc-class.tex` partial

…emplate This helps with comparing original and ours

… in its own template This is a breaking change for users who have custom templates. The new templates are more modular and easier to customize, but they now mix Pandoc's partial and Quarto's partials. Changes: - Add an new template logic dedicated to beamer as Pandoc has split beamer in its own template. Aim is to make it easy to update the beamer template in the future. - Add a new LaTeX template that is based on the default template, but it is split into several partials that are provided by Pandoc and by Quarto. - Adapt the update script to use the new partials and to update the beamer template separately. - Add two Quarto partials that are used in the new LaTeX template: `babel-lang.tex` and `biblio-config.tex`. The new organization possibly could break custom format that tweak the `pandoc.tex` partials as it now contains part of the default `common.latex` partial from Pandoc. This means that if you have a custom template that uses `pandoc.tex` you will need to update it to use the new partials.

It means previous missing package is not needed anymore. Pandoc related change: jgm/pandoc@9be0963

cscheid and others added 13 commits February 24, 2025 12:12

pandoc 3.6 lua-types update

a611fd8

pandoc 3.6 update

5a76dac

bump to pandoc 3.6

354fad2

partial work - typst template and new page-numbering

5c56f77

deps -> typst 0.12.0

1bcb66d

deps -> typst 0.13.0

a4e5d5f

update ctheorem for typst 0.13 compatibility

ac2ecdf

update tests for changed default typst font

f3e6656

Update to pandoc 3.6.3

3497654

and small change in HTML styles template

import pandoc's html style change

fb89d30

- jgm/pandoc@0a2e8dd

update to pandoc's jats template

3364332

from jgm/pandoc@1eed55f Add CRediT roles to JATS Enable annotating author roles using the Contribution Role Taxonomy (CRediT) and export this information in conformant JATS

cderv marked this pull request as draft March 11, 2025 12:41

lua - don't use nil to create empty caption

6014c12

Update docx snapshot as new Pandoc as change some Docx Writer output

2a45846

Related to this change in pandoc - jgm/pandoc@6fbb50f) to solve jgm/pandoc#10577

cderv added 4 commits March 11, 2025 16:01

Remove starting newline in RawBlock of format markdown.

0d0fc8d

They add no effect prior to Pandoc 3.6.3 are they were collapse. From Pandoc 3.6.3 they are not anymore Related Pandoc change: jgm/pandoc@d9281d1 to solve jgm/pandoc#10477

update other test snapshot

73244fb

follow up of 0d0fc8d

Revert "update other test snapshot"

41f659f

This reverts commit 73244fb.

insert new line after caption to keep the effect of previous `\n</div…

0fb4df0

…>` when caption location is bottom.

cderv commented Mar 13, 2025

View reviewed changes

cderv added 4 commits March 13, 2025 16:49

Get newest latex partials

f1509e3

pandoc template udpate - Improved babel support

42b33b0

From jgm/pandoc@9be0963 Introducing new `babeloptions` and moving option in `doc-class.tex` partial

latex template - move nocite-ids in same place as pandoc's original t…

1ddaa65

…emplate This helps with comparing original and ours

cderv added 9 commits March 13, 2025 16:49

Adapt snapshot to new LaTeX template

d1cf693

tests - adapt as Pandoc 3.6.3 brings improved babel support

7cd8906

It means previous missing package is not needed anymore. Pandoc related change: jgm/pandoc@9be0963

Add to changelog

3bdf0f2

cleanup file needed for dev only

2c4a250

typo

892c6fa

cleanup - This is stored in beamer folder now

a761f7e

Add comment to line that use authors normalization

1c221f5

Merge branch 'main' into chore/pandoc-3.6

8a62c54

Avoid trailing space by putting comment in a line above

2d3b24a

cscheid marked this pull request as ready for review March 14, 2025 19:20

cscheid and others added 2 commits March 14, 2025 15:20

Merge branch 'main' into chore/pandoc-3.6

4387e6f

Merge branch 'main' into chore/pandoc-3.6

2bc5993

cscheid merged commit 68d1239 into main Mar 26, 2025
49 checks passed

cscheid deleted the chore/pandoc-3.6 branch March 26, 2025 15:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update to Pandoc 3.6.3 and Typst 0.13.0 #12262

Update to Pandoc 3.6.3 and Typst 0.13.0 #12262

Uh oh!

cderv commented Mar 11, 2025

Uh oh!

cderv commented Mar 11, 2025 •

edited

Loading

Uh oh!

cderv commented Mar 11, 2025 •

edited

Loading

Uh oh!

cderv commented Mar 11, 2025

Uh oh!

cderv commented Mar 11, 2025

Uh oh!

cderv commented Mar 11, 2025

Uh oh!

cderv Mar 13, 2025

Uh oh!

tarleb Mar 14, 2025

Uh oh!

cderv Mar 14, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Update to Pandoc 3.6.3 and Typst 0.13.0 #12262

Update to Pandoc 3.6.3 and Typst 0.13.0 #12262

Uh oh!

Conversation

cderv commented Mar 11, 2025

Uh oh!

cderv commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cderv commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cderv commented Mar 11, 2025

Uh oh!

cderv commented Mar 11, 2025

Uh oh!

cderv commented Mar 11, 2025

Uh oh!

cderv Mar 13, 2025

Choose a reason for hiding this comment

Uh oh!

tarleb Mar 14, 2025

Choose a reason for hiding this comment

Uh oh!

cderv Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

cderv commented Mar 11, 2025 •

edited

Loading

cderv commented Mar 11, 2025 •

edited

Loading

cderv Mar 14, 2025 •

edited

Loading