Bugfix for pipe table delimiters (#48)

* Fix pipe table parsing with code spans containing pipes (#29)

Added code span recognition to pipe table cell parsing. The block grammar
now properly handles backtick code spans like `|` within table cells by
repeating the inline grammar's code span parsing logic in the block context.

Changes:
- Added CODE_SPAN_START and CODE_SPAN_CLOSE external tokens to block grammar
- Implemented parse_code_span() in block scanner with lookahead for matching delimiters
- Added _pipe_table_code_span rule to parse code spans within table cells
- Updated scanner state serialization to track code span delimiter length

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* Add tests for pipe tables with code spans containing pipes (issue #29)

These tests verify that the parser correctly handles code spans containing
pipe characters within pipe tables. The fix in the block parser now properly
parses code spans to avoid treating pipes inside backticks as table
delimiters.

Test cases cover:
- Simple code span with single pipe
- Multiple code spans with pipes in different cells
- Mixed backtick delimiters (double and triple backticks)

All tests pass and match Pandoc's output.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* Fix pipe table parsing with latex spans containing pipes

This commit applies the same technique used for code spans (issue #29)
to fix latex spans (inline math with $ delimiters) in pipe tables.

Changes:
- Added LATEX_SPAN_START and LATEX_SPAN_CLOSE external tokens
- Implemented parse_latex_span() in scanner.c to handle dollar sign
  delimiters within pipe table cells
- Added _pipe_table_latex_span grammar rule
- Fixed token check ordering in scan() to prioritize latex span parsing
  over display math state tracking when inside pipe table cells
- Added tree-sitter test case for latex spans with pipes
- Added end-to-end tests in quarto-markdown-pandoc crate

The fix ensures that pipes inside latex spans (e.g., $|$) are not
treated as table cell delimiters, matching Pandoc's behavior.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>

Author: cscheid

crates/quarto-markdown-pandoc/tests/pandoc-match-corpus/markdown/pipe-table-code-span.qmd

@@ -0,0 +1,3 @@
+| a | b |
+|---|---|
+| `|` | oh no |

crates/quarto-markdown-pandoc/tests/pandoc-match-corpus/markdown/pipe-table-latex-span.qmd

@@ -0,0 +1,3 @@
+| a | b |
+|---|---|
+| $|$ | math |

crates/quarto-markdown-pandoc/tests/pandoc-match-corpus/markdown/pipe-table-mixed-backticks.qmd

@@ -0,0 +1,5 @@
+| Test | Description |
+|------|-------------|
+| `` ` `` | backtick in code |
+| ``` | ``` | pipe in triple backtick code |
+| `a|b|c` | multiple pipes |

crates/quarto-markdown-pandoc/tests/pandoc-match-corpus/markdown/pipe-table-mixed-code-latex.qmd

@@ -0,0 +1,6 @@
+| Test | Description |
+|------|-------------|
+| `|` | backtick code with pipe |
+| $|$ | latex span with pipe |
+| `a|b` | more code |
+| $x|y$ | more math |

crates/quarto-markdown-pandoc/tests/pandoc-match-corpus/markdown/pipe-table-multiple-code-spans.qmd

@@ -0,0 +1,5 @@
+| Column 1 | Column 2 | Column 3 |
+|----------|----------|----------|
+| `|` | normal text | `a|b` |
+| regular | `||` | more text |
+| `x | y` | text | `|>` |

crates/quarto-markdown-pandoc/tests/pandoc-match-corpus/markdown/pipe-table-multiple-latex-spans.qmd

@@ -0,0 +1,5 @@
+| Column 1 | Column 2 | Column 3 |
+|----------|----------|----------|
+| $|$ | normal text | $a|b$ |
+| regular | $||$ | more text |
+| $x | y$ | text | $|>$ |

crates/tree-sitter-qmd/tree-sitter-markdown/grammar.js

@@ -454,21 +454,47 @@ module.exports = grammar({
                 ),
             ),
 
+            // Code span within pipe table cells - simplified version that only handles backticks
+            _pipe_table_code_span: $ => seq(
+                $._code_span_start,
+                repeat(choice(
+                    $._word,
+                    $._whitespace,
+                    common.punctuation_without($, []),
+                )),
+                $._code_span_close,
+            ),
+
+            // Latex span within pipe table cells - simplified version that only handles dollar signs
+            _pipe_table_latex_span: $ => seq(
+                $._latex_span_start,
+                repeat(choice(
+                    $._word,
+                    $._whitespace,
+                    common.punctuation_without($, []),
+                )),
+                $._latex_span_close,
+            ),
+
             _pipe_table_cell_contents: $ => prec.right(
                 seq(
                     choice(
                         $._word,
-                        $._display_math_state_track_marker, 
-                        $._inline_math_state_track_marker, 
+                        $._display_math_state_track_marker,
+                        $._inline_math_state_track_marker,
                         $._backslash_escape,
+                        $._pipe_table_code_span,
+                        $._pipe_table_latex_span,
                         common.punctuation_without($, ['|']),
                     ),
                     repeat(choice(
                         $._word,
-                        $._display_math_state_track_marker, 
-                        $._inline_math_state_track_marker, 
+                        $._display_math_state_track_marker,
+                        $._inline_math_state_track_marker,
                         $._whitespace,
                         $._backslash_escape,
+                        $._pipe_table_code_span,
+                        $._pipe_table_latex_span,
                         common.punctuation_without($, ['|']),
                     )))),
 
@@ -569,6 +595,14 @@ module.exports = grammar({
         // special tokens to allow external scanner serialization to happen
         $._display_math_state_track_marker,
         $._inline_math_state_track_marker,
+
+        // code span delimiters for parsing pipe table cells
+        $._code_span_start,
+        $._code_span_close,
+
+        // latex span delimiters for parsing pipe table cells
+        $._latex_span_start,
+        $._latex_span_close,
     ],
     precedences: $ => [
         [$._setext_heading1, $._block],