Bugfix for pipe table delimiters #48
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added code span recognition to pipe table cell parsing. The block grammar now properly handles backtick code spans like `|` within table cells by repeating the inline grammar's code span parsing logic in the block context. Changes: - Added CODE_SPAN_START and CODE_SPAN_CLOSE external tokens to block grammar - Implemented parse_code_span() in block scanner with lookahead for matching delimiters - Added _pipe_table_code_span rule to parse code spans within table cells - Updated scanner state serialization to track code span delimiter length 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
These tests verify that the parser correctly handles code spans containing pipe characters within pipe tables. The fix in the block parser now properly parses code spans to avoid treating pipes inside backticks as table delimiters. Test cases cover: - Simple code span with single pipe - Multiple code spans with pipes in different cells - Mixed backtick delimiters (double and triple backticks) All tests pass and match Pandoc's output. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
This commit applies the same technique used for code spans (issue #29) to fix latex spans (inline math with $ delimiters) in pipe tables. Changes: - Added LATEX_SPAN_START and LATEX_SPAN_CLOSE external tokens - Implemented parse_latex_span() in scanner.c to handle dollar sign delimiters within pipe table cells - Added _pipe_table_latex_span grammar rule - Fixed token check ordering in scan() to prioritize latex span parsing over display math state tracking when inside pipe table cells - Added tree-sitter test case for latex spans with pipes - Added end-to-end tests in quarto-markdown-pandoc crate The fix ensures that pipes inside latex spans (e.g., $|$) are not treated as table cell delimiters, matching Pandoc's behavior. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #29.
In addition, we also fix the same bug for
$|$inlines.