Fix InexactError in peek_behind_pos when skipping nested trivia nodes #604
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Explanation of bug written below by Claude :robot. Fixes JuliaLang/julia#60084
When parsing certain incomplete keywords like "do", the parser would crash
with
InexactError: trunc(UInt32, -1)instead of returning a ParseError.Root cause:
The peek_behind_pos function walks backwards through the flat output array,
subtracting byte_spans to calculate byte positions. However, when a non-terminal
trivia node contains child nodes, both the parent and children exist in the
output array with overlapping byte_spans. The old code would subtract the
byte_span of both the parent and its children, causing byte_idx to go negative.
Example with input "do":
The parser creates this output structure:
[1] TOMBSTONE (byte_span=0)
[2] "do" token (byte_span=2, is_trivia=true, covers bytes 1-2)
[3] error node (byte_span=2, is_trivia=true, node_span=1, covers bytes 1-2)
└── contains node [2] as a child
When peek_behind_pos walks backwards from next_byte=3:
Old behavior:
- Start: byte_idx = 3, node_idx = 3
- Skip error node [3]: byte_idx = 3 - 2 = 1, node_idx = 2
- Skip "do" node [2]: byte_idx = 1 - 2 = -1 ❌ (tries to convert to UInt32)
The problem: Both nodes cover the same bytes (1-2), so subtracting both
spans double-counts the same 2 bytes.
Solution:
When skipping a non-terminal node, also skip all its children without
subtracting their byte_spans, since the parent's byte_span already includes
them:
if is_non_terminal(node)
node_idx -= (1 + node.node_span) # Skip the node + all its children
else
node_idx -= 1
end
New behavior:
Why "do" was unique:
The "do" keyword is the only Julia keyword that:
Other incomplete keywords either parse successfully or fail earlier before
reaching the code path that calls peek_behind.