Skip to content

Commit d0deec3

Browse files
Earlopainmatzbot
authored andcommitted
[ruby/prism] Fix parser translator tokens for comment-only file
In ruby/prism#3393 I made a mistake. When there is no previous token, it wraps around to -1. Oops Additionally, if a comment has no newline then the offset should be kept as is ruby/prism@3c266f1de4
1 parent f56f3ea commit d0deec3

File tree

3 files changed

+23
-5
lines changed

3 files changed

+23
-5
lines changed

lib/prism/translation/parser/lexer.rb

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -281,14 +281,14 @@ def to_a
281281
location = range(token.location.start_offset, lexed[index][0].location.end_offset)
282282
index += 1
283283
else
284-
value.chomp!
285-
location = range(token.location.start_offset, token.location.end_offset - 1)
284+
is_at_eol = value.chomp!.nil?
285+
location = range(token.location.start_offset, token.location.end_offset + (is_at_eol ? 0 : -1))
286286

287-
prev_token = lexed[index - 2][0]
287+
prev_token = lexed[index - 2][0] if index - 2 >= 0
288288
next_token = lexed[index][0]
289289

290-
is_inline_comment = prev_token.location.start_line == token.location.start_line
291-
if is_inline_comment && !COMMENT_CONTINUATION_TYPES.include?(next_token&.type)
290+
is_inline_comment = prev_token&.location&.start_line == token.location.start_line
291+
if is_inline_comment && !is_at_eol && !COMMENT_CONTINUATION_TYPES.include?(next_token&.type)
292292
tokens << [:tCOMMENT, [value, location]]
293293

294294
nl_location = range(token.location.end_offset - 1, token.location.end_offset)
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
foo # Bar
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
@ ProgramNode (location: (1,0)-(1,3))
2+
├── flags: ∅
3+
├── locals: []
4+
└── statements:
5+
@ StatementsNode (location: (1,0)-(1,3))
6+
├── flags: ∅
7+
└── body: (length: 1)
8+
└── @ CallNode (location: (1,0)-(1,3))
9+
├── flags: newline, variable_call, ignore_visibility
10+
├── receiver: ∅
11+
├── call_operator_loc: ∅
12+
├── name: :foo
13+
├── message_loc: (1,0)-(1,3) = "foo"
14+
├── opening_loc: ∅
15+
├── arguments: ∅
16+
├── closing_loc: ∅
17+
└── block: ∅

0 commit comments

Comments
 (0)