Skip to content

Fix parser translator during string escaping with invalid utf-8#3583

Merged
kddnewton merged 1 commit intoruby:mainfrom
Earlopain:parser-translator-escape-invalid-encoding
Jun 11, 2025
Merged

Fix parser translator during string escaping with invalid utf-8#3583
kddnewton merged 1 commit intoruby:mainfrom
Earlopain:parser-translator-escape-invalid-encoding

Conversation

@Earlopain
Copy link
Collaborator

byte.ord can raise when it is not valid in its encoding. Instead, prefer scan_byte over get_byte since that already returns the byte as an integer, sidestepping conversion issues.

Fixes #3582

@Earlopain Earlopain force-pushed the parser-translator-escape-invalid-encoding branch from 154ed98 to 1bcf087 Compare June 11, 2025 11:31
@Earlopain
Copy link
Collaborator Author

Right, no scan_byte, it's much to recent. Just make the character binary then, seems good enough.

Copy link
Collaborator

@kddnewton kddnewton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use scan_byte and add a polyfill for when it's necessary

@Earlopain Earlopain force-pushed the parser-translator-escape-invalid-encoding branch from 1bcf087 to f2a2ebc Compare June 11, 2025 12:58
Instead, prefer `scan_byte` over `get_byte` since that already returns the byte as an integer, sidestepping conversion issues.

Fixes ruby#3582
@Earlopain Earlopain force-pushed the parser-translator-escape-invalid-encoding branch from f2a2ebc to 7f3008b Compare June 11, 2025 13:28
@Earlopain
Copy link
Collaborator Author

Right, makes sense. I changed it.

@kddnewton kddnewton merged commit 615e726 into ruby:main Jun 11, 2025
56 checks passed
@Earlopain Earlopain deleted the parser-translator-escape-invalid-encoding branch October 3, 2025 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parser Translation: tokenization error

2 participants