-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Invalid escape sequences are allowed in the current tokenization and parsing process -- strings are modeled as-is without processing escapes in the tokenizer and in both the default and round-trip parsers. An error for invalid escape sequences will only trigger an error upon serialization or deserialization.
Ideally, the processor should reject invalid escape sequences in tokenization and/or in parsing. Though, for some common applications (think linters, language servers, etc.) it may be preferable to be able to at least tokenize or get model representations of such documents, even if they are technically invalid.
Rejecting documents with invalid escape sequences should be the default behavior just as it is for other kinds of document errors (like unterminated strings, invalid line breaks, etc.). This will potentially be a breaking behavior change, but is probably still the correct thing to do even as a default behavior. Might consider adding a configuration to opt-out of this behavior in one way or another.