-
-
Notifications
You must be signed in to change notification settings - Fork 161
Open
Description
This is probably a duplicate of #500, but I'm not sure, so I'm opening this issue.
I'm experiencing some issues with this crate and have been creating a minimal POC/test to track it down.
use logos::Logos;
#[derive(Logos, Debug)]
enum TokenKind {
#[regex("[0-9]+(_+[0-9]+)*")]
DecInt,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_lexer() {
let source = "123_";
let mut lexer = TokenKind::lexer(source).spanned();
assert_eq!("", lexer.slice(), "Initial lexer slice should be empty");
assert_eq!(source, lexer.remainder(), "Initial lexer remainder should match source");
let (result, span) = lexer.next().unwrap();
let slice = &source[span];
assert_eq!("123", lexer.slice(), "Lexer slice should be '123'");
assert_eq!("_", lexer.remainder(), "Lexer remainder should be '_'");
match result {
Ok(TokenKind::DecInt) => (),
Err(_) => panic!("Unexpected error"),
}
assert_eq!(slice, "123");
}
}This should lex DecInt tokens with this regex ([0-9]+(_+[0-9]+)*):
For the input 123_, I expect a TokenKind::DecInt with slice 123 (without _), but I get Lexer slice should be '123' Left: 123 Right: 123_ and if I comment out both assert_eq before the match (
- assert_eq!("123", lexer.slice(), "Lexer slice should be '123'");
+ // assert_eq!("123", lexer.slice(), "Lexer slice should be '123'");
- assert_eq!("_", lexer.remainder(), "Lexer remainder should be '_'");
+ // assert_eq!("_", lexer.remainder(), "Lexer remainder should be '_'");) I get an Unexpected error.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels