Performance degradation between 1.7 and 1.9

[hurricup]

We have a erb language lexer. Syntax if very simple: https://github.com/ruby/erb

Most of the rules in lexer are upto matches and some dangerously looking negations:

START_LEXEME = "<%" | "<%#" | "<%="   // this is ued for upto as `~{START_LEXEME}`
BEFORE_START_LEXEME = !({ANY_CHAR}* {START_LEXEME} {ANY_CHAR}*)

And seems that they both worked perfectly up to 1.7 (lexer speed was about 170 mb/sec.
But after 1.9 lexer speed went down to 0.03 mb/sec.

I really don't like the last rule, looks like it matches everything for each check. But I'm not quite sure (it worked somehow before).

Could you please clarify what went wrong here?
Thank you.