IDLE: Wrong highlighting when previous line ends with backslash

[TheLizzard]

Bug report

Bug description:

Code like this

str. \
    format

should be coloured the same way as

str.format

Right now IDLE doesn't respect the slash before the newline and thinks that every new line is a new statement.

Similarly in:

x = match if match else \
                   match

The last match should be coloured in like a variable and not a keyword.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-140334: Fix IDLE syntax highlighting after line continuations #140335
• gh-140347: Use pyrepl's colorizer in IDLE #140337

[TheLizzard]

I found exactly where the problem is coming from (Lib/idlelib/colorizer.py@make_pat) and I have a solution that passes all tests. I am working a pull request.

[terryjreedy]

@TheLizzard Thank you for a definitely non trivial patch. Will test when I can before looking at Stan's alternative.

@StanFromIreland When I merged to patch to support t-strings, I thought about using the new tokenizer and associated PyREPL code. I made a new issue for this and changed your PR to attach it there.

[StanFromIreland]

I think we can merge the patch first, and then look at the larger issue.