-
-
Notifications
You must be signed in to change notification settings - Fork 33.1k
Closed as not planned
Labels
stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-parsertype-featureA feature request or enhancementA feature request or enhancement
Description
Bug report
Bug description:
Tab characters used as white space do not roundtrip when using untokenize
. Here's an example:
import tokenize, io
source = "a +\tb"
tokens = list(tokenize.generate_tokens(io.StringIO(source).readline))
x = tokenize.untokenize(tokens)
print(x)
# a + b
Here, the tab is replaced with a regular space. Given that untokenize tries to match the source string exactly, I think we should fix this. We can do that by getting the characters from the source line rather than always using a normal space:
Line 185 in 9c2bb7d
self.tokens.append(" " * col_offset) |
I'll send a fix in a moment :)
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs
Metadata
Metadata
Assignees
Labels
stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-parsertype-featureA feature request or enhancementA feature request or enhancement