Skip to content

Allow fixing unicode files with non-ASCII characters on Windows #252

@BayerGL

Description

@BayerGL

Description

Cannot run autoimport on a UTF-8 file with cyrillic characters on Windows.

Steps to reproduce

> autoimport <any_file_with_non_ascii>.py

Current behavior

Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\Projects\MyProject\.venv\Scripts\autoimport.exe\__main__.py", line 7, in <module>
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\autoimport\entrypoints\cli.py", line 94, in cli
    fixed_code = services.fix_files(flattened_files, config)
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\autoimport\services.py", line 28, in fix_files
    source = file_wrapper.read()
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\encodings\cp1251.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 29668: character maps to <undefined>

Desired behavior

Just works and fixes the file.

Environment

Windows 11
Python 3.9
autoimport==1.3.3

I made a workaround for myself by adding encoding='utf-8' at

files.append(click.File("r+").convert(py_file, None, None))

            files.append(click.File("r+", encoding='utf-8').convert(py_file, None, None))

and at

return [click.File("r+").convert(value, param, ctx)]

            return click.File("r+", encoding='utf-8').convert(value, param, ctx)

But it would be better to have a special encoding parameter at the command line arguments and/or in the pyproject.toml configuration file.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions