Skip to content

Commit 162c5e1

Browse files
committed
test: explicitly encode arguments for Python 3
The argument handling for `subprocess.check_output` differs between Python 2.7, [3.0, 3.6], [3.7). With Python 2.7, the list is implicitly decoded and we cannot use the `encode` method as it will attempt to decode it as ASCII which will fail with unicode data. Python [3.0, 3.6] will attempt to encode the arguments before executing, but will do so based on the system locale. In the case that the locale is `POSIX`, the encoding will be attempted in ASCII, which again fails when unicode data is used. Python [3.7) will accept the encoded strings and pass them through. It will implicitly encode the arguments using the file system encoding (normally UTF-8), which will allow it to actually work with the content encoded or decoded. In order to provide maximal compatibility, pre-encode the arguments as UTF-8 when using python 3. Rely on the interpreter encoding the argument implicitly on Python 2. This allows running the incrParse.simple test with python 2.7, python 3.6, python 3.7, python 3.8 with `LC_ALL=POSIX` as is the default on Ubuntu Server. Thanks to @tbkka for the details on how to reproduce this issue and the help with the python conversion!
1 parent fb364ae commit 162c5e1

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

utils/incrparse/test_util.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,11 @@ def run_command(cmd):
2525
if sys.version_info[0] < 3:
2626
cmd = list(map(lambda s: s.encode('utf-8'), cmd))
2727
print(' '.join([escapeCmdArg(arg) for arg in cmd]))
28-
return subprocess.check_output(cmd, stderr=subprocess.STDOUT)
28+
if sys.version_info[0] < 3:
29+
return subprocess.check_output(cmd, stderr=subprocess.STDOUT)
30+
else:
31+
return subprocess.check_output(list(map(lambda s: s.encode('utf-8'), cmd)),
32+
stderr=subprocess.STDOUT)
2933

3034

3135
def parseLine(line, line_no, test_case, incremental_edit_args, reparse_args,

0 commit comments

Comments
 (0)