Decode command output conservatively

On win32, rather than reading the temporary file used for external
command output as text, read it as bytes and convert to the codec of
the stream it will be written to, with suitable error fallback defined
so it doesn't fail. Affects Configure usags.  Fixes SCons#3529.

Signed-off-by: Mats Wichmann <[email protected]>

Author: mwichmann

CHANGES.txt

@@ -69,6 +69,9 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
   From Mats Wichmann:
     - Add support for Python 3.13 (as of alpha 2). So far only affects
       expected bytecodes in ActionTests.py.
+    - Be more cautious about encodings fetching command output on Windows.
+      Problem occurs in piped-spawn scenario, used by Configure tests.
+      Fixes #3529.
 
 
 RELEASE 4.6.0 -  Sun, 19 Nov 2023 17:22:20 -0700

RELEASE.txt

@@ -50,6 +50,8 @@ FIXES
 - MSVS: Fix the msvs project generation test scripts so that "false positive" tests
   results are not possible when the initial build is successful and the command-line
   build of the project file fails.
+- On Windows platform, when collecting command output (Configure checks),
+  make sure decoding of bytes doesn't fail.
 
 IMPROVEMENTS
 ------------

SCons/Platform/win32.py

@@ -165,16 +165,18 @@ def piped_spawn(sh, escape, cmd, args, env, stdout, stderr):
     # and do clean up stuff
     if stdout is not None and not stdoutRedirected:
         try:
-            with open(tmpFileStdoutName) as tmpFileStdout:
-                stdout.write(tmpFileStdout.read())
+            with open(tmpFileStdoutName, "rb") as tmpFileStdout:
+                output = tmpFileStdout.read()
+                stdout.write(output.decode(stdout.encoding, "replace"))
             os.remove(tmpFileStdoutName)
         except OSError:
             pass
 
     if stderr is not None and not stderrRedirected:
         try:
-            with open(tmpFileStderrName) as tmpFileStderr:
-                stderr.write(tmpFileStderr.read())
+            with open(tmpFileStderrName, "rb") as tmpFileStderr:
+                errors = tmpFileStderr.read()
+                stdout.write(errors.decode(stderr.encoding, "replace"))
             os.remove(tmpFileStderrName)
         except OSError:
             pass