Merge branch 'master' into maint/sconsign

Author: bdbaddog

CHANGES.txt

@@ -53,10 +53,32 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
     - Fix of the --debug=sconscript option to return exist statements when using return
       statement with stop flag enabled
 
+  From Prabhu S. Khalsa:
+    - Fix typo in user documentation (issue #4458)
+
+  From Andrew Morrow:
+    - The NewParallel scheduler is now the default, the `tm_v2` flag is removed,
+      and the old scheduler is opt-in under `--experimental=legacy_sched`. Additionally,
+      the new scheduler is now used for -j1 builds as well.
+    - A python interpreter with support for the `threading` package is now required,
+      and this is enforced on startup. SCons currently sets its minimum supported
+      Python to 3.6, and it was not until Python 3.7 where `threading` became
+      default supported. In practice, we expect most real world Python 3.6 deployments
+      will have `threading` support enabled, so this will not be an issue.
+    - CacheDir writes no longer happen within the taskmaster critical section,
+      and therefore can run in parallel with both other CacheDir writes and the
+      taskmaster DAG walk.
+
   From Mats Wichmann:
     - Add support for Python 3.13 (as of alpha 2). So far only affects
       expected bytecodes in ActionTests.py.
     - sconsign cleanup - remove some dead code, minor manpage tweaks.
+    - Be more cautious about encodings fetching command output on Windows.
+      Problem occurs in piped-spawn scenario, used by Configure tests.
+      Fixes #3529.
+    - Clarify/fix documentation of Scanners in User Guide and Manpage.
+      Fixes #4468.
+    - Add Pseudo() to global functions, had been omitted. Fixes #4474.
 
 
 RELEASE 4.6.0 -  Sun, 19 Nov 2023 17:22:20 -0700
@@ -7948,4 +7970,3 @@ A brief overview of important functionality available in release 0.01:
   - Linux packages available in RPM and Debian format.
 
   - Windows installer available.
-

CONTRIBUTING.rst

@@ -297,13 +297,13 @@ You may specifically list one or more tests to be run::
 
         $ python runtest.py SCons/BuilderTests.py
 
-        $ python runtest.py test/option-j.py test/Program.py
+        $ python runtest.py test/option/option-j.py test/Program.py
 
 You also use the ``-f`` option to execute just the tests listed in a specified
 text file::
 
         $ cat testlist.txt
-        test/option-j.py
+        test/option/option-j.py
         test/Program.py
         $ python runtest.py -f testlist.txt
 

RELEASE.txt

@@ -33,6 +33,7 @@ CHANGED/ENHANCED EXISTING FUNCTIONALITY
   that the generated function argument list matches the function's
   prototype when including a header file. Fixes GH Issue #4320
 - Now supports pre-release Python 3.13
+- Support for Python versions without support for the `threading` package has been removed
 
 FIXES
 -----
@@ -49,12 +50,24 @@ FIXES
 - MSVS: Fix the msvs project generation test scripts so that "false positive" tests
   results are not possible when the initial build is successful and the command-line
   build of the project file fails.
+- On Windows platform, when collecting command output (Configure checks),
+  make sure decoding of bytes doesn't fail.
+- Documentation indicated that both Pseudo() and env.Pseudo() were usable,
+  but Pseudo() did not work; is now enabled.
 
 IMPROVEMENTS
 ------------
 
 - Use of NotImplemented instead of NotImplementedError for special methods
   of _ListVariable class
+- The NewParallel scheduler is now the default, the `tm_v2` flag is removed,
+  and the old scheduler is opt-in under `--experimental=legacy_sched`. Additionally,
+  the new scheduler is now used for -j1 builds as well.
+  NOTE: This should significantly improve SCons performance for larger parallel builds
+        (Larger -j values)
+- CacheDir writes no longer happen within the taskmaster critical section, and therefore
+  can run in parallel with both other CacheDir writes and the taskmaster DAG walk.
+
 
 PACKAGING
 ---------
@@ -64,9 +77,9 @@ PACKAGING
 DOCUMENTATION
 -------------
 
-- List any significant changes to the documentation (not individual
-  typo fixes, even if they're mentioned in src/CHANGES.txt to give
-  the contributor credit)
+- Fixed the Scanner examples in the User Guide to be runnable and added
+  some more explantion. Clarified discussion of the scanner function in
+  the Scanner Objects section of the manpage.
 
 DEVELOPMENT
 -----------

SCons/Platform/win32.py

@@ -165,16 +165,18 @@ def piped_spawn(sh, escape, cmd, args, env, stdout, stderr):
     # and do clean up stuff
     if stdout is not None and not stdoutRedirected:
         try:
-            with open(tmpFileStdoutName) as tmpFileStdout:
-                stdout.write(tmpFileStdout.read())
+            with open(tmpFileStdoutName, "rb") as tmpFileStdout:
+                output = tmpFileStdout.read()
+                stdout.write(output.decode(stdout.encoding, "replace"))
             os.remove(tmpFileStdoutName)
         except OSError:
             pass
 
     if stderr is not None and not stderrRedirected:
         try:
-            with open(tmpFileStderrName) as tmpFileStderr:
-                stderr.write(tmpFileStderr.read())
+            with open(tmpFileStderrName, "rb") as tmpFileStderr:
+                errors = tmpFileStderr.read()
+                stderr.write(errors.decode(stderr.encoding, "replace"))
             os.remove(tmpFileStderrName)
         except OSError:
             pass

SCons/SConf.py

@@ -251,10 +251,9 @@ def display_cached_string(self, bi) -> None:
     def failed(self):
         # check, if the reason was a ConfigureDryRunError or a
         # ConfigureCacheError and if yes, reraise the exception
-        exc_type = self.exc_info()[0]
+        exc_type, exc, _ = self.exc_info()
         if issubclass(exc_type, SConfError):
-            # TODO pylint E0704: bare raise not inside except
-            raise
+            raise exc
         elif issubclass(exc_type, SCons.Errors.BuildError):
             # we ignore Build Errors (occurs, when a test doesn't pass)
             # Clear the exception to prevent the contained traceback

SCons/Script/Main.py

@@ -1447,6 +1447,13 @@ def main() -> None:
         sys.stderr.write("scons: *** Minimum Python version is %d.%d.%d\n" %minimum_python_version)
         sys.exit(1)
 
+    try:
+        import threading
+    except ImportError:
+        msg = "scons: *** SCons version %s requires a Python interpreter with support for the `threading` package"
+        sys.stderr.write(msg % SConsVersion)
+        sys.exit(1)
+
     parts = ["SCons by Steven Knight et al.:\n"]
     try:
         import SCons

SCons/Script/SConsOptions.py

@@ -40,7 +40,7 @@
 
 diskcheck_all = SCons.Node.FS.diskcheck_types()
 
-experimental_features = {'warp_speed', 'transporter', 'ninja', 'tm_v2'}
+experimental_features = {'warp_speed', 'transporter', 'ninja', 'legacy_sched'}
 
 
 def diskcheck_convert(value):

SCons/Script/__init__.py

@@ -343,6 +343,7 @@ def Variables(files=None, args=ARGUMENTS):
     'Local',
     'ParseDepends',
     'Precious',
+    'Pseudo',
     'PyPackageDir',
     'Repository',
     'Requires',