Merge branch 'master' into jbrill-msvc-detect

Manually resolved:
* CHANGES.txt
* RELEASE.txt

Author: jcbrill

.appveyor.yml

@@ -36,7 +36,7 @@ environment:
   matrix:
     # Test oldest and newest supported Pythons, and a subset in between.
     # Skipping 3.7 and 3.9 at this time
-    - WINPYTHON: "Python311"
+    - WINPYTHON: "Python312"
 
     - WINPYTHON: "Python310"
 
@@ -50,15 +50,15 @@ matrix:
   exclude:
     # test python 3.6 on Visual Studio 2017 image
     - image: Visual Studio 2017
-      WINPYTHON: "Python311"
+      WINPYTHON: "Python312"
     - image: Visual Studio 2017
       WINPYTHON: "Python310"
     - image: Visual Studio 2017
       WINPYTHON: "Python38"
 
     # test python 3.8 on Visual Studio 2019 image
     - image: Visual Studio 2019
-      WINPYTHON: "Python311"
+      WINPYTHON: "Python312"
     - image: Visual Studio 2019
       WINPYTHON: "Python310"
     - image: Visual Studio 2019

CHANGES.txt

@@ -9,39 +9,7 @@ NOTE: 4.3.0 now requires Python 3.6.0 and above. Python 3.5.x is no longer suppo
 
 RELEASE  VERSION/DATE TO BE FILLED IN LATER
 
-  From Ataf Fazledin Ahamed:
-    - Use of NotImplemented instead of NotImplementedError for special methods
-      of _ListVariable class
-
   From Joseph Brill:
-    - Fix issue #2755: the msvs tool no longer writes the OS environment SCONS_HOME
-      value into the SCons environment when the SCONS_HOME variable already exists
-      in the SCons environment.  Prior to this change, a valid user-defined SCons
-      environment value for SCONS_HOME would be overwritten with the OS environment
-      value of SCONS_HOME which could be None (i.e., undefined).
-    - Update the windows registry keys for detection of Visual Studio 2015 Express
-      ('14.0Exp'): the VS2015 registry key ('WDExpress') appears to be different
-      than the registry key ('VCExpress') for earlier Visual Studio express
-      versions.  The registry key value is relative to the installation root rather
-      than the VC folder and requires additional path components during evaluation.
-    - Fix the vs-6.0-exec.py test script: the msvs generated project is 'foo.dsp'
-      and the command-line invocation of the Visual Studio development environment
-      program was attempting to build 'test.dsp'.  The command-line invocation was
-      changed to build 'foo.dsp'.
-    - Update the msvs project generation test scripts: the msvs project execution
-      tests could produce a "false positive" test result when the test executable is
-      correctly built via the SConstruct env.Program() call and the command-line
-      invocation of the Visual Studio development environment program fails.  The
-      test passes due to the existence of the test executable from the initial
-      build.  The tests were modified to delete the test executable, object file,
-      and sconsign file prior to the command-line invocation of the VS development
-      binary.
-    - Method unlink_files was added to the TestCmd class that unlinks a list of
-      files from a specified directory.  An attempt to unlink a file is made only
-      when the file exists; otherwise, the file is ignored.
-    - Fix issue #4320: add an optional argument list string to configure's CheckFunc
-      method so that the generated function argument list matches the function's
-      prototype when including a header file.
     - For msvc version specifications without an 'Exp' suffix, an express installation
       is used when no other edition is detected for the msvc version.  Similarly, an
       express installation of the IDE binary is used when no other IDE edition is
@@ -96,15 +64,62 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
       and then later an environment is constructed with a user specified vswhere
       executable that detects new msvc installations.
 
+  From Mats Wichmann:
+
+    - Updated Value Node docs and tests.
+    - Python 3.13 compat: re.sub deprecated count, flags as positional args,
+      caused update-release-info test to fail.
+    - Dump() with json format selected now recognizes additional compound types
+      (UserDict and UserList), which improves the detail of the display.
+      json output is also sorted, to match the default display.
+
+
+RELEASE 4.7.0 -  Sun, 17 Mar 2024 17:22:20 -0700
+
+  From Ataf Fazledin Ahamed:
+    - Use of NotImplemented instead of NotImplementedError for special methods
+      of _ListVariable class
+
+  From Joseph Brill:
+    - Fix issue #2755: the msvs tool no longer writes the OS environment SCONS_HOME
+      value into the SCons environment when the SCONS_HOME variable already exists
+      in the SCons environment.  Prior to this change, a valid user-defined SCons
+      environment value for SCONS_HOME would be overwritten with the OS environment
+      value of SCONS_HOME which could be None (i.e., undefined).
+    - Update the windows registry keys for detection of Visual Studio 2015 Express
+      ('14.0Exp'): the VS2015 registry key ('WDExpress') appears to be different
+      than the registry key ('VCExpress') for earlier Visual Studio express
+      versions.  The registry key value is relative to the installation root rather
+      than the VC folder and requires additional path components during evaluation.
+    - Fix the vs-6.0-exec.py test script: the msvs generated project is 'foo.dsp'
+      and the command-line invocation of the Visual Studio development environment
+      program was attempting to build 'test.dsp'.  The command-line invocation was
+      changed to build 'foo.dsp'.
+    - Update the msvs project generation test scripts: the msvs project execution
+      tests could produce a "false positive" test result when the test executable is
+      correctly built via the SConstruct env.Program() call and the command-line
+      invocation of the Visual Studio development environment program fails.  The
+      test passes due to the existence of the test executable from the initial
+      build.  The tests were modified to delete the test executable, object file,
+      and sconsign file prior to the command-line invocation of the VS development
+      binary.
+    - Method unlink_files was added to the TestCmd class that unlinks a list of
+      files from a specified directory.  An attempt to unlink a file is made only
+      when the file exists; otherwise, the file is ignored.
+    - Fix issue #4320: add an optional argument list string to configure's CheckFunc
+      method so that the generated function argument list matches the function's
+      prototype when including a header file.
+
   From Thaddeus Crews:
+    - Explicitly wrap non-serializable values in json dump
     - Implemented SCons.Util.sctyping as a safe means of hinting complex types. Currently
       only implemented for `Executor` as a proof-of-concept.
 
   From William Deegan:
     - Fix sphinx config to handle SCons versions with post such as: 4.6.0.post1
 
   From Michał Górny:
-    - Remove unecessary dependencies on pypi packages from setup.cfg
+    - Remove unnecessary dependencies on pypi packages from setup.cfg
 
   From Sten Grüner:
     - Fix of the --debug=sconscript option to return exist statements when using return
@@ -144,6 +159,7 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
       harder to decode non-UTF-8 text. SCons.Util.to_Text now exists
       to convert a byte stream, such as "raw" file data.  Fixes #3569, #4462.
       The Pseudo manpage entry was updated to provide more clarity.
+    - Clarify how SCons finds the project top directory, and what that is used for.
     - The internal routine which implements the PyPackageDir function
       would fail with an exception if called with a module which is
       not found.  It will now return None.  Updated manpage entry and
@@ -152,6 +168,19 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
       describe the Microsoft C++ compiler. Update the version table slightly.
       Amplified the usage of MSVC_VERSION.
     - Improve SharedLibrary docs a bit.
+    - More consistent use of &Python; in the manpage.  A few links added.
+      A warning about overwriting env['ENV'] and one about Configure
+      checks possibly not running in in no-exec mode also added.
+    - Update warnings module: adds docstrings, drop three unused warnings
+      (DeprecatedSourceCodeWarning, TaskmasterNeedsExecuteWarning,
+      DeprecatedMissingSConscriptWarning) add two warnings to manpage
+      (cache-cleanup-error, future-reserved-variable), improve unittest, tweak
+      Sphinx build.
+    - Add locking around creation of CacheDir config file. Fixes #4489.
+    - Clarify MergeFlags usage of a dict argument.
+    - SCons documentation build can now be controlled through SKIP_DOC
+      variable - rather than just true/false can now specify
+      skip none, skip all, skip pdf docs, skip api docs.
 
 
 RELEASE 4.6.0 -  Sun, 19 Nov 2023 17:22:20 -0700

README-SF.rst

@@ -134,7 +134,7 @@ By default, the above commands will do the following:
 - Install scripts named "scons" and "sconsign" scripts in the default system
   script directory (/usr/bin or C:\\Python\*\\Scripts, for example).
 
-- Install "scons-4.5.1.exe" and "scons.exe" executables in the Python
+- Install "scons-4.7.0.exe" and "scons.exe" executables in the Python
   prefix directory on Windows (C:\\Python\*, for example).
 
 - Install the SCons build engine (a Python module) in the standard Python library directory
@@ -391,7 +391,7 @@ about `Executing SCons Without Installing`_)::
 Depending on the utilities installed on your system, any or all of the
 following packages will be built::
 
-    SCons-4.5.1-py3-none-any.whl
+    SCons-4.7.0-py3-none-any.whl
     SCons-4.7.0ayyyymmdd.tar.gz
     SCons-4.7.0ayyyymmdd.zip
     scons-doc-4.7.0ayyyymmdd.tar.gz

README-package.rst

@@ -192,5 +192,5 @@ For More Information
 Check the SCons web site at https://www.scons.org/
 
 
-Copyright (c) 2001 - 2021 The SCons Foundation
+Copyright (c) 2001 - 2024 The SCons Foundation
 

README.rst

@@ -299,4 +299,4 @@ many contributors, including but not at all limited to:
 
 \... and many others.
 
-Copyright (c) 2001 - 2021 The SCons Foundation
+Copyright (c) 2001 - 2024 The SCons Foundation

RELEASE.txt

@@ -6,20 +6,17 @@ Past official release announcements appear at:
 
 ==================================================================
 
-A new SCons release, 4.6.1, is now available on the SCons download page:
+A new SCons release, 4.7.1, is now available on the SCons download page:
 
     https://scons.org/pages/download.html
 
 
-Here is a summary of the changes since 4.6.0:
+Here is a summary of the changes since 4.7.0:
 
 NEW FUNCTIONALITY
 -----------------
 
-- Method unlink_files was added to the TestCmd class that unlinks a list of files
-  from a specified directory.  An attempt to unlink a file is made only when the
-  file exists; otherwise, the file is ignored.
-
+- List new features (presumably why a checkpoint is being released)
 
 DEPRECATED FUNCTIONALITY
 ------------------------
@@ -29,11 +26,9 @@ DEPRECATED FUNCTIONALITY
 CHANGED/ENHANCED EXISTING FUNCTIONALITY
 ---------------------------------------
 
-- Add an optional argument list string to configure's CheckFunc method so
-  that the generated function argument list matches the function's
-  prototype when including a header file. Fixes GH Issue #4320
-- Now supports pre-release Python 3.13
-- Support for Python versions without support for the `threading` package has been removed
+- Dump() with json format selected now recognizes additional compound types
+  (UserDict and UserList), which improves the detail of the display.
+  json output is also sorted, to match the default display.
 - MSVC: For msvc version specifications without an 'Exp' suffix, an express
   installation is used when no other edition is detected for the msvc version.
   This was the behavior for Visual Studio 2008 (9.0) through Visual Studio 2015
@@ -44,18 +39,6 @@ CHANGED/ENHANCED EXISTING FUNCTIONALITY
 FIXES
 -----
 
-- Fix of the --debug=sconscript option to return exist statements when using return
-  statement with stop flag enabled
-- MSVS: prevent overwriting the SCons environment variable SCONS_HOME with the OS
-  environment value of SCONS_HOME in the msvs tool.
-- MSVC: Fix the detection of Visual Studio 2015 Express ('14.0Exp') by adding a
-  registry key definition and updating the installation root-relative registry value
-  at runtime for the location of the VC folder.
-- MSVS: Fix the msvs project generation test for MSVS 6.0 to use the correct name of
-  the generated project file.
-- MSVS: Fix the msvs project generation test scripts so that "false positive" tests
-  results are not possible when the initial build is successful and the command-line
-  build of the project file fails.
 - MSVC: Visual Studio 2010 (10.0) could be inadvertently detected due to an
   sdk-only install of Windows SDK 7.1.  An sdk-only install of Visual Studio
   2010 is ignored as the msvc batch files will fail. The installed files are
@@ -85,30 +68,10 @@ FIXES
   For example, when no vswhere executable is found for the initial detection
   and then later an environment is constructed with a user specified vswhere
   executable that detects new msvc installations.
-- On Windows platform, when collecting command output (Configure checks),
-  make sure decoding of bytes doesn't fail.
-- Documentation indicated that both Pseudo() and env.Pseudo() were usable,
-  but Pseudo() did not work; is now enabled.
-- Improve handling of file data that SCons itself processes - as in
-  scanners - try harder to decode non-UTF-8 text.
-- PyPackageDir no longer fails if passed a module name which cannot be found,
-  now returns None.
 
 IMPROVEMENTS
 ------------
 
-- Use of NotImplemented instead of NotImplementedError for special methods
-  of _ListVariable class
-- The NewParallel scheduler is now the default, the `tm_v2` flag is removed,
-  and the old scheduler is opt-in under `--experimental=legacy_sched`. Additionally,
-  the new scheduler is now used for -j1 builds as well.
-  NOTE: This should significantly improve SCons performance for larger parallel builds
-        (Larger -j values)
-- CacheDir writes no longer happen within the taskmaster critical section, and therefore
-  can run in parallel with both other CacheDir writes and the taskmaster DAG walk.
-- The NewParallel scheduler now only adds threads as new work requiring execution
-  is discovered, up to the limit set by -j. This should reduce resource utilization
-  when the achievable parallelism in the DAG is less than the -j limit.
 - MSVC: Visual Studio 2015 Express (14.0Exp) does not support the sdk version
   argument. Visual Studio 2015 Express does not support the store argument for
   target architectures other than x86.  Script argument validation now takes into
@@ -120,26 +83,23 @@ IMPROVEMENTS
 PACKAGING
 ---------
 
-- Remove unecessary dependencies on pypi packages from setup.cfg
+- List changes in the way SCons is packaged and/or released
+
 
 DOCUMENTATION
 -------------
 
-- Fixed the Scanner examples in the User Guide to be runnable and added
-  some more explantion. Clarified discussion of the scanner function in
-  the Scanner Objects section of the manpage.
-- The manpage entry for Pseudo was clarified.
-- The manpage entry for SharedLibrary was clarified.
+- Updated Value Node docs.
+
+
 
 DEVELOPMENT
 -----------
 
-- Fix sphinx config to handle SCons versions with post such as: 4.6.0.post1
-- Created SCons.Util.sctyping to contain complex type information, allowing
-  for repo-wide type hinting without causing cyclical dependencies.
+- List visible changes in the way SCons is developed
 
 Thanks to the following contributors listed below for their contributions to this release.
 ==========================================================================================
 .. code-block:: text
 
-    git shortlog --no-merges -ns 4.6.0..HEAD
+    git shortlog --no-merges -ns 4.0.1..HEAD

ReleaseConfig

@@ -31,7 +31,7 @@
 # If the release type is not 'final', the patchlevel is set to the
 # release date.  This value is mandatory and must be present in this file.
 #version_tuple = (2, 2, 0, 'final', 0)
-version_tuple = (4, 6, 1, 'a', 0)
+version_tuple = (4, 6, 2, 'a', 0)
 
 # Python versions prior to unsupported_python_version cause a fatal error
 # when that version is used.  Python versions prior to deprecate_python_version

SCons/CacheDir.py

@@ -34,7 +34,7 @@
 import SCons.Action
 import SCons.Errors
 import SCons.Warnings
-import SCons
+import SCons.Util
 
 cache_enabled = True
 cache_debug = False
@@ -169,15 +169,16 @@ def _readconfig(self, path):
         """
         config_file = os.path.join(path, 'config')
         try:
+            # still use a try block even with exist_ok, might have other fails
             os.makedirs(path, exist_ok=True)
-        except FileExistsError:
-            pass
         except OSError:
             msg = "Failed to create cache directory " + path
             raise SCons.Errors.SConsEnvironmentError(msg)
 
         try:
-            with open(config_file, 'x') as config:
+            with SCons.Util.FileLock(config_file, timeout=5, writer=True), open(
+                config_file, "x"
+            ) as config:
                 self.config['prefix_len'] = 2
                 try:
                     json.dump(self.config, config)
@@ -186,9 +187,11 @@ def _readconfig(self, path):
                     raise SCons.Errors.SConsEnvironmentError(msg)
         except FileExistsError:
             try:
-                with open(config_file) as config:
+                with SCons.Util.FileLock(config_file, timeout=5, writer=False), open(
+                    config_file
+                ) as config:
                     self.config = json.load(config)
-            except ValueError:
+            except (ValueError, json.decoder.JSONDecodeError):
                 msg = "Failed to read cache configuration for " + path
                 raise SCons.Errors.SConsEnvironmentError(msg)
 

SCons/Defaults.xml

@@ -57,6 +57,14 @@ containing the
 <filename>SConstruct</filename>
 file.
 </para>
+<para>
+If variant directories are in use,
+and the configure check results should not be
+shared between variants,
+you can set &cv-CONFIGUREDIR;
+and &cv-link-CONFIGURELOG; so they are
+unique per variant directory.
+</para>
 </summary>
 </cvar>
 
@@ -71,6 +79,14 @@ containing the
 <filename>SConstruct</filename>
 file.
 </para>
+<para>
+If variant directories are in use,
+and the configure check results should not be
+shared between variants,
+you can set &cv-link-CONFIGUREDIR;
+and &cv-CONFIGURELOG; so they are
+unique per variant directory.
+</para>
 </summary>
 </cvar>