Merge branch 'master' into jbrill-msvc-detect

Manually resolved conflicts:
* CHANGES.txt
* RELEASE.txt
* test/MSVS/vs-14.3-exec.py

Author: jcbrill

CHANGES.txt

@@ -64,8 +64,26 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
       added to the default vswhere executable search list after the Chocolatey
       installation location.
 
-  From Mats Wichmann:
+  From Thaddeus Crews:
+    - GetSConsVersion() to grab the latest SCons version without needing to
+      access SCons internals.
+
+  From Raymond Li:
+    - Fix issue #3935: OSErrors are now no longer hidden during execution of
+      Actions. All exceptions during the execution of an Action are now
+      returned by value rather than by raising an exception, for more
+      consistent behavior.
+      NOTE: With this change, user created Actions should now catch and handle
+      expected exceptions (whereas previously many of these were silently
+      caught and suppressed by the SCons Action exection code).
+
+  From Ryan Carsten Schmidt:
+    - Teach ParseFlags to put a --stdlib=libname argument into CXXFLAGS.
+      If placed in CCFLAGS (the default location), it could be fed to the
+      C compiler (gcc, clang) where it is not applicable and causes a
+      warning message.
 
+  From Mats Wichmann:
     - Updated Value Node docs and tests.
     - Python 3.13 compat: re.sub deprecated count, flags as positional args,
       caused update-release-info test to fail.
@@ -74,6 +92,30 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
       json output is also sorted, to match the default display.
     - Python 3.13 (alpha) changes the behavior of isabs() on Windows. Adjust
       SCons usage of in NodeInfo classes to match.  Fixes #4502, #4504.
+    - Drop duplicated __getstate__ and __setstate__ methods in AliasNodeInfo,
+      FileNodeInfo and ValueNodeInfo classes, as they are identical to the
+      ones in parent NodeInfoBase and can just be inherited.
+    - Update manpage for Tools, and for TOOL, which also gets a minor
+      tweak for how it's handled (should be more accurate in a few situations).
+    - Test framework now uses a subdirectory named "scons" below the base
+      temporary directory. This gives something invariant to tell antivirus
+      to ignore without having to exclude tmpdir itself. Fixes #4509.
+    - MSVS "live" tests of project files adjusted to look for the generated
+      executable with an exe sufffix
+    - Documentation build now properly passes through skipping the PDF
+      (and EPUB) builds of manpage and user guide; this can also be done
+      manually if directly calling doc/man/SConstruct and doc/user/SConstruct
+      by adding SKIP_PDF=1.  This should help with distro packaging of SCons,
+      which now does not need "fop" and other tools to be set up in order to
+      build pdf versions which are then ignored.
+    - Add the ability to print a Variables object for debugging purposes
+      (provides a __str__ method in the class).
+    - Mark Python 3.6 support as deprecated.
+    - Clean up Variables: more consistently call them variables (finish the
+      old change from Options) in docstrings, etc.; some typing and other
+      tweaks.  Update manpage and user guide for Variables usage.
+    - Regularize internal usage of Python version strings and drop one
+      old Python 2-only code block in a test.
 
 
 RELEASE 4.7.0 -  Sun, 17 Mar 2024 17:22:20 -0700

RELEASE.txt

@@ -16,12 +16,13 @@ Here is a summary of the changes since 4.7.0:
 NEW FUNCTIONALITY
 -----------------
 
-- List new features (presumably why a checkpoint is being released)
+- GetSConsVersion() added to retrieve the SCons version.
 
 DEPRECATED FUNCTIONALITY
 ------------------------
 
-- List anything that's been deprecated since the last release
+- Mark Python 3.6 support as deprecated. Use --warn=no-python-version
+  to quiet the warning.
 
 CHANGED/ENHANCED EXISTING FUNCTIONALITY
 ---------------------------------------
@@ -31,6 +32,18 @@ CHANGED/ENHANCED EXISTING FUNCTIONALITY
   json output is also sorted, to match the default display.
 - Python 3.13 changes the behavior of isabs() on Windows. Adjust SCons
   usage of this in NodeInfo classes to avoid test problems.
+- Drop duplicated __getstate__ and __setstate__ methods in AliasNodeInfo,
+  FileNodeInfo and ValueNodeInfo classes, as they are identical to the
+  ones in parent NodeInfoBase and can just be inherited.
+- All exceptions during the execution of an Action are now returned by value
+  rather than by raising an exception, for more consistent behavior.
+  NOTE: With this change, user created Actions should now catch and handle
+  expected exceptions (whereas previously many of these were silently caught
+  and suppressed by the SCons Action exection code).
+- ParseFlags now sorts a --stdlib=libname argument into CXXFLAGS instead
+  of CCFLAGS; the latter variable could cause a compiler warning.
+- The implementation of Variables was slightly refactored, there should
+  not be user-visible changes.
 - MSVC: For msvc version specifications without an 'Exp' suffix, an express
   installation is used when no other edition is detected for the msvc version.
   This was the behavior for Visual Studio 2008 (9.0) through Visual Studio 2015
@@ -44,6 +57,7 @@ CHANGED/ENHANCED EXISTING FUNCTIONALITY
 FIXES
 -----
 
+- OSErrors are now no longer hidden during the execution of Actions.
 - MSVC: Visual Studio 2010 (10.0) could be inadvertently detected due to an
   sdk-only install of Windows SDK 7.1.  An sdk-only install of Visual Studio
   2010 is ignored as the msvc batch files will fail. The installed files are
@@ -77,6 +91,9 @@ FIXES
 IMPROVEMENTS
 ------------
 
+- Make the testing framework a little more resilient: the temporary
+  directory for tests now includes a component named "scons" which can
+  be given to antivirus software to exclude.
 - MSVC: Visual Studio 2015 Express (14.0Exp) does not support the sdk version
   argument. Visual Studio 2015 Express does not support the store argument for
   target architectures other than x86.  Script argument validation now takes into
@@ -99,13 +116,21 @@ DOCUMENTATION
 -------------
 
 - Updated Value Node docs.
+- Update manpage for Tools, and for the TOOL variable.
+- Update manpage and user guide for Variables usage.
 
 
 
 DEVELOPMENT
 -----------
 
-- List visible changes in the way SCons is developed
+- Documentation build now properly passes through skipping the PDF
+  (and EPUB) builds of manpage and user guide; this can also be done
+  manually if directly calling doc/man/SConstruct and doc/user/SConstruct
+  by adding SKIP_PDF=1. This should help with distro packaging of SCons,
+  which now does not need "fop" and other tools to be set up in order to
+  build pdf versions which are then ignored.
+
 
 Thanks to the following contributors listed below for their contributions to this release.
 ==========================================================================================

SCons/Action.py

@@ -892,10 +892,10 @@ def scons_subproc_run(scons_env, *args, **kwargs) -> subprocess.CompletedProcess
     kwargs['check'] = check
 
     # TODO: Python version-compat stuff: remap/remove too-new args if needed
-    if 'text' in kwargs and sys.version_info[:3] < (3, 7):
+    if 'text' in kwargs and sys.version_info < (3, 7):
         kwargs['universal_newlines'] = kwargs.pop('text')
 
-    if 'capture_output' in kwargs and sys.version_info[:3] < (3, 7):
+    if 'capture_output' in kwargs and sys.version_info < (3, 7):
         capture_output = kwargs.pop('capture_output')
         if capture_output:
             kwargs['stdout'] = kwargs['stderr'] = PIPE
@@ -1454,16 +1454,6 @@ def execute(self, target, source, env, executor: Optional[ExecutorType] = None):
                 except TypeError:
                     result.command=self.strfunction(target, source, env)
 
-                # FIXME: This maintains backward compatibility with respect to
-                # which type of exceptions were returned by raising an
-                # exception and which ones were returned by value. It would
-                # probably be best to always return them by value here, but
-                # some codes do not check the return value of Actions and I do
-                # not have the time to modify them at this point.
-                if (exc_info[1] and
-                    not isinstance(exc_info[1], EnvironmentError)):
-                    raise result
-
             return result
         finally:
             # Break the cycle between the traceback object and this

SCons/CacheDirTests.py

@@ -164,7 +164,7 @@ def __setitem__(self, name, value) -> None:
         # so that _readconfig* will try to rewrite it
         old_config = os.path.join(self._CacheDir.path, "config")
         os.remove(old_config)
-        
+
         try:
             self._CacheDir._readconfig(self._CacheDir.path)
             assert False, "Should have raised exception and did not"
@@ -315,23 +315,17 @@ def mkdir(dir, mode: int=0) -> None:
         old_warn_exceptions = SCons.Warnings.warningAsException(1)
         SCons.Warnings.enableWarningClass(SCons.Warnings.CacheWriteErrorWarning)
 
-        try:
-            cd_f7 = self.test.workpath("cd.f7")
-            self.test.write(cd_f7, "cd.f7\n")
-            f7 = self.File(cd_f7, 'f7_bsig')
-
-            warn_caught = 0
-            try:
-                f7.push_to_cache()
-            except SCons.Errors.BuildError as e:
-                assert e.exc_info[0] == SCons.Warnings.CacheWriteErrorWarning
-                warn_caught = 1
-            assert warn_caught
-        finally:
-            shutil.copy2 = save_copy2
-            os.mkdir = save_mkdir
-            SCons.Warnings.warningAsException(old_warn_exceptions)
-            SCons.Warnings.suppressWarningClass(SCons.Warnings.CacheWriteErrorWarning)
+        cd_f7 = self.test.workpath("cd.f7")
+        self.test.write(cd_f7, "cd.f7\n")
+        f7 = self.File(cd_f7, 'f7_bsig')
+
+        warn_caught = 0
+        r = f7.push_to_cache()
+        assert r.exc_info[0] == SCons.Warnings.CacheWriteErrorWarning
+        shutil.copy2 = save_copy2
+        os.mkdir = save_mkdir
+        SCons.Warnings.warningAsException(old_warn_exceptions)
+        SCons.Warnings.suppressWarningClass(SCons.Warnings.CacheWriteErrorWarning)
 
     def test_no_strfunction(self) -> None:
         """Test handling no strfunction() for an action."""

SCons/Defaults.py

@@ -291,7 +291,15 @@ def copy_func(dest, src, symlinks: bool=True) -> int:
 
     elif os.path.islink(src):
         if symlinks:
-            os.symlink(os.readlink(src), dest)
+            try:
+                os.symlink(os.readlink(src), dest)
+            except FileExistsError:
+                raise SCons.Errors.BuildError(
+                    errstr=(
+                        f'Error: Copy() called to create symlink at "{dest}",'
+                        ' but a file already exists at that location.'
+                    )
+                )
             return 0
 
         return copy_func(dest, os.path.realpath(src))

SCons/Environment.py

@@ -37,7 +37,7 @@
 import shlex
 from collections import UserDict, UserList, deque
 from subprocess import PIPE, DEVNULL
-from typing import Optional, Sequence
+from typing import Callable, Collection, Optional, Sequence, Union
 
 import SCons.Action
 import SCons.Builder
@@ -881,11 +881,11 @@ def ParseFlags(self, *flags) -> dict:
             'RPATH'         : [],
         }
 
-        def do_parse(arg) -> None:
-            # if arg is a sequence, recurse with each element
+        def do_parse(arg: Union[str, Sequence]) -> None:
             if not arg:
                 return
 
+            # if arg is a sequence, recurse with each element
             if not is_String(arg):
                 for t in arg: do_parse(t)
                 return
@@ -902,7 +902,7 @@ def append_define(name, mapping=mapping) -> None:
                 else:
                     mapping['CPPDEFINES'].append([t[0], '='.join(t[1:])])
 
-            # Loop through the flags and add them to the appropriate option.
+            # Loop through the flags and add them to the appropriate variable.
             # This tries to strike a balance between checking for all possible
             # flags and keeping the logic to a finite size, so it doesn't
             # check for some that don't occur often.  It particular, if the
@@ -926,6 +926,8 @@ def append_define(name, mapping=mapping) -> None:
             append_next_arg_to = None   # for multi-word args
             for arg in params:
                 if append_next_arg_to:
+                    # these are the second pass for options where the
+                    # option-argument follows as a second word.
                     if append_next_arg_to == 'CPPDEFINES':
                         append_define(arg)
                     elif append_next_arg_to == '-include':
@@ -1022,6 +1024,8 @@ def append_define(name, mapping=mapping) -> None:
                     else:
                         key = 'CFLAGS'
                     mapping[key].append(arg)
+                elif arg.startswith('-stdlib='):
+                    mapping['CXXFLAGS'].append(arg)
                 elif arg[0] == '+':
                     mapping['CCFLAGS'].append(arg)
                     mapping['LINKFLAGS'].append(arg)
@@ -1250,9 +1254,11 @@ def __init__(
         SCons.Tool.Initializers(self)
 
         if tools is None:
-            tools = self._dict.get('TOOLS', None)
-            if tools is None:
-                tools = ['default']
+            tools = self._dict.get('TOOLS', ['default'])
+        else:
+            # for a new env, if we didn't use TOOLS, make sure it starts empty
+            # so it only shows tools actually initialized.
+            self._dict['TOOLS'] = []
         apply_tools(self, tools, toolpath)
 
         # Now restore the passed-in and customized variables
@@ -2014,11 +2020,20 @@ def SetDefault(self, **kw) -> None:
     def _find_toolpath_dir(self, tp):
         return self.fs.Dir(self.subst(tp)).srcnode().get_abspath()
 
-    def Tool(self, tool, toolpath=None, **kwargs) -> SCons.Tool.Tool:
+    def Tool(
+        self, tool: Union[str, Callable], toolpath: Optional[Collection[str]] = None, **kwargs
+    ) -> Callable:
         """Find and run tool module *tool*.
 
+        *tool* is generally a string, but can also be a callable object,
+        in which case it is just called, without any of the setup.
+        The skipped setup includes storing *kwargs* into the created
+        :class:`~SCons.Tool.Tool` instance, which is extracted and used
+        when the instance is called, so in the skip case, the called
+        object will not get the *kwargs*.
+
         .. versionchanged:: 4.2
-           returns the tool module rather than ``None``.
+           returns the tool object rather than ``None``.
         """
         if is_String(tool):
             tool = self.subst(tool)