Detection changes for VS IDE editions and VS2008 (develop and vcforpython).

Changes:
* An express installation of the IDE binary is used when no other IDE edition is detected.
* A full development edition (e.g., Professional) of VS2008 is elected before a Visual C++ For Python edition.
* Update detection order in README.rst and remove hard tabs.
* Minor fixes lingering from original VSWHERE implementation where all call chains were not updated when the environment argument was added.

Author: jcbrill

CHANGES.txt

@@ -43,7 +43,9 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
       method so that the generated function argument list matches the function's
       prototype when including a header file.
     - For msvc version specifications without an 'Exp' suffix, an express installation
-      is used when no other edition is detected for the msvc version.
+      is used when no other edition is detected for the msvc version.  Similarly, an
+      express installation of the IDE binary is used when no other IDE edition is
+      detected.
     - VS2015 Express (14.0Exp) does not support the sdk version argument.  VS2015 Express
       does not support the store argument for target architectures other than x86.
       Script argument validation now takes into account these restrictions.
@@ -72,6 +74,9 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
       (i.e., msiexec /i VCForPython27.msi ALLUSERS=1), the registry keys are written to
       HKEY_LOCAL_MACHINE rather than HKEY_CURRENT_USER.  An entry was added to query the
       Visual C++ For Python keys in HKLM following the HKCU query, if necessary.
+    - For VS2008, a full development edition (e.g., Professional) is now selected before
+      a Visual C++ For Python edition.  Prior to this change, Visual C++ For Python was
+      selected before a full development edition when both editions are installed.
     - The registry detection of VS2015 (14.0), and earlier, is now cached at runtime and
       is only evaluated once for each msvc version.
     - The vswhere detection of VS2017 (14.1), and later, is now cached at runtime and is

RELEASE.txt

@@ -38,7 +38,8 @@ CHANGED/ENHANCED EXISTING FUNCTIONALITY
   installation is used when no other edition is detected for the msvc version.
   This was the behavior for Visual Studio 2008 (9.0) through Visual Studio 2015
   (14.0).  This behavior was extended to Visual Studio 2017 (14.1) and Visual
-  Studio 2008 (8.0).
+  Studio 2008 (8.0).  An express installation of the IDE binary is used when no
+  other IDE edition is detected.
 
 FIXES
 -----
@@ -70,6 +71,10 @@ FIXES
       msiexec /i VCForPython27.msi ALLUSERS=1
   When installed for all users, Visual Studio 2008 (9.0) Visual C++ For Python is
   now correctly detected.
+- MSVC: For Visual Studio 2008 (9.0), a full development edition (e.g., Professional)
+  is now selected before a Visual C++ For Python edition. Prior to this change,
+  Visual C++ For Python was selected before a full development edition when both
+  editions are installed.
 - MSVC: The installed vcs and visual studios lists were constructed and cached
   during their initial invocations. If a vswhere executable was specified via the
   construction variable VSWHERE and found additional msvc installations, the new

SCons/Tool/MSCommon/README.rst

@@ -36,31 +36,31 @@ installation is used only when no other installation is detected.
 ======= ======= ========================================================
 Product VCVer   Priority
 ======= ======= ========================================================
-VS2022  14.3	Enterprise, Professional, Community, BuildTools
+VS2022  14.3    Enterprise, Professional, Community, BuildTools
 ------- ------- --------------------------------------------------------
 VS2019  14.2    Enterprise, Professional, Community, BuildTools
 ------- ------- --------------------------------------------------------
 VS2017  14.1    Enterprise, Professional, Community, BuildTools, Express
 ------- ------- --------------------------------------------------------
 VS2017  14.1Exp Express
 ------- ------- --------------------------------------------------------
-VS2015  14.0	[Develop, BuildTools, CmdLine], Express
+VS2015  14.0    [Develop, BuildTools, CmdLine], Express
 ------- ------- --------------------------------------------------------
 VS2015  14.0Exp Express
 ------- ------- --------------------------------------------------------
-VS2013  12.0	Develop, Express
+VS2013  12.0    Develop, Express
 ------- ------- --------------------------------------------------------
 VS2013  12.0Exp Express
 ------- ------- --------------------------------------------------------
-VS2012  11.0	Develop, Express
+VS2012  11.0    Develop, Express
 ------- ------- --------------------------------------------------------
 VS2012  11.0Exp Express
 ------- ------- --------------------------------------------------------
 VS2010  10.0    Develop, Express
 ------- ------- --------------------------------------------------------
 VS2010  10.0Exp Express
 ------- ------- --------------------------------------------------------
-VS2008  9.0     VCForPython, Develop, Express
+VS2008  9.0     Develop, VCForPython, Express
 ------- ------- --------------------------------------------------------
 VS2008  9.0Exp  Express
 ------- ------- --------------------------------------------------------
@@ -105,20 +105,20 @@ and/or linker build failures.
 The VS2015 BuildTools ``vcvarsall.bat`` batch file dispatches to the stand-alone buildtools
 batch file under certain circumstances. A fragment from the vcvarsall batch file is:
 ::
-	if exist "%~dp0..\common7\IDE\devenv.exe" goto setup_VS
-	if exist "%~dp0..\common7\IDE\wdexpress.exe" goto setup_VS
-	if exist "%~dp0..\..\Microsoft Visual C++ Build Tools\vcbuildtools.bat" goto setup_buildsku
+    if exist "%~dp0..\common7\IDE\devenv.exe" goto setup_VS
+    if exist "%~dp0..\common7\IDE\wdexpress.exe" goto setup_VS
+    if exist "%~dp0..\..\Microsoft Visual C++ Build Tools\vcbuildtools.bat" goto setup_buildsku
 
-	:setup_VS
+    :setup_VS
 
-	...
+    ...
 
-	:setup_buildsku
-	if not exist "%~dp0..\..\Microsoft Visual C++ Build Tools\vcbuildtools.bat" goto usage
-	set CurrentDir=%CD%
-	call "%~dp0..\..\Microsoft Visual C++ Build Tools\vcbuildtools.bat" %1 %2
-	cd /d %CurrentDir%
-	goto :eof
+    :setup_buildsku
+    if not exist "%~dp0..\..\Microsoft Visual C++ Build Tools\vcbuildtools.bat" goto usage
+    set CurrentDir=%CD%
+    call "%~dp0..\..\Microsoft Visual C++ Build Tools\vcbuildtools.bat" %1 %2
+    cd /d %CurrentDir%
+    goto :eof
 
 VS2015 Express
 --------------
@@ -136,19 +136,19 @@ architecture.  The generated ``store`` library paths include directories that do
 
 The store library paths appear in two places in the ``vcvarsx86_amd64`` batch file:
 ::
-	:setstorelib
-	@if exist "%VCINSTALLDIR%LIB\amd64\store" set LIB=%VCINSTALLDIR%LIB\amd64\store;%LIB%
-	...
-	:setstorelibpath
-	@if exist "%VCINSTALLDIR%LIB\amd64\store" set LIBPATH=%VCINSTALLDIR%LIB\amd64\store;%LIBPATH%
+    :setstorelib
+    @if exist "%VCINSTALLDIR%LIB\amd64\store" set LIB=%VCINSTALLDIR%LIB\amd64\store;%LIB%
+    ...
+    :setstorelibpath
+    @if exist "%VCINSTALLDIR%LIB\amd64\store" set LIBPATH=%VCINSTALLDIR%LIB\amd64\store;%LIBPATH%
 
 The correct store library paths would be:
 ::
-	:setstorelib
-	@if exist "%VCINSTALLDIR%LIB\store\amd64" set LIB=%VCINSTALLDIR%LIB\store\amd64;%LIB%
-	...
-	:setstorelibpath
-	@if exist "%VCINSTALLDIR%LIB\store\amd64" set LIBPATH=%VCINSTALLDIR%LIB\store\amd64;%LIBPATH%
+    :setstorelib
+    @if exist "%VCINSTALLDIR%LIB\store\amd64" set LIB=%VCINSTALLDIR%LIB\store\amd64;%LIB%
+    ...
+    :setstorelibpath
+    @if exist "%VCINSTALLDIR%LIB\store\amd64" set LIBPATH=%VCINSTALLDIR%LIB\store\amd64;%LIBPATH%
 
 arm Target Architecture
 ^^^^^^^^^^^^^^^^^^^^^^^
@@ -158,19 +158,19 @@ architecture.  The generated ``store`` library paths include directories that do
 
 The store library paths appear in two places in the ``vcvarsx86_arm`` batch file:
 ::
-	:setstorelib
-	@if exist "%VCINSTALLDIR%LIB\ARM\store" set LIB=%VCINSTALLDIR%LIB\ARM\store;%LIB%
-	...
-	:setstorelibpath
-	@if exist "%VCINSTALLDIR%LIB\ARM\store" set LIBPATH=%VCINSTALLDIR%LIB\ARM\store;%LIBPATH%
+    :setstorelib
+    @if exist "%VCINSTALLDIR%LIB\ARM\store" set LIB=%VCINSTALLDIR%LIB\ARM\store;%LIB%
+    ...
+    :setstorelibpath
+    @if exist "%VCINSTALLDIR%LIB\ARM\store" set LIBPATH=%VCINSTALLDIR%LIB\ARM\store;%LIBPATH%
 
 The correct store library paths would be file:
 ::
-	:setstorelib
-	@if exist "%VCINSTALLDIR%LIB\store\ARM" set LIB=%VCINSTALLDIR%LIB\store\ARM;%LIB%
-	...
-	:setstorelibpath
-	@if exist "%VCINSTALLDIR%LIB\store\ARM" set LIBPATH=%VCINSTALLDIR%LIB\store\ARM;%LIBPATH%
+    :setstorelib
+    @if exist "%VCINSTALLDIR%LIB\store\ARM" set LIB=%VCINSTALLDIR%LIB\store\ARM;%LIB%
+    ...
+    :setstorelibpath
+    @if exist "%VCINSTALLDIR%LIB\store\ARM" set LIBPATH=%VCINSTALLDIR%LIB\store\ARM;%LIBPATH%
 
 
 Known Issues

SCons/Tool/MSCommon/sdk.py

@@ -372,7 +372,7 @@ def mssdk_setup_env(env):
             return
         msvs_version = env.subst(msvs_version)
         from . import vs
-        msvs = vs.get_vs_by_version(msvs_version)
+        msvs = vs.get_vs_by_version(msvs_version, env)
         debug('mssdk_setup_env:msvs is :%s', msvs)
         if not msvs:
             debug('mssdk_setup_env: no VS version detected, bailingout:%s', msvs)

SCons/Tool/MSCommon/vc.py

@@ -103,7 +103,7 @@ class BatchFileExecutionError(VisualCException):
 # MSVC 9.0 preferred query order:
 #     True:  VCForPython, VisualStudio
 #     False: VisualStudio, VCForPython
-_VC90_Prefer_VCForPython = True
+_VC90_Prefer_VCForPython = False
 
 # Dict to 'canonalize' the arch
 _ARCH_TO_CANONICAL = {
@@ -923,6 +923,21 @@ def msvc_version_to_maj_min(msvc_version):
     os.path.expandvars(r"%SCOOP%\shims"),
 ]]
 
+def msvc_find_vswhere():
+    """ Find the location of vswhere """
+    # For bug 3333: support default location of vswhere for both
+    # 64 and 32 bit windows installs.
+    # For bug 3542: also accommodate not being on C: drive.
+    # NB: this gets called from testsuite on non-Windows platforms.
+    # Whether that makes sense or not, don't break it for those.
+    vswhere_path = None
+    for pf in VSWHERE_PATHS:
+        if os.path.exists(pf):
+            vswhere_path = pf
+            break
+
+    return vswhere_path
+
 # normalize user-specified vswhere paths
 
 _cache_user_vswhere_paths = {}
@@ -965,13 +980,7 @@ def _vswhere_user_path(pval):
 
     return vswhere_path
 
-# normalized default vswhere path
-
-_vswhere_paths_processed = [
-    MSVC.Util.normalize_path(pval)
-    for pval in VSWHERE_PATHS
-    if os.path.exists(pval)
-]
+# normalize default vswhere path
 
 _vswhere_path_default = UNDEFINED
 
@@ -981,6 +990,12 @@ def _msvc_default_vswhere():
 
     if _vswhere_path_default == UNDEFINED:
 
+        _vswhere_paths_processed = [
+            MSVC.Util.normalize_path(pval)
+            for pval in VSWHERE_PATHS
+            if os.path.exists(pval)
+        ]
+
         if _vswhere_paths_processed:
             vswhere_path = _vswhere_paths_processed[0]
         else:
@@ -991,21 +1006,6 @@ def _msvc_default_vswhere():
 
     return _vswhere_path_default
 
-def msvc_find_vswhere():
-    """ Find the location of vswhere """
-    # For bug 3333: support default location of vswhere for both
-    # 64 and 32 bit windows installs.
-    # For bug 3542: also accommodate not being on C: drive.
-    # NB: this gets called from testsuite on non-Windows platforms.
-    # Whether that makes sense or not, don't break it for those.
-    vswhere_path = None
-    for pf in VSWHERE_PATHS:
-        if os.path.exists(pf):
-            vswhere_path = pf
-            break
-
-    return vswhere_path
-
 class _VSWhere:
 
     reset_funcs_list = []

SCons/Tool/MSCommon/vs.py

@@ -48,14 +48,22 @@
 )
 
 
+# Visual Studio express version policy when unqualified version is not installed:
+#     True:  use express version for unqualified version (e.g., use 12.0Exp for 12.0)
+#     False: do not use express version for unqualified version
+_VSEXPRESS_USE_VERSTR = True
+
+
 class VisualStudio:
     """
     An abstract base class for trying to find installed versions of
     Visual Studio.
     """
     def __init__(self, version, **kw) -> None:
         self.version = version
-        self.vernum = float(get_msvc_version_numeric(version))
+        self.verstr = get_msvc_version_numeric(version)
+        self.vernum = float(self.verstr)
+        self.is_express = True if self.verstr != self.version else False
         kw['vc_version']  = kw.get('vc_version', version)
         kw['sdk_version'] = kw.get('sdk_version', version)
         self.__dict__.update(kw)
@@ -454,9 +462,17 @@ def get_installed_visual_studios(env=None):
             if vs.get_executable(env):
                 debug('found VS %s', vs.version)
                 InstalledVSList.append(vs)
+                if vs.is_express and vs.verstr not in InstalledVSMap:
+                    if _VSEXPRESS_USE_VERSTR:
+                        InstalledVSMap[vs.verstr] = vs
                 InstalledVSMap[vs.version] = vs
     return InstalledVSList
 
+def _get_installed_vss(env=None):
+    get_installed_visual_studios(env)
+    versions = list(InstalledVSMap.keys())
+    return versions
+
 def reset_installed_visual_studios() -> None:
     global InstalledVSList
     global InstalledVSMap
@@ -519,15 +535,15 @@ def _reset_installed_vswhere_visual_studios():
 def msvs_exists(env=None) -> bool:
     return len(get_installed_visual_studios(env)) > 0
 
-def get_vs_by_version(msvs):
+def get_vs_by_version(msvs, env=None):
     global InstalledVSMap
     global SupportedVSMap
 
     debug('called')
     if msvs not in SupportedVSMap:
         msg = "Visual Studio version %s is not supported" % repr(msvs)
         raise SCons.Errors.UserError(msg)
-    get_installed_visual_studios()
+    get_installed_visual_studios(env)
     vs = InstalledVSMap.get(msvs)
     debug('InstalledVSMap:%s', InstalledVSMap)
     debug('found vs:%s', vs)
@@ -556,7 +572,7 @@ def get_default_version(env):
     """
     if 'MSVS' not in env or not SCons.Util.is_Dict(env['MSVS']):
         # get all versions, and remember them for speed later
-        versions = [vs.version for vs in get_installed_visual_studios()]
+        versions = _get_installed_vss(env)
         env['MSVS'] = {'VERSIONS' : versions}
     else:
         versions = env['MSVS'].get('VERSIONS', [])
@@ -602,7 +618,7 @@ def merge_default_version(env) -> None:
 
 # TODO: refers to versions and arch which aren't defined; called nowhere. Drop?
 def msvs_setup_env(env) -> None:
-    msvs = get_vs_by_version(version)
+    msvs = get_vs_by_version(version, env)
     if msvs is None:
         return
     batfilename = msvs.get_batch_file()
@@ -614,7 +630,7 @@ def msvs_setup_env(env) -> None:
 
         vars = ('LIB', 'LIBPATH', 'PATH', 'INCLUDE')
 
-        msvs_list = get_installed_visual_studios()
+        msvs_list = get_installed_visual_studios(env)
         vscommonvarnames = [vs.common_tools_var for vs in msvs_list]
         save_ENV = env['ENV']
         nenv = normalize_env(env['ENV'],
@@ -629,11 +645,10 @@ def msvs_setup_env(env) -> None:
         for k, v in vars.items():
             env.PrependENVPath(k, v, delete_existing=1)
 
-def query_versions():
+def query_versions(env=None):
     """Query the system to get available versions of VS. A version is
     considered when a batfile is found."""
-    msvs_list = get_installed_visual_studios()
-    versions = [msvs.version for msvs in msvs_list]
+    versions = _get_installed_vss(env)
     return versions
 
 # Local Variables:

SCons/Tool/msvsTests.py

@@ -635,9 +635,9 @@ def _TODO_test_merge_default_version(self) -> None:
         """Test the merge_default_version() function"""
         pass
 
-    def test_query_versions(self) -> None:
+    def test_query_versions(self, env=None) -> None:
         """Test retrieval of the list of visual studio versions"""
-        v1 = query_versions()
+        v1 = query_versions(env)
         assert not v1 or str(v1[0]) == self.highest_version, \
                (v1, self.highest_version)
         assert len(v1) == self.number_of_versions, v1

test/MSVC/query_vcbat.py

@@ -51,7 +51,7 @@
 #            print k, v
 #MergeMSVSBatFile(env, 9.0)
 #print(env['ENV']['PATH'])
-print(query_versions())
+print(query_versions(env=None))
 """)
 
 test.run(stderr=None)