Fix CacheDir initialization so works on 3.7 also

[mwichmann]

Although there is no indication of a change in this area of the tempfile module, it seems if the temporary directory has been renamed so the original no longer exists, the context manager fails on 3.7, though not on any later Python versions (maybe a bugfix rather than a planned behavior change?). Now use mkdtemp instead, and clean it up by hand if the rename failed (also use os.replace instead of os.rename, so it will actually fail if the destination directory existed and is nonempty, on Linux os.rename doesn't fail in this case).

Fixes #4694

Contributor Checklist:

• I have created a new test or updated the unit tests to cover the new/changed functionality.
• I have updated CHANGES.txt and RELEASE.txt (and read the README.rst).
• I have updated the appropriate documentation

[jcbrill]

FWIW: all 30 CacheDir related tests that failed in #4694 pass for all python versions 3.7 to 3.13 in both linux and Windows with the first commit of this PR locally.

[mwichmann]

Thanks for that update!

[jcbrill]

@mwichmann Stupid question follows...

The following CacheDir tests fail on a windows box with no compiler installed:

• test\CacheDir\option--cs.py
• test\CacheDir\up-to-date-q.py

Is that the expected behavior?

[bdbaddog]

@jcbrill - Is that failure new after this PR? Or was it always failing when no compiler installed?

[mwichmann]

@mwichmann Stupid question follows...

The following CacheDir tests fail on a windows box with no compiler installed:

* test\CacheDir\option--cs.py

* test\CacheDir\up-to-date-q.py

Is that the expected behavior?

I'd say yes, that would be expected. Both tests have a place where they call env.Program with a C source file, and don't have a test for skipping if there's no compiler. Without testing that - all my systems have compilers, often multiples of C compilers - that doesn't sound like good test writing

[jcbrill]

Looking at the code for one of the tests, it would have always failed when no compiler is installed.

The final test in test\CacheDir\option--cs.py is a "normal build".

If there are no known compilers, msvc is assumed under windows. I'm not sure there is an easy way to skip a test if there are no compilers installed at least under windows.

[jcbrill]

I have a few Windows VMs with only windows SDKs installed that I usually run just the msvs/msvc tests to make sure that the msvc compilers are not detected.

The CacheDir tests were setup in some test batch files that I was using to update python virtual environments and failed the two tests above.

It's not a big deal.

[mwichmann]

If there are no known compilers, msvc is assumed under windows. I'm not sure there is an easy way to skip a test if there are no compilers installed at least under windows.

There's an msvc_exists() that some tests call, though usually just to diddle flags. Maybe some combination of that with other checks?

[jcbrill]

There's an msvc_exists() that some tests call, though usually just to diddle flags. Maybe some combination of that with other checks?

Unfortunately that doesn't tell you if mingw (or other valid windows compiler) is installed.

It would have to be no msvc and the default compiler is msvc. Although, then you would have to know if it was requested by a user and just doesn't exist.

[mwichmann]

No, it's not trivial to guess beforehand what SCons is going to find once it's actually called to run the test. Various tests have all kinds of tricks to cheat some of that, but it's never ideal.

[jcbrill]

I'm ok with "if you want all the tests to pass, you need to have a compiler installed".

As described above, in the VMs with no compiler, I'm usually running a limited set of tests for msvs/msvc just to verify that a usable msvc compiler is not detected.

Sorry for the bother.

[mwichmann]

Curious, what happens to test/CC/CC.py or test/CXX/CXX.py in this scenario?

Not really ideal to be conducting a chat in a merged/closed PR, but I guess it works...

[jcbrill]

Curious, what happens to test/CC/CC.py or test/CXX/CXX.py in this scenario?

On windows without a compiler, msvc is forced to be the default compiler. cl won't be found.

Not really ideal to be conducting a chat in a merged/closed PR, but I guess it works...

Yeah. At it's core, this didn't seem to be a bug. Just call me the "Necromancer".

host=VS64-SDK-W11 
branch=scons-master 
python=3.7.9 
1/2 (50.00%) C:\Data\venv\scons-dev\3.7.9\Scripts\python.exe test\CC\CC.py
FAILED test of Z:\S\SCons\scons-master\scripts\scons.py
	at line 753 of Z:\S\SCons\scons-master\testing\framework\TestCommon.py (_complete)
	from line 867 of Z:\S\SCons\scons-master\testing\framework\TestCommon.py (run)
	from line 485 of Z:\S\SCons\scons-master\testing\framework\TestSCons.py (run)
	from line 81 of test\CC\CC.py
Z:\S\SCons\scons-master\scripts\scons.py returned 2
STDOUT =========================================================================
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
cl /Fofoo.obj /c foo.c /nologo
scons: building terminated because of errors.

STDERR =========================================================================
'cl' is not recognized as an internal or external command,
operable program or batch file.
scons: *** [foo.obj] Error 1

Test execution time: 4.5 seconds
2/2 (100.00%) C:\Data\venv\scons-dev\3.7.9\Scripts\python.exe test\CXX\CXX.py
FAILED test of Z:\S\SCons\scons-master\scripts\scons.py
	at line 753 of Z:\S\SCons\scons-master\testing\framework\TestCommon.py (_complete)
	from line 867 of Z:\S\SCons\scons-master\testing\framework\TestCommon.py (run)
	from line 485 of Z:\S\SCons\scons-master\testing\framework\TestSCons.py (run)
	from line 142 of test\CXX\CXX.py
Z:\S\SCons\scons-master\scripts\scons.py returned 2
STDOUT =========================================================================
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
cl /Fofoo.obj /c foo.cxx /TP /nologo
scons: building terminated because of errors.

STDERR =========================================================================
Traceback (most recent call last):
  File "wrapper.py", line 21, in <module>
    subprocess.run(sys.argv[1:], check=False)
  File "C:\Users\jbrill\.pyenv\pyenv-win\versions\3.7.9\lib\subprocess.py", line 488, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Users\jbrill\.pyenv\pyenv-win\versions\3.7.9\lib\subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "C:\Users\jbrill\.pyenv\pyenv-win\versions\3.7.9\lib\subprocess.py", line 1207, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
'cl' is not recognized as an internal or external command,
operable program or batch file.
scons: *** [foo.obj] Error 1

Test execution time: 4.3 seconds
Total execution time for all tests: 8.8 seconds

Failed the following 2 tests:
	test\CC\CC.py
	test\CXX\CXX.py

Edit: execution times are unreliable due to being run in a VM hosted on an external drive and with windows installing updates.