test_makefile_test_folders test failures on python 3.12

[Krock21]

Bug report

Bug description:

Hi!

I try to build python 3.12 on Ubuntu 22.04 but make test fails for me

~/tmp/cpython/build-v3.12.3$ ../configure && make && make test
...
0:01:41 load avg: 14.84 [1/1/1] test_tools failed (16 failures)
Re-running test_tools in verbose mode (matching: test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders, test_makefile_test_folders)
test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) ...
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_import/data/package3') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_import/data/package4') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_importlib/zipdata02') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_importlib/zipdata01') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_importlib/data03') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_importlib/data02') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_importlib/data02/two') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_importlib/data02/one') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_importlib/data01') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_importlib/data01/subdirectory') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_tools/test_c_analyzer') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_tools/test_c_analyzer/test_common') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_tools/test_c_analyzer/test_symbols') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_tools/test_c_analyzer/test_parser') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_tools/test_c_analyzer/test_variables') ... FAIL
  test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_tools/test_c_analyzer/test_cpython') ... FAIL

======================================================================
FAIL: test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) (relpath='test/test_import/data/package3')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aleksandr.gaev/tmp/cpython/Lib/test/test_tools/test_makefile.py", line 59, in test_makefile_test_folders
    self.assertIn(
AssertionError: 'test/test_import/data/package3' not found in ['idlelib/idle_test', 'test', 'test/audiodata', 'test/certdata', 'test/certdata/capath', 'test/cjkencodings', 'test/crashers', 'test/configdata', 'test/data', 'test/decimaltestdata', 'test/dtracedata', 'test/encoded_modules', 'test/imghdrdata', 'test/leakers', 'test/libregrtest', 'test/regrtestdata', 'test/regrtestdata/import_from_tests', 'test/regrtestdata/import_from_tests/test_regrtest_b', 'test/sndhdrdata', 'test/subprocessdata', 'test/support', 'test/support/_hypothesis_stubs', 'test/test_asyncio', 'test/test_capi', 'test/test_cppext', 'test/test_ctypes', 'test/test_dataclasses', 'test/test_doctest', 'test/test_email', 'test/test_email/data', 'test/test_future_stmt', 'test/test_gdb', 'test/test_inspect', 'test/test_import', 'test/test_import/data', 'test/test_import/data/circular_imports', 'test/test_import/data/circular_imports/subpkg', 'test/test_import/data/circular_imports/subpkg2', 'test/test_import/data/circular_imports/subpkg2/parent', 'test/test_import/data/package', 'test/test_import/data/package2', 'test/test_import/data/unwritable', 'test/test_importlib', 'test/test_importlib/builtin', 'test/test_importlib/data', 'test/test_importlib/extension', 'test/test_importlib/frozen', 'test/test_importlib/import_', 'test/test_importlib/namespace_pkgs', 'test/test_importlib/namespace_pkgs/both_portions', 'test/test_importlib/namespace_pkgs/both_portions/foo', 'test/test_importlib/namespace_pkgs/module_and_namespace_package', 'test/test_importlib/namespace_pkgs/module_and_namespace_package/a_test', 'test/test_importlib/namespace_pkgs/not_a_namespace_pkg', 'test/test_importlib/namespace_pkgs/not_a_namespace_pkg/foo', 'test/test_importlib/namespace_pkgs/portion1', 'test/test_importlib/namespace_pkgs/portion1/foo', 'test/test_importlib/namespace_pkgs/portion2', 'test/test_importlib/namespace_pkgs/portion2/foo', 'test/test_importlib/namespace_pkgs/project1', 'test/test_importlib/namespace_pkgs/project1/parent', 'test/test_importlib/namespace_pkgs/project1/parent/child', 'test/test_importlib/namespace_pkgs/project2', 'test/test_importlib/namespace_pkgs/project2/parent', 'test/test_importlib/namespace_pkgs/project2/parent/child', 'test/test_importlib/namespace_pkgs/project3', 'test/test_importlib/namespace_pkgs/project3/parent', 'test/test_importlib/namespace_pkgs/project3/parent/child', 'test/test_importlib/partial', 'test/test_importlib/resources', 'test/test_importlib/resources/data01', 'test/test_importlib/resources/data01/subdirectory', 'test/test_importlib/resources/data02', 'test/test_importlib/resources/data02/one', 'test/test_importlib/resources/data02/subdirectory', 'test/test_importlib/resources/data02/subdirectory/subsubdir', 'test/test_importlib/resources/data02/two', 'test/test_importlib/resources/data03', 'test/test_importlib/resources/data03/namespace', 'test/test_importlib/resources/data03/namespace/portion1', 'test/test_importlib/resources/data03/namespace/portion2', 'test/test_importlib/resources/namespacedata01', 'test/test_importlib/resources/zipdata01', 'test/test_importlib/resources/zipdata02', 'test/test_importlib/source', 'test/test_json', 'test/test_lib2to3', 'test/test_lib2to3/data', 'test/test_lib2to3/data/fixers', 'test/test_lib2to3/data/fixers/myfixes', 'test/test_module', 'test/test_peg_generator', 'test/test_pydoc', 'test/test_sqlite3', 'test/test_tkinter', 'test/test_tomllib', 'test/test_tomllib/data', 'test/test_tomllib/data/invalid', 'test/test_tomllib/data/invalid/array', 'test/test_tomllib/data/invalid/array-of-tables', 'test/test_tomllib/data/invalid/boolean', 'test/test_tomllib/data/invalid/dates-and-times', 'test/test_tomllib/data/invalid/dotted-keys', 'test/test_tomllib/data/invalid/inline-table', 'test/test_tomllib/data/invalid/keys-and-vals', 'test/test_tomllib/data/invalid/literal-str', 'test/test_tomllib/data/invalid/multiline-basic-str', 'test/test_tomllib/data/invalid/multiline-literal-str', 'test/test_tomllib/data/invalid/table', 'test/test_tomllib/data/valid', 'test/test_tomllib/data/valid/array', 'test/test_tomllib/data/valid/dates-and-times', 'test/test_tomllib/data/valid/multiline-basic-str', 'test/test_tools', 'test/test_ttk', 'test/test_unittest', 'test/test_unittest/testmock', 'test/test_warnings', 'test/test_warnings/data', 'test/test_zipfile', 'test/test_zipfile/_path', 'test/test_zoneinfo', 'test/test_zoneinfo/data', 'test/tokenizedata', 'test/tracedmodules', 'test/typinganndata', 'test/wheeldata', 'test/xmltestdata', 'test/xmltestdata/c14n-20', 'test/ziptestdata', 'test/test_concurrent_futures', 'test/test_multiprocessing_fork', 'test/test_multiprocessing_forkserver', 'test/test_multiprocessing_spawn'] : 'test/test_import/data/package3' is not included in the Makefile's list of test directories to install

All tests fail with the following errors:

'test/test_import/data/package3' is not included in the Makefile's list of test directories to install
'test/test_import/data/package4' is not included in the Makefile's list of test directories to install
'test/test_importlib/zipdata02' is not included in the Makefile's list of test directories to install
'test/test_importlib/zipdata01' is not included in the Makefile's list of test directories to install
'test/test_importlib/data03' is not included in the Makefile's list of test directories to install
'test/test_importlib/data02' is not included in the Makefile's list of test directories to install
'test/test_importlib/data02/two' is not included in the Makefile's list of test directories to install
'test/test_importlib/data02/one' is not included in the Makefile's list of test directories to install
'test/test_importlib/data01' is not included in the Makefile's list of test directories to install
'test/test_importlib/data01/subdirectory' is not included in the Makefile's list of test directories to install
'test/test_tools/test_c_analyzer' is not included in the Makefile's list of test directories to install
'test/test_tools/test_c_analyzer/test_common' is not included in the Makefile's list of test directories to install
'test/test_tools/test_c_analyzer/test_symbols' is not included in the Makefile's list of test directories to install
'test/test_tools/test_c_analyzer/test_parser' is not included in the Makefile's list of test directories to install
'test/test_tools/test_c_analyzer/test_variables' is not included in the Makefile's list of test directories to install
'test/test_tools/test_c_analyzer/test_cpython' is not included in the Makefile's list of test directories to install

I tried 3.12, v3.12.4, v3.12.3 and they all fail with the same

Full logs:
https://gist.github.com/Krock21/dfc5370d9a03a873f9c8e062185d7cc9

Do you know what may be the reason and what can I do about it?

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Linked PRs

• gh-121966: Fix test_makefile_test_folders test failure #124732

[vstinner]

Hi, can you still reproduce the issue?

I try to build python 3.12 on Ubuntu 22.04 but make test fails for me

Did you use a tarball from python.org?

[picnixz]

I think the issue is because the tests are not executed in the same directory:

../configure && make && make test

The configure script and the make are not at the same level so it might be a PYTHONPATH or installation issue. Nonetheless, I'm closing it as cannot reproduce since the OP did not reply for a while.