[2025.12] precision in test on exotic arch ppc64el riscv64 and s390x

[picca]

Hello,

while preparing the Debian package of cctbx, I got these two failure on ppc64el and riscv64

• ppc64el

2316s Error: the following jobs returned non-zero exit codes or suspicious stderr output:
2316s 
2316s libtbx.python "/usr/lib/python3/dist-packages/scitbx/tests/tst_minimizer_lbfgs_general.py" [FAIL] 1.5s
2316s   Time:  1.49
2316s   Return code: 1
2316s   OKs: 0
2316s   Standard error:
2316s     Traceback (most recent call last):
2316s       File "/usr/lib/python3/dist-packages/scitbx/tests/tst_minimizer_lbfgs_general.py", line 99, in <module>
2316s         run()
2316s         ~~~^^
2316s       File "/usr/lib/python3/dist-packages/scitbx/tests/tst_minimizer_lbfgs_general.py", line 61, in run
2316s         assert approx_equal(m_unbound.x[0]-res[0],0, tolerance), (m_unbound.x[0], res[0], m_unbound.x[0]-res[0])
2316s                ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2316s     AssertionError: (19.999998556205234, 19.99999855596629, 2.389448638950853e-10)
2316s 
2316s Please verify these tests manually.
2316s 
2316s Summary:
2316s   Tests run                    : 553
2316s   Failures                     : 1
2316s   Warnings (possible failures) : 0
2316s   Known Failures (  4)         : 4
2316s   Known Unstable (  3)         : 0
2316s   Stderr output (discouraged)  : 8
2316s 
2316s See run_tests_parallel_zlog for full output.
2316s 
2316s 
2316s ********************************************************************************
2316s ERROR: 1 TEST FAILURES.  PLEASE FIX BEFORE COMMITTING CODE.
2316s ********************************************************************************
2316s 
2316s autopkgtest [21:50:14]: test cctbx: -----------------------]

• riscv64

14249s Error: the following jobs returned non-zero exit codes or suspicious stderr output:
14249s 
14249s libtbx.python "/usr/lib/python3/dist-packages/scitbx/tests/tst_minimizer_lbfgs_general.py" [FAIL] 3.4s
14249s   Time:  3.44
14249s   Return code: 1
14249s   OKs: 0
14249s   Standard error:
14249s     Traceback (most recent call last):
14249s       File "/usr/lib/python3/dist-packages/scitbx/tests/tst_minimizer_lbfgs_general.py", line 99, in <module>
14249s         run()
14249s         ~~~^^
14249s       File "/usr/lib/python3/dist-packages/scitbx/tests/tst_minimizer_lbfgs_general.py", line 61, in run
14249s         assert approx_equal(m_unbound.x[0]-res[0],0, tolerance), (m_unbound.x[0], res[0], m_unbound.x[0]-res[0])
14249s                ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
14249s     AssertionError: (19.999998556205234, 19.99999855596629, 2.389448638950853e-10)
14249s 
14249s Please verify these tests manually.
14249s 
14249s Summary:
14249s   Tests run                    : 553
14249s   Failures                     : 1
14249s   Warnings (possible failures) : 0
14249s   Known Failures (  4)         : 4
14249s   Known Unstable (  3)         : 0
14249s   Stderr output (discouraged)  : 8
14249s 
14249s See run_tests_parallel_zlog for full output.
14249s 
14249s 
14249s ********************************************************************************
14249s ERROR: 1 TEST FAILURES.  PLEASE FIX BEFORE COMMITTING CODE.
14249s ********************************************************************************
14249s 
14250s autopkgtest [07:34:37]: test cctbx: -----------------------]

for info on s390x, I get a few more issues and the previous one.

I need more investigation on the nan part from my side (If I find enought time).

692s Error: the following jobs returned non-zero exit codes or suspicious stderr output:
692s 
692s libtbx.python "/usr/lib/python3/dist-packages/iotbx/regression/tst_map_manager.py" [FAIL] 1.5s
692s   Time:  1.52
692s   Return code: 1
692s   OKs: 0
692s   Standard error:
692s     Sorry: Input map file contains 'nan':  not a valid map file
692s   Standard error:
692s     Sorry: Input map file contains 'nan':  not a valid map file
692s libtbx.python "/usr/lib/python3/dist-packages/iotbx/regression/tst_data_manager.py" [FAIL] 2.9s
692s   Time:  2.89
692s   Return code: 1
692s   OKs: 0
692s libtbx.python "/usr/lib/python3/dist-packages/iotbx/regression/tst_map_model_manager_9_remove_origin_shift_and_unit_cell_crystal_symmetry.py" [FAIL] 1.5s
692s   Time:  1.48
692s   Return code: 1
692s   OKs: 0
692s   Standard error:
692s     Sorry: Input map file contains 'nan':  not a valid map file
692s   Standard error:
692s     Sorry: Input map file contains 'nan':  not a valid map file
692s libtbx.python "/usr/lib/python3/dist-packages/iotbx/regression/tst_map_model_manager_9_remove_origin_shift_and_unit_cell_crystal_symmetry_cif.py" [FAIL] 1.6s
692s   Time:  1.65
692s   Return code: 1
692s   OKs: 0
692s libtbx.python "/usr/lib/python3/dist-packages/iotbx/ccp4_map/tst.py" [FAIL] 0.7s
692s   Standard error:
692s     Sorry: Input map file contains 'nan':  not a valid map file
692s   Time:  0.71
692s   Return code: 1
692s   OKs: 0
692s libtbx.python "/usr/lib/python3/dist-packages/iotbx/regression/tst_file_reader.py" [FAIL] 1.7s
692s   Time:  1.68
692s   Return code: 1
692s   OKs: 0
692s   Standard error:
692s     Traceback (most recent call last):
692s       File "/usr/lib/python3/dist-packages/iotbx/regression/tst_file_reader.py", line 537, in <module>
692s         exercise()
692s         ~~~~~~~~^^
692s       File "/usr/lib/python3/dist-packages/iotbx/regression/tst_file_reader.py", line 529, in exercise
692s         exercise_maps()
692s         ~~~~~~~~~~~~~^^
692s       File "/usr/lib/python3/dist-packages/iotbx/regression/tst_file_reader.py", line 414, in exercise_maps
692s         assert f.file_type == "ccp4_map"
692s                ^^^^^^^^^^^^^^^^^^^^^^^^^
692s     AssertionError
692s libtbx.python "/usr/lib/python3/dist-packages/scitbx/tests/tst_minimizer_lbfgs_general.py" [FAIL] 0.2s
692s   Time:  0.18
692s   Return code: 1
692s   OKs: 0
692s   Standard error:
692s     Traceback (most recent call last):
692s       File "/usr/lib/python3/dist-packages/scitbx/tests/tst_minimizer_lbfgs_general.py", line 99, in <module>
692s         run()
692s         ~~~^^
692s       File "/usr/lib/python3/dist-packages/scitbx/tests/tst_minimizer_lbfgs_general.py", line 61, in run
692s         assert approx_equal(m_unbound.x[0]-res[0],0, tolerance), (m_unbound.x[0], res[0], m_unbound.x[0]-res[0])
692s                ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
692s     AssertionError: (19.999998556205234, 19.99999855596629, 2.389448638950853e-10)
692s 
692s Please verify these tests manually.
692s 
692s Summary:
692s   Tests run                    : 553
692s   Failures                     : 7
692s   Warnings (possible failures) : 0
692s   Known Failures (  4)         : 4
692s   Known Unstable (  3)         : 0
692s   Stderr output (discouraged)  : 14
692s 
692s See run_tests_parallel_zlog for full output.
692s 
692s 
692s ********************************************************************************
692s ERROR: 7 TEST FAILURES.  PLEASE FIX BEFORE COMMITTING CODE.
692s ********************************************************************************
692s 
693s autopkgtest [21:22:26]: test cctbx: -----------------------]
693s cctbx                FAIL non-zero exit status 1

[bkpoon]

It looks like we'll have to adjust the tolerance for that test.

Also, what is the GCC version and what level of optimization do you use? Every week, I test -01 through -03. I'll probably do an update on the compilers that we test this year.

[bkpoon]

For the nan issue, is the other architecture big endian? I'm not sure how the mrcfille module handles that when reading binary files.

[picca]

$ gcc --version
gcc (Debian 15.2.0-12) 15.2.0

[picca]

https://wiki.debian.org/ArchitectureSpecificsMemo#Summary

s390x is BE
ppc64el LE
riscv64 LE

[ndevenish]

We have also been seeing this on our x86 Mac installs

[bkpoon]

I adjusted the tolerance for running the test on platforms other than x86-64.