Additional test FAILures on Mac OS X 10.4.11, Tiger with Perl 5.34 in t/local/62_threads-ctx_new-deadlock.t and t/local/62_threads-ctx_new-deadlock.t

[ballapete]

This is a pretty old Mac and system, based on a 32bit PowerPC CPU. Here additionally these fail:

--->  Testing p5.34-net-ssleay
Executing:  cd "/opt/local/var/macports/build/_Volumes_BSD-Linux_ports_perll_p5-net-ssleay/p5.34-net-ssleay/work/Net-SSLeay-1.94" && /usr/bin/make test 
"/opt/local/bin/perl5.34" -MExtUtils::Command::MM -e 'cp_nonempty' -- SSLeay.bs blib/arch/auto/Net/SSLeay/SSLeay.bs 644
PERL_DL_NONLAZY=1 "/opt/local/bin/perl5.34" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/local/*.t t/handle/local/*.t
t/handle/local/05_use.t ..................... ok
t/local/01_pod.t ............................ ok
t/local/02_pod_coverage.t ................... skipped: These tests are for only for release candidate testing. Enable with RELEASE_TESTING=1
# 
# Testing Net::SSLeay 1.94
# 
# Perl information:
#   Version:         '5.034003'
#   Executable path: '/opt/local/bin/perl5.34'
# 
# Library version with OpenSSL_version_num():
#   OPENSSL_VERSION_NUMBER: '0x30400000'
# 
# Library information with SSLeay_version() and OpenSSL_version():
#   SSLEAY_VERSION:              'OpenSSL 3.4.0 22 Oct 2024'
#   SSLEAY_CFLAGS:               'compiler: /opt/local/bin/gcc-apple-4.2 -fPIC -fno-common -arch ppc -std=gnu9x -Wa,-force_cpusubtype_ALL -pipe -Os -arch ppc -DB_ENDIAN -DOPENSSL_PIC -D_REENTRANT -DOPENSSL_BUILDING_OPENSSL -DZLIB -DNDEBUG -isystem/opt/local/include/LegacySupport -I/opt/local/include'
#   SSLEAY_BUILT_ON:             'built on: Wed Oct 30 00:40:48 2024 UTC'
#   SSLEAY_PLATFORM:             'platform: darwin-ppc-cc'
#   SSLEAY_DIR:                  'OPENSSLDIR: "/opt/local/libexec/openssl3/etc/openssl"'
#   OPENSSL_ENGINES_DIR:         'ENGINESDIR: "/opt/local/libexec/openssl3/lib/engines-3"'
#   OPENSSL_MODULES_DIR:         'MODULESDIR: "/opt/local/libexec/openssl3/lib/ossl-modules"'
#   OPENSSL_CPU_INFO:            'CPUINFO: N/A'
#   OPENSSL_VERSION_STRING:      '3.4.0'
#   OPENSSL_FULL_VERSION_STRING: '3.4.0'
# 
# Library version information with OPENSSL_version_*():
#   OPENSSL_version_major():          '3'
#   OPENSSL_version_minor():          '4'
#   OPENSSL_version_patch():          '0'
#   OPENSSL_version_pre_release():    ''
#   OPENSSL_version_build_metadata(): ''
# 
# Library information with OPENSSL_info():
#   OPENSSL_INFO_CONFIG_DIR:             '/opt/local/libexec/openssl3/etc/openssl'
#   OPENSSL_INFO_ENGINES_DIR:            '/opt/local/libexec/openssl3/lib/engines-3'
#   OPENSSL_INFO_MODULES_DIR:            '/opt/local/libexec/openssl3/lib/ossl-modules'
#   OPENSSL_INFO_DSO_EXTENSION:          '.dylib'
#   OPENSSL_INFO_DIR_FILENAME_SEPARATOR: '/'
#   OPENSSL_INFO_LIST_SEPARATOR:         ':'
#   OPENSSL_INFO_SEED_SOURCE:            'os-specific'
Use of uninitialized value in concatenation (.) or string at t/local/03_use.t line 73.
#   OPENSSL_INFO_CPU_SETTINGS:           ''
t/local/03_use.t ............................ ok
t/local/04_basic.t .......................... ok
t/local/05_passwd_cb.t ...................... ok
t/local/06_tcpecho.t ........................ ok
t/local/07_sslecho.t ........................ ok
t/local/08_pipe.t ........................... ok
t/local/09_ctx_new.t ........................ ok
t/local/10_rand.t ........................... ok
t/local/11_read.t ........................... ok
t/local/15_bio.t ............................ ok
t/local/20_functions.t ...................... ok
t/local/21_constants.t ...................... ok
t/local/22_provider.t ....................... ok
t/local/22_provider_try_load.t .............. ok
t/local/22_provider_try_load_zero_retain.t .. ok
t/local/23_openssl_init.t ................... ok
t/local/30_error.t .......................... ok
t/local/31_rsa_generate_key.t ............... ok

#   Failed test 'X509V3_EXT_print nid=86	extended-cert.cert.pem:6'
#   at t/local/32_x509_get_cert_info.t line 273.
#          got: 'email:intermediate-ca@net-ssleay.example, URI:http://intermediate-ca.net-ssleay.example, DNS:intermediate-ca.net-ssleay.example, Registered ID:1.2.0.0, IP Address:192.168.0.1, IP Address:FD25:F814:AFB5:9873:0:0:0:1, othername: emailAddress:ica@net-ssleay.example'
#     expected: 'email:intermediate-ca@net-ssleay.example, URI:http://intermediate-ca.net-ssleay.example, DNS:intermediate-ca.net-ssleay.example, Registered ID:1.2.0.0, IP Address:192.168.0.1, IP Address:FD25:F814:AFB5:9873:0:0:0:1, othername: emailAddress::ica@net-ssleay.example'

#   Failed test 'X509V3_EXT_print nid=85	extended-cert.cert.pem:8'
#   at t/local/32_x509_get_cert_info.t line 273.
#          got: 'email:john.doe@net-ssleay.example, URI:http://johndoe.net-ssleay.example, DNS:johndoe.net-ssleay.example, Registered ID:1.2.3.4, IP Address:192.168.0.2, IP Address:FD25:F814:AFB5:9873:0:0:0:2, othername: emailAddress:jd@net-ssleay.example'
#     expected: 'email:john.doe@net-ssleay.example, URI:http://johndoe.net-ssleay.example, DNS:johndoe.net-ssleay.example, Registered ID:1.2.3.4, IP Address:192.168.0.2, IP Address:FD25:F814:AFB5:9873:0:0:0:2, othername: emailAddress::jd@net-ssleay.example'
# Looks like you failed 2 tests of 746.
t/local/32_x509_get_cert_info.t ............. 
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/746 subtests 
t/local/33_x509_create_cert.t ............... ok
t/local/34_x509_crl.t ....................... ok
t/local/35_ephemeral.t ...................... skipped: LibreSSL and OpenSSL 1.1.0 removed support for ephemeral/temporary RSA private keys
t/local/36_verify.t ......................... ok
t/local/37_asn1_time.t ...................... ok
t/local/38_priv-key.t ....................... ok
t/local/39_pkcs12.t ......................... ok
t/local/40_npn_support.t .................... ok
t/local/41_alpn_support.t ................... ok
t/local/42_info_callback.t .................. ok
t/local/43_misc_functions.t ................. ok
t/local/44_sess.t ........................... ok
t/local/45_exporter.t ....................... ok
t/local/46_msg_callback.t ................... ok
t/local/47_keylog.t ......................... ok
t/local/48_client_hello_callback.t .......... ok
t/local/50_digest.t ......................... ok
t/local/61_threads-cb-crash.t ............... ok
FATAL: TIMEOUT! at t/local/62_threads-ctx_new-deadlock.t line 36.
Perl exited with active threads:
	1 running and unjoined
	0 finished and unjoined
	0 running and detached
t/local/62_threads-ctx_new-deadlock.t ....... 
Failed 1/1 subtests 
t/local/63_ec_key_generate_key.t ............ ok
t/local/64_ticket_sharing.t ................. ok
t/local/65_security_level.t ................. ok
t/local/65_ticket_sharing_2.t ............... ok
t/local/66_curves.t ......................... ok
t/local/kwalitee.t .......................... skipped: These tests are for only for release candidate testing. Enable with RELEASE_TESTING=1

Test Summary Report
-------------------
t/local/32_x509_get_cert_info.t           (Wstat: 512 (exited 2) Tests: 746 Failed: 2)
  Failed tests:  358, 376
  Non-zero exit status: 2
t/local/62_threads-ctx_new-deadlock.t     (Wstat: 0 Tests: 0 Failed: 0)
  Parse errors: Bad plan.  You planned 1 tests but ran 0.
Files=47, Tests=2788, 120 wallclock secs ( 2.36 usr  0.81 sys + 77.68 cusr  9.98 csys = 90.83 CPU)
Result: FAIL
Failed 2/47 test programs. 2/2788 subtests failed.

I think Mac OS X 10.4.11, Tiger, has only minimal thread support. The MacPorts package manager installs npth…

[h-vn]

Can you try with prove -b --timer t/local instead of make test to see how long tests take in general. The failing threads test has a 20 second timeout which appears to be not long enough on really slow platforms. Slow would mean that all of the tests take several seconds.

[ballapete]

With OpenSSL 3.5.0 installed this failure is gone, others are left:

--->  Testing p5.34-net-ssleay
Executing:  cd "/opt/local/var/macports/build/_Users_btest_ports_perll_p5-net-ssleay/p5.34-net-ssleay/work/Net-SSLeay-1.94" && /usr/bin/make test 
"/opt/local/bin/perl5.34" -MExtUtils::Command::MM -e 'cp_nonempty' -- SSLeay.bs blib/arch/auto/Net/SSLeay/SSLeay.bs 644
PERL_DL_NONLAZY=1 "/opt/local/bin/perl5.34" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/local/*.t t/handle/local/*.t
t/handle/local/05_use.t ..................... ok
t/local/01_pod.t ............................ ok
t/local/02_pod_coverage.t ................... skipped: These tests are for only for release candidate testing. Enable with RELEASE_TESTING=1
# 
# Testing Net::SSLeay 1.94
# 
# Perl information:
#   Version:         '5.034003'
#   Executable path: '/opt/local/bin/perl5.34'
# 
# Library version with OpenSSL_version_num():
#   OPENSSL_VERSION_NUMBER: '0x30500000'
# 
# Library information with SSLeay_version() and OpenSSL_version():
#   SSLEAY_VERSION:              'OpenSSL 3.5.0 8 Apr 2025'
#   SSLEAY_CFLAGS:               'compiler: /opt/local/bin/gcc-apple-4.2 -fPIC -fno-common -arch ppc -std=gnu9x -Wa,-force_cpusubtype_ALL -pipe -Os -arch ppc -DB_ENDIAN -DOPENSSL_PIC -D_REENTRANT -DOPENSSL_BUILDING_OPENSSL -DZLIB -DNDEBUG -isystem/opt/local/include/LegacySupport -I/opt/local/include'
#   SSLEAY_BUILT_ON:             'built on: Thu Apr 10 21:43:24 2025 UTC'
#   SSLEAY_PLATFORM:             'platform: darwin-ppc-cc'
#   SSLEAY_DIR:                  'OPENSSLDIR: "/opt/local/libexec/openssl3/etc/openssl"'
#   OPENSSL_ENGINES_DIR:         'ENGINESDIR: "/opt/local/libexec/openssl3/lib/engines-3"'
#   OPENSSL_MODULES_DIR:         'MODULESDIR: "/opt/local/libexec/openssl3/lib/ossl-modules"'
#   OPENSSL_CPU_INFO:            'CPUINFO: N/A'
#   OPENSSL_VERSION_STRING:      '3.5.0'
#   OPENSSL_FULL_VERSION_STRING: '3.5.0'
# 
# Library version information with OPENSSL_version_*():
#   OPENSSL_version_major():          '3'
#   OPENSSL_version_minor():          '5'
#   OPENSSL_version_patch():          '0'
#   OPENSSL_version_pre_release():    ''
#   OPENSSL_version_build_metadata(): ''
# 
# Library information with OPENSSL_info():
#   OPENSSL_INFO_CONFIG_DIR:             '/opt/local/libexec/openssl3/etc/openssl'
#   OPENSSL_INFO_ENGINES_DIR:            '/opt/local/libexec/openssl3/lib/engines-3'
#   OPENSSL_INFO_MODULES_DIR:            '/opt/local/libexec/openssl3/lib/ossl-modules'
#   OPENSSL_INFO_DSO_EXTENSION:          '.dylib'
#   OPENSSL_INFO_DIR_FILENAME_SEPARATOR: '/'
#   OPENSSL_INFO_LIST_SEPARATOR:         ':'
#   OPENSSL_INFO_SEED_SOURCE:            'os-specific'
Use of uninitialized value in concatenation (.) or string at t/local/03_use.t line 73.
#   OPENSSL_INFO_CPU_SETTINGS:           ''
t/local/03_use.t ............................ ok
t/local/04_basic.t .......................... ok
t/local/05_passwd_cb.t ...................... ok
t/local/06_tcpecho.t ........................ ok
t/local/07_sslecho.t ........................ ok
t/local/08_pipe.t ........................... ok
t/local/09_ctx_new.t ........................ ok
t/local/10_rand.t ........................... ok
t/local/11_read.t ........................... ok
t/local/15_bio.t ............................ ok
t/local/20_functions.t ...................... ok
t/local/21_constants.t ...................... ok
t/local/22_provider.t ....................... ok
t/local/22_provider_try_load.t .............. ok
t/local/22_provider_try_load_zero_retain.t .. ok
t/local/23_openssl_init.t ................... ok
t/local/30_error.t .......................... ok
t/local/31_rsa_generate_key.t ............... ok

#   Failed test 'X509V3_EXT_print nid=103	extended-cert.cert.pem:4'
#   at t/local/32_x509_get_cert_info.t line 273.
#          got: 'Full Name:
#   URI:http://intermediate-ca.net-ssleay.example/crl1.crl
# 
# Full Name:
#   URI:http://intermediate-ca.net-ssleay.example/crl2.crl
# '
#     expected: 'Full Name:
#   URI:http://intermediate-ca.net-ssleay.example/crl1.crl
# Full Name:
#   URI:http://intermediate-ca.net-ssleay.example/crl2.crl'
# Looks like you failed 1 test of 746.
t/local/32_x509_get_cert_info.t ............. 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/746 subtests 
t/local/33_x509_create_cert.t ............... ok
t/local/34_x509_crl.t ....................... ok
t/local/35_ephemeral.t ...................... skipped: LibreSSL and OpenSSL 1.1.0 removed support for ephemeral/temporary RSA private keys
t/local/36_verify.t ......................... ok
t/local/37_asn1_time.t ...................... ok
t/local/38_priv-key.t ....................... ok
t/local/39_pkcs12.t ......................... ok
t/local/40_npn_support.t .................... ok
t/local/41_alpn_support.t ................... ok
t/local/42_info_callback.t .................. ok
t/local/43_misc_functions.t ................. ok
t/local/44_sess.t ........................... ok
t/local/45_exporter.t ....................... ok
t/local/46_msg_callback.t ................... ok
t/local/47_keylog.t ......................... ok
t/local/48_client_hello_callback.t .......... ok
t/local/50_digest.t ......................... ok
t/local/61_threads-cb-crash.t ............... ok
t/local/62_threads-ctx_new-deadlock.t ....... ok
t/local/63_ec_key_generate_key.t ............ ok
t/local/64_ticket_sharing.t ................. ok
t/local/65_security_level.t ................. ok
t/local/65_ticket_sharing_2.t ............... ok
t/local/66_curves.t ......................... ok
t/local/kwalitee.t .......................... skipped: These tests are for only for release candidate testing. Enable with RELEASE_TESTING=1

Test Summary Report
-------------------
t/local/32_x509_get_cert_info.t           (Wstat: 256 Tests: 746 Failed: 1)
  Failed test:  345
  Non-zero exit status: 1
Files=47, Tests=2789, 125 wallclock secs ( 2.46 usr  0.89 sys + 79.72 cusr  9.59 csys = 92.66 CPU)
Result: FAIL
Failed 1/47 test programs. 1/2789 subtests failed.
make: *** [test_dynamic] Error 255
•••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
pete 228 /\ prove -b --timer t/local/62_threads-ctx_new-deadlock.t
[13:12:26] t/local/62_threads-ctx_new-deadlock.t .. ok    12644 ms ( 0.01 usr  0.00 sys + 10.23 cusr  0.55 csys = 10.79 CPU)
[13:12:39]
All tests successful.
Files=1, Tests=1, 13 wallclock secs ( 0.13 usr  0.03 sys + 10.23 cusr  0.55 csys = 10.94 CPU)
Result: PASS

Is the failure with t/local/32_x509_get_cert_info.t rather trivial/insignificant?

[h-vn]

With OpenSSL 3.5.0 installed this failure is gone, others are left:

OpenSSL 3.5.0 is faster than 3.0.x, which may help finishing the test before the timeout forced failure and exit. It seems the test now takes closer to 13 seconds (12644 ms) on your system, which is getting close to 20 second timeout limit. This was just updated to 30 seconds.

> #   Failed test 'X509V3_EXT_print nid=103	extended-cert.cert.pem:4'
> #   at t/local/32_x509_get_cert_info.t line 273.
> #          got: 'Full Name:
> #   URI:http://intermediate-ca.net-ssleay.example/crl1.crl
> # 
> # Full Name:
> #   URI:http://intermediate-ca.net-ssleay.example/crl2.crl
> # '
> #     expected: 'Full Name:
> #   URI:http://intermediate-ca.net-ssleay.example/crl1.crl
> # Full Name:
> #   URI:http://intermediate-ca.net-ssleay.example/crl2.crl'

Is the failure with t/local/32_x509_get_cert_info.t rather trivial/insignificant?

It's insignificant in the sense that the reason is change in OpenSSL output instead of failure in Net::SSLeay functionality. The fix is to adapt the test. There's no need to change SSLeay.xs.

If you pull the latest version from GitHub, 32_x509_get_cert_info.t should no longer fail. The timeout in 62_threads-ctx_new-deadlock.t is also now 30 seconds instead of 20 seconds, which should help with slower systems.