CDRIVER-6071 replace build-and-test-with-toolchain with openssl-compat

[eramongodb]

Resolves CDRIVER-6071. Removes dependency on the legacy C toolchain project by replacing the build-and-test-with-toolchain task (which began inexplicably failing since 2025-07-17; did the artifact expire?) with a new openssl-compat matrix.

---

Historically, the legacy C toolchain project was meant to test both SSL library compatibility and build tool minimum requirements, as currently documented by its README:

This project exists to provide a stable set of tools and libraries at set minimum versions.

However, its purpose has long been made obsolete or superseded by better alternatives. Support for LibreSSL was dropped in CDRIVER-5693. Boost is irrelevant (C++ Driver doesn't support it since CXX-2797, nor does it use the legacy C toolchain anyways). The minimum required CMake version was raised to 3.15 in CDRIVER-4637 (a new branch as created by https://github.com/10gen/mongo-c-toolchain/pull/2 to support this new requirement); this can easily be replaced with a cmake-compat matrix (+ soon powered by uvx cmake instead of find-cmake-latest.sh). The legacy C toolchain also provides zstd, but AFAICT we do not actually test this minimum zstd compatibility requirement, or any other external library dependency's minimum version requirements (we can consider adding additional tasks to accomplish this if needed).

Therefore, the only value (that was) being provided by the build-and-test-with-toolchain task is OpenSSL library compatibility checks, which is worth preserving given the wide variety of OpenSSL library versions across platforms and distros. This PR proposes preserving and refining this compatibility coverage as the new openssl-compat matrix.

---

First, it is important to note, per OpenSSL docs:

All versions not present above are out of support and should not be used. Users of those versions are encouraged to upgrade to the newest series as soon as possible. Extended support for 1.1.1 and 1.0.2 to gain access to security fixes for those versions is available.

The versions "present above" are currently 3.0 through 3.5. This means we can/should drop compatibility coverage for any version older than 3.0 that is not either 1.0.2 and 1.1.1. Relative to the legacy C toolchain, this means dropping 0.9.8, 1.0.1, and 1.1.0.

Note

The results of an EVG distro survey on 2025-08-05 report the following distros which are currently used by the C Driver that do not have provide OpenSSL v3 or newer via default system packages:

• amazon2-arm64-latest: 1.0.2k FIPS
• debian-11: 1.1.1n
• rhel7-latest: 1.0.2k FIPS
• rhel8-latest: 1.1.1k FIPS
• rhel8-power: 1.1.1k FIPS
• rhel8-zseries: 1.1.1k FIPS
• rhel80: 1.1.1 FIPS
• rhel84: 1.1.1g FIPS
• rhel94: 1.1.1g FIPS
• ubuntu2004: 1.1.1f
• ubuntu2004-arm64: 1.1.1f

These results support the argument for compatibility-checking pre-v3 OpenSSL versions 1.0.2 and 1.1.1 only.

Furthermore, per docs:

The following OpenSSL version(s) are FIPS validated:

• 3.1.2: FIPS 140-3
• 3.0.9: FIPS 140-2
• 3.0.8: FIPS 140-2
• 3.0.0: FIPS 140-2

Other OpenSSL Releases MAY use the validated FIPS provider, but MUST NOT build and use their own FIPS provider.

Given we have plenty of OpenSSL coverage with other compile-and-test matrices, I believe we do not need to extend compatibility checks for pre-v3 OpenSSL versions more than we already have. That being said, all "special" OpenSSL-related tasks have been relocated from their prior build variants into the openssl-compat matrix: all remaining -openssl tasks simply use the system-provided OpenSSL library as-is without any special steps.

That being said, even though some system-provided OpenSSL libraries are FIPS-validated, this doesn't necessarily mean the FIPS module is enabled and used by default. Therefore, the openssl-compat matrix includes a few custom OpenSSL installations that do set the appropriate openssl.cnf properties to use the FIPS module by default. (That being said, does mongoc even use the FIPS module programmatically at all?) For now, only 3.0.9 and 3.1.2 are used for FIPS 140-2 and FIPS 140-3 coverage respectively (each being the latest patch release for its minor version series).

Important

This PR deliberately drops OpenSSL FIPS Object Module 2.0 integration checks. OpenSSL 3.0.9 already provides FIPS 140-2 compliance coverage. Per Wikipedia, "The FIPS Object Module 2.0 remained FIPS 140-2 validated in several formats until September 1, 2020, when NIST ... designated all non-compliant modules as 'Historical'. This designation includes a caution to federal agencies that they should not include the module in any new procurements." Aside from the distros listed in the survey above which provide FIPS support, I do not think we need to go out of our way to test this any longer.

The Ubuntu 24.04 distro is selected for the openssl-compat matrix primarily to avoid complications with registering trusted certificates with the custom OpenSSL installation on RHEL or Arch Linux (no special handling seems to be required on Ubuntu). RHEL is also complicated due to backports of EVP_KDF_* functions into their OpenSSL 1.1.1 package and depending on it (e.g. by libk5crypto.so.3). Since the goal of this matrix is OpenSSL version compatibility checks, platform-specific OpenSSL compatibility is deferred to other existing -openssl tasks.

The openssl-compat matrix tasks also call the run-auth-tests.sh script via the run auth tests EVG command to extend compatibility checks to runtime behavior, not just compile-time compatibility as is the case for build-and-test-with-toolchain. This should hopefully allow us to detect issues like CDRIVER-5998 a bit better than before.

---

Numerous fixes and improvements are made to authentication-related tasks and scripts:

• CDRIVER-5990: re-enable auth tests which were being skipped pending DEVPROD-16954, which was resolved in May 2025.
• CDRIVER-5995: re-enable auth tests which were being skipped pending DEVPROD-9029, which was resolved in June 2025.
• Fix authentication-* and ocsp-* task failures due to updated CMake binary directory paths.
  • Update test-aws-* scripts for consistency.
• Harden Bash scripts to ensure expanded variables are defined as expected.
  • The atlas_x509_* Evergreen project variables required by run-auth-tests.sh have been moved into the EVG project repository settings so they are inherited by the r2.1 release branch.
• Disabled several MacOS (darwinssl) OCSP tasks:
  • MacOS ARM64 only provides MongoDB Server 6.0 or newer. Filtered pre-6.0 versions from the OCSP matrix accordingly.
    • Avoid extending coverage to MacOS Intel for pre-6.0 coverage, as these distro are very limited resources.
  • mongoc does not seem to implement support for OCSP soft failure with the Secure Transport library (darwinssl). Does documentation need to be updated per spec? The tlsDisableOCSPEndpointCheck URI option is rejected by both Secure Transport (darwinssl) and Secure Channel (winssl), but neither of these behaviors seems to be publically documented.
  • OCSP Test 3 also consistently fails with the Secure Transport library (darwinssl) despite Test 1 consistently passing, yet they "should be identical" per spec when OCSP stabling is not supported (which I don't think is supported for Secure Transport...?). I've disabled this task for now, but I don't think it should be left disabled.

[eramongodb]

CDRIVER-5990 unskip auth tests on zSeries

Per this patch build, zSeries no-SASL succeeds, but zSeries SASL fails during the "Authenticating using GSSAPI" test with:

ERROR:      cluster: Authentication failed: SASL Failure: (-1): generic failure
Ping failure: SASL Failure: (-1): generic failure