FAQ: Why we do not support system-installed dependencies (Packaging Policy)

[dragon-archer]

Is your feature request related to a problem? Please describe.
Currently, cmake/package.cmake directly calls FetchContent_Declare and FetchContent_MakeAvailable, which forces users to download the dependencies from GitHub and build them from source.

Describe the solution you'd like
Check whether the dependencies are already installed on the system (e.g. using find_package) and use them if present, otherwise fallback to download from GitHub and build them from source.

Describe alternatives you've considered
Provide an option to select whether using system-installed dependencies

Additional context
N/A

[16bit-ykiko]

Hi, thanks for the suggestion.

After careful consideration, I have decided to decline this request. While I understand the rationale behind using system-installed dependencies (especially for Linux packaging guidelines), this project follows a hermetic build philosophy for several specific reasons:

1. Strict Dependency Locking:
   We intentionally lock dependency versions to guarantee stability. Allowing system libraries introduces the risk of version mismatches, where an older or newer version on the user's system behaves differently than the one we develop against. FetchContent ensures that every build uses the exact code we tested.

2. Performance and Reproducibility (Static Linking + LTO):
   I prioritize a build process that utilizes Static Linking combined with Link Time Optimization (LTO). This approach yields the maximum performance benefit by allowing the compiler to optimize across translation units. Relying on pre-built system shared libraries often prevents us from achieving these specific optimization and reproducibility goals.

3. Maintenance and Technical Constraints:

   • The "Supply Chain" Misconception: Many modern C++ libraries are heavily templated or header-only. Even if we link dynamically against a system library, the logic is often inlined into the application binary during compilation. Therefore, simply replacing a shared library (.so/.dll) on the system often does not update the code actually running in the application, negating the perceived security benefits of dynamic linking.
   • Unmaintainable Bug Reports: If we allow arbitrary system libraries, users or packagers might build dependencies with flags incompatible with our project (e.g., different C++ standards, ABI settings, or definitions). When this causes crashes or undefined behavior, the error reports come back to this repository. Debugging these issues is nearly impossible without a controlled, reproducible build environment, often leading to wasted time on issues that are strictly environmental.

For these reasons, I am enforcing the download-and-build approach to ensure the quality and stability of the project.

A note on local development:
I anticipate the argument that supporting system libraries could speed up local development cycles. However, adding such a CMake option creates a "slippery slope": once the switch exists, it will be used by packagers for production builds to comply with unbundling policies.

Since there is no technical way to restrict a CMake option to "development only," and enabling it for releases brings back all the stability/ABI issues mentioned above, I have decided not to expose this interface at all to prevent misuse.

[16bit-ykiko]

See also:
https://blogs.gentoo.org/mgorny/2021/02/19/the-modern-packagers-security-nightmare/
https://blogs.gentoo.org/mgorny/2021/02/23/why-not-rely-on-app-developer-to-handle-security/

[dragon-archer]

Hi, thank you for the detailed explanation. I understand and respect the goal of maintaining a hermetic build environment to ensure reproducibility and minimize invalid bug reports.

However, I would like to offer a few counterpoints for reconsideration regarding the strictness of this policy:

Regarding Strict Dependency Locking: While version pinning guarantees stability, most libraries relied upon here generally maintain stable APIs (i.e. LLVM and spdlog). find_package with minimum version requirements is usually sufficient for C++ ecosystems. Additionally, since the project currently supports using a system-installed LLVM, extending this flexibility to other libraries would be consistent with that existing design choice.

Performance Trade-offs: While Static Linking and LTO provide performance benefits for final release artifacts, they impose significant build-time costs. For local development or distribution packaging, the priority is often build speed and integration rather than squeezing out the last roughly 1-5% of runtime performance. While in performance‑sensitive contexts, optimizations at the algorithm or vectorization level often yield more significant improvements.

Usability vs. Security: My request isn't primarily about security, but rather about adhering to standard distribution guidelines and improving developer flexibility. Regarding the maintenance concern: we can make the CMake option OFF by default (preserving your hermetic philosophy) and clearly explicitly mark system-dependency builds as "unsupported" in the documentation. This shifts the burden of ABI incompatibility or crashes onto the packager/user, not the maintainer, while still enabling the software to be bundled in standardized environments.

Finally, I've successfully built clice using clang++ and LLVM from MSYS2/CLANG64 toolchain, and I'd like to make it into a MSYS2 package. Enabling using system-installed dependencies will make distributing clice from package managers much easier.

[16bit-ykiko]

Regarding Strict Dependency Locking: While version pinning guarantees stability, most libraries relied upon here generally maintain stable APIs (i.e. LLVM and spdlog). find_package with minimum version requirements is usually sufficient for C++ ecosystems. Additionally, since the project currently supports using a system-installed LLVM, extending this flexibility to other libraries would be consistent with that existing design choice.

Since LLVM frequently breaks its APIs, pinning the version is essential. We currently make a specific exception for LLVM, allowing the use of user-compiled versions rather than mandating a build from source, as most of our developers compile their own local copy. However, for all other libraries, I want to strictly enforce building from source.

Finally, I've successfully built clice using clang++ and LLVM from MSYS2/CLANG64 toolchain, and I'd like to make it into a MSYS2 package. Enabling using system-installed dependencies will make distributing clice from package managers much easier.

Actually, there is no need to compile a separate version of clice for MSYS2. Our current Windows build artifacts work fine with the C++ compilers inside MSYS2. This would only add extra maintenance overhead. While theoretically, the package maintainers are supposed to handle these issues, in practice, users still end up reporting them to us. Please forgive me for declining this request :(