`extra-libraries-static` of dependencies is ignored when linking fully statically

[hasufell]

Context

• pkg-config: Use --libs --static when building with using --enable-executable-static #6935 (comment)
• https://discourse.haskell.org/t/adding-template-haskell-library-produces-linker-error-on-alpine/13000

Problem description

When linking executable A fully statically (--enable-executable-static) against a Haskell library B, which has defined extra-libraries-static (either explicitly or implicitly through pkg-config), then this information is not used when linking A.

Illustration:

+ executable A
   - extra-libraries-static       <- respected
   - extra-libraries              <- respected
   + library dependency B
      - extra-libraries-static    <- ignored
      - extra-libraries           <- respected

Reproducer

https://github.com/hasufell/cabal-extra-libraries

Error

This will produce errors as such:

/home/buildozer/aports/main/curl/src/curl-8.14.1/lib/curl_ntlm_core.c:378:0: error:
    (.text+0x1f6): undefined reference to `DES_ecb_encrypt'
/usr/lib/gcc/x86_64-alpine-linux-musl/14.2.0/../../../../x86_64-alpine-linux-musl/bin/ld: /home/buildozer/aports/main/curl/src/curl-8.14.1/lib/curl_ntlm_core.c:382:(.text+0x21b): undefined reference to `DES_ecb_encrypt'
collect2: error: ld returned 1 exit status
ghc-9.6.7: `gcc' failed in phase `Linker'. (Exit code: 1)

Full build log: https://gist.github.com/hasufell/71755429d3c617701e5a2b276cea5524

[hasufell]

To confirm this, we can compare the linker line of executable A with extra-libraries-static of library B.

Linker invocation:

gcc -Wl,--no-as-needed -o dist-newstyle/build/x86_64-linux/ghc-9.6.7/cabal-extra-libraries-0.1.0.0/x/cabal-extra-libraries/build/cabal-extra-libraries/cabal-extra-libraries -lm -no-pie -Wl,--gc-sections dist-newstyle/build/x86_64-linux/ghc-9.6.7/cabal-extra-libraries-0.1.0.0/x/cabal-extra-libraries/build/cabal-extra-libraries/cabal-extra-libraries-tmp/Main.o -L/cabal-extra-libraries/dist-newstyle/build/x86_64-linux/ghc-9.6.7/curl-1.3.8/build -L/usr/lib -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/containers-0.6.7 -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/bytestring-0.11.5.4 -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/template-haskell-2.20.0.0 -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/pretty-1.1.3.6 -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/ghc-boot-th-9.6.7 -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/deepseq-1.4.8.1 -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/array-0.5.8.0 -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/base-4.18.3.0 -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/ghc-bignum-1.3 -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/ghc-prim-0.10.0 -L/root/.ghcup/ghc/9.6.7/lib/ghc-9.6.7/lib/../lib/x86_64-linux-ghc-9.6.7/rts-1.0.2 /tmp/ghc45646_0/ghc_2.o /tmp/ghc45646_0/ghc_5.o -Wl,-u,hs_atomic_add64 -Wl,-u,hs_atomic_sub64 -Wl,-u,hs_atomic_and64 -Wl,-u,hs_atomic_nand64 -Wl,-u,hs_atomic_or64 -Wl,-u,hs_atomic_xor64 -Wl,-u,hs_atomicread64 -Wl,-u,hs_atomicwrite64 -Wl,-u,base_GHCziTopHandler_runIO_closure -Wl,-u,base_GHCziTopHandler_runNonIO_closure -Wl,-u,ghczmprim_GHCziTupleziPrim_Z0T_closure -Wl,-u,ghczmprim_GHCziTypes_True_closure -Wl,-u,ghczmprim_GHCziTypes_False_closure -Wl,-u,base_GHCziPack_unpackCString_closure -Wl,-u,base_GHCziWeakziFinalizze_runFinalizzerBatch_closure -Wl,-u,base_GHCziIOziException_stackOverflow_closure -Wl,-u,base_GHCziIOziException_heapOverflow_closure -Wl,-u,base_GHCziIOziException_allocationLimitExceeded_closure -Wl,-u,base_GHCziIOziException_blockedIndefinitelyOnMVar_closure -Wl,-u,base_GHCziIOziException_blockedIndefinitelyOnSTM_closure -Wl,-u,base_GHCziIOziException_cannotCompactFunction_closure -Wl,-u,base_GHCziIOziException_cannotCompactPinned_closure -Wl,-u,base_GHCziIOziException_cannotCompactMutable_closure -Wl,-u,base_GHCziIOPort_doubleReadException_closure -Wl,-u,base_ControlziExceptionziBase_nonTermination_closure -Wl,-u,base_ControlziExceptionziBase_nestedAtomically_closure -Wl,-u,base_GHCziEventziThread_blockedOnBadFD_closure -Wl,-u,base_GHCziConcziSync_runSparks_closure -Wl,-u,base_GHCziConcziIO_ensureIOManagerIsRunning_closure -Wl,-u,base_GHCziConcziIO_interruptIOManager_closure -Wl,-u,base_GHCziConcziIO_ioManagerCapabilitiesChanged_closure -Wl,-u,base_GHCziConcziSignal_runHandlersPtr_closure -Wl,-u,base_GHCziTopHandler_flushStdHandles_closure -Wl,-u,base_GHCziTopHandler_runMainIO_closure -Wl,-u,ghczmprim_GHCziTypes_Czh_con_info -Wl,-u,ghczmprim_GHCziTypes_Izh_con_info -Wl,-u,ghczmprim_GHCziTypes_Fzh_con_info -Wl,-u,ghczmprim_GHCziTypes_Dzh_con_info -Wl,-u,ghczmprim_GHCziTypes_Wzh_con_info -Wl,-u,base_GHCziPtr_Ptr_con_info -Wl,-u,base_GHCziPtr_FunPtr_con_info -Wl,-u,base_GHCziInt_I8zh_con_info -Wl,-u,base_GHCziInt_I16zh_con_info -Wl,-u,base_GHCziInt_I32zh_con_info -Wl,-u,base_GHCziInt_I64zh_con_info -Wl,-u,base_GHCziWord_W8zh_con_info -Wl,-u,base_GHCziWord_W16zh_con_info -Wl,-u,base_GHCziWord_W32zh_con_info -Wl,-u,base_GHCziWord_W64zh_con_info -Wl,-u,base_GHCziStable_StablePtr_con_info -Wl,-u,hs_atomic_add8 -Wl,-u,hs_atomic_add16 -Wl,-u,hs_atomic_add32 -Wl,-u,hs_atomic_sub8 -Wl,-u,hs_atomic_sub16 -Wl,-u,hs_atomic_sub32 -Wl,-u,hs_atomic_and8 -Wl,-u,hs_atomic_and16 -Wl,-u,hs_atomic_and32 -Wl,-u,hs_atomic_nand8 -Wl,-u,hs_atomic_nand16 -Wl,-u,hs_atomic_nand32 -Wl,-u,hs_atomic_or8 -Wl,-u,hs_atomic_or16 -Wl,-u,hs_atomic_or32 -Wl,-u,hs_atomic_xor8 -Wl,-u,hs_atomic_xor16 -Wl,-u,hs_atomic_xor32 -Wl,-u,hs_cmpxchg8 -Wl,-u,hs_cmpxchg16 -Wl,-u,hs_cmpxchg32 -Wl,-u,hs_cmpxchg64 -Wl,-u,hs_xchg8 -Wl,-u,hs_xchg16 -Wl,-u,hs_xchg32 -Wl,-u,hs_xchg64 -Wl,-u,hs_atomicread8 -Wl,-u,hs_atomicread16 -Wl,-u,hs_atomicread32 -Wl,-u,hs_atomicwrite8 -Wl,-u,hs_atomicwrite16 -Wl,-u,hs_atomicwrite32 -Wl,-u,base_GHCziStackziCloneStack_StackSnapshot_closure -lHScurl-1.3.8-inplace -lHScontainers-0.6.7 -lHSbytestring-0.11.5.4 -lHStemplate-haskell-2.20.0.0 -lHSpretty-1.1.3.6 -lHSghc-boot-th-9.6.7 -lHSdeepseq-1.4.8.1 -lHSarray-0.5.8.0 -lHSbase-4.18.3.0 -lHSghc-bignum-1.3 -lHSghc-prim-0.10.0 -lHSrts-1.0.2 -lCffi -lcurl -lgmp -lc -lm -lc -lm -lrt -ldl -static

Library B package.conf:

/cabal-extra-libraries # cat dist-newstyle/packagedb/ghc-9.6.7/curl-1.3.8-inplace.conf
name:                   curl
version:                1.3.8
visibility:             public
id:                     curl-1.3.8-inplace
key:                    curl-1.3.8-inplace
license:                BSD-3-Clause
maintainer:             Don Stewart <dons00@gmail.com>
author:                 Sigbjorn Finne
synopsis:               Haskell binding to libcurl
description:
    libcurl is a client-side URL transfer library, supporting FTP, FTPS, HTTP,
    HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS and FILE.
    libcurl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading,
    HTTP form based upload, proxies, cookies, user+password authentication
    (Basic, Digest, NTLM, Negotiate, Kerberos4), file transfer resume,
    http proxy tunneling and more!

    This package provides a Haskell binding to libcurl.

category:               Network
abi:                    inplace
exposed:                True
exposed-modules:
    Network.Curl Network.Curl.Code Network.Curl.Debug Network.Curl.Easy
    Network.Curl.Info Network.Curl.Opts Network.Curl.Post
    Network.Curl.Types

import-dirs:
    /cabal-extra-libraries/dist-newstyle/build/x86_64-linux/ghc-9.6.7/curl-1.3.8/build

library-dirs:
    /cabal-extra-libraries/dist-newstyle/build/x86_64-linux/ghc-9.6.7/curl-1.3.8/build
    /usr/lib

library-dirs-static:
    /cabal-extra-libraries/dist-newstyle/build/x86_64-linux/ghc-9.6.7/curl-1.3.8/build
    /usr/lib

dynamic-library-dirs:
    /cabal-extra-libraries/dist-newstyle/build/x86_64-linux/ghc-9.6.7/curl-1.3.8/build
    /usr/lib

data-dir:               /cabal-extra-libraries/3rdparty/curl-1.3.8/.
hs-libraries:           HScurl-1.3.8-inplace
extra-libraries:        curl
extra-libraries-static:
    curl ssl crypto z brotlidec brotlicommon zstd ssl crypto dl psl
    unistring idn2 unistring nghttp2 cares

include-dirs:           /usr/include
depends:                base-4.18.3.0 bytestring-0.11.5.4 containers-0.6.7
haddock-interfaces:
    /cabal-extra-libraries/dist-newstyle/build/x86_64-linux/ghc-9.6.7/curl-1.3.8/doc/html/curl/curl.haddock

haddock-html:
    /cabal-extra-libraries/dist-newstyle/build/x86_64-linux/ghc-9.6.7/curl-1.3.8/doc/html/curl

[hasufell]

It turns out that this is in fact also partly a GHC issue.

• cabal passes -static (affects Haskell libs only) and -optl-static (affects system libs) to GHC and additionally consults extra-libraries-static and library-dirs-static to pass further flags to GHC:

  • cabal/Cabal/src/Distribution/Simple/GHC/Build/Link.hs

    Lines 119 to 142 in 1e3c355

    	-- TODO: Shouldn't we use withStaticLib for libraries and something else
    	-- for foreign libs in the three cases where we use `withFullyStaticExe` below?
    	linkerOpts rpaths =
    	mempty
    	{ ghcOptLinkOptions =
    	PD.ldOptions bi
    	++ [ "-static"
    	| withFullyStaticExe lbi
    	]
    	-- Pass extra `ld-options` given
    	-- through to GHC's linker.
    	++ maybe
    	[]
    	programOverrideArgs
    	(lookupProgram ldProgram (withPrograms lbi))
    	, ghcOptLinkLibs =
    	if withFullyStaticExe lbi
    	then extraLibsStatic bi
    	else extraLibs bi
    	, ghcOptLinkLibPath =
    	toNubListR $
    	if withFullyStaticExe lbi
    	then cleanedExtraLibDirsStatic
    	else cleanedExtraLibDirs

• extra-libraries-static and library-dirs-static are only considered from the current component
• cabal also passes the dependent package IDs to GHC
• GHC then gets additional linker flags from the package id's that have been passed in by cabal:
  • https://github.com/ghc/ghc/blob/867c26755e8855c6df949e65df0c2aebc9da64c3/compiler/GHC/Linker/Static.hs#L168-L172
  • https://github.com/ghc/ghc/blob/867c26755e8855c6df949e65df0c2aebc9da64c3/compiler/GHC/Linker/Unit.hs#L36-L48
• we can see that in collectLinkOpts we only consider unitExtDepLibsSys, which corresponds to the extra-libraries from a package .conf file
• GHC is entirely unaware of extra-libraries-static, so when we statically link an executable... it will just collect the libraries from extra-libraries and fail to link

[hasufell]

https://gitlab.haskell.org/ghc/ghc/-/issues/26434