Android libFoundationXML.so missing dependency on libz.so

[cobbal]

Describe the bug
In the quasi-official swift-android-sdk 6.2 release, libFoundationXML.so seems to have an unstated dependency on the system libz.so. This causes it to fail at load time.

Interestingly, libFoundationNetworking.so does state this dependency, though I didn't test loading it.

To Reproduce

run System.loadLibrary("FoundationXML") in an android app, and get

java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "deflateInit2_" referenced by "[...]/lib/arm64/libFoundationXML.so"...

Alternatively, just looking at the artifact statically:

root@54093e12c46b:~/.swiftpm/swift-sdks/swift-6.2-RELEASE-android-0.1.artifactbundle/swift-android# nm swift-resources/usr/lib/swift-aarch64/android/libFoundationXML.so  | grep deflateInit
                 U deflateInit2_

root@54093e12c46b:~/.swiftpm/swift-sdks/swift-6.2-RELEASE-android-0.1.artifactbundle/swift-android# patchelf --print-needed swift-resources/usr/lib/swift-aarch64/android/libFoundationXML.so
libFoundation.so
libFoundationInternationalization.so
libFoundationEssentials.so
lib_FoundationICU.so
libswiftDispatch.so
libdispatch.so
libBlocksRuntime.so
libswiftCore.so
libswift_Concurrency.so
libswift_StringProcessing.so
libswift_RegexParser.so
libswift_Builtin_float.so
libswift_math.so
libswiftAndroid.so
libm.so
libdl.so
libc.so

root@54093e12c46b:~/.swiftpm/swift-sdks/swift-6.2-RELEASE-android-0.1.artifactbundle/swift-android# nm ndk-sysroot/usr/lib/aarch64-linux-android/28/libz.so | grep deflateInit
00000000000021d8 T deflateInit2_
00000000000021e0 T deflateInit_

root@54093e12c46b:~/.swiftpm/swift-sdks/swift-6.2-RELEASE-android-0.1.artifactbundle/swift-android# patchelf --print-needed swift-resources/usr/lib/swift-aarch64/android/libFoundationNetworking.so
libFoundation.so
libz.so
libFoundationInternationalization.so
libFoundationEssentials.so
lib_FoundationICU.so
libswiftDispatch.so
libdispatch.so
libBlocksRuntime.so
libswiftCore.so
libswift_Concurrency.so
libswift_StringProcessing.so
libswift_RegexParser.so
libswift_Builtin_float.so
libswift_math.so
libswiftSynchronization.so
libswiftAndroid.so
libm.so
libdl.so
libc.so

Patching with patchelf --add-needed libz.so libFoundationXML.so seems to work around the issue

Expected behavior
library should load as provided

Configuration (please complete the following information):

• Swift Version: 6.2-RELEASE
• OS: Android
• OS Version: API 36

Regression information:

Additional context

[finagolfin]

Thanks for reporting this, we'll take a look. 👍

[finagolfin]

@marcprux, mind looking into this, since you added this build in the CI?

I plan to shift the libxml build to build-script, then I'll make sure it stays the same.

[marcprux]

Yeah, it looks like this is what is breaking one of my packages that tests FoundationXML:

+ adb shell cd /data/local/tmp/android-xctest &&  ./universalPackageTests.xctest  &&  ./universalPackageTests.xctest  --testing-library swift-testing && [ $? -eq 0 ] || [ $? -eq 69 ]
CANNOT LINK EXECUTABLE "./universalPackageTests.xctest": cannot locate symbol "deflateInit2_" referenced by "/data/local/tmp/android-xctest/libFoundationXML.so"...
Error: The process '/usr/bin/sh' failed with exit code 1

This was definitely working before, with Swift 6.1.1, and broke when the CI was updated to 6.2.

I wonder what changed with libFoundationXML.so? We could always have the Android build manually run patchelf --add-needed libz.so libFoundationXML.so, but it would be nice to know why libz was needed before and now it isn't.

[finagolfin]

Well, in his case, he switched from my 6.1 SDK built on my Android CI to the Docker-produced one we put out for 6.2. Since this library is built differently in each, understandable that the dependencies changed.

Looking into it now, my 6.2 SDK links against Termux's libxml2.so, which already has that dependency listed, so I didn't have to do anything. Whereas in our new Docker scripts, we are statically linking in libxml2 but not zlib, which the static linux SDK and build-script do add. Perhaps we should add that back?

[compnerd]

Perhaps we should add that back?

Yeah, I think that is a good idea. The TBC and swift.org Windows Android builds both statically link libxml2, libz, and curl for Foundation.

[marcprux]

We build libcurl and libxml2, but not libz. libz is available as a stable library (unlike libxml), so would it be better to dynamically link to it, or just statically link to ANDROID_NDK_HOME/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/libz.a?

If we statically link to it, would we encounter issues when a Swift library adds it as a dynamic link option with linkerSettings: [.linkedLibrary("z", .when(platforms: [.linux, .android]))], like SkipZip does?

I find their official guidance confusing:

The NDK always includes the latest zlib header files at the time of release, and the libz.a included in the NDK for static linking is always that same version, but the libz.so for dynamic linking comes from the device, and be whatever version happened to be released on that device. In particular, this means that the headers you built against do not match the version of zlib on the device, so the usual warnings against making assumptions about implementation details are especially valid here. We are not aware of any issues with public API, but struct layout in particular has changed over time and will likely continue to do so. Note that new API in later zlib versions will obviously not be available on OS versions that predate the API. It is possible to avoid all these problems (at the cost of increased APK size) by always using the static libz.a instead of libz.so.

[compnerd]

Yes, if you have the dynamic link, that would be an issue. However, the recommendation seems to push towards static linking due to ABI concerns.

[finagolfin]

Up to you, Marc, either approach seems fine to me, haven't dealt with these dll hell issues on Android myself.

[marcprux]

OK, I'll think on it.

@cobbal, in the short-term, does having your XML-dependent target also explicitly link to "z" work around the issue without having to resort to patchelf? It does work for me.

[cobbal]

@marcprux Good to know. patchelf is oddly enough an easier workaround for how we have things set up in maven packages.