Skip to content

Commit 74fd4e0

Browse files
daniel-grumbergdaniel-grumberg
committed
First draft of proposal for static library binary target on non apple platforms
1 parent e07c7e2 commit 74fd4e0

File tree

1 file changed

+161
-0
lines changed

1 file changed

+161
-0
lines changed

‎proposals/XXXX-swiftpm-static-library-binary-target-non-apple-platforms.md

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# Binary Static Library Dependencies
2+
3+
* Proposal: [SE-NNNN](NNNN-filename.md)
4+
* Authors:
5+
* Review Manager: TBD
6+
* Status: **Awaiting implementation**
7+
8+
<!--- *During the review process, add the following fields as needed:* --->
9+
10+
* Implementation: [swiftlang/swift-package-manager#6967](https://github.com/swiftlang/swift-package-manager/pull/6967) [swiftlang/swift-package-manager#8605](https://github.com/swiftlang/swift-package-manager/pull/8605)
11+
* Bugs: [Swift Package Manger Issue](https://github.com/swiftlang/swift-package-manager/issues/7035)
12+
13+
<!---
14+
* Decision Notes: [Rationale](https://forums.swift.org/), [Additional Commentary](https://forums.swift.org/)
15+
* Previous Revision: [1](https://github.com/swiftlang/swift-evolution/blob/...commit-ID.../proposals/NNNN-filename.md)
16+
* Previous Proposal: [SE-XXXX](XXXX-filename.md)
17+
--->
18+
19+
20+
## Introduction
21+
22+
Swift continues to grow as a cross-platform language supporting a wide variety of use cases from [programming embedded device](https://www.swift.org/blog/embedded-swift-examples/) to [server-side development](https://www.swift.org/documentation/server/) across a multitude of [operating systems](https://www.swift.org/documentation/articles/static-linux-getting-started.html).
23+
However, currently SwiftPM supports linking against binary dependencies on Apple platforms only.
24+
This proposal aims to make it possible to provide static library dependencies on non-Apple platforms.
25+
26+
Swift-evolution thread:
27+
28+
## Motivation
29+
30+
The Swift Package Manager’s [`binaryTarget` type](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0272-swiftpm-binary-dependencies.md) lets packages vend libraries that either cannot be built in Swift Package Manager for technical reasons,
31+
or for which the source code cannot be published for legal or other reasons.
32+
33+
In the current version of SwiftPM, binary targets support the following:
34+
35+
* Libraries in an Xcode-oriented format called XCFramework, and only for Apple platforms, introduced in [SE-0272](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0272-swiftpm-binary-dependencies.md).
36+
* Executables through the use of artifact bundles introduced in [SE-0305](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0305-swiftpm-binary-target-improvements.md).
37+
38+
We aim here to bring a subset of the XCFramework capabilities to non-Apple platforms in a safe way.
39+
40+
While this proposal is specifically focused on binary static library dependencies without unresolved external symbols on non-Apple platforms,
41+
it tries to do so in a way that will not prevent broader future support for static libraries and dynamically linked libraries.
42+
43+
## Proposed solution
44+
45+
This proposal extends artifact bundles introduced by [SE-0305](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0305-swiftpm-binary-target-improvements.md) to include a new kind of artifact type to represent a binary library dependency: `staticLibrary`.
46+
The artifact manifest would encode the following information for each variant:
47+
48+
* The static library to pass to the linker.
49+
On Apple and Linux platforms, this would be `.a` files and on Windows it would be a `.lib` file.
50+
* Enough information to be able to use the library's API in the packages source code,
51+
i.e., headers and module maps for libraries exporting a C-based interface.
52+
53+
Additionnaly, we propose the addition of an auditing tool that can validate the library artifact is safe to use across the platforms supported by the Swift project.
54+
Such a tool would ensure that people do not accidentally distribute artifacts that require dependencies that are not met on the various deployment platforms.
55+
56+
## Detailed design
57+
58+
This section describes the changes to artifact bundle manifests in detail, the semantic impact of the changes on SwiftPM's build infrastructure, and describes the operation of the auditing tool.
59+
60+
### Artifact Manifest Semantics
61+
62+
The artifact manifest JSON format for a static library is described below:
63+
64+
```json
65+
{
66+
"schemaVersion": "1.0",
67+
"artifacts": {
68+
"<identifier>": {
69+
"version": "<version number>",
70+
"type": "staticLibrary",
71+
"variants": [
72+
{
73+
"path": "<relative-path-to-library-file>",
74+
"headerPaths": ["<relative-path-to-header-directory-1>, ...],
75+
"moduleMapPath": "<path-to-module-map>",
76+
"supportedTriples": ["<triple1>", ... ],
77+
},
78+
...
79+
]
80+
},
81+
...
82+
}
83+
}
84+
```
85+
86+
The additions are:
87+
88+
* The `staticLibrary` artifact `type` that indicates this binary artifact is not an executable but rather a static library to link against.
89+
* The `headerPaths` field specifies directory paths relative to the root of the artifact bundle that contain the header interfaces to the static library.
90+
These are forwarded along to the swift compiler (or the C compiler) using the usual search path arguments.
91+
Each of these directories can optionally contain a `module.modulemap` file that will be used for importing the API into Swift code.
92+
* The optional `moduleMapPath` field specifies a custom module map to use if the header paths do not contain the module definitions or to provide custom overrides.
93+
94+
As with executable binary artifacts, the `path` field represents the relative path to the binary from the root of the artifact bundle,
95+
and the `supportedTriples` field provides information about the target triples supported by this variant.
96+
97+
An example artifact might look like:
98+
```json
99+
{
100+
"schemaVersion": "1.0",
101+
"artifacts": {
102+
"my-artifact": {
103+
"type": "staticLibrary",
104+
"version": "1.0.0",
105+
"variants": [
106+
{
107+
"path": "artifact.a",
108+
"headerPaths": ["include"],
109+
"supportedTriples": ["aarch64-unknown-linux-gnu"]
110+
}
111+
]
112+
}
113+
}
114+
}
115+
```
116+
117+
### Auditing tool
118+
119+
Without proper auditing it would be very easy to provide binary static library artifacts that call into unresolved external symbols that are not available on the runtime platform, e.g., due to missing linkage to a system dynamic library.
120+
121+
We propose the introduction of a new tool that can validate the "safety" of a binary library artifact across the platforms it supports and the corresponding runtime environment.
122+
123+
In this proposal we restrict ourselves to static libraries that do not have any external dependencies.
124+
To achieve this we need to be able to detect validate this property across the three object file formats used in static libraries on our supported platforms: [Mach-O](https://developer.apple.com/library/archive/documentation/Performance/Conceptual/CodeFootprint/Articles/MachOOverview.html#//apple_ref/doc/uid/20001860-BAJGJEJC) on Apple platforms, [ELF](https://refspecs.linuxfoundation.org/elf/elf.pdf) on Linux-based platforms, and [COFF](https://learn.microsoft.com/en-us/windows/win32/debug/pe-format) on Windows.
125+
All three formats express references to external symbols as _relocations_ which reside in a single section of each object file.
126+
127+
The tool would scan every object file in the static library and construct a complete list of symbols defined and referenced across the entire library.
128+
The tool would then check that the referenced symbols list is a subset of the set of defined symbols and emit an error otherwise.
129+
130+
## Security
131+
132+
This proposal brings the security implications outlined in [SE-0272](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0272-swiftpm-binary-dependencies.md#security) to non-Apple platforms,
133+
namely that a malicious attacker having access to both the server hosting the artifact and the git repository that vends the Package Manifest could provide a malicious library.
134+
Users should exercise caution when onboarding binary dependencies.
135+
136+
## Impact on existing packages
137+
138+
No current package should be affected by this change since this is only an additive change in enabling SwiftPM to use binary target library dependencies on non-Apple platforms.
139+
140+
## Future directions
141+
142+
### Extend binary compatibility guarantees
143+
144+
This proposal makes no guarantees regarding the availability of symbols in the runtime environment and therefore the auditing tool validates that all referenced symbols in the binary dependency are already resolved.
145+
In the future we would like to provide more exhaustive guarantees about core system libraries and symbols that are guaranteed to be present on supported runtime environments, e.g. `libc.so`.
146+
This is very similar to the [`manylinux`](https://peps.python.org/pep-0513/) effort in the Python community, but the standardization effort would need to extend to Apple platforms and Windows platforms.
147+
Specifically we would extend the auditing tool to allow unresolved external symbols to symbols that are known to exist.
148+
149+
### Support Swift static libraries (needs fact checking)
150+
151+
Today Swift static libraries can not be fully statically linked.
152+
However, once binary compatibility guarantees are extended to include Swift SDK symbols,
153+
static libraries exposing a purely Swift API should be supported.
154+
To do this we would extend the static library binary artifact manifest to provide a `.swiftinterface` file that can be consumed by the Swift compiler.
155+
156+
### Add support for dynamically linked dependencies
157+
158+
On Windows dynamic linking requires an _import library_ which is a small static library that contains stubs for symbols exported by the dynamic library.
159+
These stubs are roughly equivalent to a PLT entry in an ELF executable, but are generated during the build of the dynamic library and must be provided to clients of the library for linking purposes.
160+
Similarly on Linux and Apple platforms binary artifact maintainers may wish to provide a dynamic library stub to improve link performance.
161+
To support these use cases the library binary artifact manifest schema could be extended to provide facilities to provide both a link-time and runtime dependency.

0 commit comments

Comments
 (0)