AWS SDK for Swift Dependencies: an explanation

[jbelkins]

Several AWS SDK for Swift customers have noticed that recent SDK versions have come with a large number of new dependencies, and have expressed their frustration to us. This should help explain why this has happened and what the real effect of these additions is.

Recently, the AWS SDK for Swift added support for tracing, a feature that is used to track related logs, events, actions, etc. across computing systems. Tracing was introduced in smithy-swift 0.142.0 and AWS SDK for Swift 1.3.44.

The SDK's new tracing feature is based on the OpenTelemetry standard, and is implemented using OpenTelemetry's official Swift implementation.

Though we have taken OpenTelemetry-Swift as a dependency of the SDK, we do not use all of the functionality of OpenTelemetry-Swift, and not all of the dependencies that Xcode / Swift Package Manager installs when resolving OpenTelemetry-Swift's dependencies are actually compiled into your application along with AWS SDK for Swift.

AWS SDK for Swift only uses the following OpenTelemetry-Swift modules, listed with their associated external dependencies:

• OpenTelemetryApi. Has no dependencies.
• OpenTelemetrySdk. Depends on swift-atomics.
• InMemoryExporter. Has no dependencies.
• OpenTelemetryProtocolExporterHTTP. Depends on swift-log, swift-protobuf, and DataCompression.

A limitation of Swift Package Manager is that all dependencies of a package must be resolved and installed to satisfy package dependencies, even if the dependencies are not actually compiled at compile time. So, even though OpenTelemetry-Swift causes a litany of new dependencies to install, only a small subset of those are actually compiled into your application.

This isn't to say that there is no downside to having these extra, unused dependencies in your project. They must still be downloaded and stored during builds & development, taking up network bandwidth and local storage and raising the likelihood of an install error. Xcode and other editors will index them, consuming CPU and RAM when the IDE is running and rendering the development environment less responsive (i.e. your Mac may "beach ball" more.)

As a rule, we work hard to minimize the number of dependencies of the AWS SDK for Swift. When we do take dependencies, they're typically:

• Owned & maintained by AWS (i.e. smithy-swift, aws-crt-swift)
• Owned & maintained by Apple and/or the Swift core team (i.e. swift-atomics, swift-log, swift-protobuf, swift-argument-parser)
• The official Swift implementations of Web standards (i.e. opentelemetry-swift).

In summary: you will see many AWS SDK for Swift dependencies install to your development environment when it installs to your project, but at the time of this writing, these are the only ones of those dependencies that are actually compiled:

Before OpenTelemetry-Swift:

smithy-swift<https://github.com/smithy-lang/[email protected]>
aws-crt-swift<https://github.com/awslabs/[email protected]>
swift-argument-parser<https://github.com/apple/[email protected]>
swift-log<https://github.com/apple/[email protected]>

Adding OpenTelemetry-Swift results in the above four, plus the following additional four compile-time dependencies:

opentelemetry-swift<https://github.com/open-telemetry/[email protected]>
swift-atomics<https://github.com/apple/[email protected]>
swift-protobuf<https://github.com/apple/[email protected]>
DataCompression<https://github.com/mw99/[email protected]>

Please comment below if you have further questions about AWS SDK for Swift dependencies.

[zhu-xiaowei]

Although many of these added dependencies are not actually used and don't significantly impact the installation package size, for customers who are just starting to use AWS Swift SDK, the dependency list looks a bit bloated and lacks the simplicity expected of a basic SDK. Besides opentelemetry-swift, are there other options available, or are there plans to improve this issue in the future?

[jbelkins]

We haven't decided on any approach at this point because there are lots of considerations beyond the dependency graph, but we will explore:

• whether we can modularize the OpenTelemetry dependency, and
• how we can help manage the dependency tree better when we add features in the future.

[zjinhu]

Loading a large number of third-party repositories can cause the CPU load to be too high due to the compilation index. You can use a computer with an M1 chip to compile it, and the CPU load will be 200%

[dayaffe]

@zjinhu @zhu-xiaowei We are working to address this! Please see #2006, we welcome all feedback.