Skip to content

Commit 55f8620

Browse files
committed
Elaborate on a few more topic, such as API/ABI stability, governance, tools integration, and some other edits
1 parent ed96078 commit 55f8620

File tree

1 file changed

+50
-23
lines changed

1 file changed

+50
-23
lines changed

visions/swift-testing.md

Lines changed: 50 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -477,9 +477,10 @@ be unstable and terminate unexpectedly), and another which launches and monitors
477477
the first process. In this document, we'll refer to this second process as the
478478
**harness**. A two-process architecture involving a supervisory harness helps
479479
enable functionality such as relaunching after a crash, live progress reporting,
480-
or outputting results to a standard format. A modern testing library should
481-
provide any necessary APIs and runtime support to facilitate integration with a
482-
harness.
480+
or outputting results to a standard format. Some cases effectively _require_ a
481+
harness, such as when launching tests on a remotely-connected device. A modern
482+
testing library should provide any necessary APIs and runtime support to
483+
facilitate integration with a harness.
483484

484485
## Today’s solution: XCTest
485486

@@ -549,7 +550,8 @@ public macro Suite(
549550
)
550551
```
551552

552-
Here are some usage examples:
553+
Then, test authors may attach these macros to functions or types in a test
554+
target. Here are some usage examples:
553555

554556
```swift
555557
import Testing
@@ -618,24 +620,29 @@ actor (such as `@MainActor`), in accordance with standard language and type
618620
system rules. This allows tests to match the global actor of their subject and
619621
reduce the need for suspension points.
620622

621-
#### Runtime test discovery
623+
#### Test discovery
622624

623-
At runtime, tests will be represented as instances of `Test` and be retrieved
624-
using compiler-emitted metadata. The specifics of this mechanism have not yet
625-
been designed, but are planned to be covered in revisions to
626-
[SE-0385](https://github.com/apple/swift-evolution/blob/main/proposals/0385-custom-reflection-metadata.md)
627-
or a new `@section` attribute (see
628-
[pitch](https://forums.swift.org/t/pitch-low-level-linkage-control-attributes-used-and-section/65877))
629-
which will allow annotating properties as having only constant data and placing
630-
their contents in a special section of the binary.
625+
To faciliate test discovery, the attached macros above will eventually use a
626+
feature such as `@linkage`, an attribute for controlling low-level symbol
627+
linkage (see
628+
[pitch](https://forums.swift.org/t/pitch-2-low-level-linkage-control/69752)).
629+
It will allow placing variables and functions in a special section of the binary,
630+
where they can be retrieved and at runtime or inspected statically at build time.
631631

632-
However, before that support lands, the testing library will use a temporary
632+
However, before that feature lands, the testing library will use a temporary
633633
approach of iterating through types conforming to a known protocol and
634634
gathering their tests by calling a static property. The attached macros will
635635
emit code which generates the types to be discovered using this mechanism. Once
636636
more permanent support lands, the attached macros will be adjusted to adopt it
637637
instead.
638638

639+
Regardless of which technique the attached macros above use to facilitate test
640+
discovery, the APIs called by their expanded code need not be considered public
641+
or stable. In particular, code emitted by these macros may call
642+
underscore-prefixed APIs declared in the `Testing` module, which are marked
643+
`public` to facilitate use by these macros but are generally considered a
644+
private implementation detail.
645+
639646
### Traits
640647

641648
As discussed earlier, it is important to support specifying traits for a test.
@@ -853,6 +860,19 @@ leverage
853860
to capture exactly how these arrays differ and present that information to the
854861
developer as part of the test output or in the IDE.
855862

863+
### API and ABI stability
864+
865+
Once this project reaches its first major release, its API will be considered
866+
stable and will follow the same general rules as the Swift standard library.
867+
Source-breaking API changes will be considered a last resort and be preceded by
868+
a generous deprecation period.
869+
870+
However, unlike the standard library, this project will generally not guarantee
871+
ABI stability. Tests are typically only run during the development cycle and
872+
aren't distributed to end users, so the runtime testing library will not be
873+
included in OS distributions and thus its interfaces do not need to maintain
874+
ABI stability.
875+
856876
### Project governance
857877

858878
For this testing solution to stand the test of time, it needs to be well
@@ -866,22 +886,29 @@ process inspired by, but separate from,
866886
described in writing using a standard
867887
[proposal template](https://github.com/apple/swift-testing/blob/main/Documentation/Proposals/0000-proposal-template.md) and discussed in the
868888
[swift-testing](https://forums.swift.org/c/related-projects/swift-testing/103)
869-
category of the Swift Forums.
889+
category of the Swift Forums. The process for pitching and submitting proposals
890+
for review will be formalized separately and documented in the project repo.
870891

871892
A new group—tentatively named the _Swift Testing Workgroup_—will be formed to
872-
act as the primary governing body for this project. The responsibilities of
873-
members of this workgroup will include:
893+
act as the primary governing body for this project. This group will be
894+
considered a sub-group of the planned
895+
[Ecosystem Steering Group](https://www.swift.org/blog/evolving-swift-project-workgroups/)
896+
once it has been formed. The group will retroactively assume responsibility for
897+
the [swift-corelibs-xctest](https://github.com/apple/swift-corelibs-xctest)
898+
project as well. The responsibilities of members of this workgroup will include:
874899

875900
* defining and approving roadmaps for the project;
876901
* scheduling proposal reviews;
877902
* guiding community discussion;
878903
* making decisions about proposals; and
879-
* working with members of related workgroups, such as the Ecosystem, Platform,
880-
Server, or Documentation workgroups, on topics which intersect with their
881-
areas.
882-
883-
The membership of this workgroup and specifics about its role in the project
884-
will be formalized separately.
904+
* working with members of related workgroups, such as the
905+
[Platform](https://www.swift.org/platform-steering-group/),
906+
[Server](https://www.swift.org/sswg/), or
907+
[Documentation](https://www.swift.org/documentation-workgroup/) workgroups, on
908+
topics which intersect with their areas of focus.
909+
910+
The membership of this workgroup, its charter, and more specifics about its role
911+
in the project will be formalized separately.
885912

886913
### Distribution
887914

0 commit comments

Comments
 (0)