@@ -477,9 +477,10 @@ be unstable and terminate unexpectedly), and another which launches and monitors
477
477
the first process. In this document, we'll refer to this second process as the
478
478
** harness** . A two-process architecture involving a supervisory harness helps
479
479
enable functionality such as relaunching after a crash, live progress reporting,
480
- or outputting results to a standard format. A modern testing library should
481
- provide any necessary APIs and runtime support to facilitate integration with a
482
- harness.
480
+ or outputting results to a standard format. Some cases effectively _ require_ a
481
+ harness, such as when launching tests on a remotely-connected device. A modern
482
+ testing library should provide any necessary APIs and runtime support to
483
+ facilitate integration with a harness.
483
484
484
485
## Today’s solution: XCTest
485
486
@@ -549,7 +550,8 @@ public macro Suite(
549
550
)
550
551
```
551
552
552
- Here are some usage examples:
553
+ Then, test authors may attach these macros to functions or types in a test
554
+ target. Here are some usage examples:
553
555
554
556
``` swift
555
557
import Testing
@@ -618,24 +620,29 @@ actor (such as `@MainActor`), in accordance with standard language and type
618
620
system rules. This allows tests to match the global actor of their subject and
619
621
reduce the need for suspension points.
620
622
621
- #### Runtime test discovery
623
+ #### Test discovery
622
624
623
- At runtime, tests will be represented as instances of ` Test ` and be retrieved
624
- using compiler-emitted metadata. The specifics of this mechanism have not yet
625
- been designed, but are planned to be covered in revisions to
626
- [ SE-0385] ( https://github.com/apple/swift-evolution/blob/main/proposals/0385-custom-reflection-metadata.md )
627
- or a new ` @section ` attribute (see
628
- [ pitch] ( https://forums.swift.org/t/pitch-low-level-linkage-control-attributes-used-and-section/65877 ) )
629
- which will allow annotating properties as having only constant data and placing
630
- their contents in a special section of the binary.
625
+ To faciliate test discovery, the attached macros above will eventually use a
626
+ feature such as ` @linkage ` , an attribute for controlling low-level symbol
627
+ linkage (see
628
+ [ pitch] ( https://forums.swift.org/t/pitch-2-low-level-linkage-control/69752 ) ).
629
+ It will allow placing variables and functions in a special section of the binary,
630
+ where they can be retrieved and at runtime or inspected statically at build time.
631
631
632
- However, before that support lands, the testing library will use a temporary
632
+ However, before that feature lands, the testing library will use a temporary
633
633
approach of iterating through types conforming to a known protocol and
634
634
gathering their tests by calling a static property. The attached macros will
635
635
emit code which generates the types to be discovered using this mechanism. Once
636
636
more permanent support lands, the attached macros will be adjusted to adopt it
637
637
instead.
638
638
639
+ Regardless of which technique the attached macros above use to facilitate test
640
+ discovery, the APIs called by their expanded code need not be considered public
641
+ or stable. In particular, code emitted by these macros may call
642
+ underscore-prefixed APIs declared in the ` Testing ` module, which are marked
643
+ ` public ` to facilitate use by these macros but are generally considered a
644
+ private implementation detail.
645
+
639
646
### Traits
640
647
641
648
As discussed earlier, it is important to support specifying traits for a test.
@@ -853,6 +860,19 @@ leverage
853
860
to capture exactly how these arrays differ and present that information to the
854
861
developer as part of the test output or in the IDE.
855
862
863
+ ### API and ABI stability
864
+
865
+ Once this project reaches its first major release, its API will be considered
866
+ stable and will follow the same general rules as the Swift standard library.
867
+ Source-breaking API changes will be considered a last resort and be preceded by
868
+ a generous deprecation period.
869
+
870
+ However, unlike the standard library, this project will generally not guarantee
871
+ ABI stability. Tests are typically only run during the development cycle and
872
+ aren't distributed to end users, so the runtime testing library will not be
873
+ included in OS distributions and thus its interfaces do not need to maintain
874
+ ABI stability.
875
+
856
876
### Project governance
857
877
858
878
For this testing solution to stand the test of time, it needs to be well
@@ -866,22 +886,29 @@ process inspired by, but separate from,
866
886
described in writing using a standard
867
887
[ proposal template] ( https://github.com/apple/swift-testing/blob/main/Documentation/Proposals/0000-proposal-template.md ) and discussed in the
868
888
[ swift-testing] ( https://forums.swift.org/c/related-projects/swift-testing/103 )
869
- category of the Swift Forums.
889
+ category of the Swift Forums. The process for pitching and submitting proposals
890
+ for review will be formalized separately and documented in the project repo.
870
891
871
892
A new group—tentatively named the _ Swift Testing Workgroup_ —will be formed to
872
- act as the primary governing body for this project. The responsibilities of
873
- members of this workgroup will include:
893
+ act as the primary governing body for this project. This group will be
894
+ considered a sub-group of the planned
895
+ [ Ecosystem Steering Group] ( https://www.swift.org/blog/evolving-swift-project-workgroups/ )
896
+ once it has been formed. The group will retroactively assume responsibility for
897
+ the [ swift-corelibs-xctest] ( https://github.com/apple/swift-corelibs-xctest )
898
+ project as well. The responsibilities of members of this workgroup will include:
874
899
875
900
* defining and approving roadmaps for the project;
876
901
* scheduling proposal reviews;
877
902
* guiding community discussion;
878
903
* making decisions about proposals; and
879
- * working with members of related workgroups, such as the Ecosystem, Platform,
880
- Server, or Documentation workgroups, on topics which intersect with their
881
- areas.
882
-
883
- The membership of this workgroup and specifics about its role in the project
884
- will be formalized separately.
904
+ * working with members of related workgroups, such as the
905
+ [ Platform] ( https://www.swift.org/platform-steering-group/ ) ,
906
+ [ Server] ( https://www.swift.org/sswg/ ) , or
907
+ [ Documentation] ( https://www.swift.org/documentation-workgroup/ ) workgroups, on
908
+ topics which intersect with their areas of focus.
909
+
910
+ The membership of this workgroup, its charter, and more specifics about its role
911
+ in the project will be formalized separately.
885
912
886
913
### Distribution
887
914
0 commit comments