You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: proposals/NNNN-opt-in-reflection-metadata.md
+30-13Lines changed: 30 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,20 +16,25 @@ Swift-evolution thread: [Discussion thread topic for that proposal](https://foru
16
16
17
17
## Motivation
18
18
19
-
There are two kinds of Swift metadata emitted be the compiler:
19
+
There are two kinds of Swift metadata emitted by the compiler:
20
20
21
21
1. Core Metadata (type metadata record, nominal type descriptor, etc).
22
22
2. Reflection metadata (reflection metadata field descriptor).
23
23
24
-
Core metadata must constantly be emitted and may only be stripped if provenly not used. (This metadata isn't affected by this proposal)
25
-
Reflection metadata contains optional information about declarations' fields - their names and references to their types. This metadata isn't used by the language's runtime features, and the emission may be skipped if such types aren't passed to reflection-consuming APIs.
24
+
Core metadata must constantly be emitted and may only be stripped if provenly not used. (This metadata isn't affected by this proposal.)
25
+
Reflection metadata contains optional information about declarations' fields - their names and references to their types.
26
+
This metadata isn't used by the language's runtime features, and the emission may be skipped if such types aren't passed to reflection-consuming APIs.
26
27
27
-
APIs can use Reflection Metadata differently. Some like `print`, and `dump` will still work with disabled reflection, but the output will be limited. Others, like SwiftUI, rely on it and won't work correctly if the reflection metadata is missing.
28
+
APIs can use Reflection Metadata differently. Some like `print`, and `dump` will still work with disabled reflection, but the output will be limited.
29
+
Others, like SwiftUI, rely on it and won't work correctly if the reflection metadata is missing.
28
30
While the former can benefit as well, the main focus of this proposal is on the latter.
29
31
30
-
A developer can mistakenly turn off Reflection Metadata for a Swift module and won't be warned at compile-time if APIs that consume reflection are used by that module. An app with such a module won't behave as expected at runtime which may be challenging to notice and track down such bugs back to Reflection. For instance, SwiftUI implementation uses reflection metadata from user modules to trigger the re-rendering of the view hierarchy when a state has changed. If for some reason a user module was compiled with metadata generation disabled, changing the state won't trigger that behavior and will cause inconsistency between state and representation which will make such API less safe since it becomes a runtime issue rather than a compile-time one.
32
+
A developer can mistakenly turn off Reflection Metadata for a Swift module and won't be warned at compile-time if APIs that consume reflection are used by that module. An app with such a module won't behave as expected at runtime which may be challenging to notice and track down such bugs back to Reflection.
33
+
For instance, SwiftUI implementation uses reflection metadata from user modules to trigger the re-rendering of the view hierarchy when a state has changed.
34
+
If for some reason a user module was compiled with metadata generation disabled, changing the state won't trigger that behavior and will cause inconsistency between state and representation which will make such API less safe since it becomes a runtime issue rather than a compile-time one.
31
35
32
-
On the other hand, excessive Reflection metadata may be preserved in a binary even if not used, because there is currently no way to statically determine its usage. There was an attempt to limit the amount of unused reflection metadata by improving its stripability by the Dead Code Elimination LLVM pass, but in many cases, it’s still preserved in the binary because it’s referenced by Full Type Metadata which prevents Reflection Metadata from stripping. This unnecessarily increases the binary size and may affect privacy by storing more information about the code's semantics in a binary, which might be used for reverse-engineering.
36
+
On the other hand, excessive Reflection metadata may be preserved in a binary even if not used, because there is currently no way to statically determine its usage. There was an attempt to limit the amount of unused reflection metadata by improving its stripability by the Dead Code Elimination LLVM pass, but in many cases, it’s still preserved in the binary because it’s referenced by Full Type Metadata which prevents Reflection Metadata from stripping.
37
+
This unnecessarily increases the binary size and may simplify reverse-engineering.
33
38
34
39
Introducing a static compilation check can help to solve both of mentioned issues by adding to the language a way to express the requirement to have Reflection metadata at runtime.
35
40
@@ -38,7 +43,8 @@ Introducing a static compilation check can help to solve both of mentioned issue
38
43
39
44
Teaching the Type-checker and IRGen to ensure Reflection metadata is preserved in a binary if reflection-consuming APIs are used, will help to move the issue from runtime to compile time.
40
45
41
-
To achieve that, a new marker protocol `Reflectable` will be introduced. Firstly, APIs developers will gain an opportunity to express a dependency on Reflection Metadata through a generic requirement of their functions, which will make such APIs safer. Secondly, during IRGen, the compiler will be able to selectively emit Reflection symbols for the types that explicitly conform to the `Reflectable` protocol, which will reduce the overhead from reflection symbols for cases when reflection is emitted but not consumed.
46
+
To achieve that, a new marker protocol `Reflectable` will be introduced. Firstly, APIs developers will gain an opportunity to express a dependency on Reflection Metadata through a generic requirement of their functions, which will make such APIs safer.
47
+
Secondly, during IRGen, the compiler will be able to selectively emit Reflection symbols for the types that explicitly conform to the `Reflectable` protocol, which will reduce the overhead from reflection symbols for cases when reflection is emitted but not consumed.
Starting with Swift 6, we propose to enable Opt-In mode by default, to make the user experience consistent and safe. However, if full reflection isn't enabled with a new flag (`-enable-full-reflection-metadata`), the emission of reflection metadata will be skipped for all types that don't conform to the `Reflectable` protocol. This may cause changes in the behavior of the code that wasn't audited to conform to Reflectable and uses reflection-consuming APIs.
110
+
111
+
Starting with Swift 6, we propose to enable Opt-In mode by default, to make the user experience consistent and safe.
112
+
However, if full reflection isn't enabled with a new flag (`-enable-full-reflection-metadata`), the emission of reflection metadata will be skipped for all types that don't conform to the `Reflectable` protocol.
113
+
This may cause changes in the behavior of the code that wasn't audited to conform to Reflectable and uses reflection-consuming APIs.
105
114
106
115
For instance, stdlib's APIs like `dump`, `debugPrint`, `String(describing:)` will be returning limited output.
107
116
Library authors will have to prepare their APIs for Swift 6 and introduce generic requirements on `Reflectable` in their APIs.
@@ -110,14 +119,17 @@ We also propose to deprecate the compiler's options that can lead to missing ref
110
119
111
120
112
121
### Stdlib behavior changes
113
-
In Swift `Mirror(reflecting:)` is the only official way to access Reflection metadata, all other APIs are using it under the hood. We intentionally do not propose adding a Reflectable constraint on Mirror type, because it would impose restrictions on those developers who still don't want to require it. If the presence of reflection metadata is mandatory, the requirement on Reflectable protocol should be expressed in the signatures of calling functions.
122
+
123
+
In Swift `Mirror(reflecting:)` is the only official way to access Reflection metadata, all other APIs are using it under the hood.
124
+
We intentionally do not propose adding a Reflectable constraint on Mirror type, because it would impose restrictions on those developers who still don't want to require it.
125
+
If the presence of reflection metadata is mandatory, the requirement on Reflectable protocol should be expressed in the signatures of calling functions.
114
126
115
127
116
128
## Detailed design
117
129
118
130
Since Reflection metadata might be used by the debugger, we propose to always keep that metadata if full emission of debugging information isenabled (`-gdwarf-types` or `-g` flags).
119
131
120
-
**Changes in flags**
132
+
### Changes in flags
121
133
To handle behavior change between Swift pre-6 and 6, we can introduce a new upcoming feature, which will allow to enable Opt-In mode explicitly for pre-6 Swift with `-enable-upcoming-feature OptInReflection` and will set this mode by defaultin Swift 6.
122
134
123
135
A new flag `-enable-full-reflection-metadata` will also have to be introduced to allow developers to enable reflection in full if they desire in Swift 6 and later.
@@ -146,7 +158,9 @@ Casting might be a good way to improve the feature's ergonomics because currentl
146
158
147
159
To implement this feature, we propose to introduce a new runtime function `swift_reflectableCast`, and emit a call to it instead of `swift_dynamicCast`during IRGen if Reflectable is a target type.
148
160
149
-
Because of the fact that the compiler emits a call to that function at compile-time, all casts must be statically visible. All other cases like implicit conversion to `Reflectable` must be banned. This could be done at CSSimplify, when a new conversion constraint is introduced between a type variable and `Reflectable` type, the compiler will emit an error.
161
+
Because of the fact that the compiler emits a call to that function at compile-time, all casts must be statically visible.
162
+
All other cases like implicit conversion to `Reflectable` must be banned.
163
+
This could be done at CSSimplify, when a new conversion constraint is introduced between a type variable and `Reflectable` type, the compiler will emit an error.
150
164
151
165
```swift
152
166
func cast<T, U>(_ x: U) -> T {
@@ -163,7 +177,8 @@ Since reflectable casting will require a new runtime function, it should be gate
163
177
164
178
## Source compatibility
165
179
166
-
The change won’t break source compatibility in versions prior to Swift 6, because of the gating by the new flag. If as proposed, it’s enabled by defaultin Swift 6, the code with types that has not been audited to conform to the `Reflectable` protocol will fail to compile if used with APIs that consume the reflection metadata.
180
+
The change won’t break source compatibility in versions prior to Swift 6, because of the gating by the new flag.
181
+
If as proposed, it’s enabled by defaultin Swift 6, the code with types that has not been audited to conform to the `Reflectable` protocol will fail to compile if used with APIs that consume the reflection metadata.
167
182
168
183
169
184
## Effect on ABI stability
@@ -176,6 +191,8 @@ This proposal has no effect on API resilience.
176
191
177
192
## Alternatives considered
178
193
179
-
Dead Code Elimination and linker optimisations were also considered as a way to reduce the amount of present Reflection metadata in release builds. The optimiser could use a conformance to a `Reflectable` protocolas a hint about what reflection metadata should be preserved. However, turned out it was quite challenging to statically determine all usages of Reflection metadata even with hints.
194
+
Dead Code Elimination and linker optimisations were also considered as a way to reduce the amount of present Reflection metadata in release builds.
195
+
The optimiser could use a conformance to a `Reflectable` protocolas a hint about what reflection metadata should be preserved.
196
+
However, turned out it was quite challenging to statically determine all usages of Reflection metadata even with hints.
180
197
181
198
It was also considered to use an attribute `@reflectable` on nominal type declaration to express the requirement to have reflection metadata, however, a lot of logic had to be re-implemented outside of type-checker to ensure all guarantees are fulfilled.
0 commit comments