Skip to content

Commit 3b7acde

Browse files
nicolasstuckinicolasstucki
committed
Add Binary APIs SIP
1 parent 16d8674 commit 3b7acde

File tree

1 file changed

+211
-0
lines changed

1 file changed

+211
-0
lines changed

‎content/binary-api.md

Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
---
2+
layout: sip
3+
permalink: /sips/:title.html
4+
stage: pre-sip
5+
status: submitted
6+
title: SIP-52 - Binary APIs
7+
---
8+
9+
**By: Author Nicolas Stucki**
10+
11+
## History
12+
13+
| Date | Version |
14+
|---------------|--------------------|
15+
| Feb 27 2022 | Initial Draft |
16+
17+
## Summary
18+
19+
This proposal introduces the `@binaryAPI` and `@binaryAPIAccessor` annotations on term definitions. The purpose of binary APIs is to have publicly accessible definitions in generated bytecode for definitions that are private or protected.
20+
21+
22+
## Motivation
23+
24+
### Provide a sound way to refer to private members in inline definitions
25+
26+
Currently, the compiler automatically generates accessors for references to private members in inline definitions. This scheme interacts poorly with binary compatibility. It causes the following three unsoundness in the system:
27+
* Changing any definition from private to public is a binary incompatible change
28+
* Changing the implementation of an inline definition can be a binary incompatible change
29+
* Removing final from a class is a binary incompatible change
30+
31+
You can find more details in https://github.com/lampepfl/dotty/issues/16983
32+
33+
### Avoid duplication of inline accessors
34+
35+
Ideally, private definitions should have a maximum of one inline accessor, which is not the case now.
36+
When an inline method accesses a private/protected definition that is defined outside of its class, we generate an inline in the class of the inline method. This implies that accessors might be duplicated if a private/protected definition is accessed from different classes.
37+
38+
### Removing deprecated APIs
39+
40+
There is no precise mechanism to remove a deprecated method from a library without causing binary incompatibilities. We should have a straightforward way to indicate that a method is no longer publicly available but still available in the generated code for binary compatibility.
41+
42+
```diff
43+
- @deprecated(...) def myOldAPI: T = ...
44+
+ private[C] def myOldAPI: T = ...
45+
```
46+
47+
48+
## Proposed solution
49+
50+
### High-level overview
51+
52+
This proposal introduces 2 the `@binaryAPI` and `@binaryAPIAccessor` annotations, and changes adds a migration path to inline methods.
53+
54+
#### `@binaryAPI` annotation
55+
56+
A binary API is a definition that is annotated with `@binaryAPI` or overrides a definition annotated with `@binaryAPI`.
57+
This annotation can be placed on `def`, `val`, `lazy val`, `var`, `object`, and `given` definitions.
58+
A binary API will be publicly available in the bytecode.
59+
60+
This annotation cannot be used on `private`/`private[this]` definitions.
61+
62+
Removing this annotation from a non-public definition is a binary incompatible change.
63+
64+
Example:
65+
66+
~~~ scala
67+
class C {
68+
@binaryAPI private[C] def packagePrivateAPI: Int = ...
69+
@binaryAPI protected def protectedAPI: Int = ...
70+
@binaryAPI def publicAPI: Int = ... // warn: `@binaryAPI` has no effect on public definitions
71+
}
72+
~~~
73+
will generate the following bytecode signatures
74+
~~~ java
75+
public class C {
76+
public C();
77+
public int packagePrivateAPI();
78+
public int protectedAPI();
79+
public int publicAPI();
80+
}
81+
~~~
82+
83+
#### `@binaryAPIAccessor` annotation
84+
85+
A binary API with accessor is a definition that is annotated with `@binaryAPIAccessor`.
86+
This annotation can be placed on `def`, `val`, `lazy val`, `var`, `object`, and `given` definitions.
87+
The annotated definition will get a public accessor.
88+
89+
This can be used to access `private`/`private[this]` definitions within inline definitions.
90+
91+
Example:
92+
~~~ scala
93+
class C {
94+
@binaryAPIAccessor private def privateAPI: Int = ...
95+
@binaryAPIAccessor def publicAPI: Int = ...
96+
}
97+
~~~
98+
will generate the following bytecode signatures
99+
~~~ java
100+
public class C {
101+
public C();
102+
private int privateAPI();
103+
public int publicAPI();
104+
public final int C$$inline$privateAPI();
105+
public final int C$$inline$publicAPI();
106+
}
107+
~~~
108+
109+
Note that the change from `private[this]` to package private, protected or public is a binary compatible change.
110+
Removing this annotation is a binary incompatible change.
111+
112+
#### Binary API and inlining
113+
114+
If there is a reference to a binary API in an inline method we can use the definition without needing an inline accessor.
115+
116+
Example 3:
117+
~~~ scala
118+
class C {
119+
@binaryAPI protected def a: Int = ...
120+
protected def b: Int = ...
121+
inline def foo: Int = a + b
122+
}
123+
~~~
124+
before inlining the compiler will generate the accessors for inlined definitions
125+
~~~ scala
126+
class C {
127+
@binaryAPI protected def a: Int = ...
128+
protected def b: Int = ...
129+
final def C$inline$b: Int = ...
130+
inline def foo: Int = a + C$inline$b
131+
}
132+
~~~
133+
134+
Note that if the inlined member is `a` would be private, we would generate the accessor `C$inline$a`, which happens to be binary compatible with the automatically generated one.
135+
This is only a tiny mitigation of binary compatibility issues compared with all the different ways accessors can be generated.
136+
137+
### Specification
138+
139+
We must add `binaryAPI` and `binaryAPIAccessor` to the standard library.
140+
141+
```scala
142+
package scala.annotation
143+
144+
final class binaryAPI extends scala.annotation.StaticAnnotation
145+
final class binaryAPIAccessor extends scala.annotation.StaticAnnotation
146+
```
147+
148+
#### `@binaryAPI` annotation
149+
150+
* Only valid on `def`, `val`, `lazy val`, `var`, `object`, and `given`.
151+
* TASTy will contain references to non-public definitions that are out of scope but `@binaryAPI`. TASTy already allows those references.
152+
* Annotated definition will be public in the generated bytecode. Definitions should be made public as early as possible in the compiler phases, as this can remove the need to create other accessors. It should be done after we check the accessibility of references.
153+
154+
155+
#### `@binaryAPIAccessor` annotation
156+
157+
* Only valid on `def`, `val`, `lazy val`, `var`, `object`, and `given`.
158+
* An public accessor will be generated for the annotated definition. This accessor will be named `<fullClassName>$$inline$<definitionName>`.
159+
160+
#### Inline
161+
162+
* Inlining will not require the generation of an inline accessor for binary APIs.
163+
* Inlining will not require the generation of a new inline accessor, it will use the binary API accessors.
164+
* The user will be warned if a new inline accessor is automatically generated.
165+
The message will suggest `@binaryAPI` or `@binaryAPIAccessor` and how to fix potential incompatibilities.
166+
In a future version, these will become an error.
167+
168+
### Compatibility
169+
170+
The introduction of the `@binaryAPI` and `@binaryAPIAccessor` do not introduce any binary incompatibility.
171+
172+
Using references to `@binaryAPI` and `@binaryAPIAccessor` in inline code can cause binary incompatibilities. These incompatibilities are equivalent to the ones that can occur due to the unsoundness we want to fix. When migrating to binary APIs, the compiler will show the implementation of accessors that the users need to add to keep binary compatibility with pre-binaryAPI code.
173+
174+
A definition can be both `@binaryAPI` and `@binaryAPIAccessor`. This would be used to indicate that the definition used to be private, but now we want to publish it as public. The definition would become public, and the accessor would be generated for binary compatibility.
175+
176+
### Other concerns
177+
178+
* Tools that analyze inlined TASTy code might need to know about `@binaryAPI`. For example TASTy MiMa.
179+
180+
### Open questions
181+
182+
#### Question 1
183+
Should `@binaryAPIAccessor` accessors be named `<fullClassName>$$<definitionName>`? This encoding would match the names of `trait` accessor generated for private definition. We could use a single accessor instead of two. This would introduce an extra binary incompatibility with pre-binaryAPI code.
184+
185+
#### Question 2
186+
```scala
187+
class A:
188+
@binaryAPIAccessor protected def protectedDef: Int = ...
189+
class B extends A:
190+
override protected def protectedDef: Int = ...
191+
inline def inlinedDef: Int =
192+
// Should this use the accessor of generated for `A.protectedDef`? Or should we warn that `protectedDef` should be a `@binaryAPI`
193+
protectedDef
194+
```
195+
196+
## Alternatives
197+
198+
Having alternatives is not a strict requirement for a proposal, but having at least one with carefully exposed pros and cons gives much more weight to the proposal as a whole. -->
199+
200+
### Only add `@binaryAPI`
201+
This would simplify the system and the user interaction with this feature. The drawback is that we could not access `private[this]` definitions in inline code. Users would need to use `private[C]` instead, which could cause name clashes.
202+
203+
### Only add `@binaryAPIAccessor`
204+
This would simplify the system and the user interaction with this feature. The drawback is that we would add code size and runtime overhead to all uses of this feature. It would not solve the [Removing deprecated APIs](#removing-deprecated-apis) motivation.
205+
206+
## Related work
207+
208+
* Proof of concept: https://github.com/lampepfl/dotty/pull/16992
209+
* Initial discussions: https://github.com/lampepfl/dotty/issues/16983
210+
211+
<!-- ## FAQ -->

0 commit comments

Comments
 (0)