Merge branch 'main' into unroll

Author: lihaoyi

content/unroll-default-arguments.md

@@ -466,22 +466,84 @@ use them when the given `p: Product` has a smaller `productArity` than the curre
    does not matter. Trying to standardize this across all possible macros and all possible
    typeclasses is out of scope
 
-6. `@unroll` only supports `final` methods. `object` methods and constructors are naturally
+6. `@unroll` generates a quadratic amount of generated bytecode as more default parameters
+   are added: each forwarder has `O(num-params)` size, and there are `O(num-default-params)`
+   forwarders. We do not expect this to be a problem in practice, as the small size of the
+   generated forwarder methods means the constant factor is small, but one could imagine
+   the `O(n^2)` asymptotic complexity becoming a problem if a method accumulates hundreds of
+   default parameters over time. In such extreme scenarios, some kind of builder pattern
+   (such as those listed in [Major Alternatives](#major-alternatives)) may be preferable.
+
+7. `@unroll` only supports `final` methods. `object` methods and constructors are naturally
    final, but `class` or `trait` methods that are `@unroll`ed need to be explicitly marked `final`.
    It has proved difficult to implement the semantics of `@unroll` in the presence of downstream 
    overrides, `super`, etc. where the downstream overrides can be compiled against by different 
    versions of the upstream code. If we can come up with some implementation that works, we can
    lift this restriction later, but for now I have not managed to do so and so this restriction
    stays.
 
-7. `@unroll` generates a quadratic amount of generated bytecode as more default parameters
-   are added: each forwarder has `O(num-params)` size, and there are `O(num-default-params)`
-   forwarders. We do not expect this to be a problem in practice, as the small size of the
-   generated forwarder methods means the constant factor is small, but one could imagine
-   the `O(n^2)` asymptotic complexity becoming a problem if a method accumulates hundreds of
-   default parameters over time. In such extreme scenarios, some kind of builder pattern
-   (such as those listed in [Major Alternatives](#major-alternatives)) may be preferable.
-   
+### Challenges of Non-Final Methods and Overriding
+
+To elaborate a bit on the issues with non-final methods and overriding, consider the following 
+case with four classes, `Upstream`, `Downstream`, `Main1` and `Main2`, each of which is compiled
+against different versions of each other (hence the varying number of parameters for `foo`):
+
+```scala
+class Upstream{ // V2
+   def foo(s: String, n: Int = 1, @unroll b: Boolean = true, l: Long = 0) = s + n + b + l
+}
+```
+
+```scala
+class Downstream extends Upstream{ // compiled against Upstream V1
+   override def foo(s: String, n: Int = 1) = super.foo(s, n) + s + n
+}
+```
+
+```scala
+object Main1 { // compiled against Upstream V2
+   def main(args: Array[String]): Unit = {
+      new Downstream().foo("hello", 123, false, 456L)
+   }
+}
+```
+
+```scala
+object Main2 { // compiled against Upstream V1
+   def main(args: Array[String]): Unit = {
+      new Downstream().foo("hello", 123)
+   }
+}
+```
+
+
+The challenge here is: how do we make sure that `Main1` and `Main2`, who call
+`new Downstream().foo`, correctly pick up the version of `def foo` that is 
+provided by `Downstream`? 
+
+With the current implementation, the `override def foo` inside `Downstream` would only
+override one of `Upstream`'s synthetic forwarders, but would not override the actual
+primary implementation. As a result, we would see `Main1` calling the implementation
+of `foo` from `Upstream`, while `Main2` calls the implementation of `foo` from 
+`Downstream`. So even though both `Main1` and `Main2` have the same 
+`Upstream` and `Downstream` code on the classpath, they end up calling different
+implementations based on what they were compiled against.
+
+We cannot perform the method search and dispatch _within_ the `def foo` methods,
+because it depends on exactly _how_ `foo` is called: the `InvokeVirtual` call from
+`Main1` is meant to resolve to `Downstream#foo`, while the `InvokeSpecial` call
+from `Downstream#foo`'s `super.foo` is meant to resolve to `Upstream#foo`. But a
+method implementation cannot know how it was called, and thus it is impossible
+for `def foo` to forward the call to the right place.
+
+This issue only arises in the presence of version skew between `Upstream` and 
+`Downstream` code, but that scenario is precisely where `@unroll` is meant to
+provide benefits. Thus, unless we can find some solution, we cannot properly support
+virtual methods and overrides in `@unroll`.
+
+It may be possible to loosen this restriction to also allow abstract methods that
+are implemented only once by a final method. See the section about 
+[Abstract Methods](#abstract-methods) for details.
 
 ## Major Alternatives
 
@@ -605,7 +667,7 @@ are a large number of default parameters being added every version, as it would
 amount of generated code.
 
 
-### Abstract and Virtual Methods
+### Abstract Methods
 
 In [Limitations](#limitations), I mentioned that `@unroll` only supports `final` methods. 
 It is likely possible for abstract methods which are `@unrolled` to have concrete forwarder