Finish wildcards proposal.

Author: munificent

working/wildcards/feature-specification.md

@@ -1,10 +1,15 @@
-TODO: link
+# Wildcards
+
+Author: Bob Nystrom
+
+Status: In-progress
+
+Version 1.0
 
 Pattern matching brings a new way to declare variables. Inside patterns, any
 variable whose name is `_` is considered a "wildcard". It behaves like a
-variable syntactically, but it doesn't actually create a variable with that
-name. That means you can use `_` multiple times in a pattern without a name
-collision, and you can't use it as an expression to access the matched value.
+variable syntactically, but doesn't actually create a variable with that name.
+That means you can use `_` multiple times in a pattern without a name collision.
 
 This proposal extends that non-binding behavior to other variables in Dart
 programs named `_`.
@@ -19,7 +24,7 @@ The name `_` is a well-established convention in Dart for a couple of uses:
     var hundredPenguins = List.generate(100, (_) => 'penguin');
     ```
 
-*   **Used method override parameters.** When a method overrides an inherited
+*   **Unused method override parameters.** When a method overrides an inherited
     method, it must accept the same required parameters. If it doesn't use one,
     sometimes it gets named `_`. (In practice, it's more common to just use the
     same name as the inherited parameter.)
@@ -47,9 +52,9 @@ The name `_` is a well-established convention in Dart for a couple of uses:
 Since `_` *is* a binding name in Dart today, you can only use this name once in
 any given scope. If you want to ignore multiple callback parameters, the
 convention is to use a series of underscores for each name: `_`, `__`, `___`,
-etc. In practice, these are all "wildcard" names.
+etc.
 
-Making `_` actually non-binding solves a few problems:
+Making `_` non-binding outside of patterns solves a few problems:
 
 *   It makes other variable declarations consistent with how variable
     declarations in patterns behave.
@@ -59,45 +64,258 @@ Making `_` actually non-binding solves a few problems:
 
 *   It prevents code from using a variable that wasn't intended to be used.
 
-At the same time, we want to *not* prevent the idiom of using `_` as the name
-of the canonical private constructor. This means making `_` non-binding for
-*variables*, but not for *members*. Getters and setters make the line between
-those somewhat fuzzy.
+At the same time, we want to support the idiom of using `_` as the name of the
+canonical private constructor, so we don't want *all* declarations named `_`
+to be non-binding.
+
+It seems that the natural line to draw is between declarations that are local
+to a block scope versus those that are top-level declarations or members where
+library privacy comes into play.
 
 ## Proposal
 
-TODO: should silence unused variable warnings
+A *local declaration* is any of:
 
-## Breaking change
+*   Function parameters. This includes top-level functions, local functions,
+    function expressions ("lambdas"), instance methods, static methods,
+    constructors, etc. It includes all parameter kinds: simple, field formals,
+    and function-typed formals, etc.:
+
+    ```dart
+    Foo(_, this._, super._, void _(), {_}) {}
 
-This is a breaking change to code that declares variables named `_` and actually
-uses them.
+    list.where((_) => true);
+    ```
+
+*   Local variable declaration statement variables.
+
+    ```dart
+    main() {
+      var _ = 1;
+      int _ = 2;
+    }
+    ```
 
-TODO: stats
+*   For loop variable declarations.
 
-## Semantics
+    ```dart
+    for (_ = 0;;) {}
+    for (_ in list) {}
+    ```
+
+*   Catch clause parameters.
+
+    ```dart
+    try {
+      throw '!';
+    } catch (_) {
+      print('oops');
+    }
+    ```
+
+*   Generic type and generic function type parameters.
+
+    ```dart
+    class T<_> {}
+
+    takeGenericCallback(<_>() => true);
+    ```
+
+A local declaration whose name is `_` does not bind anything to that name. This
+means you can have multiple local declarations named `_` in the same namespace
+without a collision error. The initializer, if there is one, is still executed,
+but the value is not accessible.
+
+Other declarations: top-level variables, top-level function names, type names,
+member names, etc. are unchanged. They can be named `_` as they are today.
 
 We do not change how identifier *expressions* behave. Members can be named `_`
 and you can access them from inside the class where the member is declared
-without any leading `this.`. If the member is a method tear-off, it's possible
-for a `_` expression to be meaningful:
+without any leading `this.`:
+
+```dart
+class C {
+  var _ = 'bound';
+
+  test() {
+    print(_); // Prints "bounnd".
+  }
+}
+```
+
+Likewise with a top-level named `_`:
+
+```dart
+var _ = 'ok';
+
+main() {
+  print(_); // Prints "ok".
+}
+```
+
+It's just that a local declaration named `_` doesn't bind that name to anything.
+
+There are a few interesting corners and refinements:
+
+### Assignment
+
+The behavior of assignment expressions is unchanged. In a pattern assignment,
+`_` is always a wildcard. This is valid:
+
+```dart
+int a;
+(_, a) = (1, 2);
+```
+
+But in a non-pattern assignment, `_` is treated as a normal identifier. If it
+resolves to something assignable (which now must mean a member or top-level
+declaration), the assignment is valid. Otherwise it's an error:
+
+```dart
+main() {
+  _ = 1; // Error.
+}
+
+class C {
+  var _;
+
+  test() {
+    _ = 2; // OK.
+  }
+}
+```
+
+### Wildcards do not shadow
+
+Here is an interesting example:
 
 ```dart
 class C {
-  static C _() => C();
+  var _ = 'field';
 
-  static C test() {
-    var tearOff = _; // <-- Valid.
-    return tearOff();
+  test() {
+    var _ = 'local';
+
+    _ = 'assign';
   }
 }
 ```
 
-todo: should we allow instance or static fields? no: will be inconsistent if
-later allow patterns there
+This program is valid and assigns to the *field*, not the local. This code is
+quite confusing. In practice, we expect reasonable users will not name fields
+`_` and thus not run into this problem.
+
+### Initializing formals
+
+An initializing formal named `_` does still initialize a field named `_` (and
+you can still have a field with that name):
+
+```dart
+class C {
+  var _;
+
+  C(this._); // OK.
+}
+```
+
+But no *parameter* with that name is bound, which means `_` can't be accessed
+inside the initializer list. In the body is fine, since that refers to the
+field, not the parameter:
+
+```dart
+class C {
+  var _;
+  var other;
+
+  C(this._)
+    : other = _ { // <-- Error. No "_" in scope.
+    print(_); // OK. Prints the field.
+  }
+}
+```
+
+Even though the parameters no longer collide, it is still an error to have two
+initializing formals named `_`:
+
+```dart
+class C {
+  var _;
+  C(this._, this._); // Error.
+}
+```
+
+### Unused variable warnings
+
+Dart tools currently warn if you have an unused local declaration. If the
+declaration is named `_`, it now *can't* be used, so the tools should stop
+showing the warning for that name.
+
+### Multiple underscore lint and quick fix
+
+Since `_` is binding today, when you need more than one, the convention to avoid
+collisions is to use a series of underscores. With this proposal, that
+convention is no longer needed.
+
+It would be helpful if the linter would suggest that variables whose name is a
+series of more than one `_` be renamed to just `_` now that it won't collide.
+Likewise, a quick fix could perform that change automatically.
+
+## Breaking change
+
+This is a breaking change to code with parameters or other local declarations
+named `_` that actually uses them. Fortunately, code doing that is rare.
+
+I wrote [a script][scrape] to examine every identifier whose name consists only
+of underscores in a large corpus of pub packages, Flutter widgets, and open
+source Flutter applications (18,695,158 lines in 102,015 files):
+
+[scrape]: https://gist.github.com/munificent/e03728874aae4d16a9760f207aecb16c
+
+```
+-- Declaration (33074 total) --
+  21786 ( 65.870%): Parameter name
+   8093 ( 24.469%): Constructor name
+   2913 (  8.808%): Catch parameter
+    236 (  0.714%): Local variable
+     31 (  0.094%): Loop variable
+      3 (  0.009%): Extension name
+      2 (  0.006%): For loop variable
+      2 (  0.006%): Static field
+      2 (  0.006%): Type parameter
+      2 (  0.006%): Instance field
+      2 (  0.006%): Method name
+      1 (  0.003%): Enum value name
+      1 (  0.003%): Function name
+
+-- Use (17356 total) --
+  13289 ( 76.567%): Private constructor invocation                   =========
+   2640 ( 15.211%): Private superclass constructor invocation        ==
+    807 (  4.650%): Identifier expression                            =
+    522 (  3.008%): Redirection to private constructor               =
+     48 (  0.277%): Factory constructor redirecting to private name  =
+     46 (  0.265%): Assignment target                                =
+      3 (  0.017%): Field initializer                                =
+      1 (  0.006%): Type annotation                                  =
+```
+
+As expected, most declarations named `_` (or longer) are parameters and
+constructor names, the two main idioms. Catch clause parameters are fairly
+common too. All other declarations named `_` are extremely rare, less than 1% of
+the total when put together.
+
+When code *uses* a declaration named `_` (or longer), over 95% are private
+constructor calls, as expected. But there are a relatively small number of uses
+where a variable named `_` is being accessed. My simple syntactic analysis can't
+distinguish between whether those are accessing local variables (in which case
+they will break) or instance or top-level members (in which case they're OK).
+From skimming some of the examples, it does look like some users sometimes name
+lambda parameters `_` and then use the parameter in the body.
 
-todo: should we allow getters and setters? no: inconsistent with vars.
+Fixing these is easy: just rename the variable and its uses to something other
+than `_`. Since this proposal doesn't affect the behavior of externally visible
+members, these fixes can always be done locally without changing a library's
+public API.
 
-todo: should we allow methods? yes: constructors and static methods mostly
-interchangeable. allowing static methods without instance methods would be
-weird.
+However, this *is* a breaking change. If this ships in the same version as
+pattern matching, we can gate it behind a language version and only break code
+when it upgrades to that version.