specify the annotation introspection API and constant evaluation API (#3408)

Updates to annotation introspection based on recent discussions, and in anticipation of changing the rules around shadowing.

The primary concrete change here is that we only allow access to identifiers/constructors from separate strongly connected components. This works around any issues with the constants changing between phases.

I decided not to try and restrict the API to code objects that came from annotations, so this does enable general purpose const evaluation, but it doesn't seem any more problematic to me than allowing the evaluation of const expressions from annotations. Please correct me if I am wrong :).

Author: jakemac53

working/macros/aspect-macros.md

@@ -58,7 +58,7 @@ identifier for the resulting declaration.
 ### Applying an Aspect Macro
 
 Aspect Macros can only be applied by other macros (including aspect macros),
-while the macro is running. They do this through the following api:
+while the macro is running. They do this through the following API:
 
 ```dart
 Future<Identifier> applyAspect(AspectMacro macro, Declaration declaration);
@@ -110,7 +110,7 @@ macro class FromJson extends ClassDeclarationsAspectMacro {
 - The serializable aspect is enforced by not permitting Aspect Macros to have
   any fields, and they only have a single const constructor with no arguments.
   - **TODO**: Should we make this constructor explicit?
-  - **TODO**: Should the api only take the Type of the aspect instead of an
+  - **TODO**: Should the API only take the Type of the aspect instead of an
     instance?
 
 ### Ordering of Aspect Macros

working/macros/example/benchmark/src/shared.dart

@@ -1,4 +1,4 @@
-// There is no public API exposed yet, the in progress api lives here.
+// There is no public API exposed yet, the in progress API lives here.
 import 'package:_fe_analyzer_shared/src/macros/api.dart';
 import 'package:_fe_analyzer_shared/src/macros/executor/introspection_impls.dart';
 import 'package:_fe_analyzer_shared/src/macros/executor/remote_instance.dart';

working/macros/feature-specification.md

@@ -87,7 +87,7 @@ There are two things you can do with an `OmittedTypeAnnotation`:
       file as the macro annotation, so they can always do this.
   - When the final augmentation library is created, the actual type that was
     inferred will be used (or `dynamic` if no type was inferred).
-- Explicitly ask to infer the type of it through the builder apis (only
+- Explicitly ask to infer the type of it through the builder APIs (only
   available in phase 3).
   - We don't allow augmentations of existing declarations to contribute to
     inference, so in phase 3 type inference can be performed.
@@ -658,9 +658,9 @@ constructors are invoked, and their limitations.
       types in the user code instantiating the macro are not necessarily present
       in the macros own transitive imports.
 
-*Note: The Macro API is still being designed, and lives [here][api].*
+*Note: The Macro API is still being designed, and lives [here][API].*
 
-[api]: https://github.com/dart-lang/sdk/blob/main/pkg/_fe_analyzer_shared/lib/src/macros/api.dart
+[API]: https://github.com/dart-lang/sdk/blob/main/pkg/_fe_analyzer_shared/lib/src/macros/api.dart
 
 ### Writing a Macro
 
@@ -716,7 +716,7 @@ return results. For instance when generating a constructor, a macro will likely
 just iterate over the fields and create a parameter for each.
 
 We need generated augmentations to be identical on all platforms for all the
-same inputs, so we need to have a defined ordering when introspection apis are
+same inputs, so we need to have a defined ordering when introspection APIs are
 returning lists of declarations.
 
 Therefore, whenever an implementation is returning a list of declarations, they
@@ -736,41 +736,59 @@ to introspect over non-macro metadata annotations applied to declarations.
 For example, a `@JsonSerialization()` class macro might want to look for an
 `@unseralized` annotation on fields to exclude them from serialization.
 
-**TODO**: The following subsections read more like a design discussion that a
-proposal. Figure out what we want to do here and rewrite (#1930).
+Some macros may need to evaluate the real values of metadata arguments, while
+others may only need the ability to emit that same code back into the program.
 
 #### The annotation introspection API
 
-We could try to give users access to an actual instance of the annotation, or
-we could give something more like the [DartObject][] class from the analyzer.
+All declarations which can be annotated will have an
+`Iterable<MetadataAnnotation> get metadata` getter.
 
-[DartObject]: https://pub.dev/documentation/analyzer/latest/dart_constant_value/DartObject-class.html
+All `MetadataAnnotation` objects have a `Code get code` getter, which gives
+access to the annotation as a `Code` object.
+
+In addition, there will be two subtypes of `MetadataAnnotation`:
+
+- `IdentifierMetadataAnnotation`: A simple const identifier, has a single
+  `Identifier get identifier` getter.
+- `ConstructorMetadataAnnotation`: A const constructor invocation. This will
+  have the following getters:
+  - `Identifier get type`
+  - `Identifier get constructor`
+  - `Arguments get arguments`
+    - The `Arguments` class will provide access to the positional and named
+      arguments as separate `Code` objects.
 
-Since annotations may contain references to types or identifiers that the macro
-does not import, we choose to expose a more abstract API (similar to
-[DartObject][]).
+For any macro which only wants to emit code from annotations back into the
+program, these `Code` objects are sufficient.
 
-**TODO**: Define the exact API.
+For a macro which wants to access the actual _value_ of a given argument or
+the metadata annotation as a whole, they can evaluate `Code` instances as
+constants (see next section).
 
-#### Annotations that require macro expansion
+### Constant evaluation
 
-This could happen if the annotation class has macros applied to it, or if
-some argument(s) to the annotation constructor use macros.
+Macros may want the ability to evaluate constant expressions, in particular
+those found as arguments to metadata annotations.
 
-Because macros are not allowed to generate code that shadows an identifier
-in the same library, we know that if an annotation class or any arguments to it
-could be resolved, then we can assume that resolution is correct.
+We expose this ability through the `DartObject evaluate(Code code)` API, which
+is available in all phases, with the following restrictions:
 
-This allows us to provide an API for macro authors to attempt to evaluate an
-annotation in _any phase_. The API may fail (if it requires more macro
-expansion to be done), but that is not expected to be a common situation. In
-the case where it does fail, users should typically be able to move some of
-their code to a separate library (which they import). Then things from that
-library can safely be used in annotations in the current library, and evaluated
-by macros.
+- No identifier in `code` may refer to a constant which refers to any
+  system environment variable, Dart define, or other configuration which is not
+  otherwise visible to macros.
+- All identifiers in `code` must be defined outside of the current
+  [strongly connected component][] (that is, the strongly connected component
+  which triggered the current macro expansion).
 
-Evaluation must fail if there are any macros left to be expanded on the
-annotation class or any arguments to the annotation constructor.
+The `DartObject` API is an abstract representation of an object, which can
+represent types which are not visible to the macro itself. It will closely
+mirror the [same API in the analyzer][DartObject].
+
+The call to `evaluate` will throw a `ConstantEvaluationException` if the
+evaluation fails due to a violation of one of the restrictions above.
+
+[DartObject]: https://pub.dev/documentation/analyzer/latest/dart_constant_value/DartObject-class.html
 
 #### Are macro applications introspectable?
 
@@ -1088,7 +1106,7 @@ a Dart program containing macro applications:
 
 Starting at the entrypoint library, traverse all imports, exports, and
 augmentation imports to collect the full graph of libraries to be compiled.
-Calculate the [strongly connected components][] of this graph. Each component is
+Calculate the [strongly connected component][]s of this graph. Each component is
 a library cycle, and the edges between them determine how the cycles depend on
 each other. Sort the library cycles in topological order based on the connected
 component graph.
@@ -1101,7 +1119,7 @@ library cycle, it is guaranteed that all macros used by the cycle have already
 been compiled. Also, any types or other declarations used by that cycle have
 either already been compiled, or are defined in that cycle.
 
-[strongly connected components]: https://en.wikipedia.org/wiki/Strongly_connected_component
+[strongly connected component]: https://en.wikipedia.org/wiki/Strongly_connected_component
 
 #### 2. Compile each cycle
 
@@ -1560,7 +1578,7 @@ resources outside of `lib`, which has both benefits and drawbacks.
 
 **TODO**: Evaluate APIs for listing files and directories.
 
-**TODO**: Consider adding `RandomAccessResource` api.
+**TODO**: Consider adding `RandomAccessResource` API.
 
 The specific API is as follows, and would only be available at compile time:
 

working/macros/motivation.md

@@ -313,7 +313,7 @@ stated more concretely as:
 4. We propose to consider designing a quotation syntax so that macros that
    generate Dart code can generate code in a readable fashion. This also means
    code generators only depend on language *syntax* for code generation and not
-   an imperative api, which will better stand the test of time.
+   an imperative API, which will better stand the test of time.
 
 Most of this is as-yet-undesigned, but the key piece is that **we think macros
 should be written in normal imperative Dart code which is executed at compile
@@ -417,7 +417,7 @@ you should:
   * Possibly provide an opt out for this, if feasible/desirable.
 * Be able to trace errors that flow through generated code, as well as navigate
   back to the line in the macro implementation that produced the code.
-* Be able to auto-complete macro generated apis.
+* Be able to auto-complete macro generated APIs.
 
 ### Performance