[_fe_analyzer_shared] Add shared logic to support null-shorting.

This change introduces a new mixin, `NullShortingMixin`, with a type
parameter `Guard` for the data structure used by the client to desugar
null-aware accesses. The mixin maintains a stack of these guards, and
provides methods that the client can use to manipulate the stack:

- `startNullShorting` adds an entry to the stack; it should be called
  when the client encounters the `?.` part of a null-aware expression.

It also provides two hooks that the client can override if desired:

- `handleNullShortingStep`, called whenever an entry is removed from
  the stack; this will let the CFE know when it should de-sugar a null
  short using a "let" expression.

- `handleNullShortingFinished`, called whenever a sequence of entries
  is removed from the stack; this will let the analyzer know when it
  should change the static type of an expression as a result of
  null-shorting.

Also, a new optional parameter, `continueNullShorting`, is added to
`TypeAnalyzer.analyzeExpression`. If this parameter is `false` (the
default value), then any null shorting that is started during analysis
of the expression (due to the client calling `startNullShorting`) will
be terminated before returning. If it is `true`, then null shorting
won't be terminated, so it will extend to the containing
expression. For expression types that are able to extend null shorting
that appears in their target subexpression (e.g., method calls and
property accesses), the `visit` or `analyze` method should pass
`false` for this parameter when making a recursive call to analyze the
target.

Finally, the `NullShortingMixin` has a getter `nullShortingDepth`,
that `TypeAnalyzer.analyzeExpression` uses to determine when null
shorting should be terminated, and a method `finishNullShorting`, that
actually does the work of terminating null shorting. In principle,
clients don't need to invoke these parts of the `NullShortingMixin`
API. However, since the CFE doesn't always use
`TypeAnalyzer.analyzeExpression` (favoring its own internal methods
`InferenceVisitorImpl.inferExpression` and
`InferenceVisitorImpl.inferNullAwareExpression`), the CFE will need to
use them.

The "mini_ast" tests of flow analysis formerly used a method called
`nullAwareAccess` to exercise the flow analysis effects of null-aware
constructs. This was hacky and confusing, and is now unnecessary,
since the shared infrastructure now fully supports null-shorting. So
this method has been removed and replaced by the ability to mark a
method invocation or property access as null-aware.

This change only builds the infrastructure for shared analysis of
null-shorting; the analyzer and CFE still handle null shorting on
their own. In follow-up CLs I will change the analyzer and CFE to make
use of the shared mechanism.

Change-Id: Ide25a915c4d06eab751b87c1c2745749d23a8114
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/399480
Reviewed-by: Johnni Winther <[email protected]>
Commit-Queue: Paul Berry <[email protected]>

Author: stereotype441

pkg/_fe_analyzer_shared/lib/src/flow_analysis/flow_analysis.dart

@@ -2,6 +2,9 @@
 // for details. All rights reserved. Use of this source code is governed by a
 // BSD-style license that can be found in the LICENSE file.
 
+/// @docImport 'package:_fe_analyzer_shared/src/type_inference/null_shorting.dart';
+library;
+
 import 'package:_fe_analyzer_shared/src/types/shared_type.dart';
 import 'package:meta/meta.dart';
 
@@ -188,7 +191,8 @@ class ExpressionPropertyTarget<Expression extends Object>
 /// analysis knows that the "is" expression is *not* an immediate child of the
 /// "if" statement, so therefore no type promotion should occur.
 abstract class FlowAnalysis<Node extends Object, Statement extends Node,
-    Expression extends Node, Variable extends Object, Type extends Object> {
+        Expression extends Node, Variable extends Object, Type extends Object>
+    implements FlowAnalysisNullShortingInterface<Expression, Type> {
   factory FlowAnalysis(
     FlowAnalysisOperations<Variable, Type> operations,
     AssignedVariables<Node, Variable> assignedVariables, {
@@ -733,26 +737,6 @@ abstract class FlowAnalysis<Node extends Object, Statement extends Node,
   /// expression.
   void nonNullAssert_end(Expression operand);
 
-  /// Call this method after visiting an expression using `?.`.
-  void nullAwareAccess_end();
-
-  /// Call this method after visiting a null-aware operator such as `?.`,
-  /// `?..`, `?.[`, or `?..[`.
-  ///
-  /// [target] should be the expression just before the null-aware operator, or
-  /// `null` if the null-aware access starts a cascade section.
-  ///
-  /// [targetType] should be the type of the expression just before the
-  /// null-aware operator, and should be non-null even if the null-aware access
-  /// starts a cascade section.
-  ///
-  /// Note that [nullAwareAccess_end] should be called after the conclusion
-  /// of any null-shorting that is caused by the `?.`.  So, for example, if the
-  /// code being analyzed is `x?.y?.z(x)`, [nullAwareAccess_rightBegin] should
-  /// be called once upon reaching each `?.`, but [nullAwareAccess_end] should
-  /// not be called until after processing the method call to `z(x)`.
-  void nullAwareAccess_rightBegin(Expression? target, Type targetType);
-
   /// Call this method after visiting the value of a null-aware map entry.
   void nullAwareMapEntry_end({required bool isKeyNullAware});
 
@@ -2115,6 +2099,34 @@ class FlowAnalysisDebug<Node extends Object, Statement extends Node,
   }
 }
 
+/// Flow analysis interface methods used by [NullShortingMixin].
+///
+/// These are separated from [FlowAnalysis] in order to isolate
+/// [NullShortingMixin] from the type parameters of [FlowAnalysis] that aren't
+/// relevant to it.
+abstract interface class FlowAnalysisNullShortingInterface<
+    Expression extends Object, Type extends Object> {
+  /// Call this method after visiting an expression using `?.`.
+  void nullAwareAccess_end();
+
+  /// Call this method after visiting a null-aware operator such as `?.`,
+  /// `?..`, `?.[`, or `?..[`.
+  ///
+  /// [target] should be the expression just before the null-aware operator, or
+  /// `null` if the null-aware access starts a cascade section.
+  ///
+  /// [targetType] should be the type of the expression just before the
+  /// null-aware operator, and should be non-null even if the null-aware access
+  /// starts a cascade section.
+  ///
+  /// Note that [nullAwareAccess_end] should be called after the conclusion
+  /// of any null-shorting that is caused by the `?.`.  So, for example, if the
+  /// code being analyzed is `x?.y?.z(x)`, [nullAwareAccess_rightBegin] should
+  /// be called once upon reaching each `?.`, but [nullAwareAccess_end] should
+  /// not be called until after processing the method call to `z(x)`.
+  void nullAwareAccess_rightBegin(Expression? target, Type targetType);
+}
+
 /// An instance of the [FlowModel] class represents the information gathered by
 /// flow analysis at a single point in the control flow of the function or
 /// method being analyzed.

pkg/_fe_analyzer_shared/lib/src/flow_analysis/flow_analysis_operations.dart

@@ -4,6 +4,7 @@
 
 /// @docImport 'flow_analysis.dart';
 /// @docImport '../field_promotability.dart';
+/// @docImport '../type_inference/type_analyzer_operations.dart';
 library;
 
 /// Callback API used by flow analysis to query and manipulate the client's
@@ -84,6 +85,16 @@ abstract interface class FlowAnalysisTypeOperations<Type extends Object> {
   /// Returns `true` if [type] is a reference to a type parameter.
   bool isTypeParameterType(Type type);
 
+  /// Computes the nullable form of [type], in other words the least upper bound
+  /// of [type] and `Null`.
+  ///
+  /// The concrete classes implementing [TypeAnalyzerOperations] should mix in
+  /// [TypeAnalyzerOperationsMixin] and implement
+  /// [TypeAnalyzerOperations.makeNullableInternal] to receive a concrete
+  /// implementation of [makeNullable] instead of implementing [makeNullable]
+  /// directly.
+  Type makeNullable(Type type);
+
   /// Returns the non-null promoted version of [type].
   ///
   /// Note that some types don't have a non-nullable version (e.g.

pkg/_fe_analyzer_shared/lib/src/type_inference/null_shorting.dart

@@ -0,0 +1,113 @@
+// Copyright (c) 2024, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+/// @docImport 'type_analyzer.dart';
+/// @docImport 'type_analyzer_operations.dart';
+library;
+
+import '../flow_analysis/flow_analysis.dart';
+import '../flow_analysis/flow_analysis_operations.dart';
+
+/// Null shorting logic to be shared between the analyzer and the CFE.
+///
+/// This mixin should be used by the same class that mixes in [TypeAnalyzer].
+///
+/// The type parameter [Guard] should be instantiated with the data structure
+/// used by the client to desugar null-aware accesses. (The analyzer can
+/// instantiate this with `Null`, since it doesn't do desugaring.)
+mixin NullShortingMixin<Guard, Expression extends Object, Type extends Object>
+    on TypeAnalysisNullShortingInterface<Expression, Type> {
+  /// Stack of [Guard] objects associated with null-shorting operations that
+  /// haven't been terminated yet.
+  final _guards = <Guard>[];
+
+  @override
+  int get nullShortingDepth => _guards.length;
+
+  @override
+  Type finishNullShorting(int targetDepth, Type inferredType) {
+    assert(targetDepth < nullShortingDepth);
+    inferredType = operations.makeNullable(inferredType);
+    do {
+      // End non-nullable promotion of the null-aware variable.
+      flow.nullAwareAccess_end();
+      handleNullShortingStep(_guards.removeLast(), inferredType);
+    } while (nullShortingDepth > targetDepth);
+    handleNullShortingFinished(inferredType);
+    return inferredType;
+  }
+
+  /// Hook called by [finishNullShorting] after terminating all the null
+  /// shorting that needs to be terminated for a given expression.
+  ///
+  /// [inferredType] is the (nullable) type of the final expression.
+  void handleNullShortingFinished(Type inferredType) {}
+
+  /// Hook called by [finishNullShorting] after terminating a single
+  /// null-shorting operation.
+  ///
+  /// [guard] is the value that was passed to [startNullShorting] when the
+  /// null-shorting operation was started.
+  ///
+  /// [inferredType] is the (nullable) type of the final expression.
+  void handleNullShortingStep(Guard guard, Type inferredType) {}
+
+  /// Starts null shorting for a null-aware expression that participates in
+  /// null-shorting.
+  ///
+  /// [target] should be the target of the null-aware operation (the expression
+  /// to the left of the `?.`), and [targetType] should be its static type.
+  ///
+  /// [guard] should be the data structure that will be used by the client to
+  /// desugar the null-aware access. It will be passed to
+  /// [handleNullShortingStep] when the null shorting for this particular
+  /// null-aware expression is terminated.
+  void startNullShorting(Guard guard, Expression target, Type targetType) {
+    // Ensure the initializer of [_nullAwareVariable] is promoted to
+    // non-nullable.
+    flow.nullAwareAccess_rightBegin(target, targetType);
+    _guards.add(guard);
+  }
+}
+
+abstract interface class TypeAnalysisNullShortingInterface<
+    Expression extends Object, Type extends Object> {
+  /// Returns the client's [FlowAnalysis] object.
+  FlowAnalysisNullShortingInterface<Expression, Type> get flow;
+
+  /// Returns the number of null-shorting operations that haven't been
+  /// terminated yet.
+  int get nullShortingDepth;
+
+  /// The [FlowAnalysisTypeOperations], used to access types and check
+  /// subtyping.
+  ///
+  /// Typically this will be an instance of [TypeAnalyzerOperations].
+  FlowAnalysisTypeOperations<Type> get operations;
+
+  /// Terminates one or more null-shorting operations that were previously
+  /// started using [NullShortingMixin.startNullShorting].
+  ///
+  /// This method should be called at the point where the null-shorting flow
+  /// control path rejoins with the main flow control path. For example, when
+  /// analyzing the expression `i?.toString()`, this method should be called
+  /// after analyzing the call to `toString()`.
+  ///
+  /// [targetDepth] should be a value previously returned by
+  /// [nullShortingDepth]. Null-shorting operations will be terminated until the
+  /// null-shorting depth returns to this value. Caller is required to pass in
+  /// a value that is strictly less than [nullShortingDepth] (that is, there
+  /// must be at least one null-shorting operation that needs to be terminated).
+  ///
+  /// [inferredType] should be the type of the expression that was just
+  /// analyzed, prior to termination of null shorting. For example, if the
+  /// expression that is being analyzed is `i?.toString()`, [inferredType]
+  /// should represent the non-nullable type `String`.
+  ///
+  /// The return value represents the type of the full expression, after
+  /// termination of null shorting. For example, if the expression that is being
+  /// analyzed is `i?.toString()`, the return value will represent the type
+  /// `String?`.
+  Type finishNullShorting(int targetDepth, Type inferredType);
+}

pkg/_fe_analyzer_shared/lib/src/type_inference/type_analysis_result.dart

@@ -65,28 +65,15 @@ class DeclaredVariablePatternResult<
 
 /// Container for the result of running type analysis on an expression.
 ///
-/// This class keeps track of a provisional type of the expression (prior to
-/// resolving null shorting) as well as the information necessary to resolve
-/// null shorting.
-abstract class ExpressionTypeAnalysisResult<
+/// This class keeps track of the type of the expression. Derived classes expose
+/// other results of type analysis that are specific to certain expression
+/// types.
+class ExpressionTypeAnalysisResult<
     TypeStructure extends SharedTypeStructure<TypeStructure>> {
-  /// Type of the expression before resolving null shorting.
-  ///
-  /// For example, if `this` is the result of analyzing `(... as int?)?.isEven`,
-  /// [provisionalType] will be `bool`, because the `isEven` getter returns
-  /// `bool`, and it is not yet known (until looking at the surrounding code)
-  /// whether there will be additional selectors after `isEven` that should act
-  /// on the `bool` type.
-  SharedTypeView<TypeStructure> get provisionalType;
-
-  /// Resolves any pending null shorting.  For example, if `this` is the result
-  /// of analyzing `(... as int?)?.isEven`, then calling [resolveShorting] will
-  /// cause the `?.` to be desugared (if code generation is occurring) and will
-  /// return the type `bool?`.
-  ///
-  /// TODO(paulberry): document what calls back to the client might be made by
-  /// invoking this method.
-  SharedTypeView<TypeStructure> resolveShorting();
+  /// The static type of the expression.
+  final SharedTypeView<TypeStructure> type;
+
+  ExpressionTypeAnalysisResult({required this.type});
 }
 
 /// Result for analyzing an if-case statement or element in
@@ -112,7 +99,7 @@ class IfCaseStatementResult<
 /// Container for the result of running type analysis on an integer literal.
 class IntTypeAnalysisResult<
         TypeStructure extends SharedTypeStructure<TypeStructure>>
-    extends SimpleTypeAnalysisResult<TypeStructure> {
+    extends ExpressionTypeAnalysisResult<TypeStructure> {
   /// Whether the integer literal was converted to a double.
   final bool convertedToDouble;
 
@@ -316,7 +303,7 @@ class ObjectPatternResult<
 /// Container for the result of running type analysis on a pattern assignment.
 class PatternAssignmentAnalysisResult<
         TypeStructure extends SharedTypeStructure<TypeStructure>>
-    extends SimpleTypeAnalysisResult<TypeStructure> {
+    extends ExpressionTypeAnalysisResult<TypeStructure> {
   /// The type schema of the pattern on the left hand size of the assignment.
   final SharedTypeSchemaView<TypeStructure> patternSchema;
 
@@ -422,28 +409,11 @@ class RelationalPatternResult<
       required super.matchedValueType});
 }
 
-/// Container for the result of running type analysis on an expression that does
-/// not contain any null shorting.
-class SimpleTypeAnalysisResult<
-        TypeStructure extends SharedTypeStructure<TypeStructure>>
-    implements ExpressionTypeAnalysisResult<TypeStructure> {
-  /// The static type of the expression.
-  final SharedTypeView<TypeStructure> type;
-
-  SimpleTypeAnalysisResult({required this.type});
-
-  @override
-  SharedTypeView<TypeStructure> get provisionalType => type;
-
-  @override
-  SharedTypeView<TypeStructure> resolveShorting() => type;
-}
-
 /// Result for analyzing a switch expression in
 /// [TypeAnalyzer.analyzeSwitchExpression].
 class SwitchExpressionResult<
     TypeStructure extends SharedTypeStructure<TypeStructure>,
-    Error> extends SimpleTypeAnalysisResult<TypeStructure> {
+    Error> extends ExpressionTypeAnalysisResult<TypeStructure> {
   /// Errors for non-bool guards.
   ///
   /// The key is the case index of the erroneous guard.

pkg/_fe_analyzer_shared/lib/src/type_inference/type_analyzer.dart

@@ -4,6 +4,7 @@
 
 import '../flow_analysis/flow_analysis.dart';
 import '../types/shared_type.dart';
+import 'null_shorting.dart';
 import 'type_analysis_result.dart';
 import 'type_analyzer_operations.dart';
 
@@ -262,25 +263,29 @@ class SwitchStatementMemberInfo<Node extends Object, Statement extends Node,
 /// of each entry in order to verify that when an entity is popped, it has the
 /// expected kind.
 mixin TypeAnalyzer<
-    TypeStructure extends SharedTypeStructure<TypeStructure>,
-    Node extends Object,
-    Statement extends Node,
-    Expression extends Node,
-    Variable extends Object,
-    Pattern extends Node,
-    Error,
-    TypeParameterStructure extends SharedTypeParameterStructure<TypeStructure>,
-    TypeDeclarationType extends Object,
-    TypeDeclaration extends Object> {
+        TypeStructure extends SharedTypeStructure<TypeStructure>,
+        Node extends Object,
+        Statement extends Node,
+        Expression extends Node,
+        Variable extends Object,
+        Pattern extends Node,
+        Error,
+        // Work around https://github.com/dart-lang/dart_style/issues/1568
+        // ignore: lines_longer_than_80_chars
+        TypeParameterStructure extends SharedTypeParameterStructure<TypeStructure>,
+        TypeDeclarationType extends Object,
+        TypeDeclaration extends Object>
+    implements
+        TypeAnalysisNullShortingInterface<Expression,
+            SharedTypeView<TypeStructure>> {
   TypeAnalyzerErrors<Node, Statement, Expression, Variable,
       SharedTypeView<TypeStructure>, Pattern, Error> get errors;
 
-  /// Returns the client's [FlowAnalysis] object.
+  @override
   FlowAnalysis<Node, Statement, Expression, Variable,
       SharedTypeView<TypeStructure>> get flow;
 
-  /// The [TypeAnalyzerOperations], used to access types, check subtyping, and
-  /// query variable types.
+  @override
   TypeAnalyzerOperations<TypeStructure, Variable, TypeParameterStructure,
       TypeDeclarationType, TypeDeclaration> get operations;
 
@@ -553,20 +558,36 @@ mixin TypeAnalyzer<
   /// Analyzes an expression.  [node] is the expression to analyze, and
   /// [schema] is the type schema which should be used for type inference.
   ///
+  /// If [continueNullShorting] is `false` (the default), then any null shorting
+  /// that starts inside [node] will be terminated, and the returned type will
+  /// be nullable, to reflect the fact that null-aware expressions might
+  /// evaluate to `null`.
+  ///
+  /// If [continueNullShorting] is `true`, then null shorting that starts inside
+  /// [node] will be allowed to continue into the containing expression.
+  ///
   /// Stack effect: pushes (Expression).
   SharedTypeView<TypeStructure> analyzeExpression(
-      Expression node, SharedTypeSchemaView<TypeStructure> schema) {
+      Expression node, SharedTypeSchemaView<TypeStructure> schema,
+      {bool continueNullShorting = false}) {
+    int? nullShortingTargetDepth;
+    if (!continueNullShorting) nullShortingTargetDepth = nullShortingDepth;
     // Stack: ()
     if (schema is SharedDynamicTypeSchemaView<TypeStructure>) {
       schema = operations.unknownType;
     }
     ExpressionTypeAnalysisResult<TypeStructure> result =
         dispatchExpression(node, schema);
     // Stack: (Expression)
-    if (operations.isNever(result.provisionalType)) {
+    if (operations.isNever(result.type)) {
       flow.handleExit();
     }
-    return result.resolveShorting();
+    SharedTypeView<TypeStructure> type = result.type;
+    if (nullShortingTargetDepth != null &&
+        nullShortingDepth > nullShortingTargetDepth) {
+      type = finishNullShorting(nullShortingTargetDepth, type);
+    }
+    return type;
   }
 
   /// Analyzes a collection element of the form