[analyzer] Fix signature calculation when merging yaml files with "error" keys

**TL;DR**
In certain situations a new signature was created for every
analyzer start causing more work than needed to be done and filling up
the cache for no reason. The fix can cause the startup cost to improve
significantly.

**Details**
When merging analysis options yaml files (when one file includes
another), and both include data under "analyzer: errors:" we get
inconsistent signatures because of ordering issues. This causes "old
work" to be redone and more files to be written.

This CL fixes the issue by not introducing a semi-random order when
merging.

For the contexts I have open, the following files gives (gave) a bad
(inconsistent) signature:
pkg/analyzer/test/analysis_options.yaml
pkg/front_end/analysis_options.yaml
pkg/analysis_server/test/analysis_options.yaml

Looking at the time it takes from the first read/write until the last:
Before:
37.148419
37.273016
36.807384

Now:
14.164655
15.251373
15.108677

While the difference is easily visible, for good measure:

```
Difference at 95.0% confidence
        -22.2347 +/- 1.0223
        -59.9702% +/- 2.7573%
        (Student's t, pooled s = 0.45103)
```

And from the column "time" in top (note that I'm running from source so
that this includes the initial dart compile of the analyzer):
Before:
1:10.84
1:09.81
1:26.98

Now:
0:38.03
0:38.72
0:37.79

Again the difference is easily visible, but for good measure:

```
Difference at 95.0% confidence
        -37.6967 +/- 15.4529
        -49.6815% +/- 20.3658%
        (Student's t, pooled s = 6.81767)
```

If clearing the cache and opening (waiting for writes to complete),
closing and repeating, before this CL I got this with the contexts I
have open:

After 1st open:
```
$ du -chs ~/.dartServer/
186M    total
```

After the 2nd open:
```
$ du -chs ~/.dartServer/
212M    total
```

 => +26MB

After the 3rd open:
```
$ du -chs ~/.dartServer/
247M    total
$ find ~/.dartServer/ | wc -l
25713
```

 => +35MB

After the 4th open:
```
$ du -chs ~/.dartServer/
272M    total
$ find ~/.dartServer/ | wc -l
27769
```

 => +25MB; +2056 files

After the 5th open:
```
$ du -chs ~/.dartServer/
297M    total
$ find ~/.dartServer/ | wc -l
29825
```

 => +25MB; +2056 files

After the 6th open:
```
$ du -chs ~/.dartServer/
321M    total
$ find ~/.dartServer/ | wc -l
31881
```

 => +24MB; +2056 files

And we seem to have reached a point where we can guess the 7th if we want to.

And with the fix in this CL:

After 1st open:
```
$ du -chs ~/.dartServer/
186M    total
$ find ~/.dartServer/ | wc -l
20816
```

After 2nd:
```
$ du -chs ~/.dartServer/
186M    total
$ find ~/.dartServer/ | wc -l
20816
```

After the 3rd:
```
$ du -chs ~/.dartServer/
186M    total
$ find ~/.dartServer/ | wc -l
20816
```

And it seems that it's stable.

We might also want to sort the strings as ordering (probably?) doesn't
matter, but I'll leave that for another CL.

Change-Id: Id06ca84e8b063bb31a261753f56620443f341a2e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/410700
Commit-Queue: Jens Johansen <[email protected]>
Reviewed-by: Konstantin Shcheglov <[email protected]>

Author: jensjoha

pkg/analyzer/lib/src/util/yaml.dart

@@ -2,8 +2,6 @@
 // for details. All rights reserved. Use of this source code is governed by a
 // BSD-style license that can be found in the LICENSE file.
 
-import 'dart:collection';
-
 import 'package:source_span/source_span.dart';
 import 'package:yaml/src/event.dart';
 import 'package:yaml/yaml.dart';
@@ -33,8 +31,8 @@ class Merger {
   YamlNode merge(YamlNode o1, YamlNode o2) {
     // Handle promotion first.
     YamlMap listToMap(YamlList list) {
-      Map<YamlNode, YamlNode> map =
-          HashMap<YamlNode, YamlNode>(); // equals: _equals, hashCode: _hashCode
+      // We use the default linked hash map so the ordering is the same every time.
+      Map<YamlNode, YamlNode> map = {};
       ScalarEvent event =
           ScalarEvent(o1.span as FileSpan, 'true', ScalarStyle.PLAIN);
       for (var element in list.nodes) {
@@ -76,8 +74,8 @@ class Merger {
 
   /// Merge maps (recursively).
   YamlMap mergeMap(YamlMap m1, YamlMap m2) {
-    Map<YamlNode, YamlNode> merged =
-        HashMap<YamlNode, YamlNode>(); // equals: _equals, hashCode: _hashCode
+    // We use the default linked hash map so the ordering is the same every time.
+    Map<YamlNode, YamlNode> merged = {};
     m1.nodeMap.forEach((k, v) {
       merged[k] = v;
     });

pkg/analyzer/test/src/options/analysis_options_test.dart

@@ -8,6 +8,8 @@ import 'package:analyzer/file_system/memory_file_system.dart';
 import 'package:analyzer/src/analysis_options/analysis_options_provider.dart';
 import 'package:analyzer/src/dart/analysis/analysis_options.dart';
 import 'package:analyzer/src/error/codes.dart';
+import 'package:analyzer/src/file_system/file_system.dart';
+import 'package:analyzer/src/generated/source.dart';
 import 'package:test/test.dart';
 import 'package:test_reflective_loader/test_reflective_loader.dart';
 
@@ -30,8 +32,8 @@ class AnalysisOptionsTest {
   AnalysisOptionsImpl parseOptions(String content) =>
       AnalysisOptionsImpl.fromYaml(
         optionsMap: optionsProvider.getOptionsFromString(content),
-        file: resourceProvider.getFile(resourceProvider.convertPath(
-            '/project/analysis_options.yaml')),
+        file: resourceProvider.getFile(
+            resourceProvider.convertPath('/project/analysis_options.yaml')),
         resourceProvider: resourceProvider,
       );
 
@@ -414,4 +416,43 @@ code-style:
 ''');
     expect(analysisOptions.codeStyleOptions.useFormatter, true);
   }
+
+  test_signature_on_merge() {
+    var resourceProvider = MemoryResourceProvider();
+    var sourceFactory = SourceFactory([ResourceUriResolver(resourceProvider)]);
+    var optionsProvider = AnalysisOptionsProvider(sourceFactory);
+    var otherOptions = resourceProvider.getFile(
+        resourceProvider.convertPath("/project/analysis_options_helper.yaml"));
+    otherOptions.writeAsStringSync('''
+analyzer:
+  errors:
+    a: warning
+    b: ignore
+    c: ignore
+''');
+    var mainOptions = resourceProvider.getFile(
+        resourceProvider.convertPath("/project/analysis_options.yaml"));
+    mainOptions.writeAsStringSync('''
+include: analysis_options_helper.yaml
+analyzer:
+  errors:
+    d: ignore
+''');
+
+    var options = AnalysisOptionsImpl.fromYaml(
+      optionsMap: optionsProvider.getOptionsFromFile(mainOptions),
+      file: mainOptions,
+      resourceProvider: resourceProvider,
+    );
+    var sig1 = options.signature;
+    for (var i = 0; i < 100; i++) {
+      var options2 = AnalysisOptionsImpl.fromYaml(
+        optionsMap: optionsProvider.getOptionsFromFile(mainOptions),
+        file: mainOptions,
+        resourceProvider: resourceProvider,
+      );
+      var sig2 = options2.signature;
+      expect(sig1, sig2);
+    }
+  }
 }

pkg/analyzer/test/src/util/yaml_test.dart

@@ -94,6 +94,51 @@ main() {
         expect(merge(1, 'foo'), 'foo');
         expect(merge({'foo': 1}, 'foo'), 'foo');
       });
+
+      test('merged order directly from yaml', () {
+        YamlMap loadYaml(String content) {
+          return loadYamlNode(content) as YamlMap;
+        }
+
+        var aYaml = """foo:
+          a: bar""";
+        var bYaml = """foo:
+          b: baz""";
+
+        var merge1Value = mergeYamlMaps(loadYaml(bYaml), loadYaml(aYaml))
+            .valueAt("foo")
+            .toString();
+        for (int i = 0; i < 100; i++) {
+          var merge2Value = mergeYamlMaps(loadYaml(bYaml), loadYaml(aYaml))
+              .valueAt("foo")
+              .toString();
+          expect(merge2Value, merge1Value);
+        }
+      });
+
+      test('merged w/promotion order directly from yaml', () {
+        YamlMap loadYaml(String content) {
+          return loadYamlNode(content) as YamlMap;
+        }
+
+        var aYaml = """foo:
+          - a
+          - b
+          - c
+          - d""";
+        var bYaml = """foo:
+          b: true""";
+
+        var merge1Value = mergeYamlMaps(loadYaml(bYaml), loadYaml(aYaml))
+            .valueAt("foo")
+            .toString();
+        for (int i = 0; i < 100; i++) {
+          var merge2Value = mergeYamlMaps(loadYaml(bYaml), loadYaml(aYaml))
+              .valueAt("foo")
+              .toString();
+          expect(merge2Value, merge1Value);
+        }
+      });
     });
   });
 }
@@ -103,6 +148,9 @@ final Merger merger = Merger();
 Object merge(Object o1, Object o2) =>
     merger.merge(wrap(o1), wrap(o2)).valueOrThrow;
 
+YamlMap mergeYamlMaps(YamlMap defaults, YamlMap overrides) =>
+    Merger().mergeMap(defaults, overrides);
+
 YamlNode wrap(Object? value) {
   if (value is List) {
     var wrappedElements = value.map((e) => wrap(e)).toList();