Canonicalization overhaul part two (#1390)

* Squash canonicalization overhaul part2 into single commit

* Cleanup and add comments

* dartfmt

* Update changelog, reduce memory usage for link checker

* dartfmt

* delete dartdoc.iml

* Review comments

* dartfmt

* whitespace in changelog

* Comment enhancements

* Correct changelog to 'orphaned file checker'

* update test package docs -- whitespace change in SDK

Author: jcollins-g

.idea/modules.xml

@@ -1,8 +1,62 @@
-## unreleased
-
-* added a new `--footer-text` command-line option, to allow adding additional
-  text in the package name and copyright section of the footer
-* Reduced stack depth by not recomputing findCanonicalLibraryFor (#1381)
+## 0.11.0
+
+* Many cleanups to dartdoc stdout/stderr, error messages, and warnings:
+  * Display fatal errors with 'fatal error' string to distinguish them from ordinary errors
+  * Upgrades to new Package.warn system.
+    * Fully integrated all scattered "warnings" (#1369) and added new ones for the link checker.
+    * Allow for setting which warnings are errors in the library.
+    * Change location output to something IntelliJ can understand and link to
+    * Display location output for all warnings including line number plus column, when available
+      from analyzer (still some bugs in our resolution). It still doesn't do code references quite
+      right but at least gets you to the neighborhood.
+    * Add a warn method to ModelElements so they can warn on themselves without help from the
+      Package.
+    * Warn correctly and squelch duplicates across doc inheritance and canonicalization almost
+      everywhere.
+    * Change --show-warnings to show all warnings, even those that might not be useful yet.
+  * Display a count of all warnings/errors after document generation.
+  * Make the progress counter tick slower.
+* Added a built-in link checker and orphaned file checker, and tied it into Package.warn so
+  that when debugging dartdoc we can breakpoint and discover what about that ModelElement
+  caused us to create the broken link. (#1380)
+* Fix bug where canonicalEnclosingElement could return a non-canonical Class.
+* Fix bug where findCanonicalModelElementFor could return a non-canonical Class.
+* Fix overriddenElement for Accessors to generate using enclosingCombo hint to ModelElement factory.
+* Fix fullyQualifiedNameWithoutLibrary when periods are part of the library name.
+* Add an allModelElements for Classes to support comment references.
+* Make allModelElements for Libraries work using Class.allModelElements recursively.
+* Squish some bugs related to duplicate logic for instantiating inherited class members.
+  * Enum and a few other places could still generate duplicate ModelElements for the
+    same thing.  This is now fixed.
+  * EnumField is now handled by ModelElement.from factory, fixing #1239.
+  * Added hints for EnumField and Accessors (index, enclosingCombo) to offload the buggy
+    logic for figuring this out from callers to ModelElement.from.
+* Fix broken link generation when a canonical class's defining library isn't canonical.
+* Partial rewrite of GetterSetterCombo and Fields/TopLevelVariable handling
+  * Link correctly to generic types for Fields/TopLevelVariables.
+  * Use right, left, and bidirectional arrows for read-only, write-only, and read-write
+    parameters.
+* Partial rewrite of comment reference system (#1391, #1285 partial)
+  * Handle gracefully a variety of things users try in the real world, like prefixing operators
+    with 'operator', embedded newlines in comment references, and cases that shouldn't be
+    considered at all (comment refs that are really array references in sample docs, etc).
+  * Handle canonicalization correctly for comment references: point to the right places and
+    only to canonical elements.
+  * In general, warnings related to comment references should be much more useful now. (#1343)
+    * Many fewer ambiguous doc reference warnings now and the ones that exist should be more
+      easily understandable and fixable with the new warning message.
+    * Understand references to parameters even though we don't do anything useful with them just yet
+    * Generics outside square brackets (#1250) are now warned with better context information that
+      takes newlines into account, but there are so many of them in complex packages like Flutter
+      that we still only show those with --show-warnings.
+  * Cache the traversal of allModelElements.
+  * Change handling of enum constant linking in codeRefs to work properly, though warnings about
+    that aren't right in some edge cases still.
+  * Only use analyzer resolving of commentRefs as a last resort since they don't take dartdoc
+    canonicalization into account.
+* Added a new `--footer-text` command-line option, to allow adding additional
+  text in the package name and copyright section of the footer.
+* Reduced stack depth by not recomputing findCanonicalLibraryFor. (#1381)
 * Workaround for (#1367) forces on enableAssertInitializer.
 * Work around analyzer-0.29 bug where embedded SDK uri's aren't properly
   reversed.

CHANGELOG.md

@@ -55,15 +55,15 @@ main(List<String> arguments) async {
 
   var readme = args['sdk-readme'];
   if (readme != null && !(new File(readme).existsSync())) {
-    stderr
-        .write(" Error: unable to locate the SDK description file at $readme.");
+    stderr.write(
+        " fatal error: unable to locate the SDK description file at $readme.");
     exit(1);
   }
 
   Directory inputDir = new Directory(args['input']);
   if (!inputDir.existsSync()) {
     stderr.write(
-        " Error: unable to locate the input directory at ${inputDir.path}.");
+        " fatal error: unable to locate the input directory at ${inputDir.path}.");
     exit(1);
   }
 
@@ -77,7 +77,8 @@ main(List<String> arguments) async {
       args['header'].map(_resolveTildePath).toList() as List<String>;
   for (String headerFilePath in headerFilePaths) {
     if (!new File(headerFilePath).existsSync()) {
-      stderr.write(" Error: unable to locate header file: ${headerFilePath}.");
+      stderr.write(
+          " fatal error: unable to locate header file: ${headerFilePath}.");
       exit(1);
     }
   }
@@ -86,7 +87,8 @@ main(List<String> arguments) async {
       args['footer'].map(_resolveTildePath).toList() as List<String>;
   for (String footerFilePath in footerFilePaths) {
     if (!new File(footerFilePath).existsSync()) {
-      stderr.write(" Error: unable to locate footer file: ${footerFilePath}.");
+      stderr.write(
+          " fatal error: unable to locate footer file: ${footerFilePath}.");
       exit(1);
     }
   }
@@ -96,7 +98,7 @@ main(List<String> arguments) async {
   for (String footerFilePath in footerTextFilePaths) {
     if (!new File(footerFilePath).existsSync()) {
       stderr.write(
-          " Error: unable to locate footer-text file: ${footerFilePath}.");
+          " fatal error: unable to locate footer-text file: ${footerFilePath}.");
       exit(1);
     }
   }
@@ -110,7 +112,7 @@ main(List<String> arguments) async {
   if (args.rest.isNotEmpty) {
     var unknownArgs = args.rest.join(' ');
     stderr.write(
-        'Error: detected unknown command-line argument(s): $unknownArgs');
+        ' fatal error: detected unknown command-line argument(s): $unknownArgs');
     _printUsageAndExit(parser, exitCode: 1);
   }
 
@@ -120,7 +122,7 @@ main(List<String> arguments) async {
 
   if (!packageMeta.isValid) {
     stderr.writeln(
-        'Unable to generate documentation: ${packageMeta.getInvalidReasons().first}.');
+        ' fatal error: Unable to generate documentation: ${packageMeta.getInvalidReasons().first}.');
     exit(1);
   }
 
@@ -165,10 +167,11 @@ main(List<String> arguments) async {
       autoIncludeDependencies: args['auto-include-dependencies'],
       categoryOrder: args['category-order']);
 
-  var dartdoc = new DartDoc(inputDir, excludeLibraries, sdkDir, generators,
+  DartDoc dartdoc = new DartDoc(inputDir, excludeLibraries, sdkDir, generators,
       outputDir, packageMeta, includeLibraries,
       includeExternals: includeExternals);
 
+  dartdoc.onCheckProgress.listen(_onProgress);
   Chain.capture(() async {
     DartDocResults results = await dartdoc.generateDocs();
     print('\nSuccess! Docs generated into ${results.outDir.absolute.path}');
@@ -218,7 +221,8 @@ ArgParser _createArgsParser() {
   parser.addOption('footer-text',
       allowMultiple: true,
       splitCommas: true,
-      help: 'paths to footer-text files (optional text next to the copyright).');
+      help:
+          'paths to footer-text files (optional text next to the copyright).');
   parser.addOption('exclude',
       allowMultiple: true, splitCommas: true, help: 'Library names to ignore.');
   parser.addOption('include',
@@ -264,8 +268,12 @@ ArgParser _createArgsParser() {
   return parser;
 }
 
-void _onProgress(File file) {
-  if (_showProgress) stdout.write('.');
+int _progressCounter = 0;
+void _onProgress(var file) {
+  if (_showProgress && _progressCounter % 5 == 0) {
+    stdout.write('.');
+  }
+  _progressCounter += 1;
 }
 
 /// Print help if we are passed the help option.