New changelog parser + typed representation of the changelog structure.

[isoos]

• This only a small step to get improved markdown processing (see Improve markdown parsing #3331).
• The new parser is designed to parse a wide range of changelog formats, and in itself it is not a recommendation for any particular format, a recommendation for that is outside of the scope of this PR.
• While the date parsing is improved a bit, it can be better in the future (see We're markdown rendering of changelog - some headers with dates, others not #5127).
• The parsed output is integrated only in the markdown processing for now, but the typed structure enables further features to be implemented, e.g. RSS feed should contain the relevant snippet extracted from CHANGELOG #5183, Suggestion: Please add an RSS/Atom feed for each package's changelog #5267, Feed for new versions. #5532.
• Small consistency-related changes in the output on the changelog tab:
  • [] around the version are removed
  • the v before the version is removed (fixes Show versions in changelog consistently #7395)
  • the date format is yyyy-MM-dd, it may be yyyy/MM/dd in the source
  • the date is always separated by - from the version (optional in the source)

[isoos]

Note: I've downloaded, extracted and tested it on the latest versions of all packages from pub.dev. With random sampling and some manual checks, I think most of the files are nicely formatted and working as expected, and maybe only a few percent had issues like repeated versions (on the same or different header level - which is now handled in the parser) or inconsistent header structure (e.g. major and minor versions with different header level).

At this point I think the new parser will be an improvement, handles more edge cases, and for the rest we can add support for them case-by-case.

[sigurdm]

I worry that by doing recursive calls here we are subject to crash by stack-overflow...

[isoos]

Yeah, it is probably a not really necessary item here, switching to always do node.text.trim() (it is using a tree visitor inside, but it doesn't need such joins, it appends the text to a buffer).

[sigurdm]

Does the input get sanitized in this pipeline, or does that happen later?

[isoos]

Sanitization is the last step, anything we do here, will get sanitized at the end.

Note: since the parseMarkdown is no longer used (helper method in the test, and post-markdown-rendering step in the lib/ code), removing it.