Improve Changelogs (readability, discoverability, ...)

[coreyward]

Changelogs for Gatsby subpackages are missing critical, breaking updates while being version-bump-spammed. This combination makes it hard to tell what version a given feature landed in and fail to adequately communicate changes to users.

For example, the changelog for gatsby-plugin-emotion shows that there have not been any significant updates since v3.0.0 back in 2018. In reality, v5.0.0 included a significant breaking change that required all users of the package to change their code by updating to Emotion v11. That it was a breaking change was even called out in the PR title and commit title, but it never made it into the changelog at all:

Coupled with the noise of version bumps that have no changes (itself a scourge) from commits that have nearly 1k changes across hundreds of packages, it's very difficult to know when something has actually been updated, let alone what may have actually changed. That may be okay for patch versions, but when updating minor and major versions where I need to review it becomes a huge time suck.

1. Can someone look into fixing this and regenerating existing changelogs?
2. Would it be possible to look into the downstream impact of the version management process Gatsby is using to hopefully improve DX?

[vladar]

We use lerna changelog that is based on conventional commits. And it is not the best fit for us at the moment, given our release process and the way we use monorepo. For example emotion bump PR contained a prefix breaking(gatsby-plugin-emotion) which is not understood by lerna changelog. So it skipped it and didn't include it in the changelog. Re-generating won't help - it will skip any non-conventional commits.

There are also other examples when it didn't work that well for us. We are considering other tools to generate changelogs and manage monorepo. But in the meantime, we publish release notes with a high-level overview of changes in every release: https://www.gatsbyjs.com/docs/reference/release-notes/

Emotion bump was mentioned in the December release: https://www.gatsbyjs.com/docs/reference/release-notes/v2.28/

[coreyward]

I had no idea that Gatsby started publishing minor version release notes (looks like that only started as of November 2020), but in any case it's almost as difficult to track down changes as with the git folder history view on Github. It's nice to have this, and it would be great to be in a newsletter, but it's not an adequate replacement for package-level changelogs.

In the real world I am looking at outdated dependencies in my app, like [email protected], and seeing via VSCode tooltip or yarn outdated that there is a major version 6.x.x available, and heading to the package on Github to look for 1) upgrade instructions and 2) the changelog or release notes. When neither is available (typically only the case for smaller individual-maintained repos), I check the commit history. For literally every single other dependency I use this works. Gatsby has 1) no upgrade instructions, 2) a changelog that misses major events, and 3) a very difficult to follow commit history intertwined with every other one of the hundreds of packages under the Gatsby monorepo.

This is especially important as you look to keep issues to a minimum amidst tens of thousands of developers updating to v3 while others (👋 ) remain on v2 for at least another 12 months. Without exaggeration, updating the dozens of Gatsby subpackage dependencies to the latest available for Gatsby v2 will take me hours with the current setup.

[vladar]

That's fair. And we definitely want to improve the situation with changelogs. One of the best tools we've found is changesets but they don't support our release workflow at the moment. For anyone interested, you can follow changesets/changesets#548 for updates.

And let us know if you are aware of other good tools for changelog generation that work well with monorepos and branch-for-release model.

[coreyward]

Is there something wrong with asking for PRs to include an addition to the changelog, then automating the version listed in the changelog when it's published? E.g.:

Changelog for gatsby-plugin-x before:

# Changelog
## Unreleased
No changes.
## v1.0.0
- Feature A was added (link to PR)
- Bugfix (link to PR)

## v0.9.0
- Minor change to B (link to PR)

I open a PR to make gatsby-plugin-x automatically do your homework, including the following patch:

diff --git a/CHANGELOG.md b/CHANGELOG.md
 # Changelog
 ## Unreleased
-No changes.
+- Feature: automatically does your homework (link to PR)
 
 ## v1.0.0
 - Feature A was added (link to PR)
 - Bugfix (link to PR)
 
 ## v0.9.0
 - Minor change to B (link to PR)

My PR is accepted, and later a release is cut. The following automated change occurs:

diff --git a/package.json b/package.json
   "name": "gatsby-plugin-x",
-  "version": "1.0.0",
+  "version": "1.1.0",
   "license": "MIT",

diff --git a/CHANGELOG.md b/CHANGELOG.md
 # Changelog
 ## Unreleased
+No changes.
+
+## v1.1.0
+- Feature: automatically does your homework (link to PR)
 
 ## v1.0.0
 - Feature A was added (link to PR)
 - Bugfix (link to PR)
 
 ## v0.9.0
 - Minor change to B (link to PR)

This might not be off-the-shelf, but it seems straightforward enough and easy to add automated checks for in the PR.

[vladar]

The problem with this workflow is that it only works well when you always publish all releases from a single branch (so your releases are linear).

We publish actual stable releases from release branches and have backporting workflow from master to those release branches. In other words, our releases are a tree. The same change in master may actually end up in 3.x+1 (next minor) and 3.x.y (a patch in current minor line) and actually even 2.x.y (a patch in the previous major line) when backported.

So problems are:

1. A shared changelog file causes conflicts when cherry-picking/backporting. Currently, most of our cherry-picks are automated but when there is a shared changelog file - many of those will contain conflicts and require manual backporting with conflict-resolution.

2. Changelogs in master and release branches inevitably deviate. Master doesn't "know" about any stable releases in release branches. And we never merge from the release branches back to master (even if we did - merging from multiple release branches would be manual with conflict-resolution).

So this model is too simplistic and unfortunately doesn't work well with our release process. We will come up with a solution of course, but it's not that simple and requires some planning, tooling, and workflow changes/enforcement.

[vladar]

So, we've built our custom solution based on tooling from conventional-changelog.

All changelogs in the master branch are now updated. Some examples:

• gatsby
• gatsby-source-wordpress

Minor releases also link to corresponding Release Notes.

via #32886

[coreyward]

Awesome!!! Thanks for circling back. This is such a helpful improvemetn!