Automate release process #6912
Description
User Story
As a developer working on Sleeper, I want my changes to be released quickly and easily, so that I don't have to spend much time on releases, and I can release the project as often as I like.
As a user of Sleeper, I want to get important changes quickly, so that any security problems can be addressed immediately.
Description / Background
Our current release process includes a lot of steps where we need to wait for a build, or a CDK deployment, or GitHub Actions workflows, before we can move onto the next step. This means we either move to another task, which delays the next step if we don't get back at exactly the right time, or we wait and do nothing for some time. In practice we've found that when doing the release in tandem with other tasks, it takes all day to make a release, and we can't fully focus on anything else for that day.
If we release every 3 weeks, that's 17 releases in a year. If we spent 2 weeks to automate this down to half an hour for a release, then we'd break even on time spent after about a year.
That automation would also allow us to release much more often than every 3 weeks, so we could get important changes out very quickly.
We'd like to get the release process down to something that someone could do in 30 minutes without any need for context switches. This assumes that reasonable manual testing is done separately, to find problems that would then be addressed with automated tests, avoiding the need for any manual testing at the point of release.
Technical Notes / Implementation Details
Here are the parts of the release process that seem like we could do within half an hour without automating them, if the last release was quite recent:
- Look through the pull requests since the last release and write the changelog
- Check for any needed documentation updates, and anything half-implemented that we might not want to release
- Check for any backwards incompatible changes or anything that could potentially cause problems upgrading a Sleeper instance (refer to documentation of potential problems)
- Update the roadmap
- Check the system test output and add the performance test results to the documentation
If we did those in one pull request, we could set up automation so that when we merge that PR the rest of the release process would happen automatically. If something goes wrong, we can consider how to resume it after dealing with the problem.
It could be worth creating a record of our analysis during the checks above. That could be included in the same pull request.
TODO list remaining steps, plan automation