chore: Upgrade release system to the toys-release gem

[dazuma]

This is a long-awaited upgrade to the release system. It was rewritten pretty much from the ground up to be more testable and maintainable. That said, the OTel-Ruby repos exercise the system more than any other customer, so I'll be around to monitor the next releases closely to ensure things go smoothly.

Some key changes:

• The release system is now distributed as a gem, toys-release. So loading it is less sketchy than depending on a tag in another repository as we've been doing before. This change currently pins the toys-release gem version at 0.4.0 (see the file .toys/release.rb).
• Most of the interfaces are mostly the same, but backward compatibility was not a goal here. I updated the release configs (.toys/.data/releases.yml) and the tool invocations in the GitHub Actions workflows accordingly.
• I set the GitHub Actions workflows to run on Ruby 4.0 and Toys 0.19.1. In particular, toys-release requires at least Toys 0.19.0. Ruby 4.0 isn't really necessary, but toys-release uses it by default. If people are more comfortable using 3.4, I can change it to that version.

Some of the config has also been updated to work with the toys-release gem. Specifically:

• The main_branch key defaults to main, so main_branch: main is redundant and was removed.
• The commit_lint key was dropped. The commit linter in general was removed from toys-release, as it was not really maintained and no one was using it. And it has been disabled in our repos anyway. There are better dedicated conventional commit linter tools available if we want to use one.
• I removed the trailing slash hack for the opentelemetry-helpers-sql and opentelemetry-instrumentation-http gem configurations. That bug has been fixed, and the hack is no longer necessary.
• I removed version_rb_path settings for gems whose configurations did not need it.

There is quite a bit of new functionality and fixed issues in toys-release, that we might choose to take advantage of (again, as a second step after this initial migration). For example:

• There is now a more flexible way to customize which paths signal that a gem has changed, supporting shared files that are used by multiple gems. This should make hacks like chore: Fix Ruby Version Management #1765 unnecessary.
• There is now a way to customize how conventional commit tags (including scopes) map to releases and semver, which Ariel requested in Allow configurable list of conventional commits dazuma/toys#215.
• Toys-release is no longer limited to releasing from the main branch. You should be able to release from any branch, by choosing it when triggering the "open release request" workflow.

Finally, I (@dazuma) expect to be able to engage a lot more now on OTel-Ruby. I quit my job at Google a few months ago, in part because it was stifling me and I was not able to make space for open source and community work. Now that I'm out, I've had a chance to, among other things, finish this long-overdue update to the release system. I also should be able to attend the weekly SIGs more often now. So anyone working on this repo, if you have questions, issues, or feature requests regarding the release system and process, please let me know.

I'm keeping this PR in draft until I've had the chance to discuss this update with the other maintainers, and to do some more testing.

Note there is a corresponding pull request in the main opentelemetry-ruby repository: open-telemetry/opentelemetry-ruby#2006

One final note: I notice there is a release-please workflow present in this repository as well. release-please is an alternative release mechanism that could replace toys-release. (In fact, toys-release actually borrows a few of its ideas, such as release pull requests and conventional commit-based changelogs, from release-please.) I have not touched the existing release-please workflow, pending discussion with other maintainers about which direction to go. However, I do have some inside knowledge regarding release-please, so please do not make that decision without discussing with me.

[arielvalentin]

@dazuma happy to have you back!

I started experimenting with release-please since we use it at GitHub to manage our internal gems and it has a wider user base and audience than toys does. It also a few more features that I wanted to introduce in the config repo. E.g. the draft PR accumulating changes. I feel like this is makes tracking pending changes better than our existing flow of requesting a release and reviewing the PR.

I didn't know of a simple way to experiment with it that pod not require that we commit changes to directly to this repo. I did not get very far other adding the configuration and running the action a few times. I wasn't ready to commit to anything so I disabled the action and left the configs in place.

What is your preferred medium to have follow up discussions around toys vs release-please? Would GitHub discussions work? Slack? And issue? The SIG?

[dazuma]

@arielvalentin Let's discuss it in the SIG next week. Sorry I missed yesterday; I was trying to make it, but stuff happened.

Edit: Oh, and accumulating subsequent changes into an existing release PR would definitely be a feature I could add to toys-release.

[arielvalentin]

Mostly improve by trust here. Since the interface is very similar and it's using envars in some cases, I don't see anything surprising.

One thought I had was, why not add toys as a bundler dependency in the top level Gemfile? That would allow us to be able to install toys and run commands locally for testing in a more familiar way.

The actions would then execute installs using Bundler and could potentially cache gemfiles between runs.

This is all food for thought and not really necessary.

I'll also ask for your help to remove any release-please related code and configs in a follow up PR since we are moving forward with new versions of toys.

[arielvalentin]

Should we add toys to the project gemfile and use bundle install along with bundle-cache: true?

[dazuma]

@arielvalentin

One thought I had was, why not add toys as a bundler dependency in the top level Gemfile? That would allow us to be able to install toys and run commands locally for testing in a more familiar way.

The actions would then execute installs using Bundler and could potentially cache gemfiles between runs.

In general, the best practice for Toys is not to include the toys gem itself in a Gemfile, and not to run bundle exec toys ..., but instead to let Toys itself handle bundler integration. The reasons for this are described in detail in the Toys user's guide. It also could lead to conflicts if someone locally wants to run a different version of Toys than is pinned in the Gemfile.

I guess in this case, we could work around most of this by creating a custom gemfile used only for the Toys install in these workflows (pointing to it by setting $BUNDLE_GEMFILE during the setup-ruby step). So we could do that and possibly get a modest performance bump from the bundle cache. Releases aren't that common, so I'm not sure it would be worth the trouble, but we could try it out.

I'll also ask for your help to remove any release-please related code and configs in a follow up PR since we are moving forward with new versions of toys.

Yeah, will do.