Lock metadata requirements in the lockfile

[byroot]

Reopening of #4455 because it's still a major issue when trying to implement nightly ruby-head CI for an application.

---

Describe the problem as clearly as you can

Currently, if someone is using a lockfile generated with ruby version A, and then uses that lockfile with ruby version B, it can happen that the variants for gems in the lockfile have different metadata requirements between each other, resulting in the Gemfile not being instalable on ruby version B and causing an error.

Previous bundler versions also had this problem, but since platform specific variants were not locked in the lockfile, but resolved at install time, the issue was easier to fix in that case. We only needed to also consider metadata requirements when doing this install time resolution step. This was fixed by #4449.

To fix this issue for newer bundlers that lock the specific platform under the PLATFORMS sections and platform specific variants under each GEM section, we need to also include metadata requirements there, so that bundler is able to detect that the lockfile is not satisfied by the current ruby version and a re-resolve is triggered instead of bailing out.
Post steps to reproduce the problem

Create this Gemfile:

source "https://rubygems.org"

gem "nokogori"

Run bundle lock on ruby 3.0, which generates the following Gemfile.lock file:

GEM
  remote: https://rubygems.org/
  specs:
    nokogiri (1.11.2-x86_64-linux)
      racc (~> 1.4)
    racc (1.5.2)

PLATFORMS
  x86_64-linux

DEPENDENCIES
  nokogiri

BUNDLED WITH
   2.3.0.dev

Then switch to a dev build of ruby-head, and run bundle install.

Which command did you run?

bundle install after setting up the previous situation.

What were you expecting to happen?

A successful installation of the RUBY variant nokogiri-1.11.2, which supports ruby 3.1.0.dev, and the lockfile re-generated to include this variant.

What actually happened?

$ bundle install
Fetching gem metadata from https://rubygems.org/.......
nokogiri-1.11.2-x86_64-linux requires ruby version < 3.1.dev, >= 2.5, which is incompatible with the current version, ruby 3.1.0p-1

A proposal to fix this is to add new versions of the GEM sections that also lock metadata requirements, and make bundler generate both versions by default, so that the lockfile is still usable by other bundler's that don't understand the new format. Like this:

GEM
  remote: https://rubygems.org/
  specs:
    nokogiri (1.11.2-x86_64-darwin)
      racc (~> 1.4)
    racc (1.5.2)

GEMv2
  remote: https://rubygems.org/
  specs:
    nokogiri (1.11.2-x86_64-darwin)
      racc (~> 1.4)
      ruby:  (>= 2.5, < 3.1.dev)
    racc (1.5.2)

PLATFORMS
  x86_64-darwin

DEPENDENCIES
  nokogiri

BUNDLED WITH
   2.3.0.dev

Then bundler should be able to use this information to detect that the current ruby can't install that lockfile and trigger a re-resolve.

Note that this backwards compatibility tricks might not be necessary if we introduce "bundler version auto installing and switching" since by doing that we would guarante that a lockfile generated by bundler version "x.y.z" will be installed by bundler version "x.y.z".

Reticketed from #4282.