integrationservertest: Overhaul upgrade-config.yaml

[ericywl]

Problem

The current upgrade-config.yaml, especially the lazy-rollover-with-exceptions configuration is not very intuitive (see thread). We should overhaul it to have a more intuitive and flexible way of configuring expected lazy rollovers.

Features

1. We should be able to configure whether we expect lazy rollovers between minors.
2. We should be able to configure whether we expect lazy rollovers between patches of the same minor.

[ericywl]

Below are some thoughts that I have, may not make it into the final solution.

1. Fully expect that every single version upgrade (between majors, minors and patches) will have lazy rollover. Then, the configuration will only list down those with no lazy rollover (see comment).
2. Have a clearer from and to so that it's easier to reason the configuration.
3. Always use full form of <MAJOR>.<MINOR>.<PATCH>, using x to represent any e.g. 8.19.x means any patch in 8.19. (8.19 as value is interpreted in YAML as a float, not as a string)
4. Add ability to express range of patches e.g. 8.17.{1-7} means any patches between 8.17.1 and 8.17.7.

Combining the above, we will have something like:

lazy-rollover-exceptions:
  ## Same Minors
  # No lazy rollover from any patches between 8.17.1 and 8.17.7 to any other patch.
  # This means that upgrading from e.g. 8.17.2 to 8.17.8 is expected to have rollover.
  - from: 8.17.{1-7}
    to: 8.17.{1-7}
  # No lazy rollover from any 8.18 patch to 8.18 patch
  - from: 8.18.x
    to: 8.18.x
  # No lazy rollover from any 8.19 patch to 8.19 patch
  - from: 8.19.x
    to: 8.19.x
  # No lazy rollover from any 9.0 patch to 9.0 patch
  - from: 9.0.x
    to: 9.0.x
  # No lazy rollover from any 9.1 patch to 9.1 patch
  - from: 9.1.x
    to: 9.1.x
  ## Different Minors
  # No lazy rollover from any 8.18 patch to 9.0 patch
  - from: 8.18.x
    to: 9.0.x

[endorama]

I needed some advanced upgrade case for the recent tests of the apm-data bug, you can see how the upgrade-config.yaml looked here: 8efcfac

(I also added support for patch versions, should either be extracted and merged on main or included later when we implement this proposal. I would decide which to do based on when we plan to tackle this change)

[endorama]

I like the presented idea. There is a big trade off here in either being too verbose or too implicit. Both cases present issues but I think being too implicit is the one with additional risks and costs, so I'd lean a bit on being more verbose. The current proposal looks a good trade off to me. The current proposal will also force us to update the configuration when new versions are out, but I think it limits the effort by allowing range of versions and patch catch-all.

[endorama]

@ericywl a suggestion: I think it would be nice to have a YAML comment field to each entry, to be used in place of YAML comments. It would then be printed when running the test (in addition to what I added here). I found this information very helpful when troubleshooting a test failed for a rollover mismatch and I think having also a human comment on the why would be beneficial.

[ericywl]

Added PR for this issue. There are some differences from the previous comment.

# Lazy rollover is by default expected for every single upgrade (be it between
# majors, minors or patches). This configuration lists down the exceptions to
# this expectation, in the form of from-version and to-version. During the
# upgrade tests, when the upgrade e.g. 8.19.4 -> 9.0.1, matches with any of the
# defined version pairs, there will be no lazy rollover expected in that upgrade.
#
#
# There are 2 ways of defining the exception version:
#
#
#  1. Using singular version with wildcards:
#      - Match specific patch version               : <MAJOR>.<MINOR>.<PATCH>
#      - Match any versions in minor                : <MAJOR>.<MINOR>.*
#      - Match any versions in major                : <MAJOR>.*.*
#      - Match any versions in major (special rule) : <MAJOR>.x.*
#
# Note that the minor position is given a special wildcard character (x).
# This is because of a special rule when the pattern is used in both from and
# to. For example, the following uses x in both from and to, and is used to
# indicate that there is no lazy rollover from any 9.* patch to any other
# 9.* patch iff the minor is the same.
# ```
#  - from: "9.x.*"
#    to:   "9.x.*"
# ```
#
# Both major and patch positions are not given the special wildcard (x) because
# it is unlikely to be useful in our use cases. If due to some mistake, the
# special wildcard (x) is used only in one of the strings, it is treated as a
# normal wildcard (*), so the following example would be effectively the same
# as 9.*.*.
# ```
#  - from: "9.x.*"
#    to:   "9.1.2"
# ```
#
#
#  2. Using ranges:
#      - Square brackets for inclusive           e.g. [<VERSION_A> - <VERSION_B>]
#      - Round brackets for exclusive            e.g. (<VERSION_A> - <VERSION_B>)
#
# When using ranges, the versions defined in the range cannot contain any
# wildcards (*, x) and must be a full valid version with major, minor, patch.
# The brackets can be mixed to have inclusive start, exclusive end i.e. [A-B)
# or vice versa i.e. (A-B]. Here is an example:
# ```
#  - from: "(1.2.2 - 4.5.6]"
#    to:   "[2.0.0 - 3.0.0)"
# ```
#
# Based on the example above, if the upgrade test from-version is in range
# e.g. 3.4.5 and to-version is also in range e.g. 2.6.7, the upgrade will be
# expected to not have lazy rollover.
lazy-rollover-exceptions:
  # No lazy rollover from any 8.17 patch between [8.17.0, 8.17.7] to any other 8.17 patch between.
  # This means that upgrading from e.g. 8.17.2 to 8.17.8 is expected to have rollover.
  - from: "[8.17.0 - 8.17.7]"
    to:   "[8.17.0 - 8.17.7]"
  # No lazy rollover from any 8.17 patch after 8.17.8 to any other 8.17 patch after 8.17.8.
  - from: "[8.17.8 - 8.18.0)"
    to:   "[8.17.8 - 8.18.0)"
  # No lazy rollover from any 8.18 patch to any other 8.18 patch.
  - from: "8.18.*"
    to:   "8.18.*"
  # No lazy rollover from any 8.19 patch to any other 8.19 patch.
  - from: "8.19.*"
    to:   "8.19.*"
  # No lazy rollover from any 9.* patch to any other 9.* patch iff minor is the same.
  - from: "9.x.*"
    to:   "9.x.*"
  # No lazy rollover from any 9.2 patch to any other 9.3 patch.
  - from: "9.2.*"
    to:   "9.3.*"