NE-2097: Add logic to upgrade OSSM Operator #1247
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
openshift-ci-robot
added
the
jira/valid-reference
Contributor
|
@alebedev87: This pull request references NE-2097 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
Contributor
|
Skipping CI for Draft Pull Request. |
openshift-ci
bot
added
the
do-not-merge/work-in-progress
Contributor
Author
|
/test all |
alebedev87
force-pushed
the
gwapi-upgrade-ossm-next-ip
branch
from
0ede81e to
5804199
Compare
Contributor
|
@alebedev87: This pull request references NE-2097 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
alebedev87
commented
Jul 29, 2025
Comment on lines
+781
to
+922
| // Note that the progressing state remains "true" if the desired version is | ||
| // beyond the latest available in the upgrade graph. | ||
| // There is no reliable way of knowing that the end of the upgrade graph | ||
| // is reached and no next version is available. None of OLM resources | ||
| // can provide such information. | ||
| progressing = true |
Contributor
Author
There was a problem hiding this comment.
@joelanford : If you can help me here. I'm having hard times trying to find the right way to programmatically determine whether the OLM operator I manage reached its final version available in the upgrade graph. This PR modifies the logic of the cluster ingress operator to approve every next installplan (subscription.status.currentCSV) which is created by OLM until the desired version is reached. Before it was not possible to do upgrades to version greater than N+1. The desired version is coming from the code and is recorded in subscription.spec.startingCSV.
I didn't find any status in OLM's Subscription, InstallPlan or ClusterServiceVersion which would point me to the fact that there are no more versions in the upgrade graph. I tried many variants with the subscription's current, installed versions and CSV statuses. The problem is that the situation when the latest version is installed is not different from when any intermediate CSV is installed. For example, when the cluster ingress operator wants the OSSM operator (==managed one) to be in version v3.0.5 while the latest available is v3.0.3 and the currently installed is v3.0.0 the upgrade flow ends in this state: Desired: v3.0.5, Installed: v3.0.3, Current: v3.0.3, InstalledCVS: Succeeded. Which is not different from the state when an intermediate v3.0.1 is installed: Desired: v3.0.5, Installed: v3.0.1, Current: v3.0.1, InstalledCVS: Succeeded.
I start to think that this may be impossible and we should make sure we never set the desired version which doesn't exist.
Member
There was a problem hiding this comment.
My understanding is atLatestVersion := sub.Status.InstalledCSV == sub.Status.CurrentCSV
Member
There was a problem hiding this comment.
Problem is that, depending on the upgrade graph, you may skip right past your desired version with no way of installing it other than uninstalling and reinstalling using startingCSV.
Contributor
Author
There was a problem hiding this comment.
Problem is that, depending on the upgrade graph, you may skip right past your desired version with no way of installing it other than uninstalling and reinstalling using startingCSV.
Addressed in 31eeb0d.
Contributor
|
@alebedev87: This pull request references NE-2097 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
Contributor
Author
|
/test all |
Contributor
|
@alebedev87: This pull request references NE-2097 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-merge-robot
added
the
needs-rebase
alebedev87
force-pushed
the
gwapi-upgrade-ossm-next-ip
branch
from
922b728 to
35e6699
Compare
openshift-merge-robot
removed
the
needs-rebase
Contributor
Author
|
/test all |
openshift-merge-robot
added
the
needs-rebase
alebedev87
force-pushed
the
gwapi-upgrade-ossm-next-ip
branch
from
35e6699 to
8bb6826
Compare
openshift-merge-robot
removed
the
needs-rebase
alebedev87
force-pushed
the
gwapi-upgrade-ossm-next-ip
branch
from
8bb6826 to
eb88926
Compare
openshift-ci
bot
removed
the
do-not-merge/work-in-progress
Contributor
|
@alebedev87: This pull request references NE-2097 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
alebedev87
force-pushed
the
gwapi-upgrade-ossm-next-ip
branch
from
eb88926 to
b0bfd36
Compare
Contributor
Author
|
Manual run of the e2e test: |
alebedev87
force-pushed
the
gwapi-upgrade-ossm-next-ip
branch
2 times, most recently
from
3d079e1 to
a7fa4c5
Compare
Contributor
|
@alebedev87: This pull request references NE-2097 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
Contributor
|
/assign @Miciah |
alebedev87
force-pushed
the
gwapi-upgrade-ossm-next-ip
branch
from
255163d to
0b5020f
Compare
Contributor
Author
|
Increased the timeout to 2 minutes for |
Contributor
Author
|
/retest |
openshift-merge-robot
added
the
needs-rebase
…ps upgrade graph The implemented logic supports automatic approval of the next install plan from the upgrade graph, progressing sequentially toward the desired version. If the next available install plan version is semantically greater than the desired version, the upgrade stops.
This commit introduces a sematic version check for the CSV of next installplan. If the next installplan has a version which is sematically past the desired it's not considered for the upgrade.
… helpers This commit introduces the `assertOSSMOperatorWithConfig` helper function, which extends the existing `assertOSSMOperator` by allowing parameterization of polling interval, timeout, and expected OSSM version (new check). Also, this commit adds the `assertIstioWithConfig` helper function, which extends the existing `assertIstio` by allowing parameterization of the expected Istio version (new check).
The `TestOSSMOperatorUpgradeViaIntermediateVersions` test verifies that the OSSM operator correctly upgrades through intermediate versions. It ensures the upgrade logic automatically approves each InstallPlan along the upgrade graph until the desired version is reached. This test is not added to the suite of all tests because it will be run in a dedicated CI job to avoid conflicts with existing GatewayAPI jobs which install the latest supported version of OSSM operator and Istio.
This commit introduces helper functions to clean up Gateway, GatewayClass, and OLM operator after e2e tests. These functions are integrated into the GatewayAPI and GatewayAPI upgrade tests to prevent conflicts and ensure a clean environment for subsequent runs.
CCM leader election can take several minutes, delaying LB provisioning. In some clusters, leader election has been observed to take up to 8m. To accommodate this, increase the timeout for `assertGatewaySuccessful` to 10m.
By OLM design, a subscription cannot be installed unless all catalog sources are healthy, regardless of which catalog source the subscription uses. This commit updates the `assertSubscription` helper to verify catalog source health as part of its checks. The timeout was increased to 2 minutes to accommodate the additional polling time.
alebedev87
force-pushed
the
gwapi-upgrade-ossm-next-ip
branch
from
0b5020f to
d322094
Compare
openshift-merge-robot
removed
the
needs-rebase
Contributor
Author
|
Manual test of |
Contributor
Author
|
/test e2e-gcp-operator |
Contributor
|
/lgtm |
Use assertInstallPlan helper function get more insights about failed install plans for the upgrade e2e test.
Contributor
|
New changes are detected. LGTM label has been removed. |
Contributor
Author
|
Added |
Contributor
Author
|
/retest |
alebedev87
force-pushed
the
gwapi-upgrade-ossm-next-ip
branch
from
8b877cb to
1006dbe
Compare
Contributor
|
@alebedev87: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Contributor
Author
|
GCP job failure: The service account was removed by the cleanup of the previous gatewayAPI test: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces initial upgrade logic for the OSSM Operator.
The implemented logic supports automatic approval of the next install plan from the upgrade graph, progressing sequentially toward the desired version. If the next available install plan version is semantically greater than the desired version, the upgrade stops.
Status handling updates
Notes
Manual test
Result: