Skip to content

Conversation

kuiwang02
Copy link
Contributor

@kuiwang02 kuiwang02 commented Oct 9, 2025

/cc @jianzhangbjz @perdasilva @camilamacedo86

UPSTREAM: : support webhook case in disconnected

Summary

This PR enables OLMv1 webhook operator tests to run in disconnected environments by building operator bundles and catalog images using OpenShift's internal build system instead of relying on external registries.

Key Changes

  1. Disconnected Environment Support
  • Build operator bundles and catalog images locally using OpenShift BuildConfig and ImageStream
  • Use internal image registry for all webhook operator images instead of external quay.io references
  • Remove [Skipped:Disconnected] label from test suite to enable execution in air-gapped clusters
  1. Image Management
  • Added webhook operator image quay.io/olmtest/webhook-operator:v0.0.5 to the approved image list in test/extended/util/image/image.go
  • Integrated with image.LocationFor() to support automatic image mirroring in disconnected environments
  • Added image.InitializeImages(os.Getenv("KUBE_TEST_REPO")) in main.go to enable repository mapping
  1. Enhanced Resource Cleanup
  • Refactored cleanup mechanism to use Ginkgo's DeferCleanup for guaranteed resource cleanup even on test failures
  • Changed setupWebhookOperator from returning a cleanup function to registering DeferCleanup internally
  • Added deletion verification for ClusterExtension and Namespace resources
  • Cleanup now executes in LIFO order: ClusterExtension → ClusterRoleBinding → Namespace
  1. Build Automation
  • Added embedded test data using go-bindata for webhook operator bundle and catalog
  • Created pkg/bindata/webhook/bundle/bundle.go with 484 lines of generated code
  • Created pkg/bindata/webhook/index/index.go with 271 lines of generated code
  • Updated Makefile to support building webhook test images from testdata
  1. Test Improvements
  • Added unique suffixes to catalog names and namespaces to enable parallel test execution
  • Added original-name labels to preserve test identity across renames
  • Marked individual test cases as [Serial] where appropriate
  • Enhanced debugging output for test failures

Testing

INFO[0191] Found 0 must-gather tests                    
started: 0/1/5 "[sig-olmv1][OCPFeatureGate:NewOLM] OLMv1 operator installation should block cluster upgrades if an incompatible operator is installed"

started: 0/2/5 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working validating webhook"


passed: (39.6s) 2025-10-10T11:40:07 "[sig-olmv1][OCPFeatureGate:NewOLM] OLMv1 operator installation should block cluster upgrades if an incompatible operator is installed"


passed: (1m16s) 2025-10-10T11:40:44 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working validating webhook"

started: 0/3/5 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should be tolerant to tls secret deletion [Serial]"


passed: (1m20s) 2025-10-10T11:42:10 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should be tolerant to tls secret deletion [Serial]"

started: 0/4/5 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working conversion webhook [Serial]"


passed: (1m11s) 2025-10-10T11:43:32 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working conversion webhook [Serial]"

started: 0/5/5 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working mutating webhook [Serial]"


passed: (1m20s) 2025-10-10T11:45:01 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working mutating webhook [Serial]"

Shutting down the monitor
Collecting data.
INFO[0530] Starting CollectData for all monitor tests   
INFO[0530]   Starting CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0530]   Finished CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0530] Finished CollectData for all monitor tests   
Computing intervals.
Evaluating tests.
Cleaning up.
INFO[0530] beginning cleanup                             monitorTest=watch-namespaces
Serializing results.
Writing to storage.
  m.startTime = 2025-10-10 19:39:22.81484 +0800 CST m=+191.312667834
  m.stopTime  = 2025-10-10 19:45:01.59107 +0800 CST m=+530.092367084
Processing monitorTest: watch-namespaces
  finalIntervals size = 10
  first interval time: From = 2025-10-10 19:39:22.821833 +0800 CST m=+191.319661292; To = 2025-10-10 19:39:22.821833 +0800 CST m=+191.319661292
  last interval time: From = 2025-10-10 19:45:01.589561 +0800 CST m=+530.090858751; To = 2025-10-10 19:45:01.589561 +0800 CST m=+530.090858751
Writing junits.
Writing JUnit report to e2e-monitor-tests__20251010-113621.xml
5 pass, 0 flaky, 0 skip (8m40s)

Assisted-by: Claude Code

Copy link
Contributor

openshift-ci bot commented Oct 9, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: kuiwang02
Once this PR has been reviewed and has the lgtm label, please assign perdasilva for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kuiwang02
Copy link
Contributor Author

/payload-job-with-prs periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6 openshift/origin#30353

Copy link
Contributor

openshift-ci bot commented Oct 9, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/8f66c2a0-a4ce-11f0-8ccb-30af1119612e-0

@kuiwang02
Copy link
Contributor Author

/payload-job-with-prs periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview openshift/origin#30353

Copy link
Contributor

openshift-ci bot commented Oct 9, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/e7430f90-a4d0-11f0-9c09-15e0e4f20a01-0

@kuiwang02
Copy link
Contributor Author

/payload-job-with-prs periodic-ci-openshift-release-master-ci-4.21-e2e-aws-ovn-techpreview openshift/origin#30353

Copy link
Contributor

openshift-ci bot commented Oct 9, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-e2e-aws-ovn-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/20dc1620-a4d1-11f0-9afa-28161a25297b-0

@kuiwang02
Copy link
Contributor Author

/payload-job-with-prs periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview openshift/origin#30356

Copy link
Contributor

openshift-ci bot commented Oct 9, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/4830f310-a4ec-11f0-85f6-8de0f114b59e-0

@kuiwang02
Copy link
Contributor Author

/payload-job-with-prs periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview openshift/origin#30356

Copy link
Contributor

openshift-ci bot commented Oct 10, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b27282c0-a56e-11f0-997c-3c786d788ca2-0

@kuiwang02
Copy link
Contributor Author

kuiwang02 commented Oct 10, 2025

/retitle OPRUN-4156: UPSTREAM: : support webhook case in disconnected

@openshift-ci openshift-ci bot changed the title [Testing. DO not Merge]UPSTREAM: <carry>: support disconnected OPRUN-4156: UPSTREAM: <carry>: support webhook case in disconnected Oct 10, 2025
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 10, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 10, 2025

@kuiwang02: This pull request references OPRUN-4156 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.21.0" version, but no target version was set.

In response to this:

/cc @jianzhangbjz @perdasilva @camilamacedo86

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@kuiwang02
Copy link
Contributor Author

/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview

Copy link
Contributor

openshift-ci bot commented Oct 10, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/f2b31d90-a5af-11f0-9db6-906ca291b9bf-0

@kuiwang02
Copy link
Contributor Author

/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-1of2

Copy link
Contributor

openshift-ci bot commented Oct 10, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-1of2

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/21a4f010-a5b0-11f0-8acc-edce4dd7b236-0

@kuiwang02
Copy link
Contributor Author

/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-2of2

Copy link
Contributor

openshift-ci bot commented Oct 10, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-2of2

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/42203700-a5b0-11f0-86d9-7ffa31e864b1-0

@perdasilva
Copy link
Contributor

Nice! We'll want to move the image building to the helpers package and refactor the incompatible test to use it as well. But, if this works, I'm happy to merge it so we can get signal in Sippy ASAP and do that refactoring in an immediate follow-up PR.

@kuiwang02
Copy link
Contributor Author

Nice! We'll want to move the image building to the helpers package and refactor the incompatible test to use it as well. But, if this works, I'm happy to merge it so we can get signal in Sippy ASAP and do that refactoring in an immediate follow-up PR.

we could keep it for a while so that we can make webhook case works. after that, could refactor the image build. could you please wait for a few days? thanks

@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 10, 2025

@kuiwang02: This pull request references OPRUN-4156 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.21.0" version, but no target version was set.

In response to this:

/cc @jianzhangbjz @perdasilva @camilamacedo86

UPSTREAM: : support webhook case in disconnected

Summary

This PR enables OLMv1 webhook operator tests to run in disconnected environments by building operator bundles and catalog images using OpenShift's internal build system instead of relying on external registries.

Key Changes

  1. Disconnected Environment Support
  • Build operator bundles and catalog images locally using OpenShift BuildConfig and ImageStream
  • Use internal image registry for all webhook operator images instead of external quay.io references
  • Remove [Skipped:Disconnected] label from test suite to enable execution in air-gapped clusters
  1. Image Management
  • Added webhook operator image quay.io/olmtest/webhook-operator:v0.0.5 to the approved image list in test/extended/util/image/image.go
  • Integrated with image.LocationFor() to support automatic image mirroring in disconnected environments
  • Added image.InitializeImages(os.Getenv("KUBE_TEST_REPO")) in main.go to enable repository mapping
  1. Enhanced Resource Cleanup
  • Refactored cleanup mechanism to use Ginkgo's DeferCleanup for guaranteed resource cleanup even on test failures
  • Changed setupWebhookOperator from returning a cleanup function to registering DeferCleanup internally
  • Added deletion verification for ClusterExtension and Namespace resources
  • Cleanup now executes in LIFO order: ClusterExtension → ClusterRoleBinding → Namespace
  1. Build Automation
  • Added embedded test data using go-bindata for webhook operator bundle and catalog
  • Created pkg/bindata/webhook/bundle/bundle.go with 484 lines of generated code
  • Created pkg/bindata/webhook/index/index.go with 271 lines of generated code
  • Updated Makefile to support building webhook test images from testdata
  1. Test Improvements
  • Added unique suffixes to catalog names and namespaces to enable parallel test execution
  • Added original-name labels to preserve test identity across renames
  • Marked individual test cases as [Serial] where appropriate
  • Enhanced debugging output for test failures

Testing

INFO[0191] Found 0 must-gather tests                    
started: 0/1/5 "[sig-olmv1][OCPFeatureGate:NewOLM] OLMv1 operator installation should block cluster upgrades if an incompatible operator is installed"

started: 0/2/5 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working validating webhook"


passed: (39.6s) 2025-10-10T11:40:07 "[sig-olmv1][OCPFeatureGate:NewOLM] OLMv1 operator installation should block cluster upgrades if an incompatible operator is installed"


passed: (1m16s) 2025-10-10T11:40:44 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working validating webhook"

started: 0/3/5 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should be tolerant to tls secret deletion [Serial]"


passed: (1m20s) 2025-10-10T11:42:10 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should be tolerant to tls secret deletion [Serial]"

started: 0/4/5 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working conversion webhook [Serial]"


passed: (1m11s) 2025-10-10T11:43:32 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working conversion webhook [Serial]"

started: 0/5/5 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working mutating webhook [Serial]"


passed: (1m20s) 2025-10-10T11:45:01 "[sig-olmv1][OCPFeatureGate:NewOLMWebhookProviderOpenshiftServiceCA] OLMv1 operator with webhooks should have a working mutating webhook [Serial]"

Shutting down the monitor
Collecting data.
INFO[0530] Starting CollectData for all monitor tests   
INFO[0530]   Starting CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0530]   Finished CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0530] Finished CollectData for all monitor tests   
Computing intervals.
Evaluating tests.
Cleaning up.
INFO[0530] beginning cleanup                             monitorTest=watch-namespaces
Serializing results.
Writing to storage.
 m.startTime = 2025-10-10 19:39:22.81484 +0800 CST m=+191.312667834
 m.stopTime  = 2025-10-10 19:45:01.59107 +0800 CST m=+530.092367084
Processing monitorTest: watch-namespaces
 finalIntervals size = 10
 first interval time: From = 2025-10-10 19:39:22.821833 +0800 CST m=+191.319661292; To = 2025-10-10 19:39:22.821833 +0800 CST m=+191.319661292
 last interval time: From = 2025-10-10 19:45:01.589561 +0800 CST m=+530.090858751; To = 2025-10-10 19:45:01.589561 +0800 CST m=+530.090858751
Writing junits.
Writing JUnit report to e2e-monitor-tests__20251010-113621.xml
5 pass, 0 flaky, 0 skip (8m40s)

Assisted-by: Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview 5

Copy link
Contributor

openshift-ci bot commented Oct 10, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/363ec7b0-a5d0-11f0-9009-a07fc2787655-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-1of2 5

Copy link
Contributor

openshift-ci bot commented Oct 10, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-1of2

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/53d712f0-a5d0-11f0-97f9-49066ab79c1f-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-2of2 5

Copy link
Contributor

openshift-ci bot commented Oct 10, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-2of2

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/59225c10-a5d0-11f0-9c66-5393e61a3e2d-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview 5

Copy link
Contributor

openshift-ci bot commented Oct 10, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/94cf2770-a5d0-11f0-999e-2d4f577aa3df-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview-serial 5

Copy link
Contributor

openshift-ci bot commented Oct 10, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/a1ef71d0-a5d0-11f0-9f16-b55319e3ecf8-0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants