SREP-1164 : Update README file for running E2E Tests for CAD #517

ratnam915 · 2025-07-29T17:09:49Z

What type of PR is this?

(feature/bug/documentation/other)

What this PR does / Why we need it?

Special notes for your reviewer

Test Coverage

Guidelines for CAD investigations

New investgations should be accompanied by unit tests and/or step-by-step manual tests in the investigation README.
Actioning investigations should be locally tested in staging, and E2E testing is desired. See README for more info on investigation graduation process.

Test coverage checks

Added tests
Created jira card to add unit test
This PR may not need unit tests

Pre-checks (if applicable)

Ran unit tests locally
Validated the changes in a cluster
Included documentation changes with PR

openshift-ci · 2025-07-29T17:10:53Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ratnam915
Once this PR has been reviewed and has the lgtm label, please assign fahlmant for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov-commenter · 2025-07-29T17:18:55Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 32.68%. Comparing base (4b2a2f2) to head (bcdc480).
⚠️ Report is 23 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #517      +/-   ##
==========================================
+ Coverage   32.16%   32.68%   +0.51%     
==========================================
  Files          37       37              
  Lines        2459     2472      +13     
==========================================
+ Hits          791      808      +17     
+ Misses       1609     1603       -6     
- Partials       59       61       +2

see 14 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

typeid · 2025-08-01T07:07:21Z

test/e2e/README.md

+## ONLY FOR LOCAL TESTING, THIS CONFIGURATION HAS TO BE REVERTED BACK BEFORE COMMIT AND PUSHING TO THE REPOSITORY
+
+Comment out #56,57 in configuration_anomaly_detection_tests.go and replace with the following code:
+
+ocme2eCli, err = ocme2e.New(ctx, ocmToken, clientID, clientSecret, ocmEnv)
+Expect(err).ShouldNot(HaveOccurred(), "Unable to setup E2E OCM Client")
+
+ocmCli, err = ocm.New(cadOcmFilePath)
+Expect(err).ShouldNot(HaveOccurred(), "Unable to setup ocm anomaly detection client")
+
+Add below statements in #50,#53 respectively 
+
+ocmToken := os.Getenv("OCM_TOKEN")
+Expect(ocmToken).NotTo(BeEmpty(), "OCM_TOKEN must be set")
+
+## "!! PLEASE NOTE THAT SINCE OCM_TOKEN IS NOW DEPRECATED ABOVE LINES OF CODE HAVE TO BE REMOVED AND #56,57 HAVE TO BE UNCOMMENTED !!"


Can we ensure we don't need to modify the code to run this locally? That seems like a workaround ;) Ideally, running e2e tests with a target cluster should work programmatically. If you could provide a short script here, that would be very beneficial.

Hi @typeid : The reason this workaround is being proposed from us is for OCM_CLIENT_ID and OCM_CLIENT_SECRET to work from a local machine a service account is required, if the service account is available we would not need to do this workaround and the test would work fine.

The script would not be required in that case, please let us know on how to proceed with this.

I can put a note in with a link to the SOP to create service account and enable it to be able to test the code in local.

How about attempting to load the existing ocm token, and falling back to clientid and clientsecret?
That way it would work for both envs, correct?

Also, how do we do this change after building the e2e?

E.g.

import ( ocmConfig "github.com/openshift-online/ocm-common/pkg/ocm/config" ocmConnBuilder "github.com/openshift-online/ocm-common/pkg/ocm/connection-builder" ) ... cfg, err := ocmConfig.Load() if err != nil { clientID := os.Getenv("OCM_CLIENT_ID") Expect(clusterID).NotTo(BeEmpty(), "OCM_CLIENT_ID must be set") clientSecret := os.Getenv("OCM_CLIENT_SECRET") Expect(clusterID).NotTo(BeEmpty(), "OCM_CLIENT_SECRET must be set") ocme2eCli, err = ocme2e.New(ctx, "", clientID, clientSecret, ocmEnv) Expect(err).ShouldNot(HaveOccurred(), "Unable to setup E2E OCM Client") } else { // Build connection based on local config connection, err := ocmConnBuilder.NewConnection().Config(cfg).AsAgent("cad-local-e2e-tests").Build() ocme2eCli = &ocme2e.Client{Connection: connection} Expect(err).ShouldNot(HaveOccurred(), "Unable to setup E2E OCM Client") }

Thanks @typeid, this has significantly reduced overhead, the README file is fairly simple now.

I made the code changes and carried out local testing and it ran fine.

Also in other news i was comparing the Pagerduty alerts that we received https://redhat.pagerduty.com/service-directory/P4BLYHK/activity to the runtimes of the E2E pipeline -> https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/periodic-ci-openshift-osde2e-main-nightly-4.19-rosa-classic-sts

It looks like they are matching, however i don't see any logs from our test cases niether passing nor failing, should be consider closing the task for the environment variables syncing with E2E?

typeid · 2025-08-01T07:10:51Z

test/e2e/README.md

+7. Enable Debug Mode
+This enables detailed logging during test execution:
+export CAD_DEBUG=true


What is this variable for if we're not running the cadctl afterwards?

This is again required for local testing and can be removed if we are following the service account mode

CAD_DEBUG? That variable doesn't even exist anymore, and it should have had no influence on e2e :)

typeid · 2025-08-01T07:11:37Z

test/e2e/README.md

+5. AWS Credentials
+These are needed for interacting with the cluster. You can find them in the ~/.aws/credentials file.
+export AWS_ACCESS_KEY_ID=<your AWS access key ID>
+export AWS_SECRET_ACCESS_KEY=<your AWS secret access key>


Can we make this more generic to work on "target stage cluster"? The credentials in this file will not always match the cluster's AWS creds.

Added another line to ensure cluster is created with a default profile and the credentials for the same profile are set in the environment variables in this step

typeid · 2025-08-01T07:12:04Z

test/e2e/README.md

@@ -9,6 +9,42 @@ To do this, following steps are recommended

 ocm get /api/clusters_mgmt/v1/clusters/(cluster-id)/credentials | jq -r .kubeconfig > /(path-to)/kubeconfig


Not part of this PR, but the sentence above:

. Deploy your new version of operator in a test cluster

That doesn't seem to apply here.

ratnam915 · 2025-08-10T15:25:20Z

"/label tide/merge-method-squash"

openshift-ci · 2025-08-10T15:26:46Z

@ratnam915: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Update README file for running E2E Tests for CAD

f331245

openshift-ci bot requested review from rafael-azevedo and RaphaelBut July 29, 2025 17:10

typeid requested changes Aug 1, 2025

View reviewed changes

ratnam915 added 4 commits August 5, 2025 18:13

Made changes to the README file

bcdc480

Merge branch 'openshift:main' into feature/SREP-1164

aee0e75

Made OCM configuration changes to support local testing

1c04980

E2E README file changed post OCM code change

a6c139e

		@@ -9,6 +9,42 @@ To do this, following steps are recommended

		ocm get /api/clusters_mgmt/v1/clusters/(cluster-id)/credentials \| jq -r .kubeconfig > /(path-to)/kubeconfig

SREP-1164 : Update README file for running E2E Tests for CAD #517

Are you sure you want to change the base?

SREP-1164 : Update README file for running E2E Tests for CAD #517

Conversation

ratnam915 commented Jul 29, 2025

What type of PR is this?

What this PR does / Why we need it?

Special notes for your reviewer

Test Coverage

Guidelines for CAD investigations

Test coverage checks

Pre-checks (if applicable)

Uh oh!

openshift-ci bot commented Jul 29, 2025

Uh oh!

codecov-commenter commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

typeid Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ratnam915 commented Aug 10, 2025

Uh oh!

openshift-ci bot commented Aug 10, 2025

Uh oh!

Uh oh!

codecov-commenter commented Jul 29, 2025 •

edited

Loading

typeid Aug 8, 2025 •

edited

Loading