SREP-1164 : Update README file for running E2E Tests for CAD #517

ratnam915 · 2025-07-29T17:09:49Z

What type of PR is this?

(feature/bug/documentation/other)

What this PR does / Why we need it?

Special notes for your reviewer

Test Coverage

Guidelines for CAD investigations

New investgations should be accompanied by unit tests and/or step-by-step manual tests in the investigation README.
Actioning investigations should be locally tested in staging, and E2E testing is desired. See README for more info on investigation graduation process.

Test coverage checks

Added tests
Created jira card to add unit test
This PR may not need unit tests

Pre-checks (if applicable)

Ran unit tests locally
Validated the changes in a cluster
Included documentation changes with PR

codecov-commenter · 2025-07-29T17:18:55Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 33.64%. Comparing base (47a8a8e) to head (31a74e4).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #517   +/-   ##
=======================================
  Coverage   33.64%   33.64%           
=======================================
  Files          37       37           
  Lines        2476     2476           
=======================================
  Hits          833      833           
  Misses       1583     1583           
  Partials       60       60

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

typeid · 2025-08-01T07:07:21Z

test/e2e/README.md

+## ONLY FOR LOCAL TESTING, THIS CONFIGURATION HAS TO BE REVERTED BACK BEFORE COMMIT AND PUSHING TO THE REPOSITORY
+
+Comment out #56,57 in configuration_anomaly_detection_tests.go and replace with the following code:
+
+ocme2eCli, err = ocme2e.New(ctx, ocmToken, clientID, clientSecret, ocmEnv)
+Expect(err).ShouldNot(HaveOccurred(), "Unable to setup E2E OCM Client")
+
+ocmCli, err = ocm.New(cadOcmFilePath)
+Expect(err).ShouldNot(HaveOccurred(), "Unable to setup ocm anomaly detection client")
+
+Add below statements in #50,#53 respectively 
+
+ocmToken := os.Getenv("OCM_TOKEN")
+Expect(ocmToken).NotTo(BeEmpty(), "OCM_TOKEN must be set")
+
+## "!! PLEASE NOTE THAT SINCE OCM_TOKEN IS NOW DEPRECATED ABOVE LINES OF CODE HAVE TO BE REMOVED AND #56,57 HAVE TO BE UNCOMMENTED !!"


Can we ensure we don't need to modify the code to run this locally? That seems like a workaround ;) Ideally, running e2e tests with a target cluster should work programmatically. If you could provide a short script here, that would be very beneficial.

Hi @typeid : The reason this workaround is being proposed from us is for OCM_CLIENT_ID and OCM_CLIENT_SECRET to work from a local machine a service account is required, if the service account is available we would not need to do this workaround and the test would work fine.

The script would not be required in that case, please let us know on how to proceed with this.

I can put a note in with a link to the SOP to create service account and enable it to be able to test the code in local.

How about attempting to load the existing ocm token, and falling back to clientid and clientsecret?
That way it would work for both envs, correct?

Also, how do we do this change after building the e2e?

E.g.

import ( ocmConfig "github.com/openshift-online/ocm-common/pkg/ocm/config" ocmConnBuilder "github.com/openshift-online/ocm-common/pkg/ocm/connection-builder" ) ... cfg, err := ocmConfig.Load() if err != nil { clientID := os.Getenv("OCM_CLIENT_ID") Expect(clusterID).NotTo(BeEmpty(), "OCM_CLIENT_ID must be set") clientSecret := os.Getenv("OCM_CLIENT_SECRET") Expect(clusterID).NotTo(BeEmpty(), "OCM_CLIENT_SECRET must be set") ocme2eCli, err = ocme2e.New(ctx, "", clientID, clientSecret, ocmEnv) Expect(err).ShouldNot(HaveOccurred(), "Unable to setup E2E OCM Client") } else { // Build connection based on local config connection, err := ocmConnBuilder.NewConnection().Config(cfg).AsAgent("cad-local-e2e-tests").Build() ocme2eCli = &ocme2e.Client{Connection: connection} Expect(err).ShouldNot(HaveOccurred(), "Unable to setup E2E OCM Client") }

Thanks @typeid, this has significantly reduced overhead, the README file is fairly simple now.

I made the code changes and carried out local testing and it ran fine.

Also in other news i was comparing the Pagerduty alerts that we received https://redhat.pagerduty.com/service-directory/P4BLYHK/activity to the runtimes of the E2E pipeline -> https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/periodic-ci-openshift-osde2e-main-nightly-4.19-rosa-classic-sts

It looks like they are matching, however i don't see any logs from our test cases niether passing nor failing, should be consider closing the task for the environment variables syncing with E2E?

typeid · 2025-08-01T07:10:51Z

test/e2e/README.md

+7. Enable Debug Mode
+This enables detailed logging during test execution:
+export CAD_DEBUG=true


What is this variable for if we're not running the cadctl afterwards?

This is again required for local testing and can be removed if we are following the service account mode

CAD_DEBUG? That variable doesn't even exist anymore, and it should have had no influence on e2e :)

typeid · 2025-08-01T07:11:37Z

test/e2e/README.md

+5. AWS Credentials
+These are needed for interacting with the cluster. You can find them in the ~/.aws/credentials file.
+export AWS_ACCESS_KEY_ID=<your AWS access key ID>
+export AWS_SECRET_ACCESS_KEY=<your AWS secret access key>


Can we make this more generic to work on "target stage cluster"? The credentials in this file will not always match the cluster's AWS creds.

Added another line to ensure cluster is created with a default profile and the credentials for the same profile are set in the environment variables in this step

typeid · 2025-08-01T07:12:04Z

test/e2e/README.md

@@ -9,6 +9,42 @@ To do this, following steps are recommended

 ocm get /api/clusters_mgmt/v1/clusters/(cluster-id)/credentials | jq -r .kubeconfig > /(path-to)/kubeconfig


Not part of this PR, but the sentence above:

. Deploy your new version of operator in a test cluster

That doesn't seem to apply here.

ratnam915 · 2025-08-10T15:25:20Z

"/label tide/merge-method-squash"

RaphaelBut · 2025-08-11T12:53:38Z

test/e2e/README.md

+
+3. Create a ROSA Cluster which would be used for E2E testing using the below command:
+
+    rosa create cluster --cluster-name=<cluster-name> --profile <default-aws-profile>


polish: we need to export the cluster id as OCM_CLUSTER_ID here

RaphaelBut · 2025-08-11T12:55:32Z

I ran through these steps while developing another test case and it worked fine.
Thanks for adding this!

openshift-ci · 2025-08-12T17:42:02Z

@ratnam915: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

typeid

/lgtm
/approve

openshift-ci · 2025-08-13T07:27:37Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ratnam915, typeid

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [typeid]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Update README file for running E2E Tests for CAD

f331245

openshift-ci bot requested review from rafael-azevedo and RaphaelBut July 29, 2025 17:10

typeid requested changes Aug 1, 2025

View reviewed changes

ratnam915 added 4 commits August 5, 2025 18:13

Made changes to the README file

bcdc480

Merge branch 'openshift:main' into feature/SREP-1164

aee0e75

Made OCM configuration changes to support local testing

1c04980

E2E README file changed post OCM code change

a6c139e

RaphaelBut reviewed Aug 11, 2025

View reviewed changes

ratnam915 added 2 commits August 12, 2025 22:42

Merge branch 'openshift:main' into feature/SREP-1164

38466a7

ReadME file updated for CAD E2E tests

31a74e4

typeid approved these changes Aug 13, 2025

View reviewed changes

openshift-ci bot assigned typeid Aug 13, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 13, 2025

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 13, 2025

openshift-merge-bot bot merged commit 2d9a751 into openshift:main Aug 13, 2025
5 checks passed

		@@ -9,6 +9,42 @@ To do this, following steps are recommended

		ocm get /api/clusters_mgmt/v1/clusters/(cluster-id)/credentials \| jq -r .kubeconfig > /(path-to)/kubeconfig


		3. Create a ROSA Cluster which would be used for E2E testing using the below command:

		rosa create cluster --cluster-name=<cluster-name> --profile <default-aws-profile>

SREP-1164 : Update README file for running E2E Tests for CAD #517

SREP-1164 : Update README file for running E2E Tests for CAD #517

Uh oh!

Conversation

ratnam915 commented Jul 29, 2025

What type of PR is this?

What this PR does / Why we need it?

Special notes for your reviewer

Test Coverage

Guidelines for CAD investigations

Test coverage checks

Pre-checks (if applicable)

Uh oh!

codecov-commenter commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

typeid Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ratnam915 commented Aug 10, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RaphaelBut commented Aug 11, 2025

Uh oh!

openshift-ci bot commented Aug 12, 2025

Uh oh!

typeid left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Aug 13, 2025

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Jul 29, 2025 •

edited

Loading

typeid Aug 8, 2025 •

edited

Loading