OCPBUGS-59354: E2E add test to verify guaranteed pod is running after kubelet restart #1361

mrniranjan · 2025-07-11T11:37:26Z

This PR addresses issue where we are verifying
if the cpu manager state file is same after kubelet restart while we are verifying the above, we are not checking if Guranteed pod started before kubelet restart is also still running.

Refer: https://issues.redhat.com/browse/OCPBUGS-43280

This PR addresses issue where we are verifying if the cpu manager state file is same after kubelet restart while we are verifying the above, we are not checking if Guranteed pod started before kubelet restart is also still running. Refer: https://issues.redhat.com/browse/OCPBUGS-43280 Signed-off-by: Niranjan M.R <[email protected]>

shajmakh · 2025-07-15T11:05:14Z

test/e2e/performanceprofile/functests/1_performance/cpu_management.go


+				By("verify test pod comes back after kubelet restart")
+				Eventually(func() error {
+					updatedPod := &corev1.Pod{}


we can reduce code complexity and avoid using references, instead you can use like this:

var updatedPod corev1.Pod Eventually(func() error{ err := testclient.DataPlaneClient.Get(ctx, client.ObjectKeyFromObject(testpod), &updatedPod) ...

Addressed in latest commit

shajmakh · 2025-07-15T11:13:56Z

test/e2e/performanceprofile/functests/1_performance/cpu_management.go

+					for _, condition := range updatedPod.Status.Conditions {
+						if condition.Type == corev1.PodReady && condition.Status == corev1.ConditionTrue {
+							testlog.Infof("post kubelet restart pod is ready with UID: %v", updatedPod.UID)
+							return nil
+						}
+					}


I think we can make the function here more consistent with the flow, as in apply negative conditions if none is met then return nil (i.e succeeds); this can be done like this:

for _, condition := range updatedPod.Status.Conditions { if condition.Type == corev1.PodReady && condition.Status != corev1.ConditionTrue { return fmt.Errorf("pod condition is not ready after restart: conditions=%v", updatedPod.Status.Conditions) } } return nil

Addressed in latest commit.

Signed-off-by: Niranjan M.R <[email protected]>

swatisehgal

Thanks for the PR. Mostly looks good, some minor comments.

swatisehgal · 2025-07-15T11:59:48Z

test/e2e/performanceprofile/functests/1_performance/cpu_management.go

+					// Check pod ready condition
+					for _, condition := range updatedPod.Status.Conditions {
+						if condition.Type == corev1.PodReady && condition.Status != corev1.ConditionTrue {
+							return fmt.Errorf("Pod ondition is not in Ready state after kubelet restart: condition: %v", updatedPod.Status.Conditions)


NIT:

Suggested change

return fmt.Errorf("Pod ondition is not in Ready state after kubelet restart: condition: %v", updatedPod.Status.Conditions)

return fmt.Errorf("Pod condition is not in Ready state after kubelet restart: condition: %v", updatedPod.Status.Conditions)

fixed in latest commit

swatisehgal · 2025-07-15T12:16:21Z

test/e2e/performanceprofile/functests/1_performance/cpu_management.go

+					}
+					// Check pod ready condition
+					for _, condition := range updatedPod.Status.Conditions {
+						if condition.Type == corev1.PodReady && condition.Status != corev1.ConditionTrue {


In case pod condition is ready but fails the condition status check would be nice to point out condition Reason and condition Message.

fixed in latest commit

Signed-off-by: Niranjan M.R <[email protected]>

swatisehgal

/lgtm

mrniranjan · 2025-07-16T00:59:40Z

/test okd-scos-e2e-aws-ovn

mrniranjan · 2025-07-16T00:59:50Z

/test e2e-aws-ovn-techpreview

mrniranjan · 2025-07-16T01:00:01Z

/test e2e-aws-ovn

openshift-ci-robot · 2025-07-16T01:16:51Z

@mrniranjan: This pull request references Jira Issue OCPBUGS-59354, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.20.0) matches configured target version for branch (4.20.0)
bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @mrniranjan

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

This PR addresses issue where we are verifying
if the cpu manager state file is same after kubelet restart while we are verifying the above, we are not checking if Guranteed pod started before kubelet restart is also still running.

Refer: https://issues.redhat.com/browse/OCPBUGS-43280

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2025-07-16T01:16:54Z

@openshift-ci-robot: GitHub didn't allow me to request PR reviews from the following users: mrniranjan.

Note that only openshift members and repo collaborators can review this PR, and authors cannot review their own PRs.

Details

In response to this:

@mrniranjan: This pull request references Jira Issue OCPBUGS-59354, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)

bug target version (4.20.0) matches configured target version for branch (4.20.0)

bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @mrniranjan

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This PR addresses issue where we are verifying
if the cpu manager state file is same after kubelet restart while we are verifying the above, we are not checking if Guranteed pod started before kubelet restart is also still running.

Refer: https://issues.redhat.com/browse/OCPBUGS-43280

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

mrniranjan · 2025-07-16T08:15:00Z

/test e2e-aws-ovn

mrniranjan · 2025-07-16T10:53:16Z

@yanirq can you approve this PR

openshift-ci · 2025-07-21T11:10:11Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: MarSik, mrniranjan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [MarSik]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mrniranjan · 2025-07-22T11:27:03Z

/retest-required

openshift-ci · 2025-07-22T11:48:07Z

@mrniranjan: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot · 2025-07-22T11:51:38Z

@mrniranjan: Jira Issue OCPBUGS-59354: All pull requests linked via external trackers have merged:

openshift/cluster-node-tuning-operator#1361

Jira Issue OCPBUGS-59354 has been moved to the MODIFIED state.

Details

In response to this:

This PR addresses issue where we are verifying
if the cpu manager state file is same after kubelet restart while we are verifying the above, we are not checking if Guranteed pod started before kubelet restart is also still running.

Refer: https://issues.redhat.com/browse/OCPBUGS-43280

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

mrniranjan · 2025-07-22T12:21:48Z

/cherry-pick release-4.19

openshift-cherrypick-robot · 2025-07-22T12:22:47Z

@mrniranjan: new pull request created: #1365

Details

In response to this:

/cherry-pick release-4.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-bot · 2025-07-22T14:35:30Z

[ART PR BUILD NOTIFIER]

Distgit: cluster-node-tuning-operator
This PR has been included in build cluster-node-tuning-operator-container-v4.20.0-202507221345.p0.geacfe9f.assembly.stream.el9.
All builds following this will include this PR.

openshift-ci bot requested review from ffromani and jmencak July 11, 2025 11:38

shajmakh reviewed Jul 15, 2025

View reviewed changes

avoid using indirect references and test negative pod status condition

dada06a

Signed-off-by: Niranjan M.R <[email protected]>

swatisehgal reviewed Jul 15, 2025

View reviewed changes

typo fix and add more debug when pod fails

db1eddf

Signed-off-by: Niranjan M.R <[email protected]>

swatisehgal reviewed Jul 15, 2025

View reviewed changes

openshift-ci bot assigned swatisehgal Jul 15, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 15, 2025

mrniranjan changed the title ~~verify Guranteed pod is running after kubelet restart~~ OCPBUGS-59354: E2E add test to verify guaranteed pod is running after kubelet restart Jul 16, 2025

openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels Jul 16, 2025

MarSik approved these changes Jul 21, 2025

View reviewed changes

openshift-ci bot assigned MarSik Jul 21, 2025

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 21, 2025

openshift-merge-bot bot merged commit eacfe9f into openshift:main Jul 22, 2025
19 checks passed

openshift-cherrypick-robot mentioned this pull request Jul 22, 2025

[release-4.19] OCPBUGS-59633: E2E add test to verify guaranteed pod is running after kubelet restart #1365

Merged

	return fmt.Errorf("Pod ondition is not in Ready state after kubelet restart: condition: %v", updatedPod.Status.Conditions)
	return fmt.Errorf("Pod condition is not in Ready state after kubelet restart: condition: %v", updatedPod.Status.Conditions)

OCPBUGS-59354: E2E add test to verify guaranteed pod is running after kubelet restart #1361

OCPBUGS-59354: E2E add test to verify guaranteed pod is running after kubelet restart #1361

Uh oh!

Conversation

mrniranjan commented Jul 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swatisehgal left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swatisehgal left a comment

Choose a reason for hiding this comment

Uh oh!

mrniranjan commented Jul 16, 2025

Uh oh!

mrniranjan commented Jul 16, 2025

Uh oh!

mrniranjan commented Jul 16, 2025

Uh oh!

openshift-ci-robot commented Jul 16, 2025

Uh oh!

openshift-ci bot commented Jul 16, 2025

Uh oh!

mrniranjan commented Jul 16, 2025

Uh oh!

mrniranjan commented Jul 16, 2025

Uh oh!

openshift-ci bot commented Jul 21, 2025

Uh oh!

mrniranjan commented Jul 22, 2025

Uh oh!

openshift-ci bot commented Jul 22, 2025

Uh oh!

Uh oh!

openshift-ci-robot commented Jul 22, 2025

Uh oh!

mrniranjan commented Jul 22, 2025

Uh oh!

openshift-cherrypick-robot commented Jul 22, 2025

Uh oh!

openshift-bot commented Jul 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants