-
Notifications
You must be signed in to change notification settings - Fork 216
NO-ISSUE: OTA-1605 Create first e2e test OCP-42543 #1249
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 3 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,84 @@ | ||
| package cvo | ||
|
|
||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Comment not related to the code I will hold us to a higher standard regarding the git hygiene, if I may. Please see the Contributing guide for the cluster-version-operator repository. I am mainly concerned about the following points:
Currently, the PR consists of a single commit Please keep in mind that these guidelines have their practical purposes.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated the commit, because this is the first OTE PR, it contains many logic, so the description may not describes everything. I will keep PR simple and describe the logic in the future. |
||
| import ( | ||
| . "github.com/onsi/ginkgo/v2" | ||
| . "github.com/onsi/gomega" | ||
| "context" | ||
| "fmt" | ||
|
|
||
| g "github.com/onsi/ginkgo/v2" | ||
| o "github.com/onsi/gomega" | ||
| kerrors "k8s.io/apimachinery/pkg/api/errors" | ||
| metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" | ||
| "k8s.io/client-go/kubernetes" | ||
|
|
||
| v1 "github.com/openshift/client-go/config/clientset/versioned/typed/config/v1" | ||
| "github.com/openshift/cluster-version-operator/test/utilities" | ||
JianLi-RH marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ) | ||
|
|
||
| var _ = Describe(`[Jira:"Cluster Version Operator"] cluster-version-operator-tests`, func() { | ||
| It("should support passing tests", func() { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why is my sanity test being removed? :(
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Its future usefulness is questionable. However, for the moment, I am using it to validate that the openshift/origin#30316 PR is functional. Would it be okay for me rename to the component in the /cc @hongkailiu
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can then validate this PR with an integrated OTE in the CVO repository.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do you want to keep "should support passing tests" only temporarily (to validate and merge openshift/origin#30316), and then do I would not practice Maybe it is not a big deal, and it is OK delete as long as having them ignored?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't mind just keeping the test forever.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I meant "ignore" as is in, do not touch. Not as in utilizing the
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ack. Jian has reverted the deleting.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @DavidHurta I just added back the case and rename it as you mentioned. It is a independent test suite.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks. It helps with the integration. It also provides a signal of the integration working in the future. Such sanity tests are also in several other OpenShift repositories that already integrated the OTE or are in the process of the integration. |
||
| Expect(true).To(BeTrue()) | ||
| var _ = g.Describe(`[Jira:"Cluster Version Operator"] cluster-version-operator-tests`, func() { | ||
| g.It("should support passing tests", func() { | ||
| o.Expect(true).To(o.BeTrue()) | ||
| }) | ||
| }) | ||
|
|
||
| var _ = g.Describe("[Jira:Cluster Version Operator] The cluster version operator", g.Ordered, g.Label("cvo"), func() { | ||
| defer g.GinkgoRecover() | ||
| var client *v1.ConfigV1Client | ||
| var kubeclient *kubernetes.Clientset | ||
|
|
||
| g.BeforeAll(func() { | ||
| client = utilities.MustGetV1Client() | ||
| kubeclient = utilities.MustGetKubeClient() | ||
| }) | ||
|
|
||
| g.It(`should not install resources annotated with release.openshift.io/delete=true`, g.Label("Conformance", "High", "42543"), func() { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I assume that the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, it is polarion ID, I have no idea if it is common but I think it is helpful for us when we want to read the case from polarion.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What does the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Then run it:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Similarly, we can use |
||
| annotation := "release.openshift.io/delete" | ||
|
|
||
| auths, err := client.Authentications().List(context.TODO(), metav1.ListOptions{}) | ||
| o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing authentications") | ||
|
|
||
| g.By(fmt.Sprintf("checking if authentication with %s annotation exists", annotation)) | ||
| for _, auth := range auths.Items { | ||
| if _, ok := auth.Annotations[annotation]; ok { | ||
| o.Expect(ok).NotTo(o.BeTrue(), fmt.Sprintf("Unexpectedly installed authentication %s which has '%s' annotation", auth.Name, annotation)) | ||
| } | ||
| } | ||
|
|
||
| namespaces, err := kubeclient.CoreV1().Namespaces().List(context.TODO(), metav1.ListOptions{}) | ||
| o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing namespaces") | ||
|
|
||
| g.By(fmt.Sprintf("checking if special resources with %s annotation exist in all namespaces", annotation)) | ||
| for _, ns := range namespaces.Items { | ||
| namespace := ns.Name | ||
| fmt.Printf("namespace: %s\n", namespace) | ||
|
|
||
| fmt.Println(" - Test services...") | ||
| services, err := kubeclient.CoreV1().Services(namespace).List(context.TODO(), metav1.ListOptions{}) | ||
| o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing services") | ||
| for _, service := range services.Items { | ||
| if _, ok := service.Annotations[annotation]; ok { | ||
| o.Expect(ok).NotTo(o.BeTrue(), fmt.Sprintf("Unexpectedly installed service %s which has '%s' annotation", service.Name, annotation)) | ||
| } | ||
| } | ||
|
|
||
| fmt.Println(" - Test RoleBinding...") | ||
| rolebindings, err := kubeclient.RbacV1().RoleBindings(namespace).List(context.TODO(), metav1.ListOptions{}) | ||
| o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing rolebindings") | ||
| for _, rb := range rolebindings.Items { | ||
| if _, ok := rb.Annotations[annotation]; ok { | ||
| o.Expect(ok).NotTo(o.BeTrue(), fmt.Sprintf("Unexpectedly installed RoleBinding %s which has '%s' annotation", rb.Name, annotation)) | ||
| } | ||
| } | ||
|
|
||
JianLi-RH marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| fmt.Println(" - Test CronJob...") | ||
| cronjobs, err := kubeclient.BatchV1().CronJobs(namespace).List(context.TODO(), metav1.ListOptions{}) | ||
| o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing cronjobs") | ||
| for _, cj := range cronjobs.Items { | ||
| if _, ok := cj.Annotations[annotation]; ok { | ||
| o.Expect(ok).NotTo(o.BeTrue(), fmt.Sprintf("Unexpectedly installed CronJob %s which has %s annotation", cj.Name, annotation)) | ||
| } | ||
| } | ||
|
|
||
JianLi-RH marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| fmt.Println("success") | ||
| } | ||
|
Comment on lines
+91
to
+150
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chainFix List error handling to fail on real API errors, not just NotFound All the List calls currently do: auths, err := client.Authentications().List(...)
o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing ...")This has two problems:
You want the test to fail on any error from those List calls. A more robust pattern is to assert on - auths, err := client.Authentications().List(context.TODO(), metav1.ListOptions{})
- o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing authentications")
+ auths, err := client.Authentications().List(context.TODO(), metav1.ListOptions{})
+ o.Expect(err).NotTo(o.HaveOccurred(), "failed to list authentications")
- namespaces, err := kubeclient.CoreV1().Namespaces().List(context.TODO(), metav1.ListOptions{})
- o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing namespaces")
+ namespaces, err := kubeclient.CoreV1().Namespaces().List(context.TODO(), metav1.ListOptions{})
+ o.Expect(err).NotTo(o.HaveOccurred(), "failed to list namespaces")
- services, err := kubeclient.CoreV1().Services(namespace).List(context.TODO(), metav1.ListOptions{})
- o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing services")
+ services, err := kubeclient.CoreV1().Services(namespace).List(context.TODO(), metav1.ListOptions{})
+ o.Expect(err).NotTo(o.HaveOccurred(), "failed to list services")
- rolebindings, err := kubeclient.RbacV1().RoleBindings(namespace).List(context.TODO(), metav1.ListOptions{})
- o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing rolebindings")
+ rolebindings, err := kubeclient.RbacV1().RoleBindings(namespace).List(context.TODO(), metav1.ListOptions{})
+ o.Expect(err).NotTo(o.HaveOccurred(), "failed to list rolebindings")
- cronjobs, err := kubeclient.BatchV1().CronJobs(namespace).List(context.TODO(), metav1.ListOptions{})
- o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing cronjobs")
+ cronjobs, err := kubeclient.BatchV1().CronJobs(namespace).List(context.TODO(), metav1.ListOptions{})
+ o.Expect(err).NotTo(o.HaveOccurred(), "failed to list cronjobs")This way, any failure to talk to the API (including If you want to double‑check semantics, please confirm in the Kubernetes/client-go docs (or local experiments) that 🌐 Web query: 💡 Result: Short answer: List returns an empty list with err == nil (the returned List object's Items slice is just empty). IsNotFound is not used for a successful List — it's used when an operation asking for a specific named resource (e.g., Get or Delete by name) returns HTTP 404; k8s.io/apimachinery/pkg/api/errors.IsNotFound(err) detects that 404 StatusError. Use IsNotFound when you expect a single resource might not exist and you want to handle "resource missing" specially. References: k8s error helpers (IsNotFound). [1] Client List/Get semantics (List returns list object; Reader/List returns error only on failure). [2] REST practice: collection endpoints return 200 + empty array when no items. [3] Sources: Replace error assertions with proper nil checks for all List calls The web search confirms your analysis: Kubernetes This means the current code has a critical flaw: o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing ...")—will pass whenever
Your proposed fix is correct. Replace all five List error checks in -o.Expect(kerrors.IsNotFound(err)).To(o.BeFalse(), "The NotFound error should occur when listing ...")
+o.Expect(err).NotTo(o.HaveOccurred(), "failed to list ...")This ensures any API failure is caught, not silently suppressed. 🤖 Prompt for AI Agents |
||
| }) | ||
| }) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| package utilities | ||
|
|
||
| import ( | ||
| "errors" | ||
| "fmt" | ||
| "os" | ||
|
|
||
| "k8s.io/client-go/kubernetes" | ||
| "k8s.io/client-go/rest" | ||
| "k8s.io/client-go/tools/clientcmd" | ||
|
|
||
| configclientv1 "github.com/openshift/client-go/config/clientset/versioned/typed/config/v1" | ||
| ) | ||
|
|
||
| // getKubeConfig get KUBECONFIG file from environment variable | ||
| func getKubeConfig() (*rest.Config, error) { | ||
| configPath, present := os.LookupEnv("KUBECONFIG") | ||
| if !present { | ||
| return nil, errors.New("the environment variable KUBECONFIG must be set") | ||
| } | ||
| config, err := clientcmd.BuildConfigFromFlags("", configPath) | ||
| return config, err | ||
| } | ||
|
|
||
| // getKubeClient creates a kubernetes.Clientset instance. | ||
| func getKubeClient() (*kubernetes.Clientset, error) { | ||
| config, err := getKubeConfig() | ||
| if err != nil { | ||
| return nil, fmt.Errorf("unable to load build config: %w", err) | ||
| } | ||
| // Create the Clientset | ||
| clientset, err := kubernetes.NewForConfig(config) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("unable to create a Kubernetes clientset: %w", err) | ||
| } | ||
|
|
||
| return clientset, nil | ||
| } | ||
|
|
||
| // getV1Client creates a configclientv1.ConfigV1Client instance. | ||
| func getV1Client() (*configclientv1.ConfigV1Client, error) { | ||
| config, err := getKubeConfig() | ||
| if err != nil { | ||
| return nil, fmt.Errorf("unable to load build config: %w", err) | ||
| } | ||
| // Create the Clientset | ||
| clientset, err := configclientv1.NewForConfig(config) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("unable to create a configclientv1 clientset: %w", err) | ||
| } | ||
|
|
||
| return clientset, nil | ||
| } | ||
|
|
||
| // MustGetKubeClient creates a kubernetes.Clientset instance, or panics on failures. | ||
| func MustGetKubeClient() *kubernetes.Clientset { | ||
| clientset, err := getKubeClient() | ||
| if err != nil { | ||
| panic("unable to create a Kubernetes clientset: " + err.Error()) | ||
| } | ||
| return clientset | ||
| } | ||
|
|
||
| // MustGetV1Client creates a configclientv1.ConfigV1Client instance, or panics on failures. | ||
| func MustGetV1Client() *configclientv1.ConfigV1Client { | ||
| clientset, err := getV1Client() | ||
| if err != nil { | ||
| panic("unable to create a configclientv1 clientset: " + err.Error()) | ||
| } | ||
| return clientset | ||
| } |
Uh oh!
There was an error while loading. Please reload this page.