OCPBUGS-9037, OCPBUGS-64565: Use cluster wildcard certificate for ingress canary #1155

rfredette · 2024-10-15T15:33:55Z

Utilize the existing ingress controller certificate management controller to also manage the certificate for the ingress canary, and use that certificate when serving the canary endpoint.

openshift-ci-robot · 2024-10-15T15:34:01Z

@rfredette: This pull request references Jira Issue OCPBUGS-9037, which is invalid:

expected the bug to target either version "4.18." or "openshift-4.18.", but it targets "4.17.z" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Utilize the existing ingress controller certificate management controller to also manage the certificate for the ingress canary, and use that certificate when serving the canary endpoint.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

rfredette · 2024-10-15T19:51:05Z

test failures appear unrelated.
/test e2e-gcp-operator
/test e2e-hypershift

candita · 2024-11-20T16:35:17Z

/assign @Miciah
/assign

candita · 2024-11-20T22:22:42Z

/retest-required

candita

@rfredette I've had these comments pending for some time. Please let me know if I've misunderstood the assignment here, but in some places it looks like we are just reusing the default cert. Was that the plan?

candita · 2024-11-20T22:33:11Z

pkg/manifests/assets/canary/service.yaml

 apiVersion: v1
-metadata:
+metadata: {}
  # name and namespace are set at runtime.


nit:

Suggested change

# name and namespace are set at runtime.

# metadata values are set at runtime.

candita · 2024-11-20T23:04:30Z

pkg/operator/controller/canary/daemonset.go

+	secretName, err := r.canarySecretName(controller.CanaryDaemonSetName().Namespace)
+	if err != nil {
+		return false, nil, err
+	}


Why not get the secretName within desiredCanaryDaemonSet? Then you don't need to pass it in as a parameter.

In order to enable desiredCanaryDaemonSet to get the secret, we would need to pass the client to desiredCanaryDaemonSet. Passing the client to desiredCanaryDaemonSet would go against our established pattern that ensureFoo gets the necessary inputs for desiredFoo and passes those specific inputs to desiredFoo. Passing the client would also mean we would be adding a parameter to desiredCanaryDaemonSet anyway, so it wouldn't really have any benefit over the current change.

candita · 2024-11-20T23:08:02Z

pkg/operator/controller/canary/daemonset.go

 	daemonset.Spec.Template.Spec.Containers[0].Image = canaryImage
 	daemonset.Spec.Template.Spec.Containers[0].Command = []string{"ingress-operator", CanaryHealthcheckCommand}

+	daemonset.Spec.Template.Spec.Volumes[0].Secret.SecretName = secretName


It would be better to call canarySecretName right here instead of in the ensure function, and then passing it in here.

Suggested change

daemonset.Spec.Template.Spec.Volumes[0].Secret.SecretName = secretName

daemonset.Spec.Template.Spec.Volumes[0].Secret.SecretName, err := canarySecretName(name.Namespace)

And then you could test the secret generation in Test_desiredCanaryDaemonSet instead of writing a new test for ensureCanaryDaemonSet.

candita · 2024-11-20T23:27:02Z

pkg/operator/controller/certificate/controller.go

+						UID:        daemonset.UID,
+						Controller: &trueVar,
+					}
+					if _, err := r.ensureDefaultCertificateForIngress(ca, "openshift-ingress-canary", canaryRef, ingress); err != nil {


Doesn't there need to be a different function to ensure a canary cert rather than ensure a default cert? Does this ensure the correct cert?

The intent is that the canary application should use the default IngressController's default certificate. This way, as long as the default IngressController has a correctly configured default certificate, so too will the canary application. Because #978 changed the canary application to use a TLS passthrough route, the only way to have the canary application use the default IngressController's default certificate is to copy that certificate to the canary application's namespace and configure the application to use that copy of the certificate.

If I understand correctly, ensureDefaultCertificateForIngress actually generates a new server certificate, using the existing CA certificate, so this logic doesn't quite implement the intent. We could use r.ensureDefaultCertificateForIngress(ca, ingress.Namespace, ref, ingress) (note the namespace) to get the existing certificate (or create it if it's missing), but it seems simpler just to do a Get from ingress.Namespace and then Create in "openshift-ingress-canary".

candita · 2025-01-24T20:37:10Z

/restest-required

candita · 2025-02-05T18:02:58Z

Failure in infra:
--- FAIL: TestNodePool/HostedCluster2/Main/TestAdditionalTrustBundlePropagation (2271.02s)

/test e2e-hypershift

openshift-bot · 2025-08-15T01:00:24Z

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

openshift-bot · 2025-09-14T08:32:32Z

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

openshift-bot · 2025-10-27T00:00:38Z

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

openshift-ci · 2025-10-27T00:01:17Z

@openshift-bot: Closed this PR.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci-robot · 2025-10-27T00:01:18Z

@rfredette: This pull request references Jira Issue OCPBUGS-9037. The bug has been updated to no longer refer to the pull request using the external bug tracker.

In response to this:

Utilize the existing ingress controller certificate management controller to also manage the certificate for the ingress canary, and use that certificate when serving the canary endpoint.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

rfredette · 2025-11-04T15:51:51Z

/reopen

openshift-ci · 2025-11-04T15:52:03Z

@rfredette: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci-robot · 2025-11-04T15:52:06Z

@rfredette: This pull request references Jira Issue OCPBUGS-9037, which is invalid:

expected the bug to target either version "4.21." or "openshift-4.21.", but it targets "4.17.z" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Utilize the existing ingress controller certificate management controller to also manage the certificate for the ingress canary, and use that certificate when serving the canary endpoint.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2025-11-04T15:53:02Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from candita. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

rfredette · 2025-11-06T20:27:35Z

/remove-lifecycle rotten

Miciah · 2025-11-06T22:14:25Z

pkg/operator/controller/certificate/controller.go

+						log.Info("canary daemonset not found; will retry default cert sync")
+						result.RequeueAfter = 5 * time.Second
+					} else {
+						errs = append(errs, fmt.Errorf("failed to get daemonset: %v", err))


Use %w when wrapping errors, on line 120 and on line 132.

Miciah · 2025-11-06T22:19:08Z

pkg/operator/controller/certificate/controller.go

 				errs = append(errs, fmt.Errorf("failed to ensure default cert for %s: %v", ingress.Name, err))
 			}
+			if ingress.Name == manifests.DefaultIngressControllerName {
+				log.Info("ensuring canary certificate")


Suggested change

log.Info("ensuring canary certificate")

log.Info("Ensuring canary certificate")

See https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md#message-style-guidelines.

Miciah · 2025-11-06T22:20:07Z

pkg/operator/controller/certificate/controller.go

 			if _, err := r.ensureDefaultCertificateForIngress(ca, deployment.Namespace, deploymentRef, ingress); err != nil {
 				errs = append(errs, fmt.Errorf("failed to ensure default cert for %s: %v", ingress.Name, err))
 			}
+			if ingress.Name == manifests.DefaultIngressControllerName {


Please add a comment to explain why this logic only applies to the "default" IngressController. Namely, the canary check is specifically for verifying that the default IngressController is ready.

openshift-ci-robot · 2025-11-13T15:39:21Z

@rfredette: This pull request references Jira Issue OCPBUGS-9037, which is invalid:

expected the bug to target either version "4.21." or "openshift-4.21.", but it targets "4.17.z" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

This pull request references Jira Issue OCPBUGS-64565, which is invalid:

expected the bug to target the "4.21.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Utilize the existing ingress controller certificate management controller to also manage the certificate for the ingress canary, and use that certificate when serving the canary endpoint.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Miciah · 2025-11-19T00:30:19Z

/jira refresh

openshift-ci-robot · 2025-11-19T00:30:26Z

@Miciah: This pull request references Jira Issue OCPBUGS-9037, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.21.0) matches configured target version for branch (4.21.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @lihongan

This pull request references Jira Issue OCPBUGS-64565, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.21.0) matches configured target version for branch (4.21.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @lihongan

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

candita · 2025-11-19T00:36:28Z

/retest e2e-aws-operator

openshift-ci · 2025-11-19T00:36:31Z

@candita: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

/test e2e-aws-operator

/test e2e-aws-ovn

/test e2e-aws-ovn-hypershift-conformance

/test e2e-aws-ovn-serial-1of2

/test e2e-aws-ovn-serial-2of2

/test e2e-aws-ovn-upgrade

/test e2e-azure-operator

/test e2e-gcp-operator

/test e2e-hypershift

/test hypershift-e2e-aks

/test images

/test okd-scos-images

/test unit

/test verify

/test verify-deps

The following commands are available to trigger optional jobs:

/test e2e-aws-gatewayapi-conformance

/test e2e-aws-operator-techpreview

/test e2e-aws-ovn-single-node

/test e2e-aws-ovn-techpreview

/test e2e-aws-pre-release-ossm

/test e2e-azure-manual-oidc

/test e2e-azure-ovn

/test e2e-gcp-ovn

/test e2e-ibmcloud-operator

/test e2e-openstack-operator

/test okd-scos-e2e-aws-ovn

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-cluster-ingress-operator-master-e2e-aws-operator

pull-ci-openshift-cluster-ingress-operator-master-e2e-aws-ovn

pull-ci-openshift-cluster-ingress-operator-master-e2e-aws-ovn-hypershift-conformance

pull-ci-openshift-cluster-ingress-operator-master-e2e-aws-ovn-serial-1of2

pull-ci-openshift-cluster-ingress-operator-master-e2e-aws-ovn-serial-2of2

pull-ci-openshift-cluster-ingress-operator-master-e2e-aws-ovn-upgrade

pull-ci-openshift-cluster-ingress-operator-master-e2e-aws-pre-release-ossm

pull-ci-openshift-cluster-ingress-operator-master-e2e-azure-operator

pull-ci-openshift-cluster-ingress-operator-master-e2e-gcp-operator

pull-ci-openshift-cluster-ingress-operator-master-e2e-hypershift

pull-ci-openshift-cluster-ingress-operator-master-hypershift-e2e-aks

pull-ci-openshift-cluster-ingress-operator-master-images

pull-ci-openshift-cluster-ingress-operator-master-okd-scos-images

pull-ci-openshift-cluster-ingress-operator-master-unit

pull-ci-openshift-cluster-ingress-operator-master-verify

pull-ci-openshift-cluster-ingress-operator-master-verify-deps

In response to this:

/retest e2e-aws-operator

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

candita · 2025-11-19T00:38:07Z

/retest

lihongan · 2025-11-19T02:19:27Z

The bug OCPBUGS-9037 mentioned console health check as well, seems this fix is just for ingress canary, I'm wondering how to fix the console route ?

openshift-ci · 2025-11-19T03:19:45Z

@rfredette: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-ovn-serial	`a7792fa`	link	true	`/test e2e-aws-ovn-serial`
ci/prow/e2e-aws-ovn-hypershift-conformance	`768f6b8`	link	true	`/test e2e-aws-ovn-hypershift-conformance`
ci/prow/hypershift-e2e-aks	`768f6b8`	link	true	`/test hypershift-e2e-aks`
ci/prow/e2e-hypershift	`768f6b8`	link	true	`/test e2e-hypershift`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci bot requested review from Miciah and knobunc October 15, 2024 15:34

rfredette force-pushed the ocpbugs-9037-use-cluster-wildcard branch from 4f7f29d to a7792fa Compare October 15, 2024 16:38

openshift-ci bot assigned candita and Miciah Nov 20, 2024

candita reviewed Nov 26, 2024

View reviewed changes

openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 15, 2025

openshift-ci bot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Sep 14, 2025

Miciah mentioned this pull request Oct 8, 2025

OCPBUGS-58145: reload serving cert on rotation #1285

Open

openshift-ci bot closed this Oct 27, 2025

openshift-ci bot reopened this Nov 4, 2025

Use the cluster wildcard certificate in ingress canary

768f6b8

rfredette force-pushed the ocpbugs-9037-use-cluster-wildcard branch from a7792fa to 768f6b8 Compare November 6, 2025 20:27

openshift-ci bot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Nov 6, 2025

Miciah reviewed Nov 6, 2025

View reviewed changes

rfredette changed the title ~~OCPBUGS-9037: Use cluster wildcard certificate for ingress canary~~ OCPBUGS-9037, OCPBUGS-64565: Use cluster wildcard certificate for ingress canary Nov 13, 2025

openshift-ci-robot added jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. and removed jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. labels Nov 13, 2025

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Nov 19, 2025

openshift-ci bot requested a review from lihongan November 19, 2025 00:30

	# name and namespace are set at runtime.
	# metadata values are set at runtime.

	daemonset.Spec.Template.Spec.Volumes[0].Secret.SecretName = secretName
	daemonset.Spec.Template.Spec.Volumes[0].Secret.SecretName, err := canarySecretName(name.Namespace)

	log.Info("ensuring canary certificate")
	log.Info("Ensuring canary certificate")

OCPBUGS-9037, OCPBUGS-64565: Use cluster wildcard certificate for ingress canary #1155

Are you sure you want to change the base?

OCPBUGS-9037, OCPBUGS-64565: Use cluster wildcard certificate for ingress canary #1155

Uh oh!

Conversation

rfredette commented Oct 15, 2024

Uh oh!

openshift-ci-robot commented Oct 15, 2024

Uh oh!

rfredette commented Oct 15, 2024

Uh oh!

candita commented Nov 20, 2024

Uh oh!

candita commented Nov 20, 2024

Uh oh!

candita left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

candita commented Jan 24, 2025

Uh oh!

candita commented Feb 5, 2025

Uh oh!

openshift-bot commented Aug 15, 2025

Uh oh!

openshift-bot commented Sep 14, 2025

Uh oh!

openshift-bot commented Oct 27, 2025

Uh oh!

openshift-ci bot commented Oct 27, 2025

Uh oh!

openshift-ci-robot commented Oct 27, 2025

Uh oh!

rfredette commented Nov 4, 2025

Uh oh!

openshift-ci bot commented Nov 4, 2025

Uh oh!

openshift-ci-robot commented Nov 4, 2025

Uh oh!

openshift-ci bot commented Nov 4, 2025

Uh oh!

rfredette commented Nov 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Nov 13, 2025

Uh oh!

Miciah commented Nov 19, 2025

Uh oh!

openshift-ci-robot commented Nov 19, 2025

Uh oh!

candita commented Nov 19, 2025

Uh oh!

openshift-ci bot commented Nov 19, 2025

Uh oh!

candita commented Nov 19, 2025

Uh oh!

lihongan commented Nov 19, 2025

Uh oh!

openshift-ci bot commented Nov 19, 2025

Uh oh!

Reviewers

candita left a comment •

edited

Loading