Skip to content

Conversation

@jianzhangbjz
Copy link
Member

Description

If pullCasDir is "", then DockerCertPath and OCICertPath are explicitly set to "", which causes containers/image not to read /etc/docker/certs.d/. So, overwrites the CA directory only when pullCasDir is not empty.

srcContext := &types.SystemContext{
    DockerCertPath: pullCasDir,  // pullCasDir = ""
    OCICertPath:    pullCasDir,
}

Test pass, see downstream PR: openshift/operator-framework-operator-controller#264

Reviewer Checklist

  • API Go Documentation
  • Tests: Unit Tests (and E2E Tests, if appropriate)
  • Comprehensive Commit Messages
  • Links to related GitHub Issue(s)

@jianzhangbjz jianzhangbjz requested a review from a team as a code owner February 7, 2025 05:32
@netlify
Copy link

netlify bot commented Feb 7, 2025

Deploy Preview for olmv1 ready!

Name Link
🔨 Latest commit 2b8ab5b
🔍 Latest deploy log https://app.netlify.com/sites/olmv1/deploys/67a59ae1ad927e00081d8c7b
😎 Deploy Preview https://deploy-preview-1727--olmv1.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@jianzhangbjz jianzhangbjz changed the title overwrites the CA directory only when pullCasDir is not empty 🐛, patch/bug fix: overwrites the CA directory only when pullCasDir is not empty Feb 7, 2025
@jianzhangbjz jianzhangbjz changed the title 🐛, patch/bug fix: overwrites the CA directory only when pullCasDir is not empty 🐛 Fix: overwrites the CA directory only when pullCasDir is not empty Feb 7, 2025
@jianzhangbjz
Copy link
Member Author

/assign @tmshort

@codecov
Copy link

codecov bot commented Feb 7, 2025

Codecov Report

Attention: Patch coverage is 50.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 67.86%. Comparing base (f6b1130) to head (2b8ab5b).
Report is 37 commits behind head on main.

Files with missing lines Patch % Lines
catalogd/cmd/catalogd/main.go 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1727      +/-   ##
==========================================
- Coverage   67.99%   67.86%   -0.13%     
==========================================
  Files          59       59              
  Lines        4993     4995       +2     
==========================================
- Hits         3395     3390       -5     
- Misses       1358     1363       +5     
- Partials      240      242       +2     
Flag Coverage Δ
e2e 53.01% <100.00%> (-0.24%) ⬇️
unit 55.07% <0.00%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@camilamacedo86 camilamacedo86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 7, 2025
@joelanford
Copy link
Member

I am struggling to see how this has any effect. The types of the DockerCertPath and OCICertPath fields are string, which means their zero values are "". We initialize an empty types.SystemContext in the line before this, so we know those fields are set to their zero values ("").

The pullCasDir variable is set by the flag, whose default value is "". So if that flag is unset, then the value we explicitly set in DockerCertPath and OCICertPath is "", which is what they already are (as described above).

What am I missing?

@tmshort
Copy link
Contributor

tmshort commented Feb 7, 2025

Same here. It should not be any different.

@camilamacedo86
Copy link
Contributor

Hi @joelanford and @tmshort,

My bet here was that the code was not interpreting nil and "" at the same way.
Because @jianzhangbjz tested this here: openshift/operator-framework-operator-controller#264 and says that with the change works.

But looking the code: https://github.com/containers/image/blob/release-5.33/docker/docker_client.go#L149-L155 I also get confused.

@tmshort
Copy link
Contributor

tmshort commented Feb 7, 2025

My bet here was that the code was not interpreting nil and "" at the same way.

Everything is a string type. There's no nil involved here.

@jianzhangbjz
Copy link
Member Author

jianzhangbjz commented Feb 8, 2025

I tested it by using the latest payload(4.19.0-0.nightly-2025-02-07-133207) without this PR change, and it failed. Details:

  • launch cluster via the cluster-bot.
image

The cluster is https://prow.ci.openshift.org/view/gs/test-platform-results/logs/release-openshift-origin-installer-launch-aws-modern/1888038029965135872

  • create the internal registry
jiazha-mac:~ jiazha$ oc new-project jian
Now using project "jian" on server "https://api.ci-ln-24ykcsb-76ef8.origin-ci-int-aws.dev.rhcloud.com:6443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app rails-postgresql-example

to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application:

    kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname

jiazha-mac:~ jiazha$ oc  new-app --image quay.io/openshifttest/registry@sha256:1106aedc1b2e386520bc2fb797d9a7af47d651db31d8e7ab472f2352da37d1b3 REGISTRY_STORAGE_DELETE_ENABLED=true --import-mode=PreserveOriginal
--> Found container image 1106aed (1 second old) from quay.io for "quay.io/openshifttest/registry@sha256:1106aedc1b2e386520bc2fb797d9a7af47d651db31d8e7ab472f2352da37d1b3"

    * An image stream tag will be created as "registry:latest" that will track this image

--> Creating resources ...
    imagestream.image.openshift.io "registry" created
    deployment.apps "registry" created
    service "registry" created
--> Success
    Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
     'oc expose service/registry' 
    Run 'oc status' to view your app.
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc create route edge my-route --service=registry
route.route.openshift.io/my-route created
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc get route
NAME       HOST/PORT                                                                  PATH   SERVICES   PORT       TERMINATION   WILDCARD
my-route   my-route-jian.apps.ci-ln-24ykcsb-76ef8.origin-ci-int-aws.dev.rhcloud.com          registry   5000-tcp   edge          None
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc set volume deploy registry --add -t pvc  --claim-size=30G -m /var/lib/registry --overwrite
deployment.apps/registry volume updated
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc create -n openshift-config configmap trusted-ca-73124 --from-file=my-route-jian.apps.xiyun419gcp-0208.qe.gcp.devcluster.openshift.com=/tmp/tls.crt --from-file=updateservice-registry=/tmp/tls.crt
configmap/trusted-ca-73124 created
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc  patch image.config.openshift.io/cluster -p '{"spec": {"additionalTrustedCA": {"name": "trusted-ca-73124"}}}' --type=merge
image.config.openshift.io/cluster patched
jiazha-mac:~ jiazha$ oc get pods
NAME                        READY   STATUS    RESTARTS   AGE
registry-5cbbd5c7bb-8x58b   1/1     Running   0          31s
  • create the clustercatalog
jiazha-mac:~ jiazha$ oc get route
NAME       HOST/PORT                                                                  PATH   SERVICES   PORT       TERMINATION   WILDCARD
my-route   my-route-jian.apps.ci-ln-24ykcsb-76ef8.origin-ci-int-aws.dev.rhcloud.com          registry   5000-tcp   edge          None
jiazha-mac:~ jiazha$ cat cc-internal-registry.yaml 
apiVersion: olm.operatorframework.io/v1
kind: ClusterCatalog
metadata:
  name: cc-redhat-operator-index-v4-17
spec:
  availabilityMode: Available
  priority: 0
  source:
    image:
      ref: my-route-jian.apps.ci-ln-24ykcsb-76ef8.origin-ci-int-aws.dev.rhcloud.com/redhat/redhat-operator-index:v4.17
    type: Image
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc create -f cc-internal-registry.yaml 
clustercatalog.olm.operatorframework.io/cc-redhat-operator-index-v4-17 created
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc get clustercatalog
NAME                             LASTUNPACKED   SERVING   AGE
cc-redhat-operator-index-v4-17                            7s
openshift-certified-operators    19m            True      20m
openshift-community-operators    19m            True      20m
openshift-redhat-marketplace     19m            True      20m
openshift-redhat-operators       19m            True      20m
jiazha-mac:~ jiazha$ oc get clustercatalog cc-redhat-operator-index-v4-17 -o yaml
apiVersion: olm.operatorframework.io/v1
kind: ClusterCatalog
metadata:
  creationTimestamp: "2025-02-08T02:26:56Z"
  finalizers:
  - olm.operatorframework.io/delete-server-cache
  generation: 1
  labels:
    olm.operatorframework.io/metadata.name: cc-redhat-operator-index-v4-17
  name: cc-redhat-operator-index-v4-17
  resourceVersion: "33257"
  uid: e6ea57ea-ffcf-4a6a-b900-d06c30438d63
spec:
  availabilityMode: Available
  priority: 0
  source:
    image:
      ref: my-route-jian.apps.ci-ln-24ykcsb-76ef8.origin-ci-int-aws.dev.rhcloud.com/redhat/redhat-operator-index:v4.17
    type: Image
status:
  conditions:
  - lastTransitionTime: "2025-02-08T02:26:56Z"
    message: 'source catalog content: error resolving canonical reference: error creating
      image source: pinging container registry my-route-jian.apps.ci-ln-24ykcsb-76ef8.origin-ci-int-aws.dev.rhcloud.com:
      Get "https://my-route-jian.apps.ci-ln-24ykcsb-76ef8.origin-ci-int-aws.dev.rhcloud.com/v2/":
      tls: failed to verify certificate: x509: certificate signed by unknown authority'
    observedGeneration: 1
    reason: Retrying
    status: "True"
    type: Progressing

@jianzhangbjz
Copy link
Member Author

jianzhangbjz commented Feb 8, 2025

I doubt this pullCasDir var is used in a somehow way during the cluster building. This PR change will be safer.

@tmshort
Copy link
Contributor

tmshort commented Feb 10, 2025

I doubt this pullCasDir var is used in a somehow way during the cluster building. This PR change will be safer.

This PR doesn't do anything; if you compare the values before and after, they are the same.

Copy link
Contributor

@tmshort tmshort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a no-op, and I have trouble believing this actually does anything without an explanation. Everything is string type here. The default value of string type is "", so setting (or not setting) to "" should be a no-op.

@jianzhangbjz
Copy link
Member Author

This is a no-op, and I have trouble believing this actually does anything without an explanation.

Hi @tmshort , yeah, however, as I showed above, without this PR, it fail. You can test it with the Downstream openshift/operator-framework-operator-controller#264 via the cluster-bot.

@jianzhangbjz
Copy link
Member Author

jianzhangbjz commented Feb 11, 2025

Test passes without this PR on the latest 4.19 payload cluster. This cluster wasn't created by the cluster-bot. However, it failed on the same version cluster was created by the cluster-bot! Details:

  • the cluster wasn't created by the cluster-bot, pass!
jiazha-mac:~ jiazha$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.19.0-0.nightly-2025-02-10-034243   True        False         3h44m   Cluster version is 4.19.0-0.nightly-2025-02-10-034243
jiazha-mac:~ jiazha$ oc new-project jian
Now using project "jian" on server "https://api.xiyun419gcp-0211.qe.gcp.devcluster.openshift.com:6443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app rails-postgresql-example

to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application:

    kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname

jiazha-mac:~ jiazha$ oc  new-app --image quay.io/openshifttest/registry@sha256:1106aedc1b2e386520bc2fb797d9a7af47d651db31d8e7ab472f2352da37d1b3 REGISTRY_STORAGE_DELETE_ENABLED=true --import-mode=PreserveOriginal
--> Found container image 1106aed (2 seconds old) from quay.io for "quay.io/openshifttest/registry@sha256:1106aedc1b2e386520bc2fb797d9a7af47d651db31d8e7ab472f2352da37d1b3"

    * An image stream tag will be created as "registry:latest" that will track this image

--> Creating resources ...
    imagestream.image.openshift.io "registry" created
    deployment.apps "registry" created
    service "registry" created
--> Success
    Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
     'oc expose service/registry' 
    Run 'oc status' to view your app.
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc create route edge my-route --service=registry
route.route.openshift.io/my-route created
jiazha-mac:~ jiazha$ oc get route
NAME       HOST/PORT                                                             PATH   SERVICES   PORT       TERMINATION   WILDCARD
my-route   my-route-jian.apps.xiyun419gcp-0211.qe.gcp.devcluster.openshift.com          registry   5000-tcp   edge          None
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc set volume deploy registry --add -t pvc  --claim-size=30G -m /var/lib/registry --overwrite
deployment.apps/registry volume updated
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc extract secret/router-ca -n openshift-ingress-operator --to=/tmp --confirm 
/tmp/tls.crt
/tmp/tls.key
jiazha-mac:~ jiazha$ oc create -n openshift-config configmap trusted-ca-73124 --from-file=my-route-jian.apps.xiyun419gcp-0211.qe.gcp.devcluster.openshift.com=/tmp/tls.crt --from-file=updateservice-registry=/tmp/tls.crt
configmap/trusted-ca-73124 created
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc  patch image.config.openshift.io/cluster -p '{"spec": {"additionalTrustedCA": {"name": "trusted-ca-73124"}}}' --type=merge
image.config.openshift.io/cluster patched

jiazha-mac:~ jiazha$ vim cc-internal-registry.yaml 
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc get clustercatalog
NAME                            LASTUNPACKED   SERVING   AGE
openshift-certified-operators   57m            True      63m
openshift-community-operators   57m            True      63m
openshift-redhat-marketplace    57m            True      63m
openshift-redhat-operators      37m            True      63m
jiazha-mac:~ jiazha$ oc create -f cc-internal-registry.yaml
clustercatalog.olm.operatorframework.io/cc-redhat-operator-index-v4-17 created
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc get clustercatalog
NAME                             LASTUNPACKED   SERVING   AGE
cc-redhat-operator-index-v4-17                            10s
openshift-certified-operators    58m            True      63m
openshift-community-operators    58m            True      63m
openshift-redhat-marketplace     57m            True      63m
openshift-redhat-operators       38m            True      63m
jiazha-mac:~ jiazha$ oc get clustercatalog cc-redhat-operator-index-v4-17 -o yaml
apiVersion: olm.operatorframework.io/v1
kind: ClusterCatalog
metadata:
  creationTimestamp: "2025-02-11T01:41:40Z"
  finalizers:
  - olm.operatorframework.io/delete-server-cache
  generation: 1
  labels:
    olm.operatorframework.io/metadata.name: cc-redhat-operator-index-v4-17
  name: cc-redhat-operator-index-v4-17
  resourceVersion: "48842"
  uid: 4f91a68b-63df-4c50-9527-d21572c47ed5
spec:
  availabilityMode: Available
  priority: 0
  source:
    image:
      ref: my-route-jian.apps.xiyun419gcp-0211.qe.gcp.devcluster.openshift.com/redhat/redhat-operator-index:v4.17
    type: Image
status:
  conditions:
  - lastTransitionTime: "2025-02-11T01:41:40Z"
    message: 'source catalog content: error resolving canonical reference: error creating
      image source: reading manifest v4.17 in my-route-jian.apps.xiyun419gcp-0211.qe.gcp.devcluster.openshift.com/redhat/redhat-operator-index:
      manifest unknown'
    observedGeneration: 1
    reason: Retrying
    status: "True"
    type: Progressing
jiazha-mac:~ jiazha$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.19.0-0.nightly-2025-02-10-034243   True        False         2m16s   Cluster version is 4.19.0-0.nightly-2025-02-10-034243
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc new-project jian
Now using project "jian" on server "https://api.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org:6443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app rails-postgresql-example

to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application:

    kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname

jiazha-mac:~ jiazha$ oc project
Using project "jian" on server "https://api.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org:6443".
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc  new-app --image quay.io/openshifttest/registry@sha256:1106aedc1b2e386520bc2fb797d9a7af47d651db31d8e7ab472f2352da37d1b3 REGISTRY_STORAGE_DELETE_ENABLED=true --import-mode=PreserveOriginal
--> Found container image 1106aed (2 seconds old) from quay.io for "quay.io/openshifttest/registry@sha256:1106aedc1b2e386520bc2fb797d9a7af47d651db31d8e7ab472f2352da37d1b3"

    * An image stream tag will be created as "registry:latest" that will track this image

--> Creating resources ...
    imagestream.image.openshift.io "registry" created
    deployment.apps "registry" created
    service "registry" created
--> Success
    Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
     'oc expose service/registry' 
    Run 'oc status' to view your app.
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc create route edge my-route --service=registry
route.route.openshift.io/my-route created
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc get route
NAME       HOST/PORT                                                       PATH   SERVICES   PORT       TERMINATION   WILDCARD
my-route   my-route-jian.apps.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org          registry   5000-tcp   edge          None
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc set volume deploy registry --add -t pvc  --claim-size=30G -m /var/lib/registry --overwrite
deployment.apps/registry volume updated
jiazha-mac:~ jiazha$ oc extract secret/router-ca -n openshift-ingress-operator --to=/tmp --confirm 
/tmp/tls.crt
/tmp/tls.key
jiazha-mac:~ jiazha$ oc create -n openshift-config configmap trusted-ca-73124 --from-file=my-route-jian.apps.xiyun419gcp-0211.qe.gcp.devcluster.openshift.com=/tmp/tls.crt --from-file=updateservice-registry=/tmp/tls.crt
configmap/trusted-ca-73124 created
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc  patch image.config.openshift.io/cluster -p '{"spec": {"additionalTrustedCA": {"name": "trusted-ca-73124"}}}' --type=merge
image.config.openshift.io/cluster patched
jiazha-mac:~ jiazha$ vim cc-internal-registry.yaml 
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc create -f cc-internal-registry.yaml 
clustercatalog.olm.operatorframework.io/cc-redhat-operator-index-v4-17 created
jiazha-mac:~ jiazha$ oc get clustercatalog
NAME                             LASTUNPACKED   SERVING   AGE
cc-redhat-operator-index-v4-17                            5s
openshift-certified-operators    17m            True      18m
openshift-community-operators    18m            True      18m
openshift-redhat-marketplace     17m            True      18m
openshift-redhat-operators       18m            True      18m
jiazha-mac:~ jiazha$ oc get clustercatalog cc-redhat-operator-index-v4-17 -o yaml
apiVersion: olm.operatorframework.io/v1
kind: ClusterCatalog
metadata:
  creationTimestamp: "2025-02-11T02:28:22Z"
  finalizers:
  - olm.operatorframework.io/delete-server-cache
  generation: 1
  labels:
    olm.operatorframework.io/metadata.name: cc-redhat-operator-index-v4-17
  name: cc-redhat-operator-index-v4-17
  resourceVersion: "34299"
  uid: a3055882-91a9-470f-bf0c-33dbfa0f3bb1
spec:
  availabilityMode: Available
  priority: 0
  source:
    image:
      ref: my-route-jian.apps.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org/redhat/redhat-operator-index:v4.17
    type: Image
status:
  conditions:
  - lastTransitionTime: "2025-02-11T02:28:22Z"
    message: 'source catalog content: error resolving canonical reference: error creating
      image source: pinging container registry my-route-jian.apps.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org:
      Get "https://my-route-jian.apps.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org/v2/":
      tls: failed to verify certificate: x509: certificate signed by unknown authority'
    observedGeneration: 1
    reason: Retrying
    status: "True"
    type: Progressing

When you run the test again, after you add the additional CA, could you restart the operator-controller and catalogd pods to see they then pick up the additional CA? (If it hadn’t already?)

And, it still get x509 fail after recreating openshift-operator-controller and openshift-catalogd, as follows,

jiazha-mac:~ jiazha$ oc delete pods --all -n openshift-catalogd
pod "catalogd-controller-manager-64c7d4586-jqqxx" deleted

jiazha-mac:~ jiazha$ oc delete pods --all -n openshift-operator-controller 
pod "operator-controller-controller-manager-bcf9bf5fd-crq6x" deleted

jiazha-mac:~ jiazha$ oc get pods -n openshift-operator-controller
NAME                                                     READY   STATUS    RESTARTS   AGE
operator-controller-controller-manager-bcf9bf5fd-829kd   1/1     Running   0          21s
jiazha-mac:~ jiazha$ oc get pods -n openshift-catalogd
NAME                                          READY   STATUS    RESTARTS   AGE
catalogd-controller-manager-64c7d4586-74276   1/1     Running   0          45s
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc get clustercatalog cc-redhat-operator-index-v4-17 -o yaml
apiVersion: olm.operatorframework.io/v1
kind: ClusterCatalog
metadata:
  creationTimestamp: "2025-02-11T02:28:22Z"
  finalizers:
  - olm.operatorframework.io/delete-server-cache
  generation: 1
  labels:
    olm.operatorframework.io/metadata.name: cc-redhat-operator-index-v4-17
  name: cc-redhat-operator-index-v4-17
  resourceVersion: "34299"
  uid: a3055882-91a9-470f-bf0c-33dbfa0f3bb1
spec:
  availabilityMode: Available
  priority: 0
  source:
    image:
      ref: my-route-jian.apps.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org/redhat/redhat-operator-index:v4.17
    type: Image
status:
  conditions:
  - lastTransitionTime: "2025-02-11T02:28:22Z"
    message: 'source catalog content: error resolving canonical reference: error creating
      image source: pinging container registry my-route-jian.apps.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org:
      Get "https://my-route-jian.apps.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org/v2/":
      tls: failed to verify certificate: x509: certificate signed by unknown authority'
    observedGeneration: 1
    reason: Retrying
    status: "True"
    type: Progressing

Recreate it, still get x509 failure.

jiazha-mac:~ jiazha$ oc delete -f cc-internal-registry.yaml 
clustercatalog.olm.operatorframework.io "cc-redhat-operator-index-v4-17" deleted
jiazha-mac:~ jiazha$ oc get clustercatalog
NAME                            LASTUNPACKED   SERVING   AGE
openshift-certified-operators   6s             True      24m
openshift-community-operators   45s            True      24m
openshift-redhat-marketplace    35s            True      24m
openshift-redhat-operators      23s            True      24m
jiazha-mac:~ jiazha$ oc create -f cc-internal-registry.yaml 
clustercatalog.olm.operatorframework.io/cc-redhat-operator-index-v4-17 created

jiazha-mac:~ jiazha$ oc get clustercatalog cc-redhat-operator-index-v4-17 -o yaml
apiVersion: olm.operatorframework.io/v1
kind: ClusterCatalog
metadata:
  creationTimestamp: "2025-02-11T02:34:55Z"
  finalizers:
  - olm.operatorframework.io/delete-server-cache
  generation: 1
  labels:
    olm.operatorframework.io/metadata.name: cc-redhat-operator-index-v4-17
  name: cc-redhat-operator-index-v4-17
  resourceVersion: "36339"
  uid: 37301060-bb62-4f28-a257-cfa3b88335f3
spec:
  availabilityMode: Available
  priority: 0
  source:
    image:
      ref: my-route-jian.apps.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org/redhat/redhat-operator-index:v4.17
    type: Image
status:
  conditions:
  - lastTransitionTime: "2025-02-11T02:34:55Z"
    message: 'source catalog content: error resolving canonical reference: error creating
      image source: pinging container registry my-route-jian.apps.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org:
      Get "https://my-route-jian.apps.ci-ln-17fp5vb-76ef8.aws-2.ci.openshift.org/v2/":
      tls: failed to verify certificate: x509: certificate signed by unknown authority'
    observedGeneration: 1
    reason: Retrying
    status: "True"
    type: Progressing

@jianzhangbjz
Copy link
Member Author

jianzhangbjz commented Feb 11, 2025

I guess the cluster hadn't finished the CAs syncing, but I'm unsure why.
Test passes after the cluster runs about 30mins. The same version cluster was created by the cluster-bot. Waiting it runs around 30 mins, test pass. Details:

jiazha-mac:~ jiazha$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.19.0-0.nightly-2025-02-10-034243   True        False         54m     Cluster version is 4.19.0-0.nightly-2025-02-10-034243
jiazha-mac:~ jiazha$ oc get clustercatalog
NAME                             LASTUNPACKED   SERVING   AGE
cc-redhat-operator-index-v4-17                            3m52s
openshift-certified-operators    64m            True      64m
openshift-community-operators    63m            True      64m
openshift-redhat-marketplace     63m            True      64m
openshift-redhat-operators       63m            True      64m

jiazha-mac:~ jiazha$ oc new-project jian
Now using project "jian" on server "https://api.ci-ln-bch310t-76ef8.aws-2.ci.openshift.org:6443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app rails-postgresql-example

to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application:

    kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname

jiazha-mac:~ jiazha$ oc  new-app --image quay.io/openshifttest/registry@sha256:1106aedc1b2e386520bc2fb797d9a7af47d651db31d8e7ab472f2352da37d1b3 REGISTRY_STORAGE_DELETE_ENABLED=true --import-mode=PreserveOriginal
--> Found container image 1106aed (2 seconds old) from quay.io for "quay.io/openshifttest/registry@sha256:1106aedc1b2e386520bc2fb797d9a7af47d651db31d8e7ab472f2352da37d1b3"

    * An image stream tag will be created as "registry:latest" that will track this image

--> Creating resources ...
    imagestream.image.openshift.io "registry" created
    deployment.apps "registry" created
    service "registry" created
--> Success
    Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
     'oc expose service/registry' 
    Run 'oc status' to view your app.
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc create route edge my-route --service=registry
route.route.openshift.io/my-route created
jiazha-mac:~ jiazha$ oc get route
NAME       HOST/PORT                                                       PATH   SERVICES   PORT       TERMINATION   WILDCARD
my-route   my-route-jian.apps.ci-ln-bch310t-76ef8.aws-2.ci.openshift.org          registry   5000-tcp   edge          None
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc set volume deploy registry --add -t pvc  --claim-size=30G -m /var/lib/registry --overwrite
deployment.apps/registry volume updated
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc extract secret/router-ca -n openshift-ingress-operator --to=/tmp --confirm 
/tmp/tls.crt
/tmp/tls.key
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc create -n openshift-config configmap trusted-ca-73124 --from-file=my-route-jian.apps.ci-ln-bch310t-76ef8.aws-2.ci.openshift.org=/tmp/tls.crt --from-file=updateservice-registry=/tmp/tls.crt
configmap/trusted-ca-73124 created
jiazha-mac:~ jiazha$ vim cc-internal-registry.yaml 
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc  patch image.config.openshift.io/cluster -p '{"spec": {"additionalTrustedCA": {"name": "trusted-ca-73124"}}}' --type=merge
image.config.openshift.io/cluster patched
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc create -f cc-internal-registry.yaml 
clustercatalog.olm.operatorframework.io/cc-redhat-operator-index-v4-17 created
jiazha-mac:~ jiazha$ 
jiazha-mac:~ jiazha$ oc get clustercatalog cc-redhat-operator-index-v4-17 -o yaml
apiVersion: olm.operatorframework.io/v1
kind: ClusterCatalog
metadata:
  creationTimestamp: "2025-02-11T06:48:24Z"
  finalizers:
  - olm.operatorframework.io/delete-server-cache
  generation: 1
  labels:
    olm.operatorframework.io/metadata.name: cc-redhat-operator-index-v4-17
  name: cc-redhat-operator-index-v4-17
  resourceVersion: "41438"
  uid: 1c346d57-a47d-4cc6-88a3-78b57fd995f1
spec:
  availabilityMode: Available
  priority: 0
  source:
    image:
      ref: my-route-jian.apps.ci-ln-bch310t-76ef8.aws-2.ci.openshift.org/redhat/redhat-operator-index:v4.17
    type: Image
status:
  conditions:
  - lastTransitionTime: "2025-02-11T06:48:24Z"
    message: 'source catalog content: error resolving canonical reference: error creating
      image source: reading manifest v4.17 in my-route-jian.apps.ci-ln-bch310t-76ef8.aws-2.ci.openshift.org/redhat/redhat-operator-index:
      manifest unknown'
    observedGeneration: 1
    reason: Retrying
    status: "True"
    type: Progressing

@jianzhangbjz
Copy link
Member Author

I closed this PR since the cluster did work without it, but needs more much time.

@jianzhangbjz jianzhangbjz reopened this Feb 17, 2025
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 17, 2025
@openshift-merge-robot
Copy link

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm Indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants