Feature to skip management cluster's ImageDigestMirrorsSets

[hasan4791]

When the management cluster itself is deployed by Hypershift with ImageDigestMirrors configured and if we're deploying a Hosted Cluster on this Management cluster, the ImageDigestMirrors are being used even though we haven't configured the same in the HC spec. In such scenario, if there are any private mirrors configured for the upstream quay, the deployment of hosted cluster is getting failed as its not able to pull from the mirror. Currently hypershift is always honoring the ImageDigestMirrors irrespective of the current Hosted Cluster spec. Looking for the suggestions for this feature to have a control flag based on which it can be decided whether to use mirrors or not.

[jparrill]

/cc @jparrill

[hasan4791]

@jparrill What is your suggestion on this one?

[jparrill]

Hey @hasan4791, Thanks for bringing this up :).

It's true that there was an issue about having mirrors and go to fallback once they don't work properly. This is solved already in the most recent versions of Hypershift and we are backporting them to the older OCP payloads right now:

• 4.17 - [release-4.17] OCPBUGS-57122,OCPBUGS-57123: Add support for registry root entry only in the IDMS/ICSP #6230
• 4.18 - [release-4.18] OCPBUGS-56166,OCPBUGS-56955: Add support for registry root entry only in the IDMS/ICSP #6163 and [release-4.18] OCPBUGS-56166,OCPBUGS-56955: Fix CatalogSource images check when unauthorized #6237
• 4.19 - [release-4.19] OCPBUGS-55693: Add support for registry root entry only in the IDMS/ICSP #6131 and [release-4.19] OCPBUGS-56792: Fix CatalogSource image check when unauthorized #6198
• 4.20 - OCPBUGS-54483: Add support for registry root entry only in the IDMS/ICSP #6100 and OCPBUGS-56492: Fix CatalogSource image check when unauthorized #6192

By design we extract the IDMS/ICSP from the MGMT cluster in order to download the OCP payload and extract it during multiples phases of the HC deployment but this is totally different from the ImageContentSources field in the HC API which is basically to install new IDMS into the HostedCluster for the customer workloads.

In order to know if this feature would be useful for multiple customers, could you elaborate what's your use case?. If this is only because the mirrored image is not there and the controller is not falling back into the original image, as I said, this is already solved and it's being backported. If there is any other different reason, please let me know and we can discuss it within the team to have a real justification of the implementation.

[hasan4791]

@jparrill We are using ROKS 4.18 from IBM Cloud as management cluster and this cluster is being managed by the Hypershift with IDMS configured in that. And we are deploying a cluster(Hosted Cluster) using the Hypershift for Power workers nodes, but we aren't able to pull the images from quay as mirrors are configured for the same in IDMS. We are using our own subscription for the Hosted cluster but not able to pull the image. This is the error we're hitting

{"level":"error","ts":"2025-05-28T13:09:02Z","msg":"Failed to look up release image using registry mirror","controller":"hostedcluster","controllerGroup":"hypershift.openshift.io","controllerKind":"HostedCluster","HostedCluster":{"name":"test-ibmpowervsimagename-fresh","namespace":"clusters"},"namespace":"clusters","name":"test-ibmpowervsimagename-fresh","reconcileID":"8c69fc2e-241e-4e65-b2b1-486105d24fbd","registry mirror":"private.us.icr.io/ocp-release","error":"failed to extract release metadata: failed to obtain root manifest for private.us.icr.io/ocp-release-nightly@sha256:b56987c176cf5a6f87917f3ba6b1402c6d3437b17ce980c67e924dfd4e2eb2db: manifest unknown: manifest unknown","stacktrace":"github.com/openshift/hypershift/support/releaseinfo.

[mainstreamtt]

Hey @jparrill,

We are trying to install workload on the HC by using Tags in a disconnected environment. So we are facing the issue that we can't provide a ImageTagMirrorSet at the cluster creation stage to use our enterprise registry. We also tried to apply the ITMS object after cluster creation, but without any success.

Can you point us into the right direction ?

Thanks
BR

[bryan-cox]

Hey @mainstreamtt 👋🏻 - only ImageDigestMirrorSet is supported on HC. We don't currently support tags.

[hasan4791]

@jparrill We are using ROKS 4.18 from IBM Cloud as management cluster and this cluster is being managed by the Hypershift with IDMS configured in that. And we are deploying a cluster(Hosted Cluster) using the Hypershift for Power workers nodes, but we aren't able to pull the images from quay as mirrors are configured for the same in IDMS. We are using our own subscription for the Hosted cluster but not able to pull the image. This is the error we're hitting

{"level":"error","ts":"2025-05-28T13:09:02Z","msg":"Failed to look up release image using registry mirror","controller":"hostedcluster","controllerGroup":"hypershift.openshift.io","controllerKind":"HostedCluster","HostedCluster":{"name":"test-ibmpowervsimagename-fresh","namespace":"clusters"},"namespace":"clusters","name":"test-ibmpowervsimagename-fresh","reconcileID":"8c69fc2e-241e-4e65-b2b1-486105d24fbd","registry mirror":"private.us.icr.io/ocp-release","error":"failed to extract release metadata: failed to obtain root manifest for private.us.icr.io/ocp-release-nightly@sha256:b56987c176cf5a6f87917f3ba6b1402c6d3437b17ce980c67e924dfd4e2eb2db: manifest unknown: manifest unknown","stacktrace":"github.com/openshift/hypershift/support/releaseinfo.

@jparrill Any idea regarding this error?