Feature: Automatically trust CA in containerd

[RafPe]

Have you searched for this feature request?

• I searched but did not find similar requests

Problem Statement

I am doing exploration of building/hosting and using images internally ( in the cluster created ) via the gitea image registry.

What happens right now out of the box ( when running default idpbuilder without any configuration changes ) we of course would get error about unverified authority i.e.

  Warning  Failed     34s (x4 over 112s)     kubelet            Failed to pull image "gitea.raftech.localtest.me:8443/foobar/echo:v1.0.0": failed to pull and unpack image "gitea.raftech.localtest.me:8443/foobar/echo:v1.0.0": failed to resolve reference "gitea.raftech.localtest.me:8443/foobar/echo:v1.0.0": failed to do request: Head "https://gitea.raftech.localtest.me/v2/foobar/echo/manifests/v1.0.0?ns=gitea.raftech.localtest.me%3A8443": tls: failed to verify certificate: x509: certificate signed by unknown authority
  Warning  Failed     34s (x4 over 112s)     kubelet            Error: ErrImagePull

If this is something we would like to pursue I would be happy to try to contribute under guidance of the maintainers 👍

Possible Solution

My suggestion would be that we could ( since we control all the steps of the build ):

1. Create folder under /etc/containerd/certs.d/gitea.<hostname>.localtest.me:8443/ca.crt
2. Populate it with the CA we have self generated

This out of the box gives the ability to use without any problems images hosted in Gitea registry

Alternatives Considered

The only option I see to overcome this would be to provide containerd overrides via cluster config during cluster creation

[squidboylan]

Hi, this should be fixed by #542 which is available in 0.10.1 . However instead of trusting the ca it allows the registry to be insecure, if we could think of an simple way to properly trust the registry that would be a great improvement.

[RafPe]

@squidboylan I just downloaded the newest version and its skipping the verification as you mentioned.

Long term run trusting the CA cert would be great - I am just not sure what would be an easy way for us to stream it into every node.
Right now as post install setup I exported the cert via kubectl get secret -n default idpbuilder-cert -o yaml -o jsonpath="{.data['ca\.crt']}" | base64 -d | pbcopy and from there I added it to the custom CA file under my registry configuration in containerd ( /etc/containerd/certs.d/gitea.raftech.localtest.me:8443/ca.crt ). With that I was then able to modify the contianer.d configuration to use that CA via

ca ="/etc/containerd/certs.d/gitea.raftech.localtest.me:8443/ca.crt"

On top of that we also have situation where we have custom kind configurations ( i.e. https://github.com/raftechio/cnoe-stacks-custom/blob/main/kind-config/cluster-with-nfs.yml ) and that presents challenge as we do not have the dynamic mount of the certs.d directory as in the default setup

  extraMounts:
  - containerPath: /etc/containerd/certs.d
    hostPath: /var/folders/70/ktz7hyp16md5zq_1_py5dh9w0000gn/T/idpbuilder-registry-certs.d-95379839
containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry]
    config_path = "/etc/containerd/certs.d"

So I am also looking for a way to properly automate/simplify the process as there seems to be more that needs to happen for the setup to work in more advanced scenarios