Custom webhook tls error: bad certificate

[NickYadance]

I setup a simple interpreter webhook for my CRD and deploy it outside the Karmada cluster. The webhook runs well but cannot connect to karmada with TLS handshake error.

2023-02-17T17:21:04+08:00 warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted)
I0217 17:21:09.685827  545599 webhook.go:84] Registering webhooks to the webhook server
I0217 17:21:09.686055  545599 internal.go:366] "Starting server" path="/metrics" kind="metrics" addr="[::]:9090"
I0217 17:22:20.976002  545599 log.go:198] http: TLS handshake error from xx:18884: remote error: tls: bad certificate

I installed karmada with Helm in auto cert mode. The tls cert of the webhook is from karmada-webhook-cert secret.

tls.crt == decode(tls.crt of karmada-webhook-cert)
tls.key == decode(tls.key of karmada-webhook-cert)

The caBundle is equal to the server-ca.crt of karmada-cert.

---
apiVersion: config.karmada.io/v1alpha1
kind: ResourceInterpreterWebhookConfiguration
metadata:
  name: mycrd
webhooks:
  ...
    clientConfig:
      caBundle: 
# equal to (server-ca.crt of secret karmada-cert)
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURtVENDQW9HZ0F3SUJBZ0lVSTJtSjkzS1J0eU5Vb25USHhRQTNCc2JHSDNFd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1hERUxNQWtHQTFVRUJoTUNlSGd4Q2pBSUJnTlZCQWdNQVhneENqQUlCZ05WQkFjTUFYZ3hDakFJQmdOVgpCQW9NQVhneENqQUlCZ05WQkFzTUFYZ3hDekFKQmdOVkJBTU1BbU5oTVJBd0RnWUpLb1pJaHZjTkFRa0JGZ0Y0Ck1CNFhEVEl6TURJeE5EQTNNamN3T0ZvWERUSTBNREl4TkRBM01qY3dPRm93WERFTE1Ba0dBMVVFQmhNQ2VIZ3gKQ2pBSUJnTlZCQWdNQVhneENqQUlCZ05WQkFjTUFYZ3hDakFJQmdOVkJBb01BWGd4Q2pBSUJnTlZCQXNNQVhneApDekFKQmdOVkJBTU1BbU5oTVJBd0RnWUpLb1pJaHZjTkFRa0JGZ0Y0TUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGCkFBT0NBUThBTUlJQkNnS0NBUUVBNFZJWUlMaFRKdDJSL0lLVVY2U2daQU5iMCt3OFp5Z2RicmdCNXF3L0JvUVkKU0ljaktsWU01TjRkYkc0RC9YTXdCTnVuUm81bUhQeGNoMHJMQlVacjdpbTFVdkhtbEF0TWxsL2piOHM3dlFCbAo0RTVKVmw4ejdMR0ZtbUFRYVNka1dTYWwwd3R3UjVWQ0Ivc2hJWkVFVE5tb2R2KzdZeFFKL0ExeW8xems3aS9pCjZWa2dJVU5mVmZ3NmppRkd0clVqQk1EUmlFVm1nbisrZjA1Z2NtS3dQSjZWdURSZzlQZjU0dXgrV2JhUEl0cjgKRXF5VXVCR2U1Z1hFSEd3YlUrQjluOU9tZm1KeU92bE9MQUE1dnYxTk1YV2hKbTVybWFkQzgwQmkwcStlek9mcQp4L1ROek9qWEV1UldINjdMR28zYnhoQ2pENjU4bFhOMnlCTG5zZDdMTVFJREFRQUJvMU13VVRBZEJnTlZIUTRFCkZnUVU4K3pzS2FlQWtQNlZIR3N1YXBsWXlkc20zcEV3SHdZRFZSMGpCQmd3Rm9BVTgrenNLYWVBa1A2VkhHc3UKYXBsWXlkc20zcEV3RHdZRFZSMFRBUUgvQkFVd0F3RUIvekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBQ3dwbQpHRTlnenRhYTh1OTdjb2JySjNZaU5nUHVvbysvZkNvK3RYM3UyV2wxU2gvMXgrc1lmWGlVUHZrZEFHUCs2dEpyCnpDWGNmeUVrWm5vK0haZW0yTkNMTFQwUkRORDA4TC9RMXh1N2lhcWNBZDRMZXlDM0hZK0l5M1JuMk9SMy9XUzUKcktvbFBHY2U1QndSOTNIeUoyMEdKR2ZLSi81RGR0eDN0MEZLbHY4VnlQb1pUOXluV3d2aGZLbHRmM0VGU1BUaQpieDJYSms0NkQ4VzF1RFVVT3pDamhFekxkVklnVGZ5WjVUY3AySWt1Z0krbzVDMlROWFNkaUwzWUNlbTZUdTBYCnM1MUFvOTFQYkJkNkxLd2VIVU50L3gvN3hMdVcwdy9PZ0l0U1pVNWdpRnRZVE1CYkUveUl3VEhtanZ6MjhxMm8KcEpqZTZ0SWFHV3IrWkt6U3F3PT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=

I have carefully verified that the my webhook cert is exactly the same as in karmada-webhook pod generated by Helm, but it just keeps telling TLS errors. Any idea where things go wrong?

[XiShanYongYe-Chang]

About the certificate, I would like to invite @lonelyCZ to take a look.

[NickYadance]

thx for the quick reply. i just noticed that caBundle is described as PEM encoded CA bundle. But in example it is simplely the base64 encode of ca.crt and i used the base64 too. So which format is the correct one, base64 or pem ?

// `caBundle` is a PEM encoded CA bundle which will be used to validate the webhook's server certificate.
	// If unspecified, system trust roots on the apiserver are used.
	// +optional

[lonelyCZ]

http: TLS handshake error from xx:18884: remote error: tls: bad certificate

From that, it could be occured by a bad cert formant. you can check if tls.crt and tls.key are referenced correctly.

i just noticed that caBundle is described as PEM encoded CA bundle. But in example it is simplely the base64 encode of ca.crt and i used the base64 too. So which format is the correct one, base64 or pem ?

It should be base64 format.

[NickYadance]

thx for the reply @lonelyCZ

With being sure that i had the correct cert, i tried to curl to the webhook server directly in verbose mode and got illegal host error. Since i deployed my webhook outside the cluster, the cert needs to be regenerated with filling the host of webhook.

After that it works fine.