The Principal does not work with an opaque secret for argocd-agent-ca

[gnunn1]

Describe the bug

The Principal and Agent both require the argocd-agent-ca secret in order to verify the TLS certificates being used by the mTLS. The default in the documentation is for this to be a TLS secret with the CA in tls.crt and the key in tls.key however the reality is that users with their own CA will not want to provide a key.

The Agent works fine with an Opaque secret as follows:

apiVersion: v1
data:
 ca.crt: XXXXX
kind: Secret
metadata:
 name: argocd-agent-ca
 namespace: argocd-agent
type: Opaque

However using this on the Principal results in the Agent pod failing with the following repeating message:

time="2026-02-04T18:25:45Z" level=info msg="Outgoing unary call to /authapi.Authentication/Authenticate" module=Connector
time="2026-02-04T18:25:45Z" level=warning msg="Auth failure: rpc error: code = Unavailable desc = connection error: desc = \"error reading server preface: remote error: tls: unknown certificate authority\" (retrying in 1s)"
time="2026-02-04T18:25:46Z" level=info msg="Outgoing unary call to /authapi.Authentication/Authenticate" module=Connector
time="2026-02-04T18:25:46Z" level=warning msg="Auth failure: rpc error: code = Unavailable desc = connection error: desc = \"error reading server preface: remote error: tls: unknown certificate authority\" (retrying in 1.2s)"

Steps to reproduce the behaviour

1. Use an opaque secret for argocd-agent-ca with the key ca.crt

Expected behavior

The agent should be able to successfully authenticate with the principal

[tleveque69]

I had the exact same issue, it prevents the flawless deployment of the principal. Could we either unify in the code how agents and principal mounts the CA ?
Or should we change how it is deployed in kubernetes ?

Also I would be glad to Do a PR if you allow / guide me a little

[jannfis]

@tleveque69 The CA isn't actually mounted in the default installation. Instead, the principal uses the Kubernetes API to read the value from a secret. @gnunn1 and I discussed yesterday that it would probably be good to read the CA certificate from either key ca.crt (to streamline) and tls.crt (to keep existing installations working).

The code that loads certificates from a secret and builds an x509 cert pool is here. Right now, it either loads from one field in the secret (if field is non-empty) or from all fields (if field) is empty. My idea was to extend this function to accept a list of fields instead, and change the semantics as follows:

• If fields is empty, load data from all keys found in a secret
• If fields has a length of 1, and fields[0] == "", behave as if fields would be empty (basically, support former semantics)
• If fields is non-empty, load data from all keys specified in fields. Ignore fields not in the secret but in the list.

I was thinking of changing the signature of the function to become something like

func X509CertPoolFromSecret(ctx context.Context, kube kubernetes.Interface, namespace, name string, fields ...string) (*x509.CertPool, error)

And then update the tests, and change the caller in the principal to pass tls.crt and ca.crt when calling this function.

I have already written some code, but if you would like to give it a shot, you're more than welcome!

[tleveque69]

Haaaaaaaa ok my ca was also obviously in field ca.crt, I really got confused because it doesn't work the same way on the agent and the principal, i know that there are different components of argocd-agent but they seems to have some common way of working here. I think the caller should be unified on this. If i'm not mistaken the agent has a the same approach except it call the function with field being empty .
What do you think ?

[gnunn1]

Just to be clear as I forgot to mention it in the original post, the other thing the fix should incoporate is for the both the principal and agent to log if there is a problem loading the argocd-agent-ca secret. Part of what made this difficult to track down is the principal logged that it loaded the secret when it did not really load the secret.

And just to re-confirm what @jannfis said, using an opaque secret with the tls.crt field works everywhere. I do think it should support both fields, ca.crt and tls.crt, since these appear to be used interchangeably in Kubernetes.

[jannfis]

If i'm not mistaken the agent has a the same approach except it call the function with field being empty .

Yes, the agent loads the x509 certificates with an empty field-spec, and thereby is able to load multiple certificates. The reason for this is simple: If you are about to change the principal's certificate for whatever reason, and it will be signed by another CA, you can pre-load the new CA to all of your agents before you make the change on the principal and the agent will accept both, old and new. Then, you can change the certificate on the principal without service interruption.

[tleveque69]

Got it , thanks for taking the time to explain 🙏 .
I don't think i would have time to set up the repo and dev in it 😞 , also i'm not such a go developer sorry.