Issuer & Clusterissuer can not issue certificates

[sam-sre]

Hi,

We are considering EJBCA in our organization and currently running a PoC to test some features, but we encountered some issues and we look for guidance..

Problem:

The cert-manager issuer & clusterissuer can not connect to the ejbca instance.

Test Environment:

• RKE2 K8s cluster (v1.32.8)
• All nodes: Ubuntu 22.04LTS
• cert-manager deployed via helm (chart v. 1.18.2)
• cert-manager external issuer deployed like here
• Mariadb is connected to EJBCA instance. Manifest used here
• EJBCA deployed via helm using this chart (values file below) :
  helm install ejbca --set fullnameOverride=ejbca oci://repo.keyfactor.com/charts/ejbca-ce --namespace ejbca-ns

Details:

We followed the cer-manager tutorial here with some modifications (like mentioned below)

Both EJBCA and MariaDB are running in the same namespace ejbca-ns and in healthy state

ejbca-7f4b7f8854-nxgs6                   1/1     Running             0     1       n/a       n/a    954       n/a       n/a 10.42.7.190     k8s-worker-2     162m      
mariadb-deployment-757b74fc-mpkmh        1/1     Running             0     1       n/a       n/a    105       n/a       n/a 10.42.7.212     k8s-worker-2     4h5m  

1- EJBCA values
values.yaml file used:

ejbca:
  useEphemeralH2Database: false
  env:
     DATABASE_JDBC_URL: "jdbc:mariadb://ejbca-database-service:3306/ejbca?characterEncoding=utf8"
     DATABASE_USER: "ejbca"
     DATABASE_PASSWORD: "ejbca"


services:
  directHttp:
    enabled: false
  proxyAJP:
    enabled: false
  proxyHttp:
    enabled: true
    type: ClusterIP
    bindIP: 0.0.0.0
    httpPort: 8081
    httpsPort: 8082

2-Ingress for EJBCA UI
ingress.yaml used

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ejbca
  namespace: ejbca-ns
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/auth-tls-verify-client: "optional_no_ca"
    nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
spec:
  ingressClassName: nginx
  rules:
    - host: ejbca.test.local
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: ejbca
                port:
                  number: 8081

3- Issuer
issuer.yaml used:
Note: the hostname is mapping to the ejbca service FQDN

apiVersion: ejbca-issuer.keyfactor.com/v1alpha1
kind: Issuer
metadata:
  namespace: pkirules
  labels:
    app.kubernetes.io/name: issuer
    app.kubernetes.io/instance: pkirules-tls
    app.kubernetes.io/part-of: ejbca-issuer
    app.kubernetes.io/created-by: ejbca-issuer
  name: pkirules-tls
spec:
  hostname: "ejbca.ejbca-ns.svc.cluster.local"
  ejbcaSecretName: "ejbca-secret"
  certificateAuthorityName: "MyPKISubCA-G1"
  certificateProfileName: "TLS Server Profile"
  endEntityProfileName: "TLS Server Profile"
  caBundleSecretName: ejbca-ca-secret 

4- EJBCA Configuration
We followed your tutorials here. All Certificate Profiles, Certificate Authorities, Crypto Tokens and End Entity Profiles are created:

Troubleshooting

Describing the issuer via kubectl -n pkirules describe issuers.ejbca-issuer.keyfactor.com
The Error healthcheck failed: Get "https://ejbca.ejbca-ns/ejbca/ejbca-rest-api/v1/certificate/status": context deadline exceeded (Client.Timeout exceeded while awaiting headers) is leading us nowhere

Name:         pkirules-tls
Namespace:    pkirules
Labels:       app.kubernetes.io/created-by=ejbca-issuer
              app.kubernetes.io/instance=pkirules-tls
              app.kubernetes.io/name=issuer
              app.kubernetes.io/part-of=ejbca-issuer
Annotations:  <none>
API Version:  ejbca-issuer.keyfactor.com/v1alpha1
Kind:         Issuer
Metadata:
  Creation Timestamp:  2025-10-04T16:16:06Z
  Generation:          1
  Resource Version:    22576640
  UID:                 5affe53f-2b69-4127-be51-b8536394fe33
Spec:
  Ca Bundle Secret Name:       ejbca-ca-secret
  Certificate Authority Name:  MyPKISubCA-G1
  Certificate Profile Name:    TLS Server Profile
  Ejbca Secret Name:           ejbca-secret
  End Entity Profile Name:     TLS Server Profile
  Hostname:                    ejbca.ejbca-ns.svc.cluster.local
Status:
  Conditions:
    Last Transition Time:  2025-10-04T16:16:16Z
    Message:               healthcheck failed: Get "https://ejbca.ejbca-ns.svc.cluster.local/ejbca/ejbca-rest-api/v1/certificate/status": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
    Reason:                ejbca-issuer.IssuerController.Reconcile
    Status:                False
    Type:                  Ready
Events:                    <none>

BR
Sam

[primetomas]

Moved from Issues to Discussion as that is the place for troubleshooting, before determining if it is an issue with EJBCA.

Looking at the error, from the client: context deadline exceeded (Client.Timeout exceeded while awaiting headers)

This is a network error message, indicating that that network connection could not be established. If it had been a TLS handshake issue it is typically a "connection closed by peer" or something like that. Does it take some time for the error to appear? I.e. a timeout should take some time.

Since your EJBCA is up and running, and accessible as per the screenshots, this is good.

A few question to check:

1. Are you using client certificate authentication for the Admin UI? (just to verify that client cert auth is enabled and working).
2. Did you enable REST API in the protocol configuration?

From the error my guess is that there is a network issue, and the URL is routed to cyberspace without reaching the end point. Some things to try:

• access the URL (https://ejbca.ejbca-ns.svc.cluster.local/ejbca/ejbca-rest-api/v1/certificate/status) with your browser. You should see something like the attached screenshot.
• if that works, then it's a topic of the internal routing between the containers

[sam-sre]

Thanks for the auick reply @primetomas

There are no network policies in place within the K8s cluster and connectivity is working fine between the other pods.

Does it take some time for the error to appear?

Not really ,, maybe a couple seconds

Are you using client certificate authentication for the Admin UI? (just to verify that client cert auth is enabled and working).

no .. I'm accessing it via an ingress without TLS.

Did you enable REST API in the protocol configuration?

yes

access the URL (https://ejbca.ejbca-ns.svc.cluster.local/ejbca/ejbca-rest-api/v1/certificate/status) with your browser. You should see something like the attached screenshot.

The ejbca.ejbca-ns.svc.cluster.local is an internal FQDN that can not be reached from outside the cluster.

We tried to memic the tutorial here almost bit by bit ,, unless there is something to be done that is not mentioned in the tutorial, we did not change much other than the K8s distribution

BR
Sam

[primetomas]

All environments differ slightly. I don't run k8s myself so I'm trying to help from an EJBCA point of view. Somehow you should be able to access the EJBCA instance with your browser (you provided screenshots). Whatever URL you use to access the admin UI you can check that the REST API service is available by using https://your-reachable-ejbca-host/ejbca/ejbca-rest-api/v1/certificate/status