ejabberd-deployment example for kubernetes bare metal wanted

[AuqW9cQ]

Is there anyone running ejabberd in kubernetes?

Would be nice if someone could provide their configuration or have tips on mine.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ejabberd-server
  namespace: ejabberd
  labels:
    app: ejabberd
spec:
  replicas: 1
  selector:
    matchLabels:
      pod-label: ejabberd-server-pod
  template:
    metadata:
      labels:
        pod-label: ejabberd-server-pod
    spec:
      containers:
      - name: ejabberd
        image: ejabberd/ecs:22.05
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 5222
          protocol: TCP
          name: c2s
        - containerPort: 5223
          protocol: TCP
          name: c2s2
        - containerPort: 5269
          protocol: TCP
          name: s2s
        - containerPort: 5280
          protocol: TCP
          name: http
        - containerPort: 5443
          protocol: TCP
          name: http-upload
        - containerPort: 3478
          protocol: UDP
          name: stun
        volumeMounts:
        - name: ejabberd-pv
          mountPath: /home/ejabberd/conf
          subPath: conf
        - name: ejabberd-pv
          mountPath: /home/ejabberd/database
          subPath: database
        - name: ejabberd-pv
          mountPath: /home/ejabberd/logs
          subPath: logs
        - name: ejabberd-pv
          mountPath: /home/ejabberd/upload
          subPath: upload
      volumes:
      - name: ejabberd-pv
        persistentVolumeClaim:
          claimName: ejabberd-pv
---
apiVersion: v1
kind: Service
metadata:
  name: ejabberd-server
  namespace: ejabberd
  labels:
    app: ejabberd
spec:
  selector:
    pod-label: ejabberd-server-pod
  ports:
  - port: 5222
    targetPort: c2s
    protocol: TCP
    name: c2s
  - port: 5223
    targetPort: c2s2
    protocol: TCP
    name: c2s2
  - port: 5269
    targetPort: s2s
    protocol: TCP
    name: s2s
  - port: 5280
    targetPort: http
    protocol: TCP
    name: http
  - port: 5443
    targetPort: http-upload
    protocol: TCP
    name: http-upload
  - port: 3478
    targetPort: stun
    protocol: UDP
    name: stun
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ejabberd-ingress
  namespace: ejabberd
  annotations:
    kubernetes.io/ingress.class: "public"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"

spec:
  tls:
  - hosts:
    - example.com
    secretName: ejabberd-secret
  rules:
  - host: example.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ejabberd-server
            port:
              name: http

[sando38]

Hello,

I have a kustomize example on bare metal here:
https://github.com/sando38/docker-ejabberd-multiarch/tree/master/kustomize

• It uses a custom image, however, you may exchange it with the ecs version.
  • With the ecs version, you should ignore all the environment variable stuff from the linked example and just mount your ejabberd.yml file into the container.
  • You may need to double check the security context as well in the statefulset, as I am not sure, if the ecs version can run unpriviledged.
• It does use traefik with ingressroutes, however, that should be easy to adapt to kubernetes ingress kind.

In case of questions, just let me know.

[AuqW9cQ]

Edit says:
I think I have found the problem. The error seems to be with the certificate of my main domain, with ejabberdctl request-certificate I could generate the certificates for the subdomains:

bin/ejabberdctl request-certificate conference.example.com,upload.example.com,pubsub.example.com
~ $ bin/ejabberdctl list-certificates
conference.example.com       /home/ejabberd/database/ejabberd@ejabberd-0/acme/live/8486eb8f0ea7dd8aaabf6a0ef6f4182cb4a19f5f  true
pubsub.example.com   /home/ejabberd/database/ejabberd@ejabberd-0/acme/live/8486eb8f0ea7dd8aaabf6a0ef6f4182cb4a19f5f  true
upload.example.com   /home/ejabberd/database/ejabberd@ejabberd-0/acme/live/8486eb8f0ea7dd8aaabf6a0ef6f4182cb4a19f5f  true
~ $ bin/ejabberdctl request-certificate example.com
Error: error
Error: "Challenge failed for domain example.com: ACME server reported: 159.60.58.110: Invalid response from http://example.com/.well-known/acme-challenge/Fy_UnK_XUaglsie87yOZys9DWtOqF83ANSO5tp4v_5HY: 404 (error type: unauthorized)"

Only how do I get the certificate for the main domain now? My hoster wants that still itself, becaus there still runs my website.

---

Before Edit:

Thanks for your reply @sando38

I tried the whole day, but i don´t get it work.

Nevertheless, I think I've made a lot of progress and learned from your example. e.g. I patched Ingress for non http(s) services.

But unfortunately I don't get the certificate thing yet.

This is my current DNS configuration at my hoster

_xmpp-client._tcp.example.com. 	300 	SRV 	Priorität: 0 Gewicht: 5 -Port: 5222 Ziel: xmpp.example.com
_xmpp-client._tcp.example.com. 	300 	SRV 	Priorität: 0 Gewicht: 5 -Port: 5223 Ziel: xmpp.example.com
_xmpp-client._tcp.example.com. 	300 	SRV 	Priorität: 0 Gewicht: 5 -Port: 443 Ziel: xmpp.example.com
_xmpp-server._tcp.example.com. 	300 	SRV 	Priorität: 0 Gewicht: 5 -Port: 5269 Ziel: xmpp.example.com
_xmpp-server._tcp.example.com. 	300 	SRV 	Priorität: 0 Gewicht: 5 -Port: 5270 Ziel: xmpp.example.com
conference.example.com. 	300 	CNAME 	xmpp.example.com	
pubsub.example.com. 	300 	CNAME 	xmpp.example.com	
upload.example.com. 	300 	CNAME 	xmpp.example.com	
_acme-challenge.xmpp.example.com. 	14400 	TXT 	8GkMePzD4DfdljNujty4drqslkrmxNzONGKOYxy3ssoM
xmpp.example.com. 	300 	A 	97.113.134.60 <- my external IP from my home updadet dynamically
example.com. 	300 	A 	159.20.58.110     <- IP from my hoster
localhost.example.com. 	300 	A 	127.0.0.1
example.com. 	300 	MX 	Priorität: 0 Ziel: example.com

and here are the important iptables from my ipfire to cluster node 192.168.1.235

	0 	0 	LOG 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp spt:443 dpt:5223 limit: avg 10/sec burst 20 LOG flags 0 level 4 prefix "FORWARDFW "
	0 	0 	ACCEPT 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp spt:443 dpt:5223
	1 	60 	LOG 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5222 limit: avg 10/sec burst 20 LOG flags 0 level 4 prefix "FORWARDFW "
	1 	60 	ACCEPT 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5222
	1 	60 	LOG 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5223 limit: avg 10/sec burst 20 LOG flags 0 level 4 prefix "FORWARDFW "
	1 	60 	ACCEPT 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5223
	0 	0 	LOG 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5443 limit: avg 10/sec burst 20 LOG flags 0 level 4 prefix "FORWARDFW "
	0 	0 	ACCEPT 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5443
	0 	0 	LOG 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5269 limit: avg 10/sec burst 20 LOG flags 0 level 4 prefix "FORWARDFW "
	0 	0 	ACCEPT 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5269
	0 	0 	LOG 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5270 limit: avg 10/sec burst 20 LOG flags 0 level 4 prefix "FORWARDFW "
	0 	0 	ACCEPT 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5270
	0 	0 	LOG 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5280 limit: avg 10/sec burst 20 LOG flags 0 level 4 prefix "FORWARDFW "
	0 	0 	ACCEPT 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:5280
	1 	60 	LOG 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:443 limit: avg 10/sec burst 20 LOG flags 0 level 4 prefix "FORWARDFW "
	1 	60 	ACCEPT 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:443
	0 	0 	LOG 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:80 limit: avg 10/sec burst 20 LOG flags 0 level 4 prefix "FORWARDFW "
	0 	0 	ACCEPT 	tcp 	-- 	red0 	* 	0.0.0.0/0 	192.168.1.235 	tcp dpt:80

Here is my current confi:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: ejabberd
  namespace: ejabberd
  labels:
    type: statefulset
spec:
  serviceName: ejabberd
  replicas: 1
  selector:
    matchLabels:
      app: ejabberd
  template:
    metadata:
      labels:
        app: ejabberd
    spec:      
      containers:
      - name: ejabberd
        image: ejabberd/ecs:22.05 #sando38/docker-ejabberd-multiarch:latest
        ports:
        - name: mqtt
          containerPort: 1883
        - name: epmd
          containerPort: 4369
        - name: c2s
          containerPort: 5222
        - name: c2ss
          containerPort: 5223
        - name: s2s
          containerPort: 5269
        - name: s2ss
          containerPort: 5270
        - name: http
          containerPort: 5280
        - name: https
          containerPort: 5443
        - name: stun-udp
          protocol: UDP
          containerPort: 3478 #3748
        - name: stun
          protocol: TCP
          containerPort: 3478 #3748
        - name: stuns
          containerPort: 5349
        readinessProbe:
          tcpSocket:
            port: epmd
          initialDelaySeconds: 30
          periodSeconds: 15
        volumeMounts:
        - name: ejabberd-pv
          mountPath: /home/ejabberd/conf
          subPath: conf
        - name: ejabberd-pv
          mountPath: /home/ejabberd/database
          subPath: database
        - name: ejabberd-pv
          mountPath: /home/ejabberd/logs
          subPath: logs
        - name: ejabberd-pv
          mountPath: /home/ejabberd/upload
          subPath: upload
      volumes:
      - name: ejabberd-pv
        persistentVolumeClaim:
          claimName: ejabberd-pv

---
apiVersion: v1
kind: Service
metadata:
  name: ejabberd
  namespace: ejabberd
  labels:
    app: ejabberd
spec:
  type: ClusterIP
  ports:
  - name: mqtt
    port: 1883
    targetPort: 1883
  - name: epmd
    port: 4369
    targetPort: 4369
  - name: c2s
    port: 5222
    targetPort: 5222
  - name: c2ss
    port: 5223
    targetPort: 5223
  - name: s2s
    port: 5269
    targetPort: 5269
  - name: s2ss
    port: 5270
    targetPort: 5270
  - name: http
    port: 5280
    targetPort: 5280
  - name: https
    port: 5443
    targetPort: 5443
  - port: 3478
    targetPort: 3478
    protocol: UDP
    name: stun-turn-udp
  - port: 3478
    targetPort: 3478
    protocol: TCP
    name: stun-turn-tcp
  - port: 5349
    targetPort: 5349
    protocol: TCP
    name: stuns-turns-tcp
  selector:
    app: ejabberd
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-ingress-tcp-microk8s-conf
  namespace: ingress
data:
  1883: "ejabberd/ejabberd:1883" #mqtt
  4369: "ejabberd/ejabberd:4369" #epmd
  5222: "ejabberd/ejabberd:5222" #c2s
  5223: "ejabberd/ejabberd:5223" #c2ss
  #443: "ejabberd/ejabberd:5223" #c2ss
  5269: "ejabberd/ejabberd:5269" #s2s
  5270: "ejabberd/ejabberd:5270" #s2ss
  3478: "ejabberd/ejabberd:3478" #stun-turn
  5349: "ejabberd/ejabberd:5349" #stuns-turns
  5443: "ejabberd/ejabberd:5443" #https
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress-microk8s-controller
  namespace: ingress
spec:
  template:
    spec:
      containers:
      - name: nginx-ingress-microk8s
        ports:
        - containerPort: 80
        - containerPort: 443
        - name: prox-tcp-5222
          containerPort: 5222
          hostPort: 5222
          protocol: TCP
        - name: prox-tcp-5223
          containerPort: 5223
          hostPort: 5223
          protocol: TCP
        - name: prox-tcp-5269
          containerPort: 5269
          hostPort: 5269
          protocol: TCP
        - name: prox-tcp-5270
          containerPort: 5270
          hostPort: 5270
          protocol: TCP
        - name: prox-tcp-5443
          containerPort: 5443
          hostPort: 5443
          protocol: TCP
---
apiVersion: networking.k8s.io/v1 
kind: Ingress
metadata:
  name: ejabberd-ingress
  namespace: ejabberd
  annotations:
    kubernetes.io/ingress.class: "public"
    #kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"

spec:
  rules:
  - host: example.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ejabberd
            port:
              number: 5280
              #name: http
  - host: conference.example.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ejabberd
            port:
              number: 5280
              #name: http
  - host: pubsub.example.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ejabberd
            port:
              number: 5280
              #name: http
  - host: upload.example.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ejabberd
            port:
              number: 5280
              #name: http

here is the log:

kubectl logs -n ejabberd             pod/ejabberd-0
2022-08-03 22:25:58.871456+00:00 [info] Loading configuration from /home/ejabberd/conf/ejabberd.yml
2022-08-03 22:25:59.312772+00:00 [info] Configuration loaded successfully
2022-08-03 22:25:59.506797+00:00 [info] Got no NOTIFY_SOCKET, notifications disabled
2022-08-03 22:25:59.637584+00:00 [info] Loading modules for example.com
2022-08-03 22:25:59.659996+00:00 [warning] Mnesia backend for mod_mam is not recommended: it's limited to 2GB and often gets corrupted when reaching this limit. SQL backend is recommended. Namely, for small servers SQLite is a preferred choice because it's very easy to configure.
2022-08-03 22:25:59.717300+00:00 [info] Going to offer STUN/TURN service: 10.1.84.172:3478 (udp)
2022-08-03 22:25:59.824540+00:00 [info] Building MQTT cache for example.com, this may take a while
2022-08-03 22:25:59.873804+00:00 [info] Waiting for Mnesia synchronization to complete
2022-08-03 22:25:59.932884+00:00 [warning] No certificate found matching example.com
2022-08-03 22:25:59.933081+00:00 [warning] No certificate found matching pubsub.example.com
2022-08-03 22:25:59.933202+00:00 [warning] No certificate found matching upload.example.com
2022-08-03 22:25:59.933341+00:00 [warning] No certificate found matching conference.example.com
2022-08-03 22:25:59.933523+00:00 [warning] No certificate found matching proxy.example.com
2022-08-03 22:25:59.933780+00:00 [info] ejabberd 22.5.0 is started in the node 'ejabberd@ejabberd-0' in 1.16s
2022-08-03 22:25:59.934014+00:00 [info] Requesting new certificate for example.com, pubsub.example.com and 3 more hosts from https://acme-v02.api.letsencrypt.org/directory
2022-08-03 22:25:59.935595+00:00 [info] Start accepting TCP connections at [::]:1883 for mod_mqtt
2022-08-03 22:25:59.935595+00:00 [info] Start accepting UDP connections at [::]:3478 for ejabberd_stun
2022-08-03 22:25:59.935732+00:00 [info] Start accepting TCP connections at 10.1.84.172:7777 for mod_proxy65_stream
2022-08-03 22:25:59.935827+00:00 [info] Start accepting TCP connections at [::]:5280 for ejabberd_http
2022-08-03 22:25:59.935851+00:00 [info] Start accepting TLS connections at [::]:5443 for ejabberd_http
2022-08-03 22:25:59.935927+00:00 [info] Start accepting TLS connections at [::]:5223 for ejabberd_c2s
2022-08-03 22:25:59.935914+00:00 [info] Start accepting TCP connections at [::]:5269 for ejabberd_s2s_in
2022-08-03 22:25:59.935965+00:00 [info] Start accepting TCP connections at [::]:5222 for ejabberd_c2s
2022-08-03 22:26:00.077560+00:00 [warning] Description: "Authenticity is not established by certificate path validation"
     Reason: "Option {verify, verify_peer} and cacertfile/cacerts is missing"

2022-08-03 22:26:03.459226+00:00 [error] Failed to request certificate for example.com, pubsub.example.com and 3 more hosts: ACME server reported: Error creating new order :: too many failed authorizations recently: see https://letsencrypt.org/docs/failed-validation-limit/ (error type: rateLimited)
2022-08-03 22:27:07.055610+00:00 [info] (<0.713.0>) Accepted connection [::ffff:10.1.84.156]:44418 -> [::ffff:10.1.84.172]:5223
2022-08-03 22:27:07.056223+00:00 [info] (<0.714.0>) Accepted connection [::ffff:10.1.84.156]:44598 -> [::ffff:10.1.84.172]:5222
2022-08-03 22:27:07.072055+00:00 [warning] (tcp|<0.713.0>) Failed to secure c2s connection: TLS failed: no_certfile

possibly other interesting output:

kubectl get all,pv,pvc,endpoints,secrets,challenges,orders,ingress,endpoints,configmaps --all-namespaces -o wide | grep 'ejabberd'
ejabberd             pod/ejabberd-0                                   1/1     Running   0               12m     10.1.84.172       cluster-bs-dg01   <none>           <none>
ejabberd             service/ejabberd                    ClusterIP   10.152.183.220   <none>        1883/TCP,4369/TCP,5222/TCP,5223/TCP,5269/TCP,5270/TCP,5280/TCP,5443/TCP,3478/UDP,3478/TCP,5349/TCP   21m     app=ejabberd
ejabberd    statefulset.apps/ejabberd   1/1     21m   ejabberd     ejabberd/ecs:22.05
            persistentvolume/ejabberd-pv                                10Gi       RWO            Retain           Bound       ejabberd/ejabberd-pv                                             21m     Filesystem
ejabberd             persistentvolumeclaim/ejabberd-pv      Bound     ejabberd-pv                                10Gi       RWO            microk8s-hostpath   21m     Filesystem
ejabberd             endpoints/ejabberd                    10.1.84.172:5223,10.1.84.172:5269,10.1.84.172:5222 + 8 more...   21m
ejabberd    ingress.networking.k8s.io/ejabberd-ingress    <none>   example.com,conference.example.com,pubsub.example.com + 1 more...   127.0.0.1   80        21m
ejabberd             configmap/kube-root-ca.crt                     1      21m

It would be nice if you, or anyone else, could tell me where the error is.

Thanks in advance

[sando38]

For the first answer, I can see, that your kubectl logs output shows, you hit the lets encrypt rate limit. You may try the staging environment. You need to set it in the eturnal.yml file in the ACME part.

https://acme-staging-v02.api.letsencrypt.org/directory

Do you plan to to use the integrated TURN Service from ejabberd as well?

In the service part, I am pretty sure you do not need the epmd port, because this would be only interesting to "publish" within your cluster if you plan clustering with ejabberd.

[AuqW9cQ]

Since I have not managed to delegate the acme challenge for my main domain in such a way that both -my hoster and home hosted kubernetes cluster- can resolve it, I have set up a kubernetes CronJob and copy all the certificates from my hoster to the drive mounted by ejabberd.

However, I still get the Kubernetes Ingress Controller Fake Certificate delivered.

What is wrong with my configuration and how can I fix this?

Here is my actual ejabberd-deployment:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: ejabberd
  namespace: ejabberd
  labels:
    type: statefulset
spec:
  serviceName: ejabberd
  replicas: 1
  selector:
    matchLabels:
      app: ejabberd
  template:
    metadata:
      labels:
        app: ejabberd
    spec:
      containers:
      - name: ejabberd
        image: ejabberd/ecs:19.09 #ejabberd/ecs:22.05 #sando38/docker-ejabberd-multiarch:latest
        ports:
        - name: mqtt
          containerPort: 1883
        #- name: epmd
        #  containerPort: 4369
        - name: c2s
          containerPort: 5222
        - name: c2ss
          containerPort: 5223
        - name: s2s
          containerPort: 5269
        - name: s2ss
          containerPort: 5270
        - name: http
          containerPort: 5280
        - name: https
          containerPort: 5443
        - name: stun-udp
          protocol: UDP
          containerPort: 3478 #3748
        - name: stun
          protocol: TCP
          containerPort: 3478 #3748
        - name: stuns
          containerPort: 5349
        volumeMounts:
        - name: ejabberd-pv
          mountPath: /home/ejabberd/conf
          subPath: conf
        - name: ejabberd-pv
          mountPath: /home/ejabberd/database
          subPath: database
        - name: ejabberd-pv
          mountPath: /home/ejabberd/logs
          subPath: logs
        - name: ejabberd-pv
          mountPath: /home/ejabberd/upload
          subPath: upload
      volumes:
      - name: ejabberd-pv
        persistentVolumeClaim:
          claimName: ejabberd-pv
---
apiVersion: v1
kind: Service
metadata:
  name: ejabberd
  namespace: ejabberd
  labels:
    app: ejabberd
spec:
  type: ClusterIP
  ports:
  - name: mqtt
    port: 1883
    targetPort: 1883
  #- name: epmd
  #  port: 4369
  #  targetPort: 4369
  - name: c2s
    port: 5222
    targetPort: 5222
  - name: c2ss
    port: 5223
    targetPort: 5223
  - name: s2s
    port: 5269
    targetPort: 5269
  - name: s2ss
    port: 5270
    targetPort: 5270
  - name: http
    port: 5280
    targetPort: 5280
  - name: https
    port: 5443
    targetPort: 5443
  - port: 3478
    targetPort: 3478
    protocol: UDP
    name: stun-turn-udp
  - port: 3478
    targetPort: 3478
    protocol: TCP
    name: stun-turn-tcp
  - port: 5349
    targetPort: 5349
    protocol: TCP
    name: stuns-turns-tcp
  selector:
    app: ejabberd
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-ingress-tcp-microk8s-conf
  namespace: ingress
data:
  1883: "ejabberd/ejabberd:1883" #mqtt
  #4369: "ejabberd/ejabberd:4369" #epmd
  5222: "ejabberd/ejabberd:5222" #c2s
  5223: "ejabberd/ejabberd:5223" #c2ss
  #443: "ejabberd/ejabberd:5223" #c2ss
  5269: "ejabberd/ejabberd:5269" #s2s
  5270: "ejabberd/ejabberd:5270" #s2ss
  3478: "ejabberd/ejabberd:3478" #stun-turn
  5349: "ejabberd/ejabberd:5349" #stuns-turns
  5443: "ejabberd/ejabberd:5443" #https
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress-microk8s-controller
  namespace: ingress
spec:
  template:
    spec:
      containers:
      - name: nginx-ingress-microk8s
        ports:
        - containerPort: 80
        - containerPort: 443
        - name: prox-tcp-5222
          containerPort: 5222
          hostPort: 5222
          protocol: TCP
        - name: prox-tcp-5223
          containerPort: 5223
          hostPort: 5223
          protocol: TCP
        - name: prox-tcp-5269
          containerPort: 5269
          hostPort: 5269
          protocol: TCP
        - name: prox-tcp-5270
          containerPort: 5270
          hostPort: 5270
          protocol: TCP
        - name: prox-tcp-5443
          containerPort: 5443
          hostPort: 5443
          protocol: TCP
---
apiVersion: networking.k8s.io/v1 
kind: Ingress
metadata:
  name: ejabberd-ingress
  namespace: ejabberd
  annotations:
    kubernetes.io/ingress.class: "public"
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"

spec:
  rules:
  - host: example.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ejabberd
            port:
              number: 5280
              #name: http
  - host: conference.example.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ejabberd
            port:
              number: 5280
              #name: http
  - host: proxy.example.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ejabberd
            port:
              number: 5280
              #name: http
  - host: pubsub.example.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ejabberd
            port:
              number: 5280
              #name: http
  - host: upload.example.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ejabberd
            port:
              number: 5280
              #name: http

@sando38

Do you plan to to use the integrated TURN Service from ejabberd as well?

Yes, but first the other things have to run smoothly. It would be nice if you could help me with my certificate problem.

In the service part, I am pretty sure you do not need the epmd port, because this would be only interesting to "publish" within your cluster if you plan clustering with ejabberd.

OK, i've commented it out.

[sando38]

However, I still get the Kubernetes Ingress Controller Fake Certificate delivered.

What is wrong with my configuration and how can I fix this?

Sounds a bit as if the ingress controller terminates/ tries to terminate the TLS connections instead of ejabberd itself. Are the TCP routers configured with something like ssl pass-through? I am not much familiar with nginx ingress controller and the kubernetes ingress kind itself.

Additionally:

• Did you specify the cert file paths in the eturnal.yml configuration file?

[sando38]

For documentation:
I have created a helm chart for ejabberd. It is under development, but I run my server successfully with it.
https://github.com/sando38/helm-ejabberd

The goal is to merge it upstream at some point if wanted of course.