Cloudflared Cloudflare Tunnel

[gunnars04]

Hi,

Regarding:
"Use cloudflared to expose a Kubernetes app to the Internet"
https://developers.cloudflare.com/cloudflare-one/tutorials/many-cfd-one-tunnel/

Have you tried it with this project? Would it be wise to implement regarding scale and having multiple k8s clusters in Hetzner Cloud?

[mysticaltech]

Thanks, very interesting, did not know about it! I do not see any reason why it will not work with this project.

[maggie44]

Did you ever try this out? Question here becomes what should be disabled when using this solution?

It runs from within and seems it points directory to the service which could handle the load balancing? https://github.com/cloudflare/argo-tunnel-examples/blob/master/named-tunnel-k8s/app.yaml

Is disabling the hetzner load balancer as simple as removing those lines? What about the ingress controller, should cloudflared point to the ingress or to the service? Seems like it is documented for the service so then what is the running ingress used for? Then would need to disable access on the firewall which would no longer be needed.

https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/deploy-cloudflared-replicas/#about-cloudflared-replicas

[mysticaltech]

@maggie44 I can only answer part of your question right now (maybe I'll dive deeper later), just set ingress_controller=none in your kube.tf to open the way for the above tools. That will basically clear the way and disable LBs etc.

[mysticaltech]

And definitely, cloudflared should point to a service, as an ingress needs an ingress controller (which itself is a service) and in this case we have none.

[mysticaltech]

The service exposes your pods/deployment/daemonset/statefulset... etc.

[maggie44]

Thanks, this is really helpful.

I think where it gets more blurry is how the load balancing would then work for the service it points to. Cloudflared doesn’t handle that. And can a service be expected to perform as efficient load balancing as the Hetzner load balancer? Hard to see how to get the best of both worlds.

[mysticaltech]

@maggie44 You can double check, but from my understanding, the service will automatically load balance incoming requests to the pods, and that can be configured even more. And here's what my good old friend GPT-4 has to say about this:

---

You:

Thanks, this is really helpful.

I think where it gets more blurry is how the load balancing would then work for the service it points to. Cloudflared doesn’t handle that. And can a service be expected to perform as efficient load balancing as the Hetzner load balancer? Hard to see how to get the best of both worlds.

ChatGPT:

You're right, cloudflared itself doesn't handle load balancing. However, you can combine it with the load balancing capabilities provided by Kubernetes to achieve a balance between security and performance.

In Kubernetes, the primary way to distribute load between multiple replicas of a deployment is to use a Kubernetes Service of type ClusterIP. This service type exposes the deployment internally and uses a stable IP address. Kubernetes uses round-robin load balancing by default when multiple Pods match the service selector.

Here's how you can get the best of both worlds:

1. Create a deployment with multiple replicas for your application:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app-container
        image: my-app-image
        ports:
        - containerPort: 80

2. Create a ClusterIP service for the deployment:

apiVersion: v1
kind: Service
metadata:
  name: my-app-service
spec:
  selector:
    app: my-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: ClusterIP

3. Update the cloudflared deployment to point to the ClusterIP service:

Replace the --url flag value in the cloudflared-deployment.yaml file with the DNS name of the ClusterIP service you created (e.g., http://my-app-service.namespace.svc.cluster.local:80).

With this setup, the traffic will first pass through the cloudflared tunnel, ensuring security, and then be distributed by the ClusterIP service to the replicas of your application deployment. Kubernetes will handle the load balancing between these replicas, so you don't need to worry about load balancing at the cloudflared level.

Regarding your question about the efficiency of Kubernetes load balancing compared to Hetzner load balancer, it's important to note that Kubernetes' built-in load balancing capabilities are generally sufficient for most use cases, especially when combined with horizontal scaling (i.e., increasing the number of replicas). However, it might not be as feature-rich or customizable as a dedicated load balancer like Hetzner's.

If you need more advanced load balancing features or better performance, you can use an external load balancer, like Hetzner's, in combination with cloudflared. In this case, you would expose your application using a Kubernetes Service of type LoadBalancer, which provisions an external load balancer for your application. The cloudflared tunnel would then point to the external load balancer's IP address instead of the ClusterIP service. This way, you can benefit from the advanced load balancing features provided by Hetzner while still maintaining the security of a Cloudflare tunnel.

[mysticaltech]

Here's what our good friend GPT-4 has to say about it, maybe this can get you started:

---

K3s Cluster with Cloudflare

You:

I disabled all ingress controllers on my k3s cluster, how do I use cloudflared (that I can have as a deployment in the cluster) make my deployments accessible through cloudflared tunnels?

ChatGPT:

To use Cloudflare's cloudflared to create tunnels for your deployments, you can follow these steps:

1. Install cloudflared:

First, you need to have cloudflared installed on your local machine. Download the appropriate version for your operating system from the official Cloudflare downloads page and follow the installation instructions.

2. Authenticate cloudflared:

Run the following command to authenticate cloudflared with your Cloudflare account:

cloudflared login

This command will open a browser window where you can log in to your Cloudflare account and authorize cloudflared. It will then generate a certificate file that you'll use later in the Kubernetes deployment.

3. Create a Kubernetes Secret with the certificate:

Create a Kubernetes Secret containing the cert.pem file generated in the previous step. Replace /path/to/cert.pem with the actual path to your cert.pem file:

kubectl create secret generic cloudflared-cert --from-file=cert.pem=/path/to/cert.pem

4. Create a cloudflared deployment:

Create a Kubernetes deployment for cloudflared. Create a file named cloudflared-deployment.yaml with the following contents:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cloudflared
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cloudflared
  template:
    metadata:
      labels:
        app: cloudflared
    spec:
      containers:
      - name: cloudflared
        image: cloudflare/cloudflared:latest
        args:
        - tunnel
        - --url
        - http://your-service.namespace.svc.cluster.local:your-service-port
        - --hostname
        - your-subdomain.example.com
        - --origincert
        - /etc/cloudflared/cert.pem
        volumeMounts:
        - name: cloudflared-cert
          mountPath: /etc/cloudflared
      volumes:
      - name: cloudflared-cert
        secret:
          secretName: cloudflared-cert

Replace the following placeholders with your own values:

• http://your-service.namespace.svc.cluster.local:your-service-port: The URL of the Kubernetes service you want to expose through the Cloudflare tunnel.
• your-subdomain.example.com: The hostname you want to use for the tunnel. This should be a subdomain of a domain managed by Cloudflare.

5. Deploy the cloudflared deployment:

Apply the deployment configuration file to your cluster:

kubectl apply -f cloudflared-deployment.yaml

6. Check the status of your deployment:

kubectl get deployments

Once the deployment is ready and running, your service should be accessible via the specified hostname (e.g., your-subdomain.example.com) through the Cloudflare tunnel.

Note that if you have more than one service you want to expose, you can create additional deployments for each service by modifying the cloudflared-deployment.yaml file and changing the --url and --hostname arguments.

[maggie44]

So far so good. Here are some docs from Cloudflare on an implementation: https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/deployment-guides/kubernetes/

I generated the token from Cloudflare Zero dashboard, added it as a secret, then put it in to the container as an environment variable TUNNEL_TOKEN which allows Cloudflared to pick it up:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cloudflared-ingress
  namespace: my-namespace
  labels:
    app: cloudflared-ingress
spec:
  selector:
    matchLabels:
      app: cloudflared-ingress
  replicas: 3
  template:
    metadata:
      labels:
        app: cloudflared-ingress
    spec:
      containers:
      - name: cloudflared-ingress
        image: cloudflare/cloudflared:2023.7.3
        imagePullPolicy: IfNotPresent
        args:
          - tunnel
          - --no-autoupdate
          - --metrics
          - 0.0.0.0:2000
          - run
        envFrom:
          - secretRef:
              name: cloudflared-ingress-token
        env:
          - name: TZ
            value: UTC
        livenessProbe:
          httpGet:
            path: /ready
            port: 2000
          failureThreshold: 1
          initialDelaySeconds: 10
          periodSeconds: 10
      restartPolicy: Always
      terminationGracePeriodSeconds: 60

The final piece of the puzzle allowing all the inbound ports to be closed.

[mysticaltech]

Wow, amazing! Thanks for sharing.