|
| 1 | +--- |
| 2 | +title: "Running RKE2 Ingress-NGINX with an External LoadBalancer" |
| 3 | +date: 2025-04-10T00:00:00-05:00 |
| 4 | +draft: false |
| 5 | +tags: ["RKE2", "Ingress", "LoadBalancer", "AWS", "MetalLB"] |
| 6 | +categories: |
| 7 | +- Kubernetes |
| 8 | +- RKE2 |
| 9 | +author: "Matthew Mattox - mmattox@support.tools" |
| 10 | +description: "Guide for configuring RKE2 ingress-nginx behind a LoadBalancer on AWS, GCP, Azure, or MetalLB." |
| 11 | +more_link: "yes" |
| 12 | +url: "/rke2-ingress-nginx-external-lb/" |
| 13 | +--- |
| 14 | + |
| 15 | +This article provides step-by-step guidance for configuring RKE2’s built-in `ingress-nginx` controller to run behind an external LoadBalancer. It supports environments like AWS, GCP, Azure, and MetalLB, and replaces the default hostNetwork + DaemonSet configuration with a Deployment and LoadBalancer service. |
| 16 | + |
| 17 | +<!--more--> |
| 18 | + |
| 19 | +# Running RKE2 Ingress-NGINX with an External LoadBalancer |
| 20 | + |
| 21 | +## Section 1: Summary and Use Case |
| 22 | + |
| 23 | +### Summary |
| 24 | + |
| 25 | +By default, RKE2 deploys the ingress controller as a DaemonSet with host networking. This is ideal for bare-metal setups but not suitable for cloud-native ingress via a LoadBalancer. This guide helps configure the controller to run as a Deployment with a LoadBalancer service, suitable for environments like AWS ELB or MetalLB. |
| 26 | + |
| 27 | +### Use Case |
| 28 | + |
| 29 | +You may want to: |
| 30 | +- Integrate ingress-nginx with your cloud provider's LoadBalancer |
| 31 | +- Use MetalLB for bare-metal LoadBalancer IP assignment |
| 32 | +- Disable host networking for better isolation |
| 33 | +- Scale ingress-nginx using Deployments |
| 34 | + |
| 35 | +--- |
| 36 | + |
| 37 | +## Section 2: Configuration Instructions |
| 38 | + |
| 39 | +### Prerequisites |
| 40 | + |
| 41 | +- A running RKE2 cluster |
| 42 | +- Access to the control plane node(s) |
| 43 | +- A LoadBalancer integration (e.g., AWS, GCP, Azure, or MetalLB) |
| 44 | + |
| 45 | +--- |
| 46 | + |
| 47 | +### Step 1: Create a HelmChartConfig Override |
| 48 | + |
| 49 | +Create the following file on all RKE2 server nodes: |
| 50 | + |
| 51 | +```bash |
| 52 | +cat > /var/lib/rancher/rke2/server/manifests/ingress-nginx.yaml << 'EOF' |
| 53 | +apiVersion: helm.cattle.io/v1 |
| 54 | +kind: HelmChartConfig |
| 55 | +metadata: |
| 56 | + name: rke2-ingress-nginx |
| 57 | + namespace: kube-system |
| 58 | +spec: |
| 59 | + valuesContent: | |
| 60 | + controller: |
| 61 | + hostNetwork: false |
| 62 | + kind: Deployment |
| 63 | + replicaCount: 3 |
| 64 | + service: |
| 65 | + enabled: true |
| 66 | + type: LoadBalancer |
| 67 | +EOF |
| 68 | +``` |
| 69 | + |
| 70 | +This config: |
| 71 | +- Converts the controller to a Deployment |
| 72 | +- Disables host networking |
| 73 | +- Enables a LoadBalancer-type service |
| 74 | + |
| 75 | +--- |
| 76 | + |
| 77 | +### Step 2: Wait for RKE2 to Apply Changes |
| 78 | + |
| 79 | +RKE2 automatically reconciles manifests from this directory. To confirm deployment: |
| 80 | + |
| 81 | +```bash |
| 82 | +kubectl -n kube-system get deploy,svc -l app.kubernetes.io/name=ingress-nginx |
| 83 | +``` |
| 84 | + |
| 85 | +You should see a Deployment and a LoadBalancer service. |
| 86 | + |
| 87 | +--- |
| 88 | + |
| 89 | +### Step 3: Verification |
| 90 | + |
| 91 | +Check that a LoadBalancer was successfully provisioned: |
| 92 | + |
| 93 | +```bash |
| 94 | +kubectl get svc -n kube-system ingress-nginx-controller |
| 95 | +``` |
| 96 | + |
| 97 | +#### Example Output (AWS): |
| 98 | + |
| 99 | +``` |
| 100 | +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE |
| 101 | +ingress-nginx-controller LoadBalancer 10.43.248.207 a1b2c3d4e5f6g7h8-1234567890.us-west-2.elb.amazonaws.com 80:31014/TCP,443:32478/TCP 2m |
| 102 | +``` |
| 103 | + |
| 104 | +You can now route ingress traffic to the ELB hostname or associate a custom domain. |
| 105 | + |
| 106 | +--- |
| 107 | + |
| 108 | +### Step 4: Example Ingress Resource |
| 109 | + |
| 110 | +Here is a simple Ingress manifest for testing: |
| 111 | + |
| 112 | +```yaml |
| 113 | +apiVersion: networking.k8s.io/v1 |
| 114 | +kind: Ingress |
| 115 | +metadata: |
| 116 | + name: example |
| 117 | + namespace: default |
| 118 | + annotations: |
| 119 | + nginx.ingress.kubernetes.io/rewrite-target: / |
| 120 | +spec: |
| 121 | + rules: |
| 122 | + - host: example.yourdomain.com |
| 123 | + http: |
| 124 | + paths: |
| 125 | + - path: / |
| 126 | + pathType: Prefix |
| 127 | + backend: |
| 128 | + service: |
| 129 | + name: your-service |
| 130 | + port: |
| 131 | + number: 80 |
| 132 | +``` |
| 133 | +
|
| 134 | +Point the DNS for `example.yourdomain.com` to the ELB address or external IP. |
| 135 | + |
| 136 | +--- |
| 137 | + |
| 138 | +### Notes |
| 139 | + |
| 140 | +- Adjust `replicaCount` as needed |
| 141 | +- Ensure firewall/security groups allow ports 80/443 |
| 142 | +- For MetalLB, ensure address pools are properly configured |
| 143 | +- This override is persistent across cluster upgrades |
| 144 | + |
| 145 | +--- |
| 146 | + |
| 147 | +### References |
| 148 | + |
| 149 | +- RKE2 Docs: https://docs.rke2.io |
| 150 | +- Kubernetes Ingress: https://kubernetes.io/docs/concepts/services-networking/ingress/ |
| 151 | +- MetalLB Setup: https://metallb.universe.tf/ |
0 commit comments