Networking Configuration

This document covers the networking setup for the Turing RK1 Kubernetes cluster, including MetalLB load balancing and NGINX Ingress Controller.

Network Architecture

                    ┌─────────────────────────────────────────┐
                    │           External Network              │
                    │            10.10.88.0/24                │
                    └───────────────┬─────────────────────────┘
                                    │
                    ┌───────────────┴─────────────────────────┐
                    │         Turing Pi 2 BMC                 │
                    │           10.10.88.70                   │
                    └───────────────┬─────────────────────────┘
                                    │
        ┌───────────┬───────────────┼───────────────┬───────────┐
        │           │               │               │           │
   ┌────┴────┐ ┌────┴────┐    ┌────┴────┐    ┌────┴────┐      │
   │ Node 1  │ │ Node 2  │    │ Node 3  │    │ Node 4  │      │
   │   CP    │ │ Worker  │    │ Worker  │    │ Worker  │      │
   │ .73     │ │ .74     │    │ .75     │    │ .76     │      │
   └─────────┘ └─────────┘    └─────────┘    └─────────┘      │
                                                               │
                    ┌─────────────────────────────────────────┘
                    │
                    │  MetalLB Virtual IPs
                    │  ┌─────────────────────────────────┐
                    │  │ 10.10.88.80 - Ingress Controller│
                    │  │ 10.10.88.81 - Portainer Agent   │
                    │  │ 10.10.88.82-89 - Available      │
                    │  └─────────────────────────────────┘

IP Address Allocation

Static Assignments

Purpose	IP Address
BMC	10.10.88.70
Control Plane	10.10.88.73
Worker 1	10.10.88.74
Worker 2	10.10.88.75
Worker 3	10.10.88.76

MetalLB Pool

IP Range	Purpose
10.10.88.80	NGINX Ingress Controller
10.10.88.81	Portainer Agent
10.10.88.82-89	Available for services

Kubernetes Networks

Network	CIDR
Pod Network	10.244.0.0/16
Service Network	10.96.0.0/12

MetalLB Installation

MetalLB provides LoadBalancer functionality for bare-metal clusters.

Step 1: Add Helm Repository

helm repo add metallb https://metallb.github.io/metallb
helm repo update

Step 2: Create Namespace

kubectl create namespace metallb-system

# Label as privileged for Talos
kubectl label namespace metallb-system pod-security.kubernetes.io/enforce=privileged

Step 3: Install MetalLB

helm install metallb metallb/metallb -n metallb-system --wait

Step 4: Configure IP Pool

# metallb-config.yaml
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: default-pool
  namespace: metallb-system
spec:
  addresses:
    - 10.10.88.80-10.10.88.89
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default
  namespace: metallb-system
spec:
  ipAddressPools:
    - default-pool

Apply:

kubectl apply -f metallb-config.yaml

Step 5: Verify

kubectl get pods -n metallb-system
kubectl get ipaddresspools -n metallb-system

NGINX Ingress Controller

Step 1: Add Helm Repository

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

Step 2: Create Namespace

kubectl create namespace ingress-nginx
kubectl label namespace ingress-nginx pod-security.kubernetes.io/enforce=privileged

Step 3: Install

helm install ingress-nginx ingress-nginx/ingress-nginx \
  -n ingress-nginx \
  --set controller.service.type=LoadBalancer \
  --set controller.ingressClassResource.default=true \
  --wait

Step 4: Verify

kubectl get svc -n ingress-nginx
kubectl get pods -n ingress-nginx

Expected output:

NAME                       TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)
ingress-nginx-controller   LoadBalancer   10.111.58.78   10.10.88.80   80:32072/TCP,443:30100/TCP

Creating Ingress Resources

Basic HTTP Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app-ingress
  namespace: default
spec:
  ingressClassName: nginx
  rules:
    - host: myapp.local
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-service
                port:
                  number: 80

Ingress with TLS

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app-ingress-tls
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - myapp.example.com
      secretName: myapp-tls
  rules:
    - host: myapp.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-service
                port:
                  number: 80

Path-Based Routing

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: multi-path-ingress
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  ingressClassName: nginx
  rules:
    - host: apps.local
      http:
        paths:
          - path: /api(/|$)(.*)
            pathType: ImplementationSpecific
            backend:
              service:
                name: api-service
                port:
                  number: 8080
          - path: /web(/|$)(.*)
            pathType: ImplementationSpecific
            backend:
              service:
                name: web-service
                port:
                  number: 80

Configured Ingress Resources

Longhorn UI

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: longhorn-ingress
  namespace: longhorn-system
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
spec:
  ingressClassName: nginx
  rules:
    - host: longhorn.local
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: longhorn-frontend
                port:
                  number: 80

Access: Add to /etc/hosts:

10.10.88.80  longhorn.local

LoadBalancer Services

View All LoadBalancer Services

kubectl get svc -A --field-selector spec.type=LoadBalancer

Create a LoadBalancer Service

apiVersion: v1
kind: Service
metadata:
  name: my-lb-service
  namespace: default
spec:
  type: LoadBalancer
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 8080

Request Specific IP

apiVersion: v1
kind: Service
metadata:
  name: my-lb-service
  namespace: default
  annotations:
    metallb.universe.tf/loadBalancerIPs: "10.10.88.85"
spec:
  type: LoadBalancer
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 8080

DNS Configuration

Local Development

Add entries to /etc/hosts on your workstation:

10.10.88.80  longhorn.local
10.10.88.80  myapp.local
10.10.88.80  dashboard.local

Production DNS

For production, configure DNS A records:

*.apps.example.com  ->  10.10.88.80

Useful Annotations

NGINX Ingress Annotations

annotations:
  # SSL/TLS
  nginx.ingress.kubernetes.io/ssl-redirect: "true"
  nginx.ingress.kubernetes.io/force-ssl-redirect: "true"

  # Timeouts
  nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
  nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
  nginx.ingress.kubernetes.io/proxy-connect-timeout: "60"

  # Body size
  nginx.ingress.kubernetes.io/proxy-body-size: "100m"

  # Authentication
  nginx.ingress.kubernetes.io/auth-type: basic
  nginx.ingress.kubernetes.io/auth-secret: basic-auth
  nginx.ingress.kubernetes.io/auth-realm: "Authentication Required"

  # CORS
  nginx.ingress.kubernetes.io/enable-cors: "true"
  nginx.ingress.kubernetes.io/cors-allow-origin: "*"

  # WebSocket
  nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
  nginx.ingress.kubernetes.io/upstream-hash-by: "$request_uri"

Troubleshooting

No External IP Assigned

Check MetalLB pods:

kubectl get pods -n metallb-system
kubectl logs -n metallb-system -l app.kubernetes.io/name=metallb

Check IP pool configuration:

kubectl get ipaddresspools -n metallb-system -o yaml

Check L2Advertisement:

kubectl get l2advertisements -n metallb-system -o yaml

Ingress Not Working

Check ingress controller:

kubectl get pods -n ingress-nginx
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx

Check ingress resource:

kubectl describe ingress <name> -n <namespace>

Test directly:

curl -H "Host: myapp.local" http://10.10.88.80/

Connection Refused

Verify service exists:

kubectl get svc -n <namespace>

Check endpoints:

kubectl get endpoints <service-name> -n <namespace>

Check pod is running:

kubectl get pods -n <namespace> -l <selector>

Security Considerations

Network Policies

Restrict traffic between namespaces:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

Rate Limiting

annotations:
  nginx.ingress.kubernetes.io/limit-rps: "10"
  nginx.ingress.kubernetes.io/limit-connections: "5"

IP Whitelisting

annotations:
  nginx.ingress.kubernetes.io/whitelist-source-range: "10.0.0.0/8,192.168.0.0/16"

FilesExpand file tree

NETWORKING.md

Latest commit

History

NETWORKING.md

File metadata and controls

Networking Configuration

Network Architecture

IP Address Allocation

Static Assignments

MetalLB Pool

Kubernetes Networks

MetalLB Installation

Step 1: Add Helm Repository

Step 2: Create Namespace

Step 3: Install MetalLB

Step 4: Configure IP Pool

Step 5: Verify

NGINX Ingress Controller

Step 1: Add Helm Repository

Step 2: Create Namespace

Step 3: Install

Step 4: Verify

Creating Ingress Resources

Basic HTTP Ingress

Ingress with TLS

Path-Based Routing

Configured Ingress Resources

Longhorn UI

LoadBalancer Services

View All LoadBalancer Services

Create a LoadBalancer Service

Request Specific IP

DNS Configuration

Local Development

Production DNS

Useful Annotations

NGINX Ingress Annotations

Troubleshooting

No External IP Assigned

Ingress Not Working

Connection Refused

Security Considerations

Network Policies

Rate Limiting

IP Whitelisting