How to use SessionAffinity to allow consecutive client requests to reach the same Pod while Using YARP as Ingress Controller?

[derbach]

Hi,
we are thinking about replacing our Traefik ingress controller with YARP, however, we are unsure whether the YARP Ingress Controller's Session Affinity will allow us to have sticky pods..

The examples we found were:
https://github.com/dotnet/yarp/tree/main/samples/KubernetesIngress.Sample#session-affinity
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/servers/yarp/session-affinity?view=aspnetcore-9.0

If we understand correctly, the K8s Service is Layer 3 / 4 and cannot check cookies/headers.
How could we make sure that our consecutive client requests are routed to the same pod?

[MihaZupan]

YARP is a layer 7 HTTP proxy, everything is processed on a per-request basis, so you do have access to headers.

[derbach]

Is there also a way to manage sticky sessions with this setup?
Traefik Ingress Controller -> YARP as API Gateway -> K8s Service as Destination -> K8s Backend Pods

All the examples I found have traefik point directly to the services, but we have our YARP lying in between.

[MihaZupan]

In that setup I would assume the the "YARP as API gateway" is doing session affinity.

Affinity is based on the destinations defined in the YARP config. If you have a destination entry for every pod in your service, sessions will be sticky per pod. If you're running YARP as the ingress controller I believe this is the case.

But if your YARP config only has 1 destination defined (e.g. just the service name where you then rely on DNS to map that to individual pods), YARP won't know how to stick with the same pod. In such cases you would have to expand the destination into the actual list of pods.

(Note that ingress support in YARP is a bit experimental right now)

[derbach]

We have deployed YARP as ingress controller in our cluster and it looks very promising so far. However, we ran into an issue when defining an ingress with a service that has multiple port entries with the same targetPort.

The error looks like this:
[15:58:08 ERR] Failed to apply the new config.
System.AggregateException: The proxy config is invalid. (Duplicate route 'dummy.dev/')
 ---> System.ArgumentException: Duplicate route 'dummy.dev/'

This is only happening for services that have more than 1 port defined. Our Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  labels:
    app.kubernetes.io/instance: dummy
    app.kubernetes.io/name: dummy
  name: dummy
spec:
  rules:
  - host: dummy.dev
    http:
      paths:
      - backend:
          service:
            name: dummy
            port:
              number: 80
        path: /
        pathType: Prefix

Did we miss something in the documentation?

[derbach]

Our service looks like below. Please not that it works perfectly after I remove one of the ports. Once I add the second port back, I get the duplicate Error again. If we use different targetPorts, it works as expected aswell

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: server
    app.kubernetes.io/instance: argocd
    app.kubernetes.io/name: argocd-server
  name: argocd-server
  namespace: argocd
spec:
  clusterIP: 172.20.66.83
  clusterIPs:
  - 172.20.66.83
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports: //Please note: 1 entry here works fine, more than 1 results in the error
  - name: http
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: https  
    port: 443
    protocol: TCP
    targetPort: 8080
  selector:
    app.kubernetes.io/instance: argocd
    app.kubernetes.io/name: argocd-server
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

[MihaZupan]

The issue with ports is now #2819