SessionAffinity on k3s

[cameronelliott]

Hi, I'm using k3s to develop and prototype OBS WebRTC broadcasting.

I'm pretty new to k3s.

One issue I am struggling with is getting sessionAffinity to work.
My http requests are moving between different backends even though I have sessionAffinity on the service.

Thanks for any ideas/pointers. Cameron

Here is my yaml

apiVersion: v1
kind: Pod
metadata:
  name: deadsfu-in
spec:
  hostNetwork: true
  containers:
  - name: deadsfu
    image: x186k/deadsfu:latest
    command: ["/app/main"]
    args: ["https://cam95.ddns5.com:8443","--z-debug"]
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deadsfu-out
spec:
  selector:
    matchLabels:
      app: deadsfu-out
  replicas: 2 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: deadsfu-out
    spec:
      containers:
      - name: deadsfu
        image: x186k/deadsfu:latest
        command: ["/app/main"]
        args: ["http://:8080","--ice-candidate-srflx","192.168.86.3","--dial-ingress","https://cam95.ddns5.com:8443/sub"]
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: deadsfu-service
spec:
  sessionAffinity: ClientIP
  selector:
    app: deadsfu-out
  ports:
    - protocol: TCP
      port: 8090
      targetPort: 8080

[ChristianCiach]

How do clients access your service? Are you using the integrated Traefik?

If you are using the integrated traefik and try to handle external requests, the issue is probably that the Service doesn't see the real IP of the client, but only the IP of the integrated load-balancer (klipper-lb) of K3s. This is an issue that pops up on the issue tracker all the time.

If my assumption is right and you're using the integrated Traefik, then there are tons of workarounds to handle this issue - and none of them are really satisfactory. A simple workaround is to customize the bundled Traefik to not use a Service of type "LoadBalancer", but deploying Traefik as a DaemonSet and binding it directly to the host-ports instead. You can do this by deploying this manifest:

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    deployment:
      kind: 'DaemonSet'
    ports:
      web:
        port: 8000
        hostPort: 80
      websecure:
        port: 8443
        hostPort: 443
    service:
      enabled: false

[ChristianCiach]

It may be even more complicated than I thought. If you are using IngressRoute, have a look at the sticky attribute of an IngressRoute object: https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/#stickiness-and-load-balancing