Upcoming deprecation of ingress-nginx

[minrk]

ingress-nginx is going to end support in March.

2i2c discussion on the same topic: 2i2c-org/infrastructure#7106

I see 3 main options:

• switch to nginx ingress maintained by nginx itself: https://docs.nginx.com/nginx-ingress-controller/install/helm/open-source/
• traefik ingress controller (builtin to k3s by default, so we could just not disable it) https://doc.traefik.io/traefik/migrate/nginx-to-traefik/
• Full migration to Gateway API via envoy-gateway https://gateway.envoyproxy.io

Summary from my experiments:

• Gateway migration seems to be "the future" but it doesn't really seem like the future is now. In a lot of ways, the Gateway API is worse for our use case than Ingress, because we don't have any of the problems it tries to solve. Also, BinderHub doesn't support Gateway API yet (it should and isn't hard), but everything else we deploy does.
• F5 nginx-ingress chart seems least disruptive, since it's the same thing, with config expressed a little differently
• traefik would probably be simplest, since it's builtin to k3s, but hardest to customize if we do it that way (since it's builtin to k3s, not part of our CD pipeline), but it's vulnerable to being run by a single VC-backed company, which have a habit of pulling the rug out from under users.

I have deployed a JupyterHub migration to traefik and another to envoy-gateway. Both went fine, traefik was easy while envoy-gateway took some figuring out and a lot more manual and redundant configuration. But what I actually think we should do here is switch to F5 nginx, which also happens to be the only one I haven't done.

[minrk]

Actually, reading through F5 nginx's use of CRDs, I actually don't want to do that. It's too much like migrating to Gateway API without being the official API. So it's either traefik, which is a really simple drop-in replacement, or Gateway, I think.

The other option is Contour, which is envoy-based, but has essentially been replaced with envoy-gateway, so is already in a semi-deprecated state.

[manics]

Option 5: try compiling the Chainguard fork mentioned in 2i2c-org/infrastructure#7106 (comment) ?

[minrk]

Yeah, there are technically more options (e.g. contour), but that was my summary of the viable options. Since chainguard doesn't appear to publish images or charts, it doesn't seem particularly useful to anyone but their paying customers. I don't think maintaining our own build pipeline for the chainguard sources is super feasible. I think the only really viable options are traefik (supports ingress and gateway) or envoy-gateway at this point.