"Cooperative Traffic Offload" proposal, and the future of Blixt with L7 ingress

[shaneutt]

Previously in this project we have been open to adding L7 ingress support. Recently, the Ingate project has emerged for L7 ingress with Gateway API. So first off I propose we update our README to officially declare L7 as a non-goal, and let L7 be focused on in Ingate so we're not stepping on each other's toes.

Secondly: This project was originally conceived to be capable of deployment as standalone OR being integrated with another Gateway API implementation. We've referred to the integrated mode as "cooperative traffic offload", and the basic idea is that any GatewayClass that doesn't support TCPRoute or UDPRoute could use Blixt to handle those route types for it, but in a way that's fairly seemless for the user (e.g. they use the same GatewayClass and Gateway that they otherwise would use, but the offload to Blixt happens under the hood).

I want to propose that we make this an explicit goal of the project, and work with the Gateway API community for any API-enablement that would be needed to support it.

[EandrewJones]

Quick thoughts:

we update our README to officially declare L7 as a non-goal

+ 1 from me. Although I am personally interested implementing L7 features in rust, I think the project would benefit from refining it's scope and clarifying goals.

create a pre-v1 issue to spike/explore "cooperative traffic offload", and aim to see how Ingate and Blixt might be able to be packaged together and complement each other

I very much like the idea of treating blixt as a (or the?) L4 implementation for ingate for numerous reasons:

1. I personally believe projects benefit greatly from having a clear, focused and pragmatic use case. Blixt by itself may have very limited appeal, but ingate will surely have healthy adoption, so that's a boon for us.
2. "Cooperative traffic offload" is also an interesting concept to explore from a Gateway API perspective. If we can achieve it with ingate, that can serve as an example from which to build a standard for doing this more broadly.
3. The above reason provides a raison d'etre for Blixt to continue existing in sig-network which I know we've been grappling with
4. If ingate can benefit from L4 work we've already done, awesome!

But I do have a few questions:

1. Would be the L4 implementation for ingate or just something users opt into?
2. Would blixt be "dataplane-only" under this scenario and the controlplane logic stays in the primary gateway implementation the user is using? A "headless" implementation so to speak...

[shaneutt]

The above reason provides a raison d'etre for Blixt to continue existing in sig-network which I know we've been grappling with

Exactly, this was one of the motivations for this proposal 👍

Would be the L4 implementation for ingate or just something users opt into?

I can't speak for James and the other InGate folks, but I'm thinking what we could propose is a future where they could simply package in Blixt to add seamless (the users never have to know its Blixt) L4 support, and see what they think.

Would blixt be "dataplane-only" under this scenario and the controlplane logic stays in the primary gateway implementation the user is using? A "headless" implementation so to speak...

So I looked and I don't think we ever documented the "cooperative traffic offload" idea in the early days, we only discussed it maybe on zooms or on Slack, but to recap: the idea is that any particular GatewayClass could configure a deployed Blixt control-plane in such a way that it would be responsible for TCPRoute and UDPRoute (and maybe even TLSRoute) but from the API and user perspective that's opaque they just see it as the implementation as named in the GatewayClass. So the answer is technically "no", because the control-plane would indeed exist and we wouldn't require the implementation (e.g. InGate) to add controllers and integration with the Blixt data-plane, but they would operate on the same resources so the user doesn't perceive it.

[hargut]

Some thoughts on the topic:

• L4 TCP/UDP in eBPF seems to be a reasonable set of features
• L7 with TLS in eBPF seems not to be feasible or the L7 component would likely require standard socket usage

Where I can imagine blixt to fit in well is:

• L7 gateway controller extended with blixt for L4 (aka. cooperative traffic offloading)
• probably also use cases where L7 is handled externally (e.g. Api Management, Api Gateway, ...)
  • with potentially directly load-balancing onto the pods without an ingress/gateway controller within k8s
• TCP/UDP load-balancing to L7 gateway pods (e.g. DNS entries to a limited/static node-pool running blixt, and a scalable set of L7 gateway pods that could dynamically scale)

An important point will be to have L4 stickyness for connections on TCP and UDP (QUIC connection IDs) to allow for correctly routing TLS encrypted connections onto L7 backends. TLSRoute is likely only partially interesting due to the future usage of ECH (Encrypted Client Hello).

[EandrewJones]

Harald,

TCP/UDP load-balancing to L7 gateway pods
Could you speak more to the business case for this? For example, under what conditions might someone opt for this?

[hargut]

So far I've not thought about a real business case and was just trying to think about possibly interesting use cases.

In many cloud environments this approach could allow to drop the cloud providers L4 balancer which might be an interesting option for cost sensitive or hybrid cloud projects (removing an external cloud provider dependency).

L7 scalable would be interesting when TLS is terminated at the L7 pods which would likely be the first point where resource bottlenecks could occur (for example using something like caddy or traefik which already offer ACME integration (not sure if this is already supported within the Gateway API integration)).

[shaneutt]

L7 with TLS in eBPF seems not to be feasible or the L7 component would likely require standard socket usage
TLSRoute is likely only partially interesting due to the future usage of ECH (Encrypted Client Hello).

I agree. We've been throwing out the maybe TLSRoute thing for a few years. The thinking has been: we don't necessarily have to do everything in eBPF (originally I was considering whether we could implement a minimal TLSRoute proxy with io_uring). However given that this project is small and needs a tight scope and focus, and the implementation requirements would pull us pretty far away from home base, I do think we can probably just agree we need to leave TLSRoute alone for now. 👍

Where I can imagine blixt to fit in well is:

L7 gateway controller extended with blixt for L4 (aka. cooperative traffic offloading)
probably also use cases where L7 is handled externally (e.g. Api Management, Api Gateway, ...) with potentially directly load-balancing onto the pods without an ingress/gateway controller within k8s
TCP/UDP load-balancing to L7 gateway pods (e.g. DNS entries to a limited/static node-pool running blixt, and a scalable set of L7 gateway pods that could dynamically scale)

👍

[hargut]

My thought was more in the direction of L7 Gateway features that need to parse the HTTP request (e.g. Path, Headers, ...) for routing decisions when using TLS. In order to access the information TLS needs to be terminated which is not really feasible within eBPF.

TLSRoute may even be possible within eBPF. As far as I see the packet data is available within tc programs (https://docs.ebpf.io/linux/program-context/__sk_buff/). So it could work to identify Client Hello messages, extract the SNI host and to setup the TCP LB accordingly. I think this is not really L7 but probably L5 and would only work when Encrypted Client Hello (ECH) is not in use.

[shaneutt]

I hear you, but I guess where I was trying to head was: We've struggled to keep this project going, so given how small the project is it maybe wiser to explicitly avoid TLSRoute for the moment (I know I was the one that brought it up in the first place, but I think my mind is changing) and focus strongly on the offload support for L4 🤔