From f53169e2cce0970a6d52c97f4b7af83b1c687e37 Mon Sep 17 00:00:00 2001 From: Ricardo Pchevuzinske Katz Date: Wed, 3 Sep 2025 18:01:36 -0300 Subject: [PATCH 1/4] GEP-2643: TLSRoute --- geps/gep-2643/index.md | 235 ++++++++++++++++++++++++++++++++++++ geps/gep-2643/metadata.yaml | 16 +++ 2 files changed, 251 insertions(+) create mode 100644 geps/gep-2643/index.md create mode 100644 geps/gep-2643/metadata.yaml diff --git a/geps/gep-2643/index.md b/geps/gep-2643/index.md new file mode 100644 index 0000000000..8402c42bb5 --- /dev/null +++ b/geps/gep-2643/index.md @@ -0,0 +1,235 @@ + +# GEP-2643: TLS based passthrough Route / TLSRoute + +* Issue: [#2643](https://github.com/kubernetes-sigs/gateway-api/issues/2643) +* Status: Standard + +## TLDR + +This GEP documents the implementation of a route type that uses the Server Name +Indication attribute (aka SNI) of a TLS handshake to make the routing decision. + +This is also known as TLS passthrough, where after the server name is identified, +the gateway does a full encrypted passthrough of the communication. + +## Goals + +* Provide a TLS passthrough route type, based on the SNI identification. +* Provide a load balancing of a passthrough’d route, allowing a user to define +a route, based on the SNI identification that should pass the traffic to N load +balanced backends + +## Longer Term Goals +* Implement capabilities for [ESNI](https://www.cloudflare.com/learning/ssl/what-is-encrypted-sni/) + +## Non-Goals +* Provide an interface for users to define different listeners or ports for the +`TLSRoute` - This will be covered by the `ListenerSet` enhancement. +* Replace `TCPRoute` for cases where terminated frontend encryption is not needed. +* When using `TLSRoute` passthrough, support `PROXY` protocol on the gateway listener. +* When using `TLSRoute` passthrough, support `PROXY` protocol on the communication +between the Gateway and the backend. +* Allow a `TLSRoute` to have the encryption termination on the `Gateway` side and +do the unencrypted passthrough. This case is covered by using a `TCPRoute`. + * The feature of TLSRoute termination is tracked on its own [issue](https://github.com/kubernetes-sigs/gateway-api/issues/2111) + and should be covered on a specific GEP. +* Support TLS over UDP or UDP-based protocols such as `QUIC`. +* Support attributes other than the SNI/hostname for the route selection + +## Introduction + +While many application routing cases can be implemented using HTTP/L7 matching +(the tuple protocol:hostname:port:path), there are some specific cases where direct, +encrypted communication to the backend may be required, without further assertion. For example: +* A backend that is TLS based but not HTTP based (e.g., a Kafka service, or a +Postgres service, with its listener being TLS enabled). +* Some WebRTC solutions. +* Backends that can require direct client-certificate authentication (e.g., OAuth). + +For the example cases above, it is desired that the routing is made as a passthrough +mode, where the `Gateway` passes the packets to the backend without terminating TLS. + +While this routing is possible using other mechanisms such as a `TCPRoute` or a +Kubernetes service of type `LoadBalancer`, it may be desired to have a single point +of traffic convergence (like one single `LoadBalancer`) for reasons like cost, +traffic control, etc. + +An implementation of `TLSRoute` achieves this end using the +[server_name TLS attribute](https://datatracker.ietf.org/doc/html/rfc6066#section-3) +to determine what backend should be used for a given request. `TLSRoute` thereby +enables the use of a single gateway listener to handle traffic for multiple routes. + +## API + +```golang +type TLSRouteSpec struct { + // Hostnames defines a set of SNI hostnames that should match against the + // SNI attribute of TLS ClientHello message in TLS handshake. This matches + // the RFC 1123 definition of a hostname with 2 notable exceptions: + // + // 1. IPs are not allowed in SNI hostnames per RFC 6066. + // 2. A hostname may be prefixed with a wildcard label (`*.`). The wildcard + // label must appear by itself as the first label. + // + // If a hostname is specified by both the Listener and TLSRoute, there + // must be at least one intersecting hostname for the TLSRoute to be + // attached to the Listener. For example: + // + // * A Listener with `test.example.com` as the hostname matches TLSRoutes + // that have specified at least one of `test.example.com` or + // `*.example.com`. + // * A Listener with `*.example.com` as the hostname matches TLSRoutes + // that have specified at least one hostname that matches the Listener + // hostname. For example, `test.example.com` and `*.example.com` would both + // match. On the other hand, `example.com` and `test.example.net` would not + // match. + // + // If both the Listener and TLSRoute have specified hostnames, any + // TLSRoute hostnames that do not match the Listener hostname MUST be + // ignored. For example, if a Listener specified `*.example.com`, and the + // TLSRoute specified `test.example.com` and `test.example.net`, + // `test.example.net` must not be considered for a match. + // + // If both the Listener and TLSRoute have specified hostnames, and none + // match with the criteria above, then the TLSRoute is not accepted. The + // implementation must raise an 'Accepted' Condition with a status of + // `False` in the corresponding RouteParentStatus. + // +required + // +kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MaxItems=16 + Hostnames []Hostname `json:"hostnames,omitempty"` + // Rules are a list of actions. + // + // +required + // +kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MaxItems=1 + Rules []TLSRouteRule `json:"rules,omitempty"` +} + +type TLSRouteRule struct { + // Name is the name of the route rule. This name MUST be unique within a Route if it is set. + // + // +optional + Name *SectionName `json:"name,omitempty"` + // BackendRefs (same as other BackendRef here) + // +required + // +kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MaxItems=16 + BackendRefs []BackendRef `json:"backendRefs,omitempty"` +} +``` + + +## Request flow + +A simple `TLSRoute` implementation manifest would look like: + +```yaml +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: gateway-tlsroute +spec: + gatewayClassName: "my-class" + listeners: + - name: somelistener + port: 443 + protocol: TLS + hostname: "*.example.com" + allowedRoutes: + namespaces: + from: Same + kinds: + - kind: TLSRoute + tls: + mode: Passthrough +--- +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: TLSRoute +metadata: + name: my-tls-route +spec: + parentRefs: + - name: gateway-tlsroute + hostnames: + - abc.example.com + rules: + - backendRefs: + - name: tls-backend + port: 443 +``` + +A typical [north/south](https://gateway-api.sigs.k8s.io/concepts/glossary/#northsouth-traffic) +API request flow for a gateway implemented using a `TLSRoute` is: +* A client makes a request to https://foo.example.com. +* DNS resolves the name to a `Gateway` address. +* The reverse proxy receives the request on a `Listener` and uses the +[Server Name Indication](https://datatracker.ietf.org/doc/html/rfc6066#section-3) +attribute to match an `TLSRoute`. +* The reverse proxy passes through the request directly to one or more objects, +i.e. `Service`, in the cluster based on `backendRefs` rules of the `TLSRoute`. + +## Conflict management and precedences + +The following conflict situations are covered by TLSRoute and TLS passthrough cases: + +* When a Gateway contains a listener with `protocol=TLS`, the Gateway MUST NOT +allow any other kind of listener on the same port and the Gateway SHOULD be +marked as `Accepted=False` in case there are different kinds of listeners on +the same port. +* When a Gateway contains a listener with `protocol=TLS` and `tls.mode=Passthrough`, +the `Gateway` MUST NOT allow another listener on the same port with a different +`tls.mode` and the `Gateway` SHOULD be marked as `Accepted=False`. +* Any violating Listener should have a Condition `Conflicted=True`. +* If a hostname is specified by both the `Listener` and `TLSRoute`, there must +be at least one intersecting hostname for the `TLSRoute` to be attached to the +`Listener`. + * A `Gateway listener` with `test.example.com` as the hostname matches a `TLSRoute` that + specifies `test.example.com` but does not match a `TLSRoute` that specifies `*.example.com` + * A `Gateway listener` with `*.example.com` as the hostname matches a `TLSRoute` that + specifies at least one hostname that matches the `Gateway listener` hostname. + For example, `test.example.com` and `*.example.com` would both match. On the + other hand, `example.com` and `test.example.net` would not match. + * If both the `Gateway listener` and `TLSRoute` specify hostnames, any `TLSRoute` + hostnames that do not match the `Gateway listener` hostname MUST be ignored + for that Listener. For example, if a `Gateway listener` specified `*.example.com`, + and the `TLSRoute` specified `test.example.com` and `test.example.net`, + the later must not be considered for a match. + * In any of the cases above, the `TLSRoute` should have a `Condition` of `Accepted=True`. + +## Conformance Details + +### Feature Names + +* TLSRoute + +### Conformance tests + +| Description | Outcome | Notes | +| :---- | :---- | :---- | +| A single TLSRoute in the gateway-conformance-infra namespace attaches to a Gateway in the same namespace | A request to a hostname served by the TLSRoute should be passthrough directly to the backend. Check if the termination happened, if no additional Gateway header was added | Already [implemented](https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/tests/tlsroute-simple-same-namespace.go) needs review | +| A single TLSRoute in the gateway-conformance-infra namespace, with a backendRef in another namespace without valid ReferenceGrant, should have the ResolvedRefs condition set to False | TLSRoute conditions must have a Not Permitted status | Already [implemented](https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/tests/tlsroute-invalid-reference-grant.go) needs review | +| A TLSRoute trying to attach to a gateway without a “tls/passthrough” listener should be rejected | Condition on the TLSRoute that it was rejected (discuss with community the right condition to be used here) | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | +| A TLSRoute with a hostname that does not match the Gateway hostname should be rejected (eg.: route with hostname [www.example.com](http://www.example.com), gateway with hostname www1.example.com) | Condition on the TLSRoute that it was rejected | | +| A TLSRoute with an IP on its hostname should be rejected | Condition on the TLSRoute that it was rejected | | +| (impl specific) A Gateway containing a Listener of type TLS/Passthrough and a Listener of type HTTPS/Terminate should be accepted, and should multiplex the requests to TLSRoute and HTTPRoute correctly | Being able to do a request to a HTTP route being terminated on gateway (eg.: terminated.example.tld/xpto) and to a TLS Passthrough route on the same gateway, but different host (passthrough.example.tld) | | +| A Gateway with \*.example.tld on a TLS listener should allow a TLSRoute with hostname some.example.tld to be attached to it (and the same, but with a non wildcard hostname) | TLSRoute should be able to attach to the Gateway using the matching hostname, a request should succeed | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | +| A Gateway with something.example.tld on a TLS listener hostname should not allow a TLSRoute of \*.example.tld to be attached | TLSRoute should be rejected with invalid hostname (we should NOT support wildcard hostnames on a TLSRoute spec) | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | +| Invalid TLSRoute with invalid BackendObjectReference performs no default forwarding | | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | +| For a [Listener](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) setting mode: "terminate", TLSRoute should not be present in [ListenerStatus.SupportedKinds](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) | | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | +| Attempting to attach a TLSRoute to a [Listener](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) setting mode: "terminate" should yield [RouteConditionType](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) Accepted with status: false and [RouteConditionReason](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) NotAllowedByListeners on the TLSRoute | | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | +| For a [Listener](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) setting mode: "terminate", attempting to set TLSRoute in [AllowedRoutes.kinds](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.AllowedRoutes) would yield a [ListenerConditionType](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) Accepted with status: false and [ListenerConditionReason](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) InvalidRouteKinds | | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | + +Pending conformance verifications: + +* [https://github.com/kubernetes-sigs/gateway-api/issues/3466](https://github.com/kubernetes-sigs/gateway-api/issues/3466) +* [https://github.com/kubernetes-sigs/gateway-api/issues/2153](https://github.com/kubernetes-sigs/gateway-api/issues/2153) + +### References +* [Existing API](https://github.com/kubernetes-sigs/gateway-api/blob/main/apis/v1alpha3/tlsroute_types.go) +* [TLS Terminology - Needs update](https://github.com/kubernetes-sigs/gateway-api/blob/d28cd59d37887be07b879f098cff7b14a87c0080/geps/gep-2907/index.md?plain=1#L29) +* [TLSRoute promotion issue](https://github.com/kubernetes-sigs/gateway-api/issues/3165) +* [TLS Multiplexing issue discussion](https://github.com/kubernetes-sigs/gateway-api/issues/623) +* [TLSRoute intersecting hostnames issue](https://github.com/kubernetes-sigs/gateway-api/issues/3541) +* [TLSRoute termination feature request](https://github.com/kubernetes-sigs/gateway-api/issues/2111) +* [GatewayAPI TLS Use Cases](https://docs.google.com/document/d/17sctu2uMJtHmJTGtBi_awGB0YzoCLodtR6rUNmKMCs8) diff --git a/geps/gep-2643/metadata.yaml b/geps/gep-2643/metadata.yaml new file mode 100644 index 0000000000..b629463c21 --- /dev/null +++ b/geps/gep-2643/metadata.yaml @@ -0,0 +1,16 @@ +apiVersion: internal.gateway.networking.k8s.io/v1alpha1 +kind: GEPDetails +number: 2643 +name: TLS based passthrough Route / TLSRoute +status: Provisional +# Note: The feature already exists, the GEP is a retroactive collection of information +authors: + - candita + - Miciah + - rikatz +relationships: + seeAlso: + - number: 2907 + name: TLS Configuration Placement and Terminology +references: + - https://datatracker.ietf.org/doc/html/rfc6066#section-3 \ No newline at end of file From 066499d342f6d5d391bf043a7a4dc4526c5ce571 Mon Sep 17 00:00:00 2001 From: Ricardo Pchevuzinske Katz Date: Wed, 3 Sep 2025 18:21:19 -0300 Subject: [PATCH 2/4] Add multiplexing to TLSRoute --- geps/gep-2643/index.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/geps/gep-2643/index.md b/geps/gep-2643/index.md index 8402c42bb5..b76b7310e5 100644 --- a/geps/gep-2643/index.md +++ b/geps/gep-2643/index.md @@ -197,6 +197,34 @@ be at least one intersecting hostname for the `TLSRoute` to be attached to the the later must not be considered for a match. * In any of the cases above, the `TLSRoute` should have a `Condition` of `Accepted=True`. +## Multiplexing support + +It may be desired that on a `Gateway` a single TLS listener provides termination +for `HTTPRoutes` and passthrough to `TLSRoutes`. + +This can be achieved with a `Gateway` that specifies the following listeners: + +```yaml + listeners: + - name: somelistener + port: 443 + protocol: TLS + hostname: "*.example.tld" + tls: + mode: Passthrough + - name: httpslistener + port: 443 + protocol: HTTPS + hostname: "*.anotherexample.tld" + tls: + mode: Terminate + certificateRefs: ... +``` + +With the specification above, any request to hostnames on `*.example.tld` should be +attached to `TLSRoutes` while requests to `*.anotherexample.tld` are terminated +on the `Gateway` and attached to a `HTTPRoute`. + ## Conformance Details ### Feature Names From 033815bdc1d8ff40fc3fec792541a26a57fbe60d Mon Sep 17 00:00:00 2001 From: Ricardo Pchevuzinske Katz Date: Mon, 29 Sep 2025 16:09:37 -0300 Subject: [PATCH 3/4] Update TLSRoute GEP with more information --- geps/gep-2643/index.md | 37 ++++++++++++++++++++++++++----------- 1 file changed, 26 insertions(+), 11 deletions(-) diff --git a/geps/gep-2643/index.md b/geps/gep-2643/index.md index b76b7310e5..044fc90bb8 100644 --- a/geps/gep-2643/index.md +++ b/geps/gep-2643/index.md @@ -21,6 +21,7 @@ balanced backends ## Longer Term Goals * Implement capabilities for [ESNI](https://www.cloudflare.com/learning/ssl/what-is-encrypted-sni/) +* Implement matchers on TLSRoute rules like [ALPN](https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation) ## Non-Goals * Provide an interface for users to define different listeners or ports for the @@ -71,6 +72,7 @@ type TLSRouteSpec struct { // 2. A hostname may be prefixed with a wildcard label (`*.`). The wildcard // label must appear by itself as the first label. // + // // If a hostname is specified by both the Listener and TLSRoute, there // must be at least one intersecting hostname for the TLSRoute to be // attached to the Listener. For example: @@ -85,20 +87,27 @@ type TLSRouteSpec struct { // match. // // If both the Listener and TLSRoute have specified hostnames, any - // TLSRoute hostnames that do not match the Listener hostname MUST be + // TLSRoute hostnames that do not match any Listener hostname MUST be // ignored. For example, if a Listener specified `*.example.com`, and the // TLSRoute specified `test.example.com` and `test.example.net`, // `test.example.net` must not be considered for a match. // // If both the Listener and TLSRoute have specified hostnames, and none - // match with the criteria above, then the TLSRoute is not accepted. The + // match with the criteria above, then the TLSRoute is not accepted for that + // Listener. If the TLSRoute does not match any Listener on the parent, the // implementation must raise an 'Accepted' Condition with a status of // `False` in the corresponding RouteParentStatus. + // + // A Listener MUST be of type TLS when a TLSRoute attaches to it. The + // impolementation MUST raise an 'Accepted' Condition with a status of + // `False` in the corresponding RouteParentStatus with the reason + // of "UnsupportedValue" in case a Listener of the wrong type is used. + // // +required // +kubebuilder:validation:MinItems=1 // +kubebuilder:validation:MaxItems=16 Hostnames []Hostname `json:"hostnames,omitempty"` - // Rules are a list of actions. + // Rules is a list of TLS matchers and actions. // // +required // +kubebuilder:validation:MinItems=1 @@ -173,14 +182,13 @@ i.e. `Service`, in the cluster based on `backendRefs` rules of the `TLSRoute`. The following conflict situations are covered by TLSRoute and TLS passthrough cases: -* When a Gateway contains a listener with `protocol=TLS`, the Gateway MUST NOT -allow any other kind of listener on the same port and the Gateway SHOULD be -marked as `Accepted=False` in case there are different kinds of listeners on -the same port. -* When a Gateway contains a listener with `protocol=TLS` and `tls.mode=Passthrough`, -the `Gateway` MUST NOT allow another listener on the same port with a different -`tls.mode` and the `Gateway` SHOULD be marked as `Accepted=False`. -* Any violating Listener should have a Condition `Conflicted=True`. +* When a Gateway supports [Multiplexing](#multiplexing-support) it CAN allow multiple +listeners on the same port, as soon as they have different protocols (TLS or HTTPS), +and the related `tls.mode`, and different hostnames. +* When a Gateway does not support [Multiplexing](#multiplexing-support) and contains +a listener with `protocol=TLS`, the Gateway MUST NOT allow any other kind of +listener on the same port, and any violating Listener should have a Condition `OverlappingTLSConfig=True` +with the reason `OverlappingProtocols`. * If a hostname is specified by both the `Listener` and `TLSRoute`, there must be at least one intersecting hostname for the `TLSRoute` to be attached to the `Listener`. @@ -225,11 +233,18 @@ With the specification above, any request to hostnames on `*.example.tld` should attached to `TLSRoutes` while requests to `*.anotherexample.tld` are terminated on the `Gateway` and attached to a `HTTPRoute`. +The support for Multiplexing on TLS is `Implementation Specific`, and implementations +that support this feature MUST announce it on `GatewayClass.Status.SupportedFeatures`. + +Implementations that don't support multiplexing SHOULD mark the violating listeners with +a Condition `Conflicted=True`. + ## Conformance Details ### Feature Names * TLSRoute +* TLSMultiplexing ### Conformance tests From 23c275e1b508cb9bea1108c16a7f7f9b36341482 Mon Sep 17 00:00:00 2001 From: Ricardo Pchevuzinske Katz Date: Tue, 30 Sep 2025 15:09:54 -0300 Subject: [PATCH 4/4] Add support for TLSRoute termination --- geps/gep-2643/index.md | 198 +++++++++++++++++++++++++++++++----- geps/gep-2643/metadata.yaml | 3 +- 2 files changed, 174 insertions(+), 27 deletions(-) diff --git a/geps/gep-2643/index.md b/geps/gep-2643/index.md index 044fc90bb8..6714f85e5a 100644 --- a/geps/gep-2643/index.md +++ b/geps/gep-2643/index.md @@ -1,5 +1,5 @@ -# GEP-2643: TLS based passthrough Route / TLSRoute +# GEP-2643: TLS/SNI based routing / TLSRoute * Issue: [#2643](https://github.com/kubernetes-sigs/gateway-api/issues/2643) * Status: Standard @@ -9,15 +9,17 @@ This GEP documents the implementation of a route type that uses the Server Name Indication attribute (aka SNI) of a TLS handshake to make the routing decision. -This is also known as TLS passthrough, where after the server name is identified, -the gateway does a full encrypted passthrough of the communication. +While this feature is also known sometimes as TLS passthrough, where after the +server name is identified, the gateway does a full encrypted passthrough of the +communication, this GEP will also cover cases where a TLS communication is +terminated on the Gateway and passed unencrypted to a backend. ## Goals -* Provide a TLS passthrough route type, based on the SNI identification. -* Provide a load balancing of a passthrough’d route, allowing a user to define -a route, based on the SNI identification that should pass the traffic to N load -balanced backends +* Provide a TLS route type, based on the SNI identification. +* Support both Termination and Passthrough TLS modes for TLSRoute +* Provide load balancing of a TLS route, allowing a user to define a route, +based on the SNI identification that should pass the traffic to N load balanced backends ## Longer Term Goals * Implement capabilities for [ESNI](https://www.cloudflare.com/learning/ssl/what-is-encrypted-sni/) @@ -30,10 +32,6 @@ balanced backends * When using `TLSRoute` passthrough, support `PROXY` protocol on the gateway listener. * When using `TLSRoute` passthrough, support `PROXY` protocol on the communication between the Gateway and the backend. -* Allow a `TLSRoute` to have the encryption termination on the `Gateway` side and -do the unencrypted passthrough. This case is covered by using a `TCPRoute`. - * The feature of TLSRoute termination is tracked on its own [issue](https://github.com/kubernetes-sigs/gateway-api/issues/2111) - and should be covered on a specific GEP. * Support TLS over UDP or UDP-based protocols such as `QUIC`. * Support attributes other than the SNI/hostname for the route selection @@ -41,7 +39,7 @@ do the unencrypted passthrough. This case is covered by using a `TCPRoute`. While many application routing cases can be implemented using HTTP/L7 matching (the tuple protocol:hostname:port:path), there are some specific cases where direct, -encrypted communication to the backend may be required, without further assertion. For example: +encrypted communication to the backend may be required, without further assertion. For example: * A backend that is TLS based but not HTTP based (e.g., a Kafka service, or a Postgres service, with its listener being TLS enabled). * Some WebRTC solutions. @@ -50,6 +48,10 @@ Postgres service, with its listener being TLS enabled). For the example cases above, it is desired that the routing is made as a passthrough mode, where the `Gateway` passes the packets to the backend without terminating TLS. +On some other cases, it is desired that the termination is done on the `Gateway` +and the traffic passes the unencrypted packets to the backend without caring about +attributes other than a TCP communication. + While this routing is possible using other mechanisms such as a `TCPRoute` or a Kubernetes service of type `LoadBalancer`, it may be desired to have a single point of traffic convergence (like one single `LoadBalancer`) for reasons like cost, @@ -99,7 +101,7 @@ type TLSRouteSpec struct { // `False` in the corresponding RouteParentStatus. // // A Listener MUST be of type TLS when a TLSRoute attaches to it. The - // impolementation MUST raise an 'Accepted' Condition with a status of + // implementation MUST raise an 'Accepted' Condition with a status of // `False` in the corresponding RouteParentStatus with the reason // of "UnsupportedValue" in case a Listener of the wrong type is used. // @@ -128,10 +130,13 @@ type TLSRouteRule struct { } ``` - ## Request flow -A simple `TLSRoute` implementation manifest would look like: +Following are some of the requests flow covered by TLSRoute + +### TLSRoute + TLS Passthrough +In this workflow, the TLS traffic will be matched against the `SNI attribute` of +the request and then directed to the backends. ```yaml apiVersion: gateway.networking.k8s.io/v1 @@ -161,7 +166,7 @@ spec: parentRefs: - name: gateway-tlsroute hostnames: - - abc.example.com + - foo.example.com rules: - backendRefs: - name: tls-backend @@ -178,17 +183,159 @@ attribute to match an `TLSRoute`. * The reverse proxy passes through the request directly to one or more objects, i.e. `Service`, in the cluster based on `backendRefs` rules of the `TLSRoute`. +### TLSRoute + TLS Termination +In this workflow, the TLS traffic will be matched against the `SNI attribute` of +the request, terminated on the `Gateway` and passed unencrypted as a `TCP` connection +to the backends. + +```yaml +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: gateway-tlsroute +spec: + gatewayClassName: "my-class" + listeners: + - name: somelistener + port: 443 + protocol: TLS + hostname: "*.example.com" + allowedRoutes: + namespaces: + from: Same + kinds: + - kind: TLSRoute + tls: + mode: Terminate + certificateRefs: + - name: listener + kind: Secret +--- +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: TLSRoute +metadata: + name: my-rtmp-route +spec: + parentRefs: + - name: gateway-tlsroute + hostnames: + - rtmp.example.com + rules: + - backendRefs: + - name: rtmp-backend + port: 12345 +``` + +A typical [north/south](https://gateway-api.sigs.k8s.io/concepts/glossary/#northsouth-traffic) +API request flow for a gateway implemented using both `TLSRoute` is: +* A client makes a request to `rtmps://rtmp.example.com:443`. +* DNS resolves the name to a `Gateway` address. +* The reverse proxy receives the request on a `Listener` and uses the +[Server Name Indication](https://datatracker.ietf.org/doc/html/rfc6066#section-3) +attribute to match an `TLSRoute`. +* The reverse proxy terminates the TLS negotiation on the `Gateway`. +* The reverse proxy passes unencrypted request to one or more objects, +i.e. `Service`, in the cluster based on `backendRefs` rules of the `TLSRoute`. + +### TLSRoute + Mixed Termination +In this workflow, the TLS traffic will be matched against the `SNI attribute` of +the request, and based on the SNI attribute be directed to the backends on Passthrough +mode or be terminated on the `Gateway` and passed unencrypted to the backends. + +```yaml +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: gateway-tlsroute +spec: + gatewayClassName: "my-class" + listeners: + - name: terminatelistener + port: 443 + protocol: TLS + hostname: "rtmp.example.com" + allowedRoutes: + namespaces: + from: Same + kinds: + - kind: TLSRoute + tls: + mode: Terminate + certificateRefs: + - name: listener + kind: Secret + - name: passthroughlistener + port: 443 + protocol: TLS + hostname: "direct.example.com" + allowedRoutes: + namespaces: + from: Same + kinds: + - kind: TLSRoute + tls: + mode: Passthrough +--- +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: TLSRoute +metadata: + name: my-tls-route +spec: + parentRefs: + - name: gateway-tlsroute + hostnames: + - direct.example.com + rules: + - backendRefs: + - name: tls-backend + port: 443 +--- +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: TLSRoute +metadata: + name: my-rtmp-route +spec: + parentRefs: + - name: gateway-tlsroute + hostnames: + - rtmp.example.com + rules: + - backendRefs: + - name: rtmp-backend + port: 12345 +``` + +A typical [north/south](https://gateway-api.sigs.k8s.io/concepts/glossary/#northsouth-traffic) +API request flow for a gateway implemented using a `TLSRoute` is: +* A client makes a request to `rtmps://rtmp.example.com:443`. +* DNS resolves the name to a `Gateway` address. +* The reverse proxy receives the request on a `Listener` and uses the +[Server Name Indication](https://datatracker.ietf.org/doc/html/rfc6066#section-3) +attribute to match an `TLSRoute`. +* The reverse proxy terminates the TLS negotiation on the `Gateway`. +* The reverse proxy passes unencrypted request to one or more objects, +i.e. `Service`, in the cluster based on `backendRefs` rules of the `TLSRoute`. +* A new request is made to `direct.example.com` and the same identification flow +happens. +* The reverse proxy receiving the request identifies that this is a Passthrough request +and passes through the request directly to one or more objects, +i.e. `Service`, in the cluster based on `backendRefs` rules of the `TLSRoute`. + +### TLSRoute + HTTPs multiplexing +In the case of TLSRoute + HTTPs multiplexing, it should operate similarly to the +mixed TLSRoute, but instead of decrypting and passing the TCP connection as is to +the backend, the proper HTTPRoute workflow should happen. + ## Conflict management and precedences -The following conflict situations are covered by TLSRoute and TLS passthrough cases: +The following conflict situations are covered by TLSRoute cases: * When a Gateway supports [Multiplexing](#multiplexing-support) it CAN allow multiple -listeners on the same port, as soon as they have different protocols (TLS or HTTPS), -and the related `tls.mode`, and different hostnames. +listeners on the same port, as soon as they do not conflict on `hostnames` and `tls.mode`. * When a Gateway does not support [Multiplexing](#multiplexing-support) and contains a listener with `protocol=TLS`, the Gateway MUST NOT allow any other kind of -listener on the same port, and any violating Listener should have a Condition `OverlappingTLSConfig=True` -with the reason `OverlappingProtocols`. +listener on the same port, and any violating Listener should have a Condition +`OverlappingTLSConfig=True` with the reason `OverlappingProtocols`. * If a hostname is specified by both the `Listener` and `TLSRoute`, there must be at least one intersecting hostname for the `TLSRoute` to be attached to the `Listener`. @@ -252,16 +399,15 @@ a Condition `Conflicted=True`. | :---- | :---- | :---- | | A single TLSRoute in the gateway-conformance-infra namespace attaches to a Gateway in the same namespace | A request to a hostname served by the TLSRoute should be passthrough directly to the backend. Check if the termination happened, if no additional Gateway header was added | Already [implemented](https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/tests/tlsroute-simple-same-namespace.go) needs review | | A single TLSRoute in the gateway-conformance-infra namespace, with a backendRef in another namespace without valid ReferenceGrant, should have the ResolvedRefs condition set to False | TLSRoute conditions must have a Not Permitted status | Already [implemented](https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/tests/tlsroute-invalid-reference-grant.go) needs review | -| A TLSRoute trying to attach to a gateway without a “tls/passthrough” listener should be rejected | Condition on the TLSRoute that it was rejected (discuss with community the right condition to be used here) | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | +| A TLSRoute trying to attach to a gateway without a “tls” listener should be rejected | Condition on the TLSRoute that it was rejected (discuss with community the right condition to be used here) | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | | A TLSRoute with a hostname that does not match the Gateway hostname should be rejected (eg.: route with hostname [www.example.com](http://www.example.com), gateway with hostname www1.example.com) | Condition on the TLSRoute that it was rejected | | | A TLSRoute with an IP on its hostname should be rejected | Condition on the TLSRoute that it was rejected | | -| (impl specific) A Gateway containing a Listener of type TLS/Passthrough and a Listener of type HTTPS/Terminate should be accepted, and should multiplex the requests to TLSRoute and HTTPRoute correctly | Being able to do a request to a HTTP route being terminated on gateway (eg.: terminated.example.tld/xpto) and to a TLS Passthrough route on the same gateway, but different host (passthrough.example.tld) | | +| A Gateway containing a Listener of type TLS/Passthrough and a Listener of type HTTPS/Terminate should be accepted, and should multiplex the requests to TLSRoute and HTTPRoute correctly | Being able to do a request to a HTTP route being terminated on gateway (eg.: terminated.example.tld/xpto) and to a TLS Passthrough route on the same gateway, but different host (passthrough.example.tld) | | +| A Gateway containing a Listener of type TLS/Passthrough and a Listener of type TLS/Terminate should be accepted, and should direct the requests to the right TLSRoute | Being able to do a request to a TLS route being terminated on gateway (eg.: terminated.example.tld/xpto) and to a TLS Passthrough route on the same gateway, but different host (passthrough.example.tld) | | | A Gateway with \*.example.tld on a TLS listener should allow a TLSRoute with hostname some.example.tld to be attached to it (and the same, but with a non wildcard hostname) | TLSRoute should be able to attach to the Gateway using the matching hostname, a request should succeed | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | | A Gateway with something.example.tld on a TLS listener hostname should not allow a TLSRoute of \*.example.tld to be attached | TLSRoute should be rejected with invalid hostname (we should NOT support wildcard hostnames on a TLSRoute spec) | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | | Invalid TLSRoute with invalid BackendObjectReference performs no default forwarding | | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | -| For a [Listener](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) setting mode: "terminate", TLSRoute should not be present in [ListenerStatus.SupportedKinds](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) | | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | -| Attempting to attach a TLSRoute to a [Listener](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) setting mode: "terminate" should yield [RouteConditionType](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) Accepted with status: false and [RouteConditionReason](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) NotAllowedByListeners on the TLSRoute | | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | -| For a [Listener](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) setting mode: "terminate", attempting to set TLSRoute in [AllowedRoutes.kinds](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.AllowedRoutes) would yield a [ListenerConditionType](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) Accepted with status: false and [ListenerConditionReason](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) InvalidRouteKinds | | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | +| For a [Listener](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) setting mode: "terminate", TLSRoute should be present in [ListenerStatus.SupportedKinds](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io%2fv1beta1.ListenerStatus) in case TLSRoute termination is supported | | [https://github.com/kubernetes-sigs/gateway-api/issues/1579](https://github.com/kubernetes-sigs/gateway-api/issues/1579) | Pending conformance verifications: diff --git a/geps/gep-2643/metadata.yaml b/geps/gep-2643/metadata.yaml index b629463c21..2a54e716c2 100644 --- a/geps/gep-2643/metadata.yaml +++ b/geps/gep-2643/metadata.yaml @@ -13,4 +13,5 @@ relationships: - number: 2907 name: TLS Configuration Placement and Terminology references: - - https://datatracker.ietf.org/doc/html/rfc6066#section-3 \ No newline at end of file + - https://datatracker.ietf.org/doc/html/rfc6066#section-3 + - https://github.com/kubernetes-sigs/gateway-api/issues/2111 \ No newline at end of file